A couple of articles from zdnet.com:
Big data's biggest problem: It's too hard to get the data in: "According to a study by data integration specialist Xplenty, a third of business intelligence professionals spend 50% to 90% of their time cleaning up raw data and preparing to input it into the company's data platforms. ... The data cleansing problem also means that some of the most widely sought after professionals in the tech field right now are spending a big chunk of their time doing the mind-numbing work of sorting through and organizing data sets before they ever get analysed."
The dirtiest little secret about big data: Jobs: "As a result, many companies have tried to put machine learning and artificial intelligence to use in doing some of the data sorting and data cleansing... Still, a lot of these companies quickly run up against the limitations of AI. It's excellent for very narrow, specific purposes. But, it's not very good at making judgment calls or deciding on something that falls into a gray area... Because of that, many companies have discovered that humans do the work of data sorting much better than algorithms, and so they are putting people to work behind the scenes to help their big data projects succeed."
Big data's biggest problem: It's too hard to get the data in: "According to a study by data integration specialist Xplenty, a third of business intelligence professionals spend 50% to 90% of their time cleaning up raw data and preparing to input it into the company's data platforms. ... The data cleansing problem also means that some of the most widely sought after professionals in the tech field right now are spending a big chunk of their time doing the mind-numbing work of sorting through and organizing data sets before they ever get analysed."
The dirtiest little secret about big data: Jobs: "As a result, many companies have tried to put machine learning and artificial intelligence to use in doing some of the data sorting and data cleansing... Still, a lot of these companies quickly run up against the limitations of AI. It's excellent for very narrow, specific purposes. But, it's not very good at making judgment calls or deciding on something that falls into a gray area... Because of that, many companies have discovered that humans do the work of data sorting much better than algorithms, and so they are putting people to work behind the scenes to help their big data projects succeed."
By Maish Nichani
RSS Feed