Really sharp post from Andrew McAfee today on when data analytics trumps expert intuition. Data systems can see patterns over long periods, where human learning from feedback favours short cycles. The critical dependency to my mind however, is that in big data using multiple data sources, you have to have a reliable means to resolve different labels for relevant entities and events to the same concepts. Without that the variability of vocabulary will stymie your attempts to see the patterns over the multiple sources ( and time periods). In case you think this is a minor issue, check out the 2001 deportation of Australian citizen Vivian Alvarez "as an unlawful non-citizen" - all because the various immigration department databases could not resolve different variations of her name.
By Patrick Lambe
RSS Feed