Our great leap into the world of data has come with a giant leap of faith that the core tenets of statistics no longer apply when one works with sufficiently large datasets. As Twitter demonstrates, this assumption could not be further from the truth.
In the era before “big data” became a household name, the small sizes of the datasets most researchers worked with necessitated great care in their analysis and made it possible to manually verify the results received. As datasets became ever larger and the underlying algorithms and workflows vastly more complex, data scientists became more and more reliant on the automated nature of their tools.
How is it that data science as a field has become OK with the idea of suspending its disbelief and just trusting the results of the myriad algorithms, toolkits and workflows that modern large analysis entails? As data analytics is increasingly accessed through turnkey workflows that require neither programming nor statistical understanding to use, a growing wave of data scientists hail from disciplinary fields in which they understand the questions they wish to ask of data but lack the skillsets to understand when the answers they receive are misleading.
Analytic pipelines that once connected open source implementations of published algorithms are increasingly turning to closed proprietary instantiations of unknown algorithms that lack even the most basic of performance and reliability statistics. Eager to project a proprietary edge, companies wrap known algorithms in unknown preprocessing steps to obfuscate their use but in doing so introduce unknown accuracy implications.
Singapore Latest News, Singapore Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Household net worth falls by largest amount since the Great Recession, new Fed data showsThe drop is because of the poor performance of the stock market in the fourth quarter of 2018.
Read more »
How To Solve Data Management Challenges In A Microservices WorldBy using the proven advanced features of Oracle Database, organizations can more easily realize the benefits of microservices-based applications while avoiding much of the complexity and cost.
Read more »
Square Capital head: Leveraging data to extend credit to small businesses that lack access to loansJim Cramer sits down with Jackie Reses, the head of Square Inc.'s Square Capital, to discuss how the lending arm is helping businesses that have trouble getting traditional loans.
Read more »