Algorithmic Data Analytics, Small Data Matters and Correlation versus Causation

Zusammenfassung

This is a review of aspects of the theory of algorithmic information that may contribute to a framework for formulating questions related to complex, highly unpredictable systems. We start by contrasting Shannon entropy and Kolmogorov-Chaitin complexity, which epitomize correlation and causation respectively, and then surveying classical results from algorithmic complexity and algorithmic probability, highlighting their deep connection to the study of automata frequency distributions. We end by showing that though long-range algorithmic prediction models for economic and biological systems may require infinite computation, locally approximated short-range estimations are possible, thereby demonstrating how small data can deliver important insights into important features of complex “Big Data”.

The chapter is based an invited talk delivered to UNAM-CEIICH via videoconference from The University of Sheffield in the U.K. for the Alan Turing colloquium “From computers to life” (http://www.complexitycalculator.com/TuringUNAM.pdf) in June, 2012.

Hector Zenil is a Principal Investigator and Assistant Professor affiliated to the Department of Computer Science, University of Oxford in the UK; and the Unit of Computational Medicine and SciLifeLab of the Karolinska Institute in Sweden. After a PhD in Theoretical Computer Science from the University of Lille 1 in France and a PhD in Philosophy and Epistemology awarded by the Sorbonne (Paris 1), he joined the Behavioural and Evolutionary Lab, University of Sheffield in the UK. He is also the head of the Algorithmic Nature Group and has been a visiting scholar and professor at MIT/NASA, Carnegie Mellon University and the National University of Singapore.