all bits considered data to information to knowledge


Data Scientists or… Psychohistorians?

Before the Big Data, social Data Science/Data Mining and Machine Learning there was … Psychohistory!

The concept was introduced in 1951 by Isaac Asimov in his monumental Sci-Fi trilogy “ The Foundation”, and is very closely correlated with this “new” phenomenon of statistical modeling of the social interactions.

Proof? The definition from Encyclopedia Galactica quoted at the beginning of the 4th Chapter of The Foundation Trilogy:

Gaal Dornick, using non-mathematical concepts, has defined psychohistory to be that branch of mathematics which deals with reaction of human conglomerates to fixed social and economic stimuli …

… Implicit in all these definitions is the assumption that the human conglomerate being dealt with is sufficiently large for valid statistical treatment. The necessary size of such conglomerate may be determined by Seldon’s First Theorem which… A further necessary assumption is that the human conglomerate be itself unaware of psychohistoric analysis in order for its reactions to be truly random…

The basis of all valid psychohistory lies in the development of the Seldon Functions which exhibit properties congruent to these of such social and economic forces as …”


Asimov correctly points out the boundary conditions  of this statistical analysis – for this to work the society must be unaware of the analysis taking place and/or how it works as this would skew the distribution curve. After all, if the people stop clicking on these links and like-me-buttons, and stop sharing their information  (or worse – start feeding in some garbage data) all these sophisticated models would go haywire.

To continue analogy, the "Mule" character represents the "Black Swan" event that invalidates the entire premise based on normal distribution.