Sunday, February 19, 2017

Thoughts on Silver's Signal & the Noise - Easy-to-read tidbits on practical statistical prediction in many fields

I read Nate Silver's book "The Signal and the Noise" with great interest. The author is a renowned popular statistician who has become quite famous in the world of political forecasting. I was keen to see his insights more generally on predictions. The book gives a nice overview of statistical prediction in many contexts, far more than just in political science or economics. In fact, I was most interested in the sections on weather forecasting and earthquake prediction. The book talks about how weather forecasting was one of the earliest places where people tried to do prediction using computers -- doing it, in fact, through simulation of physical models. At this point they found out that the predictions did not work very well because of the complexity of simulating such complicated systems; the butterfly effect was discovered. However, Silver talks about how weather forecasting has grown into a very successful tool where people now routinely make predictions that are quite useful to much of the world's population yet the predictions are still quite statistical in nature. These powerful predictions come from fusing physical models with lots of real world data, collected through sensors and satellites orbiting the sky. This is in great contrast to what happens in earthquake prediction where in a sense one also has a similar situation of an underlying physical model but one cannot readily observe and collect data since most of the forces and factors in earthquake prediction happen far underground. There are many other predication realms that the book discusses in great detail and overall I would highly recommend this book to anyone interested in practical statistical prediction.
The Signal and the Noise: Why So Many Predictions Fail--but Some Don't
by Nate Silver