James Gleick's book on information theory is very enjoyable to read as it provides intuitive explanations for this very tricky and information-dense field. Gleick goes through the history of information expounding on different personalities starting with Charles Babbage and Ada Byron, the Countess of Lovelace, progressing through Alan Turing and moving onto more present day actors, such as Claude Shannon at Bell Labs and Charles Bennett at IBM. Most importantly, beyond the personalities, the book gives a good intuitive explanation of information theory. In particular, it explains how information can be measured by -log p where p denotes the rarity (i.e. "surprisal") of an event (getting a particular "symbol," such that a symbol from a larger alphabet would carry more information than one from a smaller one). It also has a nice explanation of the Kolmogorov complexity of a message (as the shortest possible computer program that can describe the message) and a good accounting of a variety of logical head-twisters such as Russell's recursion and GĂ¶del's paradox. Finally, the book delves into name spaces and the importance of very simple printing interventions such as alphabetization in coming up with information organizing principles. I would highly recommend this book to anyone who desires to learn about information theory. It is both entertaining and quite educational.

Another interesting tidbit: Charles Bennett at I.B.M had some interesting connections to biology. He developed a subject called "logical depth." It seems that logical depth quantifies information but does so in a way that purely random information or purely "uniform" sequences are down weighted and things that get most strongly up weighted are those "in the middle," with a non-trivial but hidden pattern in them. I thought this was actually a kind of good match to what people are interested

in Genome Annotation. Bennett also goes into how chain letters can mutate and evolve like biological sequences.

http://www.amazon.com/Information-History-Theory-Flood/dp/1400096235

The Information: A History, A Theory, A Flood

James Gleick

Did a review (http://www.amazon.com/gp/cdp/member-reviews/A255VTLSSZIVT9/ref=cm_cr_dp_auth_rev?ie=UTF8&sort_by=MostRecentReview)

My tag: infoth0mg

Related links:

http://aminotes.tumblr.com/post/4872331472/what-defines-a-meme-james-gleick-our-world-is-a

http://en.wikipedia.org/wiki/Minimum_description_length

http://en.wikipedia.org/wiki/Jacquard_loom