Sunday, August 20, 2023

Some Random Thoughts on Scaling Laws (Moore's, Eroom's, Lockheed's, &c)

I enjoyed this recent news story in The Economist on Lockheed's Law (https://www.economist.com/finance-and-economics/2023/06/22/can-the-west-build-up-its-armed-forces-on-the-cheap). The article describes an almost inverse scaling behavior where defense and military orders are getting more and more inexpensive, mostly because the allocation of the military budget to manpower is progressively decreasing, and it is possible for manufacturers to make munitions more efficiently. Lockheed's Law is extremely important in terms of how the overall market opportunities (using the language of Kolter et al.) will change in the future. One can imagine that war will become less manpower-intensive and more capital-intensive because of this type of scaling. 

It is interesting to compare the scaling that we see in Lockheed's Law to other famous scaling relationships that have changed markets dramatically. The most famous is Moore's Law, which describes the exponential scaling of computer chips and computation in general over the last couple of decades. This scaling powered the computer and electronics industry. There are several related laws, such as Kryder's Law, which describes the exponential scaling in disk drive capacity. These laws made possible new market opportunities. The extreme case is the handheld or wristwatch computer and the fact that people now carry around computers that are much more powerful than the massive supercomputers of the past.

An associated law describes an opposite trend in the pharmaceutical industry: this "law," dubbed Eroom's Law (i.e., Moore's Law in reverse), states that drugs are becoming exponentially more expensive to put on the market. This is a great comparison to Lockheed's Law. The idea is that it is becoming much more expensive to make pharmaceuticals due to the ever-increasing requirements for research development and larger and more complicated clinical trials. A related scaling law describes the ability to sequence genomic material. This indicates a hyper-exponential increase that, at some points, is even faster than Moore's law. The scaling for DNA sequencing enables very large-scale biomedical research and diagnostic testing, but it hasn't made an impact on new drug discoveries because of the inverse scaling described in Eroom's law. Combined, Eroom's Law and the related law for DNA sequencing are reshaping the biomedical and pharmaceutical industry, making drugs comparatively more expensive but creating new opportunities for diagnostics, which are very cheap. This, in turn, is creating market opportunities for companies that are marketing only drug targets and associated research but don't have the capital to develop a drug themselves.

Altogether, different scaling laws illuminate large-scale trends that are dramatically impacting production as well as market opportunities. What was inconceivable 30 years ago is now trivial because of the exponential scaling in Moore's Law. Perhaps Lockheed's Law will lead to a type of warfare in the future that is so capital-intensive that it will be fundamentally different from the type of engagement we saw in the First and Second World Wars, and this, in turn, will reshape the market for defense products. 

References

Can the West build up its armed forces on the cheap?
https://www.economist.com/finance-and-economics/2023/06/22/can-the-west-build-up-its-armed-forces-on-the-cheap

Breaking Eroom’s Law
https://www.nature.com/articles/d41573-020-00059-3

Moore's Law
https://en.wikipedia.org/wiki/Moore%27s_law

DNA Sequencing Costs
https://www.genome.gov/about-genomics/fact-sheets/DNA-Sequencing-Costs-Data

Other Stuff Related to Scaling





Nice scaling graph for the size of transformer models
Labels: , , -- 20.8.23

Saturday, June 25, 2022

Thoughts on 3 of Kandel's books on the Brain (The Disordered Mind, In Search of Memory & Reductionism in Art and Brain Science): Different Views of the Elephant

Here, I summarize my thoughts on three books by Eric Kandel: The Disordered Mind, In Search of Memory, and Reductionism in Art and Brain Science: Bridging the Two Cultures. Eric Kandel is a great American scientist and winner of the Nobel Prize. Interestingly, he started his career as a humanities major at Harvard, and he writes very much in that tradition. 

    The books cover various topics, including psychiatric disorders such as schizophrenia, the molecular basis of memory, and the relationship between the subconscious and art, and incorporate his recollections of his life journey and world history, particularly related to Vienna. While the books focus on different things, they all look at different aspects of the same subject, like taking different views of one elephant.

    When discussing psychiatric diseases, I like how Dr. Kandel described the root causes and history of schizophrenia and autism. These diseases trace much of their early history to Vienna and some famous early brain scientists there, such as Kraepelin and Asperger. Within the topic of memory, I liked Dr. Kandel's reflection on how memory is stored in synapses from inhibitory and excitatory neurons, and, in parallel fashion, these synaptic memories turn into molecular events and gene expression through activating and repressing transcription factors of the CREB family. Kandel also talks about his own memories. It was striking how Vienna was such a center of scholarship in the early 20th century and so quickly fell into tragedy with the advent of the Nazis and has changed dramatically since then.

    Finally, I enjoyed reading about Dr. Kandel's relationship between the subconscious and art. He talked about how many recent artists have tried to move beyond the conscious representation of the figure and harness their subconscious and how abstract art can play into our deep mental processes, such as face recognition.

    Overall, I found these books very interesting reads that give an encompassing picture of both the mind and a great person.


The Disordered Mind
In Search of Memory
Reductionism in Art and Brain Science: Bridging the Two Cultures
by Eric Kandel

Tagged

Labels: , , , , , , , -- 25.6.22

Saturday, April 30, 2022

Thoughts on Schonbrun's Performance Cortex: A great account of how sport is as much about mind as muscle

I enjoyed reading the Performance Cortex by Zach Schonbrun. This book makes an interesting case that athletic performance has as much to do with the mind as with the body and the muscles. The book quotes many famous neuroscientists who argue that the point of the brain is to direct movement and that movement is integral to the brain. It surveys several new companies trying to make "neural" the latest thing in sports, much like sabermetrics, Moneyball, and statistics were the hot thing a decade ago. 

The book also goes through the pioneering work in neuroscience to explain movement, starting with the work of the Cambridge scientist Sherington, who unraveled the mysteries of the spinal cord and reflexes. The book also talks about the importance of how habits are created and how one teaches habits to other parts of the brain than those immediate to consciousness. In particular, it highlights how important it is to chunk and group various movements into an overall, automatic progression. However, it is ironic that many of these things don't have to do with the cortex itself. So in this sense, the book's title is a bit of a misnomer. 

Overall, it is a great book that I would highly recommend.


The Performance Cortex: How Neuroscience Is Redefining Athletic Genius 
Hardcover – April 17, 2018
by Zach Schonbrun
Labels: , , , , , -- 30.4.22

Saturday, September 04, 2021

Thoughts on MacIntyre's Spy & the Traitor: A Gripping Account of a Russian Agent that's Hard to Believe is Non-fiction

I read Ben MacIntyre's The Spy and the Traitor with great interest. 

While reading the book, I was immediately drawn to the thought, "this must be an amazing fictional novel describing a James Bond-like character." But, in fact, this is a work of nonfiction describing the amazing escapades of Oleg Gordievsky, a famous Russian agent who ended up working for MI6. 

Some of Gordievsky’s exploits are truly amazing and hard to believe, such as his escape from the Soviet Union in what was called Operation Pimlico. At another point, he was working as a double agent for the KGB; while in their London office, he was simultaneously advising the UK Prime Minister, Margaret Thatcher, and Gorbachev on their negotiations. 

Overall, this was a gripping read that I would highly recommend.

The Spy and the Traitor: The Greatest Espionage Story of the Cold War:
Ben Macintyre: 9781101904190: Amazon.com: Books

twitter: @BenMacintyre1

Tagged

Labels: , , , , -- 4.9.21

Sunday, August 15, 2021

Thoughts on Herman's Freedom's Forge: the people behind the huge dams & planes of WWII

I read Freedom's Forge by Arthur L. Herman with great interest. This book describes how the United States scaled up to produce many weapons in World War II, becoming the arsenal for democracy. This book is interesting in the current context where the country is scaling up to make a vast number of vaccines to combat the coronavirus.

The book profiles the major industrial figures, such as automobile magnate William S. Knudsen and shipbuilder Henry J. Kaiser, who helped lead this war effort. The book takes a very pro-business and almost hero-worshiping attitude towards these figures, but it is nevertheless entertaining. The mindset is that the scale-up worked well because it followed market mechanisms, used existing supply chains, and utilized a more controlled form of production through a war production board.

I found two things to be notable in the book. First, I was surprised by the degree to which the United States was utterly unprepared for the Second World War in terms of overall production and levels of mobilization, and how many of the major industrial figures, such as Henry Ford, staunchly opposed getting into the war effort. Second, I was impressed by the descriptions of some of the colossal construction projects undertaken, particularly some of the large dams produced by Henry Kaiser, such as the Grand Coulee and the Bonneville Dam, and the huge planes developed. One of them was the B-29 Superfortress, the aircraft that delivered the atomic bomb to Japan. Overall, this was an entertaining read about a huge scale-up.

Freedom's Forge: How American Business Produced Victory in World War II
by Arthur Herman 
https://www.amazon.com/dp/B005X0JG48


Labels: , , , , -- 15.8.21

Friday, July 23, 2021

Thoughts on Kean's Tale of the Dueling Neurosurgeons: Researchers, Patients & Great Debates about the Brain

I read The Tale of the Dueling Neurosurgeons: The History of The Human Brain as Revealed by True Stories of Trauma, Madness, and Recovery by Sam Kean with great interest. 

The title of this book is long and cumbersome, and I didn't find it appealing -- initially. However, it's an accurate description of the book.

The book covers key figures in the "history of the human brain," particularly "neurosurgeons." For instance, it talks about how neurosurgeon Wilder Penfield discovered that the brain could be stimulated by electrical current and how the great neurosurgeon Harvey Cushing performed pioneering work on the pituitary gland. It also discusses work by many other non-surgical researchers, such as Roger Sperry's work on lateralized brains and Prusiner and Gajdusek's work on kuru and protein misfolding diseases.

At first, I found the word "dueling" in the title to be cryptic, but it turns out to nicely summarize how the book presents each of the key figures as addressing a debate in neuroscience. These debates included whether nerve impulses were transmitted through Cajal's neurons or through a non-neural alternative (propagated by Golgi), whether the mode of transmission is electrical or chemical (the soup vs. spark debate), and whether we have just one or multiple types of memory (addressed by Brenda Milner in relation to the famous patient, H.M.). One of the debates I found particularly interesting was Broca and Wernicke's, whether brain functions such as language were localized to specific brain regions.

The final part of the title, "the true stories of trauma, madness, and recovery," illustrates how the book is also about the patients who have given their life to provide information about the brain. These include H.M., and also  W.J. (who was instrumental in understanding the corpus callosum), Mary Rafferty (who died demonstrating that an electrical shock could stimulate various brain regions) and the conjoined brain twins, Tatiana and Krista (who have illuminated much about the concept of consciousness and the self). 

Altogether, I thought this was an entertaining work that taught me a lot about the brain. Its mode of instruction in discussing individual people throughout history and the specific conflicts that they engaged in was effective in getting across key scientific ideas. 


The Tale of the Dueling Neurosurgeons: The History of the Human Brain as Revealed by True Stories of Trauma, Madness, and Recovery 
by Sam Kean (Author)


Labels: , , , , , -- 23.7.21

Saturday, July 03, 2021

Thoughts on López-Alt's Food Lab: Useful Facts about Food

We enjoyed reading The Food Lab by J. Kenji López-Alt. This book illustrates the appreciation of the science behind everyday objects in the kitchen. We particularly liked the explanation of why a water drop speeds around on a very hot pan but boils on a slightly cooler one (because of the Leidenfrost effect), the explanation of why it is so difficult to properly melt cheese (because cheese is a tricky combination of oils, proteins, and other components that melt in different ways), and the discussion of why fish rots so much more quickly and smells worse than meat (because fish is accustomed to being kept at much colder temperatures than beef and placing it in a refrigerator keeps it somewhat warmer than its normal temperature). 

The book also includes many useful facts. For instance, below 38 degrees, which is the temperature of a refrigerator, bacteria stop moving around; above 300 degrees, the Maillard browning reaction occurs; and at 131 degrees, bacteria start to die off. There is also a nice listing of saturated, unsaturated, and trans fats and an explanation of what unsaturation means in terms of keeping the fat oily. The book also describes multiple ways to cook a single dish and why some ways work better than others (i.e., reverse searing steak). 

Aside from the interesting scientific facts and discussion in the book, the layout and structure of the book are easy to follow. For example, after informing the reader about the scientific relationship between heat and eggs, the author provides many egg recipes. In general, a wide variety of recipes are presented in the book, with attention given to a range of cuisines. The book also provides extensive descriptions of what kinds of kitchen appliances and tools are most useful, extending the breadth of topics covered. 

Overall, the book provides a very comprehensive perspective on many topics that would be interesting to those interested in the science of cooking or introductory level cooking recipes for the home.

Mark Gerstein (with Jason Liu)

Amazon.com: The Food Lab: Better Home Cooking Through Science (9780393081084): López-Alt, J. Kenji

https://www.amazon.com/Food-Lab-Cooking-Through-Science/dp/0393081087

My tag

https://linkstream2.gerstein.info/tag/foodlab0mg

Labels: , , , , -- 3.7.21
Subscribe to: Posts (Atom)