Leaky Abstractions

A popular way of dealing with complexity in software is to encapsulate lower-level functionality behind layers of abstraction:

• A file is a useful abstraction over the bits stored on a hard drive.

• An HTTP Request is a great abstraction over bits zooming along the internet

• In the physical world, your TV's remote is a great abstraction over sending channels or volume commands as infrared signals

A perfect abstraction hides all its internal details from the outside. It only exposes an interface at a higher level of abstraction: Operating a remote control involves pressing buttons, not concerning oneself with the operational characteristics of an infra-red LED.

In practice, there's always some leakage. The TV remote is all about buttons, but the physical limitations of infrared beams become apparent as soon as one of your kids is blocking the path between the remote and the TV.

In cases of mild leakage, we can often lift the leaked detail from the lower to the higher level of abstraction. Without dwelling on the details of electromagnetic radiation, the interface for the TV remote is:

• Press buttons to tell the TV what you want

• Make sure there's a clear path from remote to TV

Unfortunately, there are situations where the lower-level details leak out in a way that can't be incorporated into the higher level without deeply understanding the lower level.

Which brings us, of course, to AI systems in general and large language models in particular. More on that tomorrow!