A day at the races

#### Nine. Billions of nanoscale switches

On the topic of growth I would like to do a deeper dive by taking hardware as a starting point.

Feynman’s lecture in December 29th 1959, “There’s Plenty of Room at the Bottom” is the seminal work on nanotechnology. In it Feynman imagines a future where manufacturing would involve the direct manipulation of atoms being able to work at scale and develop processes that would be radically different from the chemical processes of his time. Feynman’s ideas of being able to create synthetic products at vastly smaller scales turned out to be prophetic. In his lecture he also explored the logical ramification of such abilities, such as the development of the electron microscope or the ability to create microelectronics at a scale that scarcely few people could imagine at the time. The room that Feynman referred to, is the bottom end of the scale of matter. The vast space between the hand operated machinery of his time “to the billion tiny factories” that he envisioned operating at nanoscales, composed of just a handful of atoms.

One of the growth factors of digital consumer electronics, as foretold by Feynman, has been miniaturization. The ability to pack ever larger quantities of processing power into tiny Silicon Oxyde dyes. The development of new materials and photolythographic processes allowed increasingly small component sizes in microcircuitry. The average size of a transistor in 2022 is barely 4nm across, roughly 40 atoms across in size. The modern MOSFET transistor renders Feynman’s idea of the nanomachine that powers the tiny digital factories of today’s microprocessors. 

The theoretical minimum size of a transistor has been tested experimentally, and depending on sources, there’s been success with transistor designs that are three atoms across. Some more recent experiments even seemed to manage with just one atom. One order of magnitude smaller than today’s processes. It seems that at least theoretically, there’s still some room at the bottom. Processes for these tiniest of transistors are not yet a viable for large scale manufacturing. There are however practical limitations to reach these tiniest of scales. Tiny transistors, just a few nanometers in size are subject to quantum forces that increase parasitic effects, meaning that at scales below 30nm it gets increasingly hard to get transistors to behave predictably as they trigger due to these quantum effects, making further miniaturization increasingly harder. This is the reason why processor clock speeds have not increased significantly in the last decade. Increasing clock speeds in processors using such small scaled transistors would render them unstable and useless.

Since the release of the AMD Athlon X2 microprocessor about 15 years ago, whatever increase in performance you have experienced in computers, comes not just by increasingly arduous miniaturization but also by adding cores. Processors with eight cores are now common even in smartphones. Parallel multiprocessor computing, once accessible only to elite institutions is now the norm in smartphones. With clock speeds unable to increase, multi-core architectures are another vector of growth for digital circuitry.

Ten. The “30 million lines of code” problem

To send an email from the average desktop email client today. It is estimated that about 30 million lines of code must work reasonably flawlessly for that to happen, this is a low ballpark estimate. One could argue that not all possible logical paths will be activated, so not all 30 millions lines of code must work flawlessly in harmonious simultaneity.

The point of this statement is not to quantify precisely the number of lines of code for a given activity but to illustrate the level of complexity that we are discussing when taking about daily activities on a regular computer. 

These millions of lines of code were written cumulatively by thousands of different programmers, often in periods of time spanning a decade or more. 

To the end user, an operating system and a browser or email client, give the appearance of a smooth surface with a uniform interface, concealing the colossal human effort that it took to build it.

Under the hood, to the programmer lay thousands of APIs at different levels of abstraction, each with it’s own culture, each with it’s own conventions and bugs, it’s own quirks, placed there by individuals or through an arduous process of consensus by communities. The uniformity and appearance of unity of a system, disappears the moment one approaches it from the developer’s perspective.

Eleven. Known unknown

We know that at any given time it is not possible to ascertain with absolute certainty what is the state of a modern desktop machine. (remember what Ash said)

In a best case scenario you might be running a Linux distribution with perfectly reproducible builds. That means that the makers can trace back their built products to the exact source of every single piece of code that goes into that specific build. This makes the build verifiable from source to binary, meaning that nothing unexpected can be found in it. Archlinux and NixOS are among the few Linux distributions that are verifiable or can be made verifiable. 

Even when you have a reproducible build of your OS, which can only be said of the most meticulously built operating systems and a minority of security sensitive software. All other software running on your machine right now, is built with much more lax standards of reproducibility, which means that the makers can’t guarantee that a specific version of the software contains a specific version of the code.

This effectively means that you can’t know what you are running in a system with no reproducibility. Even a best informed person, can’t possibly know if the code has been tampered with, replaced with less amicable versions or if the software effectively does what it says it does. If the makers of the OS can’t even provide this level of verification, the average user stands no chance. The average home desktop runs an ecosystem of software that interacts in unknowable ways, even whole components of the original OS might have been compromised by a graphics driver install and there would be no way for the end user to tell.

The npm software repository underpins much of modern web application development. In the last few years it has been infiltrated numerous times, people have been injecting malware in packages that are downloaded millions of times every month. These packages are then incorporated into other web applications while the authors might be unaware that a dependency they need has been compromised. Sometimes these packages stay in repositories for weeks before anybody finds out about the infiltration.

The only way we can possibly find out if our system is running a malicious piece of software is by looking at footprints that this software might leave behind in parts of our system. This is how antivirus software works, checking for specific footprints in the system that the virus leaves behind, e.g. looking at anomalous network traffic, patterns of memory usage and process allocations, file hashes, binary patterns in files, etc. In short, by looking at its physical symptoms and its behaviours. We approach complex computational systems the same way medicine and psychology approach the human subject. Any diagnostic is based on observations of a hugely complex system whose parts interact in complex, sometimes even mysterious ways.

In popular culture computers are deemed as deterministic systems, cultural commentators often use this perceived quality to dismiss them as useless because “they can only give you answers”. When one adopts a certain way of looking at them they becomes as mysterious as the human body or the human mind. There’s a near infinite number of questions that can spring from accepting that they are human designs.

Twelve. Is it getting better or worse?

Douwe argues that software is getting better. His argument is that in 2022 a person with relatively little knowledge can install Yunohost and run a social network on their own with a couple of hours of work.

The ability to do this however comes at a cost, and the cost is exploding unmanageable complexity. At the end of that rather user-friendly and seemingly straight-forward installation process you are now running a base operating system that racks up some 30M lines of code plus a huge number of dependencies to setup the social network of your choice.

In doing so, you are giving that server a new function, it has become more useful at the cost of compromising what can be known about that server. Was a new version of a low-level library installed in the process? how else has the system been affected by that upgrade?

From a developer’s perspective, the general sentiment is that software is getting worse, much worse and fast. The average Electron app occupies 180MB of diskspace even if it’s just to open a blank window with no content. To produce an Electron binary, millions of lines of code from thousands of authors are pulled together into the build process.

In my lifetime I have seen how the complexity of this process has exploded beyond the cognitive abilities of a single human. A little over twenty years ago it was possible to open a binary file and understand how it had been assembled and have an overview of each of its parts. Today we build web applications that get pushed out to millions of users and we have absolutely no idea what each component part of that web application’s code is doing at any given time and how.

The job of the software developer today consists of crafting arcane recipes that bring together colossal assemblages of odd-shaped pieces to build a tittering edifice of functionalities. The only way to determine if the system is doing what is meant, is by observing its behaviour and making sure that this behaviour closely matches a set of ever evolving requirements. Nothing in this entire chain of events ever stands still for a period of time long enough to audit systems in their entirety.

All these layers of abstraction add up to a complex stack that requires ever more modern hardware to run, with only marginal gains in usability for the user. Creating an ever more intensive life cycle of software updates, demanding hardware upgrades. Phasing out support of systems that serve people just fine and demands ever so slight tweaks to their training, creating a drain on organizations and the humans that work in them.

> Alice never could quite make out, in thinking it over afterwards, how it was that they began: all she remembers is, that they were running hand in hand, and the Queen went so fast that it was all she could do to keep up with her: and still the Queen kept crying “Faster! Faster!” but Alice felt she could not go faster, though she had not breath left to say so.

The person that had more than enough word processing capabilities in Word 6 cannot easily opt to stay with that software, which from the optics of that person is perfectly fine for them, a tool that they could use for life. Instead we trap these people in a cycle of updates that are unnecessary to them, updates that sometimes force larger cognitive loads on them as individuals, retraining them, putting greater demands on them such as fast internet connections to use software that just three versions ago was working fine in their offline desktops. 

> “Well, in our country,” said Alice, still panting a little, “you’d generally get to somewhere else—if you run very fast for a long time, as we’ve been doing.”

> 

> “A slow sort of country!” said the Queen. “Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”

If a user were to opt out of this race at the hand of the Red Queen’s of technology, they would have to live in the very fringes of consumer computing. Forced to maintain old hardware, keeping a stock of spare parts, never connect their computer to the internet so that upgrades are not pushed unwillingly onto their puny aging machines. You would have to learn how to repair it yourself as tools and knowledge of dated hardware becomes less commercially exploitable by repair shops in the high streets.

The wealthiest corporations on Earth today are all software companies, in one way or another. If software is not their primary product, they rely on software to a very large extent in their operations to achieve the levels of efficiency and market penetration that makes them profitable. This race of the Red Queen of upgrades that all of us are subject to, is most suitable to them. It generates a virtually hegemonic computing landscape in which they decide what gets support, until when and on what platforms. What hardware is reasonable to run what version of their software and what hardware people will need to run the next version of their platforms. Neither you nor governments get a say in that, it is the invisible hand of the market that you are holding in this desperate race.

The interdependence between software and hardware is such that it is hard to determine which drives which. What is clear is that from the business perspective ultrafast obsolescence is profitable to both hardware and software vendors, on this their incentives for obsolescence are in perfect lockstep. And these are the incentives that drive the invisible hand of that market.

Thirteen. The curious case of the pocket computer that could

Many of us today carry a smartphone in our pocket. There are 5.6 billion smartphones on planet Earth today, it is by far the most widely available form of computation available. At all of those smartphone, run either one of two operating systems. I cannot think of any other product produced by the thermo-industrial complex that exerts such a push towards uniformity. The differences between one smartphone and another are ridiculously incremental.

The latest generation of these devices are general purpose computers running on eight cores at multi-gigahertz speeds, with custom hardware for graphics, sound and video decompression. They are veritable multimedia wonders that pack an incredible amount of computational power. They are about the most useful computers we own and their life-cycles are the shortest of all. The average smartphone is replaced within four years while the average desktop gets a six year life span.

One could argue that with an efficient and diverse software ecology that enabled more experimentation in the kinds of things these devices could do and with a handful of spare parts, most of these devices life spans could be extended well into the decade if not longer. And by efficient and diverse software ecology, I do not mean another build of Linux to replace Android or iOS, or some other postmarket OS. But software built from a perspective of scarcity and not abundance. Software that gets designed from the ground up from the perspective that “this is the only personal computing device you are ever going to own” and let human creativity build diversity from there.

There’s no good reason why our smartphone couldn’t become able to run all of the Playstation games ever, and still be able to make calls. We could host our own personal websites or social media profiles on our phones, without the need for platforms that mediate this. All of this and a lot more could potentially be built if we were to abandon the paradigm of the smartphone as a closed software ecosystem with two large corporate gatekeepers and one single paradigm of utility.

A sustainable digital ecology is one in which the smartphone you carry with you everywhere is torn open and rethought from the perspective of early personal computing.

Fourteen. The security red herring

In a conversation during the last 17th meeting with our young computer science student. I proposed to him a scenario where a computer system could be finished. By finished I meant that no hardware or software updates were needed. It seemed genuinely striking to him – but you would need security updates!? – he said. We have become so accustomed to the status quo of the upgrade race that we can scarcely imagine an alternative. But we are all in fact surrounded by such devices that can be called finished.

Network infrastructure, from specialized switches to telephony base stations, hardly if ever, get updates, although this might not hold true for a lot longer as it is becoming more common these days.

The Commodore 64 that some people keep in their attic, probably still boots up and if it does, it’s running the exact same version of the software that was in its memory in 1982 or whenever we bought it.

Cars, televisions, washing machines, synthesizers that used to have software running on their microcontrollers, have also become general computation devices. And with this questionably necessary improvement they have also taken up the upgrade race. It wasn’t that far back in the past that these devices used to be finished, at least as per the above definition of finished.

At the last LIMITS conference a number of the papers presented possible solutions to the over-consumption of hardware, at least within the context of educational institutions. One of the most often repeated bullet points was the utilization of older hardware, keep old machines circulating and extend their usable lifetime. 

This seems reasonable, after all most CO2 emissions produced by a laptop, take place during the manufacturing process. We can call the energy investment that goes into creating a new device its embodied energy budget. A new machine costs the planet a lot more than extending the life of a machine that is already around. 

A paper from 2019 discusses how “material contents of key laptop components such as the motherboard have been demonstrated to be fairly constant per laptop over time”. So the embodied energy of a laptop is static over time. [2]

The idea of putting old hardware back into service seems at first quite appealing for the environmentally conscious DIYer. Old computers can be used perfectly fine as music streaming stations, software routers, domain controllers and many other roles that require a fairly constant computational load. But for those who are more security aware this might not seem such a good idea.

Remember the Meltdown and Spectre bugs? What is special about these security vulnerabilities is that they were found in the hardware and affected all computers that followed a specific hardware architecture, including mobile devices, servers, desktops and laptops. Basically all computers manufactured since 1993 were exposed to these design bugs.

Patches for these kinds of extremely serious bugs aren’t always backported to older operating systems. Bringing old hardware back into service on potentially sensitive infrastructures can have problematic security implications. Systems are often marked as obsolete in the name of cybersecurity because patching them becomes impractical, sometimes even impossible. Security is another vector contributing to the upgrade arms race. Many organizations would actively resist the deployment of older hardware knowingly on this basis alone.