32-bit fabric

“I fed a picture of donald duck to my computer and asked it to identify said duck. I then asked my computer to draw donald duck’s girlfriend in svg format on my shitty decade-old gtx 1080. It ran into an infinite loop since the number of polka dots kept exceeding its context window. So finally I decided to generate a textured 3D mesh from donald duck and it did so within 5 minutes without much error, then I approved for my computer to send it to my 3D printer”

This comment was not something I would have made 10 years ago, nor would have expected 5 years ago, even 3 years ago once LLM quantizatization largely enabled LLMs to be run locally on 8 gb of vram. Nor did I think it would have been a comment that I would have made 6 months ago. It’s not exactly surprising, except for how quickly I was dragged into it, within the past few months, with my limited resources.

If we split the comment into sentences associated with years that it became plausible for an enthusiast with unspecialized hardware:

“… decade-old gtx 1080 – 2016
“… asked my computer to identify said duck”¹ – 2021
“… draw donald duck’s girlfriend in svg format¹” – 2022
“… on my decade-old gtx 1080” – ~2023 - 2024
“… generate a textured 3D mesh from donald duck” – ~2023
“… within 5 minutes” – 2025

Try saying that in 2006, a decade before the gtx 1080 was released, just as PCs were tipping from 32-bit hardware to 64-bit.

We habitually declare computational niches dead far before they flourish to their full capabilities. We can revive decade-old hardware, and extend their capabilities retroactively beyond what was imagined at their conception. The effect flows in both directions in the arrow of time – as compute capability increased, we observed software becoming less efficient at performing the same functions, which demonstrates that there are already more optimal solutions to the same problems that are no longer reached (by choice).

There’s way more out there in computational space to reach extreme capability improvements over the same hardware than what we can comprehend. In 2019 Donald Knuth, in response to explaining his intuition about the possibility of exploring P equals NP, said in his interview with Lex Fridman:

[…] The popular idea is if an algorithm exists, then somebody will find it. And it’s just a matter of writing it down. But many more algorithms exist than anybody can understand or ever make use of, or discover, because they’re just way beyond human comprehension. The total number of algorithms is More than mind-boggling. So, we have situations now where we know that algorithms exist, but we don’t have the foggiest idea what the algorithms are.

By the end of 2026, frontier LLMs have already demonstrated their capabilities to find vulnerabilities and proofs for problems that have taken years if not decades of work, within weeks. In other words, they have already passed the threshold of human energy and comprehension for certain kinds of problems².

In his 1977 essay Intelligent Machines Hans Moravec wrote:

With more processing power, competing presently impractical schemes could be compared by experiment, with the outcomes often suggesting incremental or revolutionary improvements. Computationally expensive highly optimizing compilers would permit efficient code generation at less human cost. The expanded abilities of existing systems such as Macsyma, the symbolic mathematics system from MIT, which can be used as a desk calculator for doing algebra and trigonometry as well as arithmetic, along with new experimental results, would accelerate theoretical development. Gains made this way would improve the very systems being used, causing more speedup. The intermediate results would be inefficient kludges busily contributing to their own improvement. The end result is systems as efficient and clever as any designed by more theoretical approaches, but sooner, because more of the labor has been done by machines.

In that vein, much of Moravec’s writing considers machines to form an evolving ecology. That is, for their communication, navigation, and processing to exhibit the same level of dynamism that we find with natural systems in proximity with each other in a physical space – from proteins, biological organisms, topographical shifts and weather patterns, the evolution of societies and cultures, and so on.

With all these big ideas, I’m just sitting at home by my computers; digesting, breathing, spinning up my slow and sloggish neural circuitry. What do I do with this thought? Justify my computer hoarding tendencies for at least another decade, naturally, and wait for 32-bit machines have their own “GTX 1080” moment.

To a machine, bits are just threads in a fabric. We don’t expect animals to understand cloth-weaving to master the full array of possiible shapes of clothing that we do. Why would we expect ourselves to understand memory space to see more than a sliver of the myriad of possibilities in-which it could be arranged?

I think that there are only a few paradigm shifts left to push the revival period towards compute hardware capabilities through the 2000s³, and we won’t necessarily understand them. Maybe one day my thinkpad T60 will.

• 8 months since this writing, we’ve already seen increadible improvements in LLM performance and efficiency
• Tiny local models in 6502 processors