Vibe coding toward the incident horizon
=======================================

We are living through a golden age of generative AI: a time when
progress is both breathtaking and somehow still unable to reliably
open a PDF without hallucinating the author's middle name. The curve
is real, the funding is real, and the demos are so real that they
must be watched on a stage with dramatic lighting, because ordinary
lighting reveals too much. The modern model does many things
remarkably well--summarization, translation, code generation--and
then it will confidently assert that 9 is a prime number
"depending on your threat model," which is how you know you're
witnessing history.

A lot of this is because we trained the future on the internet, which
is like training a gourmet chef by locking them in a gas station for
three years with unlimited energy drinks and a copy of "Culinary
Theory (Unofficial Fan Wiki)." The corpus contains Shakespeare,
medical textbooks, and the kind of Reddit thread where a man explains
that welding is safer without a mask because "my uncle never wore one
and he only glows on weekends." The model digests all of it with
equal serenity. It does not learn truth so much as it learns
/the shape of sentences that arrive after confidence/. This is why it
can produce a flawless explanation of distributed consensus while simultaneously insisting that TCP stands for
"Total Cloud Persistence." The system is not lying; it is performing
a statistically accurate reenactment of a person online.

Naturally, companies respond with marketing claims about "PhD-level
reasoning." And sure: the model can generate a literature review at a
speed that implies it is either brilliant or committing a crime. It
can draft a grant proposal that contains all required sections, three
new sections you didn't ask for, and a concluding paragraph that
reads like it was written by a very polite fog machine. Yet the same
system will fail at tasks that toddlers solve using pure spite, like
"put the triangle in the triangle hole." It can explain category
theory and then forget what a category is mid-sentence, the way a
browser can throw seven errors and keep rendering anyway, because
consequences are a legacy feature.

The most reliable promise is also the simplest: AGI is always one
year away. Not /a/ year--*the* year, a mythical constant like ã or
"next quarter," eternally approached and never reached. This is
convenient for everyone involved. Executives can announce imminent transcendence while remaining safely employed in the
pre-transcendence economy. Investors can fund the revolution without
the awkward part where the revolution arrives and asks for
accountability. And the rest of us can enjoy the comfort of knowing
that whatever is happening now is not the real thing; the real thing
will arrive next year, fully aligned, thoroughly audited, and
carrying a tasteful slide deck.

Meanwhile we are told that non-programmers can build serious
applications through vibe coding, which is true in the same way that
non-pilots can land a plane if you redefine "land" as "become part of
the landscape." Vibe coding produces software the way a s‚ance
produces architectural drawings: everyone is emotionally involved,
nobody can reproduce the result, and at the end a table has moved
slightly. The app usually works in the demo because the demo is the
model's natural habitat: a carefully curated universe in which users
behave correctly and the network never blinks. Then you ship it, and
a user does something monstrous like entering an emoji into a
phone-number field, and the system responds by inventing a new
category of error that HR cannot classify.

From here the conclusion is always delivered with the seriousness of
a prophecy: there is no future in programming; programming jobs will
disappear. This is plausible if you define "programming" as "typing
code" and define "disappear" as "mutate into ten other jobs that
still require engineers." Code will be generated. So will bugs. So
will security vulnerabilities that look like they were handcrafted by
a bored demon with an excellent CI pipeline. The remaining human work
will be debugging, governance, and the ancient art of explaining to stakeholders that "it worked in the demo" is not a compliance
framework.

And then come the humanoid robots, inevitably described as replacing
all human labor because they have hands, feet, and the haunting,
unblinking optimism of a product page. Humanoids are appealing
because the world is built for humans: stairs, doorknobs, forklifts,
keyboards, the entire category of "things that were designed without
asking permission from physics." But reality is rude. A robot can
carry boxes until it meets a wet floor, a narrow hallway, a dog, or a toddler--each of which is a chaos engine disguised as a small
problem. The robot will do your job flawlessly right up to the moment
it encounters the office kitchen, where it will stare into the sink
like it is reading an ancient prophecy, and then it will file a
ticket: *Cannot proceed: dishes are unstructured data*.

So yes: progress is astonishing. Models write code, images, prose,
and occasional small pieces of legal doctrine that would get a real
lawyer gently escorted out of the building. The failures are also
astonishing, because they are not subtle failures; they are
cliff-edge failures, the kind that make you wonder if the system is
brilliant, broken, or simply experiencing a deep and personal
disagreement with the concept of "Monday." The future will arrive,
probably one year from now, and it will be amazing. In the meantime,
we will keep shipping systems that can draft a dissertation but
cannot reliably tell whether "seven" is greater than "elephant," and
we will call this "reasoning," because optimism is a powerful
runtime. *

* With due reverence to James Mickens's talent for weaponized hyperbole
and to ChatGPT's tireless production of statistically plausible
sentences.

From: <https://www.spinellis.gr/blog/20260302/>

--- PyGate Linux v1.5.13
* Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)