LLMs Aren't Inevitable

The more pertinent question in my mind is.. why does it still matter? Your customers don’t care about elegant code, they care about working products. There is some relation to good code and good customer experience e.g in terms of performance, but in general your customers won’t feel the impact from poor code. That assumes both pieces of code are functional though, and you do need to babysit LLMs to make sure it actually remains functional.

At this point I think the only reason someone would do better without an LLM is an unwillingness to use it effectively. Even if you’re going to write all code by hand and only use an agent for validation and debugging, that’s a significant boost.

I’m happy to engage in discourse on the impact and risks to environment, society, jobs, mental health, etc. There’s plenty of room for thoughtful exploration there.

The one thing I have no patience for is that LLMs produce nothing of value or have no technical worth.

I have to keep reminding myself though, many of those pessimists are people “getting paid to not understand” as the saying goes.

It took 30 seconds with that Davinci model to predict where we are today, where we’ll be next year, and the eventual destruction if not outright extinction of humanity as we know it. Maybe the future is so grim many are simply in the denial stage still.

So what has an "LLM" truly improved in your life outside of simplistic automations/code generation/textual work? It's definitely not improved support. The only support interaction that I've noted to be somewhat better was one (of a few) with Amazon where the bot was able to refund a purchase without talking to a real person. But even then it was a case wherein a human would have probably done it just as quick. Is there a win there? Sure: for Amazon but it didn't "delight" me. And a chatbot is the lowest of low hanging fruit all the "optimists" claim an LLM can do so much better than humans or deterministic code.

How would you say LLMs are impacting the markets? Do you think they're useful to the economy thus far? Or are you looking at the spend of a small fraction of companies that could cause a broader recession when the cyclical funding dries up? If you're going to go to open weight models - well those don't appear out of thin air. Is your focus on the win/loss truly that myopic?

Again, your statement seems absurd to me. I think most people can agree there are useful modalities with LLMs. But it's always at a cost and I, like the author of the article, don't think that LLMs are as universally useful as many "optimists" believe. But to say that the pessimists can't actually rationalize their thoughts, when you provide none of your own, is truly foot in mouth garbage.

  yep. This is the choice quote.
  
  Even most humans cannot produce work to the highest human standards. The fact is, most work does not need to be done to the highest human standards, it simply needs to be done to the lowest possible standard that someone would pay for, at the cheapest price.

  The first point is the question of whether an LLM can do work not markedly inferior to that of a human doing the same task or not. This one is debatable. An LLM absolutely cannot produce work to the highest human standards, or even a good human standard at the moment, and it's unlikely that it ever will.  Most human work, however, is not done at a good standard. At best it's mediocre, and at worst it's downright horrible. The bulk of human knowledge work done in an awful of settings, then, is unfortunately more or less interchangeable with that of an LLM (seriously, so much human work is just awful: I wish I could say otherwise). On this point, then, LLMs probably pass the criterion.

“Can produce”. Is doing a lot of heavy lifting there. It can, but then they also regularly write truly bafflingly bad code that no decent experienced dev would ever produce.

I’ll give you a few examples from last week. I had claude whip up a Kafka consumer. After a lot of back and forth I was pretty confident it worked and then I decided to dig into the code. I spotted some real head scratchers.

When starting up the consumer it wanted to pass a specific rebalance strategy, but then it realized that the library it was using didn’t support that strategy, so it needed to use the default.

Instead of just deleting the config option (and maybe leaving a comment), it created a function that removed that key from the map and used it like:

    foo: “bar”,
    baz: drop_strategy_key(
      strategy: “blort”,
      bit: “bang”
    )

Another frontier model reviewer was supposed to prevent that kind of thing but it didn’t catch it either.

I had 10 public functions that were only called in tests throughout the whole project. Essentially the tests were useless because they weren’t actually testing anything.

Clearly current frontier models are useful. And maybe they produce code that is better than the median dev by many metrics. But I don’t think they are anywhere near consistently producing better quality than a say a 70the percentile dev with 10 years experience.

And even when the quality is better than the median dev, I’m finding that the bugs are not the kind of thing that humans or LLMs are good at spotting.

Many people have incredible ability to ignore evidence.

So it's not really about trying it out, I think. It's something more emotional.

Nothing that AI will ever be valuable, everything AI does is steal, nothing it creates is new, everything that the big labs publishes is a scam or hype. Oh and "it's just doing X", where you can substitute "X" with some kind of surface level description of how the output layer in an LLM works.

These tenets are already pretty much formed, all it needs is to be formalized by some kind of name or manifesto similar to Q-anon, and we've arrived at a durable state of permanent denial.

Maybe an Orange Catholic Bible.

They were not just fighting for their livelihoods, though; the fight was at least as much about dignity and fair treatment.

And they were correct to do so: skilled workers who lost their work and ended up in industrial employment did not have better lives. It took a century to a century-and-a-half for industrial quality of life to get back to pre-industrial quality of life.

Loss of dignity is something that the AI industry is not considering as it renders entire swathes of the economy button-pushers for chatbots.

When loss of livelihood appears it will be too late and at too great a scale to fight back in the workplace; the fightback will be somewhere else.

Personally, I'm against AI but even if it could never be smarter than us and if it were at the state that it was last year, I'd still be against it due to the environmental cost.

Good luck saying LLMs are useful but still a net negative in the filter bubble this author is in. No, you also need to think it has zero utility or you're the enemy.