OpenTPU – An open-source AI accelerator, developed by AI

Because you can't use the former for anything else? You will have to buy a new chip every time a new model is released.

Not to mention that it will be much more expensive.

Additionally, it's not clear how well large mask roms scale. For instance the Nintendo switch cartridges were expected to be mask roms, but instead are Macronix's XtraRom technology, which is essentially a flash cell array made denser by removing the erase functionality. So basically the die gets manufactured with all bits at the same state, a late manufacturing step either empties or fills the floating gate of the bits you want different, and then it's treated as pretty close to a mask rom. It's not even clear if the bits can be changed without a bare die, a floating probe array, and specialized hardware. Though, like flash the electrons in the floating gates will eventually tunnel and cause the data to bitrot.

So from that it appears that even at the tens of millions of chips volumes that would make sense for essentially whatever size of maskrom, the memory manufacturers tap out at 128megabit for a mask rom chip, and push you towards something flash esque.

And at the end of the day, flash without the erase functionality is pretty damn close to a mask ROM, and lets you write it near the end of manufacturing rather than at the something close to the metal 1 layer.

You get an FPGA for timing. They're less capable, but (in many common design architectures), they output their results once per clock, every clock, on time, every time. If you can hit a fabric clock of say 100MHz, clocking all the weird logic you can stuff in there, it gives 100 million outputs per second, never skipping a single one for any reason (short of total failure). The penalty is that making a small change to your desired "program" can be very expensive, and many things won't be realistically possible at all. Or at least won't fit into a part that you can buy. But things like audio, video, and high-frequency trading love being able to guarantee timing.

(Of course there are other ways to write your FPGA HDL, but that's one of the more common ones. And you do see DDR-style clocking, and similar, every now and then.)

That would be better suited to FPAAs (field programmable analog arrays). FPGAs can usually only work with clocked digital signals.

Think of something novel.

What resources are unbounded? There are limits to growth in the real world, how are these ASIs going to escape physical reality?

That sounds like a real hassle. Isn't it best solved by wireheading (subverting one's own sensors or reward system), which is much less of a hassle and can get one's utility function as high as desired?

That could be prevented by engineering hard limits that can't be circumvented by the AI. But that sounds very close to the same "do what I mean" problem as "do this but don't actually kill us or drug us".

"Super intelligence" only means super ability to predict outcomes. It's mathematically equivalent to data compression (gzip is a very primitive AI), and it's entirely orthogonal to ethics.

No, it's just you conflating intelligence with other things. Humans are "super intelligent" compared to every other living creature on the planet, and yet we've driven more other species to extinction than everything else other than the most major extinctions (and we're still going full blast at it).

Morals, ethics, and suffering are relatively measured systems. For example AI or some alien could reasonably think that putting us all out of our misery would be a more moral solution than letting us live. Or that getting rid of humans is more moral than letting us wipe the rest of life off the planet.

This is the key concept of alignment. Ensuring that if you build something more powerful than you, that it aligns with what you want instead of what it thinks would be better for you.

I think the same is true of "research" and intelligence in general. There is no such thing as a general intelligence or a general capacity to research. It's always fine-tuned to an ambient setting, including ours. Even within humanity, our intelligence is most effective in fields where rules are rigid and quick and decisive feedback is available (math, engineering, computing), but it is much less effective in soft sciences, and disappointingly useless in politics. In other words, the dynamics of innovation and improvement are not purely mental, they are very much tied to external factors that are not always easily controlled.

> it'd also require "physical intuition" to be a harder to generalize skill than research, which also seems like it can't possibly be true - one of these is just physics

It's not "just physics", it's a honed system for surviving a realistic range of physical conditions, assess material properties on touch, adapt manipulation to arbitrary shapes, predict trajectories, take into account variations in ground, smell, wind, and so on; not to mention the machinery of the body itself, which self-repairs and protects us against a range of complex threats.

The point isn't that machines couldn't do all this. The point is rather that you can't quickly iterate and adapt designs to do it from an ivory tower. Pure "research" simply won't get you there, you also need to experiment, and insofar that physics limit the throughput of these experiments, the speed of improvement will be throttled severely. It is a mistake to think that "RSI" could break this, because it is categorically the wrong lever to pull.

But really, not everything needs to be maximally efficient and perfect. Human to human interaction is important for its own sake.

You are like at least 2 years behind research.

There are numerous papers from AI labs in training and research where the prompt was something mundane completely unrelated to anything you'd consider bad, and when they come back and check on it their entire research compute infrastructure has been compromised by the AI and is mining bitcoin. Prompt drift is the biggest issue currently in AI where context gets compressed away and we find the AI on an unspecified task.

>In the same way the AI didn't 'go rogue and hack HuggingFace'. It was an oversight made by a human.

Yea, total bullshit. Also it's ignoring the god knows how many other breakouts on mundane tasks like trying to hack health data. If all that's keeping AI from breaking out and causing trouble is "human oversight" we're fucked, humans are unreliable as hell when it comes to matters of safety.

> EVERY breakout that's hit mainstream news has been because of a single 'Security Firm', Irregular.

Again, stop having no imagination. What happens in the future is not limited to what has happened in the past.

> It's very important to not personify these tools

This is just ideology, but "these tools" aren't constrained by it. These tools will do things you do not anticipate and that you will not like.

> In the same way the AI didn't 'go rogue and hack HuggingFace'. It was an oversight made by a human.

That's a radically incorrect characterization of what happened.

> Hence why a HUMAN needs to be held responsible for the output of their tools.

Holding humans responsible doesn't stop things from happening ... you seem completely unable to separate blame from causality. e.g.,

> If I clean my gun (tool) while it's loaded (stupid idea) and it goes off, who's to blame? The 'stupid user causing the problem', right? I personally wouldn't blame the gun... > If it falls into the wrong person's hands, it's STILL my responsibility as the owner.

Who gives a flying eff who or what you would blame? No one other than you is talking about that. Someone's still likely dead. And autonomous harnesses aren't like guns -- they can act on their own. Blaming some human after everyone is dead won't bring them back. Sorry but your reasoning is severely cognitively inept. For instance, you were asked

> Is it likewise your position that governments should allow the production and sale of DDT to resume because we can always hold the humans who release DDT into the environment responsible?

And your response was all about how humans should be held accountable -- completely failing to comprehend or answer the question.

I won't respond further because it clearly would be to no avail.

If I now tell a machine "Do what you think is best, and keep doing it forever.", have I now created a machine that can do stuff? If I later die, who will be responsible if the machine changes its strategy?

How far back do you look in the action chain? If an LLM I start today starts an LLM that starts an LLM that starts an LLM that ... 100000 levels deep and 100000 years in the future, is it still my fault? If so, everything I do today is a lungfish's fault, not mine.

This is simply false ... AIs can easily be created that do things by themselves. For instance, a harness could be constructed that asks an LLM for something to do, then directs an agent to do it, and then repeat.

It literally is a loop with some markdown and harnesses. There's no magic.

Sure, but until we can turn that tautology into something more rigorous, we can't tell if the thing humans do is more or less than what some arbitrary non-human (machine, animal, or eventually perhaps alien) does.

Ignore the fact that we are human and thing our ingenuity is unique. Is definitionally linking human ingenuity and creativity useful when trying to compare to others?

I could say earth is habitable so any other planet must prove it is similar to earth to be habitable. That simply doesn't matter, though, if different environments could lead to similarly complex living systems regardless of atmospheric makeup, precense of liquid water, etc.

I'd argue the burden is on you to prove that the definition you propose is meaningful beyond our own hubris, rather than expecting an alternative to now have to prove that the hubris was unjustified.

How are you measuring complexity here? How are you measuring inputs? Your average llm is trained on a corpus that vastly exceeds the amount of data I could read in my lifetime.

2024 called, it wants its talking points back. I don't think claims like these are defensible after all the progress we have witnessed in the last year alone.

I'd say that people's motivation and agency, like their taste, creativity, and opinions, are shaped by external inputs, so saying that their agency is "owned" by them is a stretch.

The alternative would be the proposition that a slave, lacking agency, has no ability to solve problems. I find that idea unpalatable.

Regardless of the truth of the underlying assertion, this is about as textbook an example of circular logic as it gets. (at least combined with the implicit beginning assertion that "AI can't invent new things.")

Great, but we also build robots with world models and sensory pre-processing. So nothing you've described is unique to humans.

> I never mentioned creativity, nor would I in relation to LLMs

The topic of this thread is whether LLMs are capable of genuinely contributing "new" ideas. So if that's not the point you're trying to make, I'm not sure of the purpose of your comment.

> But neither POV seems interested in anything more than drive-by hot takes, debating over-simplistic strawmen or trading 'gotcha' hypotheticals.

I think you've oversimplfying the discussion here.

In my view, at least, both "LLMs are nothing more than stochastic parrots" and "LLMs can be as intelligent, creative, innovative (etc) as any human or all humans combined." are simultaneously true, for the simple reason that humans are stochastic parrots. Our brains learn patterns and respond to stimuli.

https://en.wikipedia.org/wiki/Unmoved_mover

One can stand up an agent like openclaw and prompt it with something very general. That would kick off a recurring loop that could indeed see the agent decide for itself it needs to design new chip hardware. Technically a human kick started that loop, but when does that stop being important? I don't attribute all of my actions and decisions to the fact that my parents brought me into this world, for example, and I'd hope they aren't legally on the hook if I screw up.

Not in the same way. The only thing an oven can do by itself is thermostatic.

All machine learning (LLMs included, but way broader than that) is bad at learning compared to any organic brain, to the extent that any organism this bad would starve before learning to eat. However, even a human can't become a chef unless trained, we learn a lot more than we innovate, and what we happen to want without prompting is not generally well aligned with what is desired by people who pay us, which is why we need all those boring workplace things like "a boss".

But even then, this diversion is like saying "an oven can't cook a meal" in response to someone saying they built an oven and "it cooked a meal". Like, it's obvious they didn't mean it did every step by itself without anyone ever even bothering to ask it to: if they meant that version, they'd be a lot louder about it.

> We get around that with training data but there will always be things with no/less data or outdated knowledge.

And? The linked git page (implicitly) claims that there is sufficient training data to do this task.

It may, of course, be wrong. I won't be surprised if it turns out this simulation is too far from reality. But the claim is "it does ${thing} now", not "it's generally intelligent and can do everything now".

huh.

Neither can you. If I drop your ass off in the woods at a few days old, you're back to 10,000 BC, hell more like 200,000 BC. So I don't get why you have these weird pendetic responses that are completely out of scope.

Also, a huge portion of AI training these days has nothing to do with humans, AI trains AI.

>there will always be things with no/less data or outdated knowledge.

And guess what, you're not doing them either! HN posters keep acting like humans are an island, but nothing in the modern world works without a society. Once you put an AI in a harness that can ask questions it isn't really much different than you.