sigh; this argument is the new Chinese Room; easily described, utterly wrong. ht...

razorbeamz · 2026-03-24T06:18:57 1774333137

Next-token-prediction cannot do calculations. That is fundamental.

It can produce outputs that resemble calculations.

It can prompt an agent to input some numbers into a separate program that will do calculations for it and then return them as a prompt.

Neither of these are calculations.

gf000 · 2026-03-24T07:35:19 1774337719

So you don't think 50T parameter neural networks can encode the logic for adding two n-bit integers for reasonably sized integers? That would be pretty sad.

razorbeamz · 2026-03-24T08:15:35 1774340135

They do not. The fundamental technology behind LLMs does not allow that to be the case. You are hoping that an LLM can do something that it cannot do.

gf000 · 2026-03-24T09:28:27 1774344507

https://arxiv.org/html/2502.16763v2

You are wrong. Especially that we are talking about models with 50T parameters.

Can they do arbitrary computations for arbitrarily long numbers? Nope. But that's not remotely the same statement, and they can trivially call out to tools to do that in those cases.

int_19h · 2026-03-25T03:01:42 1774407702

You do realize that training a neural net to do addition is a beginner level exercise in ML?

parasubvert · 2026-03-24T06:22:03 1774333323

Humans can't do calculations either, by your definition. Only computers can.

datsci_est_2015 · 2026-03-24T09:30:21 1774344621

Third things can exist. In other words, you’re implying a false dichotomy between “human computation” and “computer computation” and implying that LLMs must be one or the other. A pithy gotcha comment, no doubt.

Edit: the implication comes from demanding that the OP’s definition must be rigorous enough to cover all models of “computation”, and by failing to do so, it means that LLMs must be more like humans than computers.

gpderetta · 2026-03-24T10:46:26 1774349186

After dismissing it for a long time, I have come around to the philosophical zombie argument. I do not believe that LLMs are conscious, but I also no longer believe that consciousness is a prerequisite for intelligence. I think at this point it is hard to deny that LLMs do not possess some form of intelligence (although not necessarily human-like). I think P-zombies is a fitting description.

zeroonetwothree · 2026-03-25T01:03:24 1774400604

I don't think P-zombies can exist. There must be some perceptible difference between an intelligence w/ consciousness and one without. The only way there wouldn't be a difference is if we are mistaken about the consciousness (either both have it or neither do).

gpderetta · 2026-03-25T10:26:05 1774434365

> There must be some perceptible difference between an intelligence w/ consciousness and one without

I think there are differences, and I think we can make good guesses, but I'm not sure we can reliably classify a P-zombie from a normal human from their behaviour with 100% accuracy..