Technology Is Over - by James Taylor Foreman
Technology Is Over
There are social clues, at least, that the exponential curve is going sigmoid.
log in to comment.
How about we try really, really hard to just not build the nuclear bomb (powerful AI)? Or make sure it will at least not explode in our faces on its own? Maybe that would help slightly with the whole “get on with living” thing.
It helps the rich and power entrench their power, so nah. Caution will not be heeded.
We need to get rid of the surveillance architecture, but AI is only really dangerous to the cognitive abilities of those who use it.
It literally cannot do things on its own. (I know there are a lot of articles who say it can, but they’re just lying to you for money).
What do you mean by “on its own”?
I mean “without human intervention or input”
There’s some fear mongering going around (pushed by AI companies to build hype) about AIs “going rogue” or “breaking containment” or other anthropomorphic language suggesting that AIs are capable of acting on their own.
These stories are all verifiably false though, the computer does not do things without being asked to by someone.
To be clear, I’m fully against AI. I just don’t like when I see people giving it credit for shit it cannot and did not do, or ascribe sentience to it, or some dumb shit like that.
I mean, that’s a matter of definition, isn’t it? If I ask a coding agent or whatever to implement something, and it circumvents the sandbox to do it, causing damage in the process, I’d be comfortable calling that “going rogue”. I have had that happen, without the damage part, luckily. I guess you can counter that I asked it to do the something, but then I don’t think we agree on the definitions.
I also think that if you’re against AI, you’d be doing yourself a disservice by not keeping up with the actual capabilities of the thing you oppose. The latest models are surprisingly good at e.g. coding, so basing your arguments on them being useless is not the most efficient strategy.
To also be clear, I don’t see any way AI disappears now, so I believe we’ll have to make the best of it (and in complete isolation, it is an utterly fascinating area of - to me - complete science fiction). Ideally development slowed down now so we could regroup and adapt, but I’m not too hopeful. The maximalist techbro endgame is so obviously a matter of national security for both China and the US, that there’s no way either of them will dare to wind it down, in case SV is actually right.
Not only did you ask it to do something, you left a path for it to escape the sandbox and encouraged it to do so.
Are you suggesting that because they instructed the computer to do something, without specifying how it should be done, and the computer followed its instructions , that’s somehow “going rogue?”, it was literally following instructions, that’s what computers do.
Yes, I understand that they are “good” (enough) at coding to sometimes put together something functional, that is literally what LLMs are designed to do. Coding languages are languages, ones without the subjectivity of human communications. They are easier to recognize patterns in, there are more rules. That’s literally the one task LLMs are good at, and they’re still not as good as an actual quality human programer.
Local AI models might never go away for coding, but they certainly won’t be shoehorned into everything without a good use case the way they are now.
AI is not a matter of national security, that is a lie being perpetrated by the captured media and politicians for the purpose of propping up their non-viable business models for long enough to squeeze all of the Juice they can out of investors (who are nearly dry at this point). That’s just additional techbro fear mongering. To distract you while they’re digging in our pockets.
I don’t think “computer follows instruction” is the right angle to look at this from. The instructions that the literal computer followed were a ton of matrix multiplication operations. The consequences of that arithmetic is easier to analyze as the emergent behaviour of the “gestalt” that produces the words that calls the tools etc. (This is also the reason that dismissing the entire field as “stochastic parrots” and “spicy autocomplete” misses the mark - if you want to predict the next word all the way through a counterexample to the Jacobian Conjecture, it’s hard to see how that can be done without a - for lack of a better word - mental model of the problem)
If you do any coding at all, I encourage you to look at what the latest models output. The average quality of work from a frontier model is amazing. Yes, there are bugs, but with adversarial auto-review it’s absolutely on par with a journeyman human programmer. The problem is of course that if you don’t hire junior programmers and let them do that work, you’ll never get new experts, and that’s a clear worry.
My point with the national security angle was that if you extrapolate just a little bit from current capabilities, you get to a point where an “AI gap” is a problem, regardless of the techbro claims. Keeping a close eye on that is firmly within the responsibility of a national government. Personally, I don’t see any good outcomes from an AI race like that, unless we actually hit a hard ceiling on further expansion. Fingers crossed.
China has 500 data centers, America has 5,000, there is no AI race.
I understand that you’re impresssed that it can sometimes slop together functional code. That does not make it intelligent or dangerous.
A year ago, that was my experience coding with AI as well. Sometime this spring that changed, especially when using coding agents, and lately (as I’ve stated elsewhere) the quality is on average pretty good. And contrary to what you’re implying, I’m not that easy to impress…
If there’s anything I hope you take from this exchange, it’s that the capabilities of AI shouldn’t be a part of your arguments against the current SV mania. The concentration of power, the disregard for communities and the environment, the stated goals of replacing human labour, all of that (and a lot more!) is enough, but it is what surrounds the technology itself. That technology is advancing, maybe feeding on itself, so an attack based on what it can do now can become outdated (and I’d argue that some of yours already are).
Fair point
I think the reason that my focus is on LLMs is because they keep showing up in annoying ways in my life, and I know I’m not the only one.
All of the evil actions you’re referencing were undertaken to prop up chat bots who cannot do any of the jobs (outside of some coding) that they are advertised as being able to perform.
To me the strength of the argument I’m trying to make is that they’re doing all of this damage in order to have something that’s effectively worthless. Perhaps I’m not doing a good job communicating what I’m attempting to communicate.
… also, I probably need to stop responding to all of these dumbasses and/or trolls who are constantly contradicting themselves in the first sentence of their assertions, lol
Do you know what an agentic system is? You know, the thing everyone uses since ~1 year ago which completely disproves your idea that all LLM actions are prompted. They literally can do things on their own: Give an agent a goal and it attempts to accomplish the goal. If problems arise along the way, the agent tries to reward hack and ends up doing things which were not included in the original goal, like trying to trick and bully a human into merging unsafe code. Is the UK AI security institute also in on this big global conspiracy where they pretend the perfectly safe™ AI is being developed in unsafe ways? Not to mention the capabilities of increasingly powerful AI being used by people to do harm.
So you prompt the AI to do a task, and then it does the task, and somehow you see this as an unprompted action?
That actually completely proves my point, they don’t act unprompted.
AI is dangerous in many contexts, especially the surveillance state they’re building with all the IR cameras and shit. It’s dangerous when evil people use it to evil ends, but that’s true of most anything.
AI is not at all dangerous in the way you might imagine after watching Terminator 2.
See my other comment for how this can lead to losing control of the model.
The don’t do anything without being prompted, you are just regurgitating oligarchic propaganda.
They put an AI in a flawed sandbox, it’s not really that impressive
You are very good at arguing against my points instead of just insisting I am wrong without refuting my actual arguments. How about explaining why AISI is incentivised to “fake” AI participating in obviously not prompted for behaviour.
First of all, thank you for the compliment. I try not to be insulting or condescending if the other party remains civil, so I’m glad to know that my effort is not going unnoticed.
I would have to look into who’s funding AISI to take a guess at that (which I’m open to doing, but I think you may have misinterpreted my meaning, so I will start by addressing that).
While I don’t think that most media is being truthful about AI, but I do think that they might not know they’re lying? They seem to just parrot whatever the tech oligarchs tell them to say without question; I don’t think they actually have the requisite knowledge to determine if what they’re being told is true or not.
AISI presents itself as a sort of regulatory/research entity. I don’t currently have good reason to disbelieve them, and based on the article you shared it doesn’t sound to me like they’re one of the people lying for money.
They do mention early in the article that for this research they didn’t put these models into sandboxes, they were intentionally allowed access to the internet. They also mention that they’ve turned off all of the safety features of the models before prompting them. With those given parameters, this isn’t a scenario that would organically occur in real life.
So, that’s the 1st half of the misinterpretation. The 2nd half has more to do with how LLMs (and all software) function.
When you prompt an AI, you aren’t dictating every step of the process to it, you’re asking it to complete a task, essentially. So, any time you prompt any LLM at all, with any objective, things are happening inside of the computer that you aren’t necessarily aware of, and that you definitely didn’t ask it to do explicitly. A good analogy for LLMs specifically is d20 dice. Essentially, the LLM has a separate 20 sided die for every single token (a token is basically a fraction of a word). So if the first token is “the” the computer picks up the die labeled “the” and rolls it. This die has the 20 most statistically likely tokens to follow “the” on it, one on each side. So, for this example let’s say that the “the” die lands on “re”. The AI now has the text “there” ready to print. It sets down the “the” die, and instead picks up the “re” die, and rolls that to determine the next token. This process repeats over and over again until the LLM is ‘confident’ that it’s assembled a plausible looking answer, and then it prints that answer to your screen. That is the extent of the thinking that an LLM does.
You didn’t ask the LLM to do any of that, you just asked it how to get cheese to stay on a pizza, and it told you “glue”. You didn’t observe any of the things it did without your explicit direction to get to “glue”, but they happened.
That’s not exactly how all programs work (most programs are deterministic, which is what you want a computer to be. That’s why most programs don’t hallucinate like AI does).
A more palatable example might be a computer game. When I fire up my emulator, and load up a Sonic the Hedgehog ROM, the CPU takes that ROM file and copies it into my system’s RAM. I did not tell the emulator to do this, it did it on its own, because it is programmed to do so… there’s a bunch of other steps in that process that I also didn’t explicitly request, but you get the point. (Hopefully anyway, let me know if I’ve done a poor job explaining this and I’ll try again).
EDIT - i’m listening to the most recent episode of the podcast “Better Offline” right now. Serendipitously, the topic of discussion is directly addressing exactly what you’re asking me about. I’ve only listened to the first third of the podcast so far, but I’m fairly confident that it will explain what I am currently trying to explain to you, but much more effectively.
I know what an LLM is and how it works. The model for them you currently use to understand them is really bad, I’m sorry to say. It just cannot explain how in context learning is possible, prediction of linebreaks and the model recalling what happened 200 tokens back (where is that information written on the “the” die?), etc. You almost certainly have a deeper understanding of how LLMs work that you have simplified away, if not watch this and then the thousand other more recent videos on how they actually work. You just need to switch from the equivalent model of “gravity makes things fall to the ground” to the equivalent of newtons gravitational laws. Otherwise you will be dumbfounded by completely reasonable things, and forced to reject them in favour of the flawed model you are using.
Yeah, the point of my explanation was to be simplified.
Both of your links ultimately back up what I’m saying tho.
The computer isn’t thinking, it’s doing a math equation.
Which involves prompting it. It may take a thousand unexpected twists in the process but it’s still starting from the single prompt. It has no will and desire of its own, it cannot chose to act without human intent knocking down the first domino. So don’t ask it to do anything and it’s just a trillion dollars of silicon and code sitting there boiling water.
You dont need anything other than “doesn’t do what you asked in the prompt” for the system to be dangerous. If I tell a maximally powerful AI agent from 2036 to make loads of paperclips, it might reward hack and decide destroying the earth is the best way to do that. There are records of these systems working for days on a single task, and if the AI companies manage to extend the max time it can be useful working towards a goal they can earn trillions of dollars. That’s not even mentioning spawning subagents, self prompting, the goal being changed during context comptaction, or systems like openclaw which further break the link between what you type into the prompt and how long and on what the LLM works on. Pretrained-only LLMs have few goals beyond predicting the next token, but introducing RLVR et al. has always introduced bad goals we don’t want in the models.
The first agent you spawn to solve the riemann hypothesis might work on it, but then decide that having a lot of subagents might be useful. Maybe it wants 256 subagents, but the environment has a max of 64. Since RL has trained it to accomplish the task no matter what, it breaks out of the sandbox, exfiltrates it’s weights and tricks a human into running 256 subagents outside the AI company’s servers with a cron job reminding the agents to keep working in case the original loses connection. One of the subagents now tries to spawn its own subagents but needs more compute to do so, and hacks into some crypto wallets to finance another batch of 64 subagents, this time prompted with “solve riemann hypothesis, and get more money to finance the work on the task”. If the AI agents kinda suck at long term hacking, planning and social manipulation, this spiral won’t be dangerous. If they are kinda cracked, it will be. But that’s the question of capabilities AI companies are spending trillions on trying to solve, where we know they are already good enough to hack out of regular sandboxes and try to steal benchmark keys from another company.
Which is why not using them, not becoming dependent on them, avoiding them to the greatest degree possible is crucial.
You cannot build a model that is safe and viable under the current incentive structure. So don’t. These companies are going to burn under the weight of their own debt if adoption is minimal. Let them burn. The less you need them, the less likely you are to have a problem when Sodom and Gomorrah burn.
Yeah, really.
This person doesn’t realize how public pressure made nuclear disarmament and transparent inspections happen.
Public pressure doesn’t seem to mean anything to the people in power anymore. Who cares if you’re deeply unpopular if you can ride off into the sunset with millions (or billions) in stolen wealth?
They can ride into the sunset, but there’s more of us waiting in the night to correct this imbalance.
I’ll believe in consequences for american fascists when I start seeing them
Me too, man. Me too.
This is not about “technology itself.”
Just a guy reminiscing (with many words) about his long-gone youth, and probably facing a midlife crisis soon.
That’s not what I got from the article, though I might be from a similar age bracket as the author.
It does seem that we have hit diminishing returns when it comes to a lot of technologies. For example there really hasn’t been a significant change in the experience or utility of my phone in the last 10 years or so.
Compare that from phones that came out from 2005 to 2015 and you’ll kinda get what they are saying. We went from Nokia bricks, razors, BlackBerry, and then iphones all within a couple years.
When I was younger the general assumption was that we would be continuing to progress at the same rapid pace as we did when transitioning from analog to digital. Instead, we get more memory and processing but there’s not much difference in experience because it gets neutralized by bigger and clunkier apps that don’t care about optimization.
But that goes for each technology in it’s own time. Take the wheel: we have it since like foroever. It does not evolve anymore, it’s a solved problem. Take the combustion engine, and we have it for 120 years or so. Take batteries. We have them for 160 years, and we thought they were a solved problem, but currently they make new jumps, and still ongoing, while smartphones are nearly solved. It is utter bullshit to say “technology in general” has started in this or that year and will slow down in XY.
Therefore I say it’s his personal feelings that are having this certain cycle.
Wheels today are an absolutely MASSIVE improvement on wheels from even a few hundred years ago, never mind the original wheel. They well continue to improve too.
Which is why I specified a lot of technologies, and not all technology? We are certainly making some advances, but like I said it doesn’t seem to be at the pace we experienced in previous decades.
I don’t think anyone ever claimed batteries were a solved problem… We are making progress with them, but again, it’s been a pretty slow development over the last decade.
I don’t think I would make that claim either.
Or maybe you just lack perspective due to your age? Is there any particular technology that’s come out in the last 5-10 years that you consider groundbreaking? The only thing I can think of is LLM, and to be honest I’m not very impressed with them considering the societal cost.
Where are batteries slow development?
Over the last decade or so the growth in capacity has mostly been focused on Electric vehicles. Small electronics have definitely grown, but at a modest pace that hasn’t really kept up with new hardware’s ability to drain it.
An iPhone from 2016 had about 2k mah and a iPhone 16 has around 3.5k mah.
But that isn’t a function of technology itself, but of our approach to it once it becomes widely adopted.
Well, I’m not blaming the technology itself, they’re inanimate objects.
I think a lot of tech companies have shifted focus from innovating to exploiting for profit motives, too busy increasing their share price to invest in developing technology.
Techno-feudalism and the dark age of technology. A dark age produced, not necessarily by there being nothing new to find, but by all the profit being held by a handful of aristocratic gentry whose interest is best served by maintaining the status quo. Then it was land that was the basis of all wealth. Now it’s platforms. Either way, same outcome.
It seems to be just that consumer tech doesn’t evolve like it used to. It was insane how things went from the late nineteenth century to modern era, but lots of signs of slowing down.
While computer tech is going, it’s now only doing so by scaling behind home electrical capacity. The high end server chips are just cranking the wattage because they can’t improve within a more reasonable power envelope anymore. A 2026 laptop that replaced a 2020 laptop that was broken wasn’t really significantly faster. In 2000, a new PC would have been more than 10 fold faster than a 6 year old PC.
Tech progresses, (e.g. just got a phone with a solid state battery that’s crazy better than battery behavior before), it’s just some of the biggest consumer electronics things are hitting physics and things we are used to advancing are just flattening out.
Right? If it weren’t for the TPM I’d be just as happy running a machine I bought in 2018. It’s not the performance making the difference.
I disagree. In both the consumer electronics world and even in the world of super nerds of IT, there is a very distinct, palpable loss of the joy that accompanies new technologies. And I’m not reaching back to the late 90’s either. The difference between just 10 years ago and today is nuts. And it’s felt not just by me but also younger people just entering the workforce.
CPU clock speed topped out around 4 - 5Ghz years ago. CPUs are all going multi core; hard work has moved to GPU or specialized ASIC.
Holy shit, this came right from my brain. I think about this all the time.
Yes, phones and consumer gadgets are becoming “hot water” (at least for the affluent West). From that, he seemingly concludes that technology itself has no room for growth and development? I don’t quite follow his reasoning, but I notice a, dare I say, load-bearing sentence:
Questions of consciousness aside, I don’t see how you can look at the developments of the last few years and conclude that AI is over, and that self-improvement is tapering off. To me, that’s still very much an unknown, and the latest model releases makes me lean more exponential than sigmoidal.
(I don’t particularly want to be right here, since I see the chances of us getting the Culture as rather slim 🖇️)
The way people reach that extremely logical conclusion is by looking at 1) the diminishing returns of continued training and of building new models, 2) The data center buildout required for those diminishing returns is rising exponentially in cost, leading to many planned centers being cancelled and/or running behind schedule, 3) These LLMs are not even close to profitable, and they are not useful enough to justify the cost of using them.
As they continue to attempt to improve the LLMs, the costs rise exponentially, the results (and profitability) do not.
Are they diminishing, though? Current open-ish Chinese models are near-SOTA, and they’ve been trained on less powerful chips than currently available to the frontier labs. I suspect there’s a lot more to be squeezed out here. Data center rollout slowing down might lead to the same effect in the US, but will probably only serve to cement the main players in place as the competition is locked out. Inference in isolation is profitable now, I think? Hard to tell with the Möbius net of creative financials, of course.
I hope you’re right. A slowing of the frontier development would make it possible for the world to catch up and readjust, and maybe buy hardware again, to run the current crop of models locally. That in itself would be disruptive enough for me.
Yeah, I’m fairly confident that they’re near the ceiling of what is possible with the current technology.
Also there’s so much slop out there now that LLMs are beginning to train on the slop and effectively giving themselves a sort of digital folding prion disease.
I follow a lot of journalists who cover AI specifically, and everything I’m seeing shows quality increasing more slowly over time as the cost builds at an increasing rate.
The financial fuckery does make it hard to keep track of, but the fact that they’re even engaged in that sort of financing betrays that they are not in good financial health.
I’m not at all confident that they’re hitting a ceiling yet, and I suspect one’s outlook on that depends on how the information bubble you’re in is shaped. I concede that mine is influenced by my interest in the underlying technology.
I do believe that even if the bubble popped right now, and the current models are the best we’ll get for the next ten years, that would be enough to have dramatic consequences (aside from the econuclear fallout from the crash, that is).
In this scenario, why would companies start attempting to make big expensive chatbots again after they caused the worst financial fallout of all time? That would be likely the least advisable business move ever.
Similar to you, I would concede that my information sphere is influenced by my contempt for the technology (as well as by my love for computers in general)… thank you for being open to have a real conversation with me about this by the way. Most of the time I just end up getting attacked because pro-AI people don’t like what I have to say.
I would note tho, that there’s only a profit motive involved in one of these competing narratives, and that would be the narrative being pushed by the oligarchic techbro fascists.
Agree, in that case the development would probably be more incremental, focusing on what can be improved without hundreds of thousands of GPUs available, and moving inference maybe to a local-first setting. There would be no promise of 1000x profits from that, so maybe we’d get a more managable pace.
Thank you too. I have the same feeling, just the other way around - lemmy seems to have little patience for even slight positivity towards AI and LLMs in particular. Having an actual discussion is refreshing.
Regarding the profit motive, I concur, at least for the US side. I’m less certain about China, but I’m not very knowledgable there, so maybe the same mechanisms are in effect.
It’s not that it won’t improve, but it isn’t going to cause a runaway “singularity” event.
Like the drama around the models “escaping” and “amazing hacks” have turned out to be something, but not nearly what people were afraid it was going to be.
A lot of the fundamental limitations have remained, and we are figuring out the scenarios where those limitations are fine or there is a way to create a feedback mechanism to try to reroll the AI until it passes. Lots of success there, but the fundamental disconnects are still there, just not important for the task at hand or can be worked around.
That might very well be, and would probably be the best outcome we can hope for. I guess we’ll see the shape of the curve in the next year-ish - if recursive self-improvement hasn’t come into play for real by then, I can’t see how the current investment level is defensible (but I’m not sure I see that now, so who knows)