37 comments

log in to comment.

  • @_chris@lemmy.world · 26 Oct 2026

    “Where are the engineers” to manage this on prem hardware?

    Well, you laid them off, you fuck.

    • @red_tomato@lemmy.world · 26 Oct 2026

      They’re going to use AI to install the AI.

      Recursive Self Installation

    • @lobut@lemmy.ca · 26 Oct 2026 · edited

      IBMs a bit weird. They fired a shitload of them a while back.

      Then they said they’re only replacing junior roles.

      Then I think they’re hiring a lot more juniors because they need to build the next seniors.

      I didn’t really follow too closely but that last statement is at least something I can respect.

      [edit] I think mushroommunk is absolutely on point [/edit]

      • @mushroommunk@lemmy.today · 26 Oct 2026

        It’s not really something you should respect. It’s PR shell games. They fired senior staff to replace them with cheaper juniors. Not because they actually think they’re training the future but because they think they can get the same work out of them+AI

    • @WesternInfidels@feddit.online · 26 Oct 2026

      I don’t like it, but that would just make them cheaper to hire back as AI babysitters.

  • @Freeposity@lemmy.world · 26 Oct 2026

    Of course IBM has a hardware solution for you

    Krishna also predicted demand for on-prem AI will spur sales of Big Blue’s Spyre mainframe accelerator cards.

    • @brucethemoose@lemmy.world · 26 Oct 2026 · edited

      Yep. This is why Nvidia is preaching open weights literally every chance they get, too.

      Still. It’s a better stance, even if for selfish reasons.

      • @p03locke@lemmy.dbzer0.com · 26 Oct 2026

        Nvidia is the shovel maker. They don’t care who buys the shovels.

        • @shalafi@lemmy.world · 27 Oct 2026

          Made a comment the other day about hardware companies making out while the software companies lose. Somebody figured out how to sell razor blades instead of the new super-hyped MEGA™ razors.

      • @madmantis24@lemmy.wtf · 26 Oct 2026

        We were only ever going to steer this neoliberal hellship towards the better stance once these business idiots finally found a selfish reason to do the right less bad thing

  • @BackgrndNoize@lemmy.world · 26 Oct 2026

    Guy selling fruit says fruit is juicy more breaking news at 11

  • @Syndication@lemmy.today · 26 Oct 2026 · edited

    Maybe I’m just sleep deprived, cynical, or thinking too deep into it, but I don’t even like how they are using the words “on-prem” throughout the article instead of simply saying they’re running local LLMs. Seems like such a weird choice of wording that is meant to subtly steer less informed people away from looking into alternatives to the subscription based LLM services, as most casual people wont associate it as the same thing and ignore it and move on. Even if that’s the correct terminology for that specific field, I feel like most people would just naturally say “Yes, we are running the LLM servers locally here” as that’s the only way I’ve ever heard people say it even online.

    • @davad@lemmy.world · 26 Oct 2026 · edited

      For a company “on-prem” might not mean the same as “local”. You might run them in your company data center or in a colocated server (rented space in a data center). In either of those cases, the data center might be physically close to you, but it could be on a different continent.

      “On-prem” implies that the company has physical access/control of the server.

      • @Syndication@lemmy.today · 26 Oct 2026

        Ah okay thank you for clearing that up for me. Makes me feel a bit less crazy now haha it was a bit off-putting with the amount of times “on-prem” was used here!

        • @partial_accumen@lemmy.world · 27 Oct 2026 · edited

          “On-prem” is the Enterprise IT antonym term of “cloud”, as in public cloud, aka hyperscalers (AWS, Azure, GCP, OCI).

    • @coolie4@lemmy.world · 26 Oct 2026

      “On-premises” as it refers to enterprise hw/sw, has been a phrase much longer than “local LLM”

      Even though everyone moved to cloud, most companies keep some portion of their infrastructure on-prem, like prod env in the cloud and on-prem test env/backups, or vice versa

      Maybe most consumers wouldn’t recognize the phrase, but industry professionals certainly would

    • @anon_8675309@lemmy.world · 26 Oct 2026

      It was written by Claude

  • @qaz@lemmy.world · 26 Oct 2026

    I mean, obviously, but I guess the only reason they’re saying this is because their own models aren’t that great (IBM granite sucks) and this way they can still sell consulting services.

  • @floquant@lemmy.dbzer0.com · 27 Oct 2026

    Whose premises? Release the GPUs and make them cheap again you motherfuckers

  • @Skullgrid@lemmy.world · 26 Oct 2026 · edited

    BTW, IBM/RHEL is hoping you might go for some red hat, IBM solutions for local distributed AI stuff :

    But you know, the CEO isn’t biased. /s

    At least they’re working on letting you have access to the open source (ish) local way of working with AIs instead of just the subscription model that the others are pushing.

    I did some digging into this before giving the fuck up since my computer is a potato, and I have too many fucking things to do with my time.

    • @p03locke@lemmy.dbzer0.com · 26 Oct 2026

      If I could pick a company that has fucked up more than Microsoft or EA, it would be IBM. For the past 20 years, the whole company has been on a wild bender of buying good products, and shoving them headfirst into the ground.

      • @partial_accumen@lemmy.world · 27 Oct 2026

        For all their faults, IBM has treated Red Hat and Hashicorp better than Broadcom has treated VMware.

  • @danielfm123@lemmy.zip · 27 Oct 2026

    It was always the future no big tech wanted… That’s why they enbeded models everywhere.

  • @saltesc@lemmy.world · 26 Oct 2026 · edited

    “It’s a simple question for any enterprise. Would you like something that’s 10 percent of the cost if it’s 90 percent as good as the best, or do you want to pay that 10 times higher cost?”

    That really depends…

    90% is disasterous. It may sound good because it’s “just” 10% off 100%, but it’s awful. Two iterations reduces that to ~0.01%< but that’s still pretty bad. Obviously the more complex, the higher the risk and the more demand and time per iteration. The machine also has no concept of criticality, so how much is being saved with three or even four iterations to assure failure is basically zero?

    “Then the question is: how many tasks require that? I would tell you some do, but the vast majority, summarizing an email, examining a contract, answering a customer call, composing an email, all of those can be done as well with the much cheaper answer.”

    Imagine 10,000 emails an hour through your enterprise and 1,000 times It fucks up. Anything from missing info to misinterpreting to just straight up modifying/deleting. No concept of criticality per email. 8,000 in a 9–5 office. 24,000 a day for a global enterprise. That’s what 90% looks like.

    Then you got “examining contracts” and all the other stuff he mentioned…

    • @brucethemoose@lemmy.world · 26 Oct 2026 · edited

      It doesn’t literally mean “90% failure rate vs 100%”

      In the HLE benchmark, for example, we may be talking about 45% test score for MiMo Pro or GLM or something, vs 55% for Sonnet and 61% for maxed out Claude.

      But further complicating that, those particular open weight models do better than Claude in one particular hallucination benchmark, meaning Claude/ChatGPT might be worse at RAG/extraction type workloads. It’s like this across the board, with different models trading blows in different areas.

      By “90%,” he really means “it’s so close the better model depends on the particular application, workload, harness, sampling configuration” and such, and that none of these models are close to “100% accuracy” across the board.


      One other thing: Anthropic, OpenAI, and Grok have all been accused of “dumbing down” models under high demand to speed up inference. If your application depends on them, and they are the only provider, that could throw a wrench in the workflow.

      But open weights? Pick your provider. You get an API as consistent as you are willing to pay for.

      In other words, that “100%” isn’t really representing the potential inconsistency one gets being locked to a single provider.

      • @saltesc@lemmy.world · 26 Oct 2026

        It’s his examples that I’m going off. But maybe he doesn’t know what he’s talking about and an IBM engineer is losing their shit at upper management right now. I used to work for them a long time ago under Virginia and that was standard… Though, so has been every other place since.

        I give IBM credit because so many of the people there are ridiculously smart. Not their business people.

        • @DickFiasco@sh.itjust.works · 26 Oct 2026

          I’m willing to bet that “90%” and “100%” have absolutely no concrete meaning here.

          • @Blisterexe@lemmy.zip · 26 Oct 2026 · edited

            “90% as good for a fraction of the cost” is a set/premade phrase and so the numbers mean nothing, since the person who said it didn’t come up with it.

            Like, he extrapolates off of it but I’m willing to bet the number is from the expression.

      • @L0rdMathias@sh.itjust.works · 26 Oct 2026

        No one said 90% failure rate. You brought that up. Reread the post.

        You might be lost, Shillsville is 20 miles that way.

        • @brucethemoose@lemmy.world · 26 Oct 2026 · edited

          You said:

          Imagine 10,000 emails an hour through your enterprise and 1,000 times It fucks up.

          So I interpreted that as an implication of “90% accuracy.”

          I’m sorry I misinterpreted you, but I’m not sure why you have to be so abrasive calling that out? “Shillsville?” ???

    • @hperrin@lemmy.ca · 26 Oct 2026 · edited

      Don’t assume the expensive model is 100% accurate. The cheap model is 90% as accurate as the expensive model, which is already less than 100% accurate.

    • @sudoMakeUser@sh.itjust.works · 26 Oct 2026

      I thought of it as grading as in an academic setting. If the top commercial models are putting out essays that are graded at 100%. The on prem models are putting out results that are graded at 90%. In the case of support emails, the results won’t be much different.