36

Software could be the easiest fix for hyperscalers' AI power squeeze, researchers say — data center demand is expected to rival Japan's electricity usage by 2030

#technology
Tom's Hardware Software could be the easiest fix for hyperscalers' AI power squeeze, researchers say — data center demand is expected to rival Japan's electricity usage by 2030 Squeezing every ounce of performance per-watt www.tomshardware.com

check post at origin

5 comments

log in to comment.

  • @avidamoeba@lemmy.ca · 10 Oct 2026

    Thats kinda how the whole Chinese competition started - optimization to make due with limited hw.

    Also I just changed the inference engine I use for local models and Qwen 3.8 27B went from 50tps to 150tps. The new engine is optimized for my hw.

    • @Lydia_K@lemmy.world · 11 Oct 2026

      What did you switch to?

      • @avidamoeba@lemmy.ca · 11 Oct 2026 · edited

        Radiance. Was using the latest llama.cpp. This thread caught my eye as simple enough to try.

        • @Agent641@lemmy.world · 11 Oct 2026

          What are the specs of the machine that runs it?

          • @avidamoeba@lemmy.ca · 11 Oct 2026

            2x R9700, Ryzen 7700, 64GB RAM. The absolute best I’ve seen from llama.cpp was 60tps but the typical was 45-55tps. Radiance gave me 150tps on the first try. It can do parallel output while keeping 80-120tps with 2-4 streams. It can do more in parallel but I heven’t tested what the numbers look like. As far as I understand these numbers are possible on a single R9700 with the MXFP4 model. Haven’t tried it.