198 comments

log in to comment.

  • @TehWorld@lemmy.world · 7 Oct 2026

    I’ve been wanting to do this, even as a testbed. I just want computers to do cool shit, and am getting tired of being a Linux sysadmin in my free time. How large of a model do you have assigned to this?

    • @percent@infosec.pub · 8 Oct 2026 · edited

      Just Qwen3.8-27B. That little model is surprisingly smart.

      NixOS also gives you the ability to roll back to previous config versions. I have not needed to do that (yet), but it reduces some risk.

      • @TehWorld@lemmy.world · 8 Oct 2026

        “Just” and a 27B model don’t exactly belong in the same sentence if you ask my wallet. 💸

        • @percent@infosec.pub · 8 Oct 2026

          Yeahhh sorry 😕

          I was fortunate to have bought my graphics card almost a year ago, and for a nice discount (a Black Friday sale, IIRC). I’d love to buy another, but they now cost >2x the price I paid.

          FWIW, you can probably run it on CPU if you have enough RAM. I assume it will be really slow though.

          Might also be worth trying Qwen3.6 35B-A3B. It’s not as “smart” as Qwen3.8 27B, but can be much faster on limited resources.