@trougnouf@lemmy.world
@trougnouf@lemmy.world
-
on Is anyone self-hosting their own AI? in c/selfhosted@lemmy.world · 19 Oct 2026
-
on Is anyone self-hosting their own AI? in c/selfhosted@lemmy.world · 19 Oct 2026
I get about 45 t/s after some tweaking, getting the thermals under control helped a lot; mainly by reducing the power consumption to 261 W (and downclocking a little bit).
What vision model do you use?
-
on Is anyone self-hosting their own AI? in c/selfhosted@lemmy.world · 18 Oct 2026
I use Qwen 3.8 27b Q4 with 100k context (llama.cpp and the Mistral AI harness) on a 24 GB AMD Radeon 7900 XTX and it’s been great. It can handle pretty much anything I throw at it and I can let it work indepedently overnight / over days on long tasks.
It uses 261 W, probably more expensive than getting a 2nd Mistral subscription with the cost of power where I live (Belgium), but it will be free heat once I have solar panels and a battery.