The original post: /r/selfhosted by /u/schaka on 2025-02-22 11:20:47.
I’m currently looking into buying a older GPU to run locally in my server, where it will be idling most of the time. I’d be curious about your setups and/or experiences.
I’m looking to use it with HomeAssistant for voice control via Whisper but ideally also as a local LLM and with functionary, so after my voice commands are interpreted, they also result in the correct action.
Power cost is 38ct/kWh and I’m hoping the GPU can idle at 10-15W with models loaded.
The following GPUs are available at the given prices. They seem to be shooting up signifcantly too:
- Radeon Instinct Mi50 16GB - 150-200€
- RX 6800 - 300-350€
- Tesla P40 - 400€+
- Tesla P100 - 250€
I can potentially get some of these cheaper buy haggling on AliBaba, but no guarantee.
Given the cost, it seems the P40 just isn’t worth it. This likely means 24GB GPUs are just out of my budget. Can I even fit all that in 16GB.
Which leaves me wondering, the P100 with CUDA and HBM2, despite its older feature set and relatively slow compute doesn’t seem like such a bad option compared to the RX 6800 and the hassle that is ROCm. Does anyone have a comparison of the two?