RTX 3090 vs RTX PRO 4000 (Blackwell): Inference, Training, and Power Cost
I wanted to know how my local RTX 3090 (used, ~€700) compares to one or more RTX PRO 4000 Blackwell cards (145W, new ~€1400) for both LLM inference (single user / scripts / my own agents) and fine-tuning image models (AI-toolkit).
My takeaway up front: with llama.cpp the 3090 is often faster at decoding; for training the RTX PRO 4000 is slightly ahead for me. In terms of power cost per token/epoch the RTX PRO 4000 is much nicer.