Local Hardware Costs
- Different hardware can imply different operation cost for the same value.
- For example, choice to run a LM locally on an external graphic card can incur big expenses, as shown in The HARD Truth About Hosting Your Own LLMs (opens in a new tab)
- high cost of the graphic card itself
- high cost of the electricity it consumes (imply it running 24/7 by default)
From Local LLM Challenge | Speed vs Efficiency (opens in a new tab)
Energy consumption
...
Heat
...
Noise
...
Required cooling
Cost
...