Hardware Rent in Cloud
DigitalOcean (opens in a new tab)
Calculating monthly costs
One possible way to calculate average costs for using LMs is (from The HARD Truth About Hosting Your Own LLMs (opens in a new tab)):
With Runpod (opens in a new tab) (capable of running Llama 3.1 70B Versatile 128k):
For A40 (opens in a new tab) (datasheet (opens in a new tab)) (48GB VRAM, 50GB RAM, 9 vCPUs):
$0.39/hr * 24 (hours in a day) * 30 (days in a month) = 280.8 $ / month