I spent $3,400 last year training models before I realized I was doing it completely wrong. Turns out I was paying for A100s when H100s would’ve finished the job in a third of the time for less total cost. Math is funny that way — faster GPUs often save you money.
Here’s the uncomfortable truth about GPU rentals: everyone obsesses over hourly rates, but nobody calculates total project cost. A cheap GPU that takes three times longer isn’t actually cheap. Let me break down the actual costs across different platforms so you don’t make my expensive mistakes.
Before we dive into platforms, let’s address the elephant in the room. Should you just buy a GPU?
Quick math on ownership:
If you’re training models occasionally or trying different GPU types for different projects, renting makes way more sense. I only recommend buying if you’re running 24/7 for months and know exactly what GPU you need.
Here’s how to actually calculate what you’ll spend:
Total Cost = (Hourly Rate × Training Hours) + (Storage Cost × Days) + (Data Transfer Fees)
Most people only look at hourly rates. Big mistake. A platform with high hourly rates but fast GPUs and cheap storage often beats “budget” options.
Scenario: Training a vision model, estimated 40 hours on an A100
Option A — “Cheap” Platform:
Option B — “Expensive” Platform:
The “expensive” option costs almost half. This is why hourly rates lie.
Amazon’s GPU instances are everywhere for a reason — they work, they’re available, and enterprises trust them. But you pay for that reliability.
AWS offers the full GPU spectrum from cheap T4s to cutting-edge H100s. The infrastructure is rock-solid, and integration with other AWS services is seamless.
GPU options:
The Spot Instances feature can save you 70% if you can tolerate interruptions. I run all my experimental training on Spot — yeah, it sometimes gets killed, but checkpointing handles that.
AWS is expensive. Like, really expensive if you’re not careful. On-demand rates are brutal, and the pricing calculator requires a PhD to understand.
Hidden costs that add up:
Best for: Enterprises with existing AWS infrastructure, teams needing guaranteed availability, projects where compliance matters more than cost.
Skip if: You’re budget-conscious or don’t need the AWS ecosystem integration.
Google’s GPU offering is what I reach for when training time matters more than hourly cost. Their infrastructure is fast, networking is excellent, and TPUs are an option if you’re in the TensorFlow world.
The networking between instances is noticeably faster than AWS in my testing. For distributed training, this matters a lot.
GPU lineup:
Plus they have TPU v4 and v5 options if you’re all-in on TensorFlow. TPUs are weird and proprietary but genuinely fast for specific workloads.
GCP is generally cheaper than AWS for raw compute, but more expensive than specialized GPU clouds. The sustained use discounts help if you’re running longer jobs.
What I like:
Watch out for:
Best for: Teams already on GCP, TensorFlow-heavy workloads, projects needing fast networking for distributed training.
Lambda feels like it was built by ML practitioners who got tired of AWS bills. They focus exclusively on GPU compute for deep learning, and it shows.
No complex instance types, no confusing pricing tiers. Just GPUs with clear hourly rates.
What they offer:
Look at those H100 prices compared to AWS. That’s $1.99/hour vs. $98/hour. Even accounting for the fact that Lambda’s are shared instances, the math is insane.
Availability is the big one. Lambda’s cheap because they’re efficient with capacity, but that means GPUs sell out. I’ve waited days for specific GPU types during busy periods.
Also, it’s bare-metal instances — you’re managing more yourself. No managed services, no hand-holding. If you need that, look elsewhere.
Pricing advantages:
Best for: Individual researchers, startups, teams comfortable with infrastructure, projects where you can be flexible on timing.
Not ideal for: Enterprises needing SLAs, projects requiring guaranteed immediate availability, teams wanting managed services.
Vast.ai is like Airbnb for GPUs. People with idle GPUs rent them out, you pay less than traditional clouds. It’s clever, but comes with quirks.
You bid on GPU time from a marketplace of providers. Prices are incredibly competitive because you’re renting someone’s idle hardware.
What you’ll find:
The prices are genuinely shocking. I’ve rented RTX 4090s for less than AWS charges for T4s.
Here’s the thing — you’re renting from random people. Sometimes the connection drops. Sometimes the host goes offline. Sometimes you get a flaky GPU.
What works:
What doesn’t:
I use Vast.ai for experiments and initial testing. When I need reliability, I move to Lambda or GCP.
Best for: Hobbyists, students, extremely budget-conscious projects, experimentation and testing.
RunPod sits between Vast.ai’s chaos and Lambda’s simplicity. They provide both marketplace GPUs and their own managed infrastructure.
You can choose secure cloud (RunPod’s own GPUs) or community cloud (marketplace, like Vast.ai).
Secure cloud pricing:
Community cloud: Similar to Vast.ai, significantly cheaper but less reliable.
RunPod’s container-based approach is actually nice. You work with Docker containers, which makes environment management cleaner.
What I appreciate:
The serverless GPUs are underrated. You only pay for actual compute time, not idle time. For inference or sporadic training, this saves serious money.
Best for: Teams comfortable with containers, projects with variable compute needs, developers wanting good UX without AWS complexity.
Paperspace wants to be the complete ML platform — notebooks, training, deployment, the works. The GPU rental is just part of it.
If you want a managed Jupyter environment with GPU access and don’t want to think about infrastructure, Paperspace delivers.
GPU options:
The free tier is genuinely useful for learning and small experiments. Not many platforms offer free GPU access.
Paperspace pushes you toward their ecosystem. Notebooks, workflows, deployments — they want you using their tools for everything.
Pros of the platform:
Cons:
Best for: Teams wanting managed ML platform, individuals learning deep learning, projects valuing convenience over cost optimization.
Jarvis is newer but making waves with competitive pricing and good availability. They’re focused purely on deep learning workloads.
Jarvis offers dedicated instances at prices closer to shared platforms. That’s unusual and valuable.
Current lineup:
The dedicated instances mean you’re not sharing the GPU. For training, this consistency matters. Shared GPUs can have variable performance depending on what else is running.
Jarvis doesn’t have the track record of Lambda or the infrastructure of AWS. If they go out of business, you’re scrambling. This is a real consideration for long-term projects.
What I like:
Best for: Projects needing dedicated GPUs at reasonable prices, teams willing to take some platform risk for cost savings.
Let me run real numbers for a common scenario: training a large vision model requiring approximately 80 hours on an A100.
AWS EC2 p4d (On-Demand):
GCP A100:
Lambda Labs:
RunPod (Secure Cloud):
Vast.ai (Marketplace):
The difference between AWS and Lambda is $2,547. For the same GPU. That’s not a typo.
Forget what’s “best” universally. Here’s how to actually decide:
Go with AWS or GCP. Yes, it’s expensive. But you get SLAs, compliance certifications, integration with existing infrastructure, and someone to call when things break. The premium is insurance.
Start with Lambda Labs. If GPUs aren’t available when you need them, try RunPod or Jarvis. The cost savings over cloud giants is just too massive to ignore.
Paperspace free tier or Vast.ai marketplace. You don’t need reliability for tutorials and experiments. Save your money.
Lambda Labs or GCP with preemptible/spot instances disabled. AWS is reliable but often overkill unless you’re already deep in their ecosystem.
RunPod Serverless or GCP with appropriate instance sizes. Pay-per-second billing matters more for inference than training.
The real cost optimization isn’t choosing the cheapest platform — it’s reducing training time.
Ways to actually save money:
I’ve saved more money optimizing training efficiency than I ever did hunting for cheap GPUs. A 2x speedup on Lambda ($1.10/hour) saves more than switching to Vast.ai ($0.80/hour) with 20% overhead from instability.
Stop overthinking this. Here’s what you do:
Most people do this backwards — they obsess over saving $0.20/hour while running inefficient code that wastes hours. Fix your code first, then optimize your platform choice.
And for the love of all that is holy, use spot/preemptible instances for anything that can handle interruptions. The 70% discount is too good to pass up.
Now stop reading comparison articles and go train something. The best GPU platform is the one you’re actually using to build things.
Comments
Post a Comment