Hardware and billing¶
Start with a small model and training_mode="low", and use
estimate_model before trying larger models.
| Model size | Mode | Single GPU |
|---|---|---|
| 0.5B - 3B | low, medium | Yes |
| 3B - 7B | low | Yes |
| 7B | medium | Yes |
| 13B | low | Tight |
| 30B+ | low | Unlikely |
Each job runs on one GPU. Maximum duration is set per workspace or cohort.
Choosing a GPU¶
client.gpus() lists the GPUs you can use right now. Each row shows the GPU's memory,
the minimum RAM and vCPUs that come with it, and its price per GPU-hour today.
Offering GPU VRAM RAM vCPU $/GPU-hr Now
----------- ------------- ----- ----- ---- -------- -------------
od-v100-1 V100 32GB 32 GB 24 GB 8 0.09 available
od-v100-2 * V100 32GB 32 GB 16 GB 4 0.10 busy, 1 waiting
od-l4-2 L4 24GB 24 GB 16 GB 8 0.39 available
...
The list changes during the day, so read it before you pick. You choose a GPU in one of three
ways, on pretrain(), finetune(), generate() and evaluate():
| You pass | The job runs on |
|---|---|
| Nothing | DataSpires' own V100 32GB (marked *). It runs one job at a time, so your job may wait for it. |
gpu_type="l4" |
The cheapest L4 on the list, preferring one that can start now |
offering="od-l4-2" |
Exactly that GPU from the list |
job = client.finetune(
model="qwen2.5-0.5b",
training_mode="low",
data=data,
gpu_type="l4", # or offering="od-l4-2"
)
gpu_type accepts these values. Any other value is rejected.
gpu_type |
Hardware | bf16 |
|---|---|---|
"v100" |
NVIDIA V100 32GB | No. evaluate() switches to fp16 on its own. |
"l4" |
NVIDIA L4 24GB | Yes |
"4090" |
NVIDIA RTX 4090 24GB | Yes |
"5090" |
NVIDIA RTX 5090 32GB | Yes |
"a100" |
NVIDIA A100 80GB | Yes |
If you are in a pilot group, a job without gpu_type or offering runs on any eligible
GPU your group has.
Pricing¶
Prices are per GPU-hour and follow live supply, so the same GPU type can cost different
amounts on different rows of client.gpus().
- The price is locked when you submit. The job prints the GPU and its price, for
example
GPU: L4 24GB (od-l4-2) at $0.39/GPU-hour, locked for this job. - You pay for the time the job runs. Time spent waiting for a GPU is free.
- Submitting needs credits for one hour at the GPU's price. If your credits run out while the job runs, it stops and can be resumed from its latest checkpoint after a top-up.
- A busy GPU queues your job. It starts on its own when the GPU frees up. A job that has not started within 24 hours is cancelled without charge.
- Machine failures are not charged. If the GPU machine fails, the job restarts on a new one from its latest checkpoint, and the failed run is free.
job.estimated_cost_usd() shows the most a job can cost: its time_limit at the price of
the GPU it would run on now.
job = client.finetune(model="qwen2.5-0.5b", data=data, time_limit="02:00:00")
print(f"At most ${job.estimated_cost_usd():.2f}")
See current prices on the Compute page and invoices on Billing.