Skip to content

Hardware and billing

Start with a small model and training_mode="low", and use estimate_model before trying larger models.

Model size Mode Single GPU
0.5B - 3B low, medium Yes
3B - 7B low Yes
7B medium Yes
13B low Tight
30B+ low Unlikely

Each job runs on one GPU. Maximum duration is set per workspace or cohort.

Choosing a GPU

client.gpus() lists the GPUs you can use right now. Each row shows the GPU's memory, the minimum RAM and vCPUs that come with it, and its price per GPU-hour today.

client.gpus()
  Offering     GPU            VRAM   RAM    vCPU  $/GPU-hr  Now
  -----------  -------------  -----  -----  ----  --------  -------------
  od-v100-1    V100 32GB      32 GB  24 GB  8     0.09      available
  od-v100-2 *  V100 32GB      32 GB  16 GB  4     0.10      busy, 1 waiting
  od-l4-2      L4 24GB        24 GB  16 GB  8     0.39      available
  ...

The list changes during the day, so read it before you pick. You choose a GPU in one of three ways, on pretrain(), finetune(), generate() and evaluate():

You pass The job runs on
Nothing DataSpires' own V100 32GB (marked *). It runs one job at a time, so your job may wait for it.
gpu_type="l4" The cheapest L4 on the list, preferring one that can start now
offering="od-l4-2" Exactly that GPU from the list
job = client.finetune(
    model="qwen2.5-0.5b",
    training_mode="low",
    data=data,
    gpu_type="l4",          # or offering="od-l4-2"
)

gpu_type accepts these values. Any other value is rejected.

gpu_type Hardware bf16
"v100" NVIDIA V100 32GB No. evaluate() switches to fp16 on its own.
"l4" NVIDIA L4 24GB Yes
"4090" NVIDIA RTX 4090 24GB Yes
"5090" NVIDIA RTX 5090 32GB Yes
"a100" NVIDIA A100 80GB Yes

If you are in a pilot group, a job without gpu_type or offering runs on any eligible GPU your group has.

Pricing

Prices are per GPU-hour and follow live supply, so the same GPU type can cost different amounts on different rows of client.gpus().

  • The price is locked when you submit. The job prints the GPU and its price, for example GPU: L4 24GB (od-l4-2) at $0.39/GPU-hour, locked for this job.
  • You pay for the time the job runs. Time spent waiting for a GPU is free.
  • Submitting needs credits for one hour at the GPU's price. If your credits run out while the job runs, it stops and can be resumed from its latest checkpoint after a top-up.
  • A busy GPU queues your job. It starts on its own when the GPU frees up. A job that has not started within 24 hours is cancelled without charge.
  • Machine failures are not charged. If the GPU machine fails, the job restarts on a new one from its latest checkpoint, and the failed run is free.

job.estimated_cost_usd() shows the most a job can cost: its time_limit at the price of the GPU it would run on now.

job = client.finetune(model="qwen2.5-0.5b", data=data, time_limit="02:00:00")
print(f"At most ${job.estimated_cost_usd():.2f}")

See current prices on the Compute page and invoices on Billing.