Skip to content

Generate

Use client.generate() to run a model without changing its weights.

job = client.generate(
    model="Qwen/Qwen2.5-0.5B",
    data=[{"text": "Write one short sentence about GPUs."}],
    max_new_tokens=48,
    gpus=1,
)
result = job.run(wait=True)

if result["status"] == "completed":
    job.download("./generate-output")

Generated text is saved to the output folder. Use job.get_logs() to view progress while the job runs.

Vision-language models

With a vision-language model and a local image manifest:

job = client.generate(
    model="smolvlm-256m",
    data="./manifest_dir/",
    outputs=["text", "confidence"],
    max_new_tokens=64,
    temperature=0.0,
)
result = job.run(wait=True)
if result["status"] == "completed":
    job.download("./vlm-output")