Generate¶
Use client.generate() to run a model without changing its weights.
job = client.generate(
model="Qwen/Qwen2.5-0.5B",
data=[{"text": "Write one short sentence about GPUs."}],
max_new_tokens=48,
gpus=1,
)
result = job.run(wait=True)
if result["status"] == "completed":
job.download("./generate-output")
Generated text is saved to the output folder. Use job.get_logs() to view progress while
the job runs.
Vision-language models¶
With a vision-language model and a local image manifest: