Setting Up a RunPod A100 Sandbox in Under 5 Minutes

This guide gets you from "signed into RunPod" to "a benchmark run in GPUOPs with real metrics" in a single sitting.

Prerequisites

A RunPod account with billing enabled
A GPUOPs account (the free trial is enough)
A model you want to benchmark — fine-tuned or base

Step 1 — Spin up the instance

In RunPod → Deploy, choose On-Demand or Interruptible.
Select the NVIDIA A100 80GB PCIe template.
Pick the official RunPod PyTorch 2.x image.
Set disk to 100GB (room for the model + logs).
Click Deploy On-Demand.

You should be in the pod within ~60 seconds.

Step 2 — Copy the generated script

GPUOPs Sandbox Wizard produces a benchmark script tailored to your hardware and framework. Open the Deploy page, choose your runbook, and copy the

bench.sh

to your clipboard.

Step 3 — Drop it in the pod

Open the pod's web terminal and paste:

pip install -q transformers accelerate
bash bench.sh

That script handles warmup, timed generation, and a metrics emission in the exact JSON shape GPUOPs expects.

Step 4 — Submit metrics back

Paste the JSON line the script prints into the Manual run submission box on the Runs page. The platform parses:

```
throughput_tokens_sec
```
```
latency_p50_ms
```
/
```
latency_p99_ms
```
```
vram_used_gb
```
/
```
vram_total_gb
```
```
sm_occupancy_pct
```
```
power_draw_watts
```

…and you'll see your run in the dashboard with AI optimization suggestions populated automatically.

Step 5 — Tear down

When you're done, Stop (not Terminate) if you'll rerun later — terminated pods can't snapshot.

Common gotchas

Cold cache: First run is always slow due to disk paging. Run the script twice and discard the first.
PCIe vs SXM: Bandwidth-limited benchmarks (memory bandwidth tests, KV cache heavy) will look 30% better on SXM-sponsored machines.
Interruptible vs on-demand: Interruptible instances can be reclaimed mid-run. Don't use them for sweeps.

Making it repeatable

Save the configuration as a template in GPUOPs (

is_template: true

). Your teammates can clone it, change the model name, and reproduce your exact setup in seconds.

The Sandbox Wizard handles Steps 1-4 automatically for RunPod, Lambda Labs, Vast.ai, and the three hyperscalers. Try it on the Deploy page.

Setting Up a RunPod A100 Sandbox for Benchmarking in Under 5 Minutes