GPU

GPU inference, anywhere there is idle silicon.

Run LLM, vision, and audio workloads on consumer NVIDIA cards and Apple Silicon MLX. Per-second billing, pre-flight benchmark, bring your own weights.

Run inference See pricing

What it is

iogrid GPU is a workload-scheduler for CUDA and MLX containers running on idle consumer-class GPUs — NVIDIA 4090 and 5090 class cards, plus Apple Silicon with MLX. The same SDK and API as Docker compute; the scheduler just additionally matches on VRAM and vendor.

Designed for batch inference, embedding generation, fine-tuning, and other GPU-bursty workloads where 5–60 second scheduling latency is acceptable in exchange for ~10× lower per-hour cost than a reserved datacenter GPU.

How it works

Step 1

Declare your GPU requirement

Pick a VRAM floor (e.g. 24 GB+ for a 13B model in 4-bit) and optionally an allowed-vendor list (NVIDIA, Apple Silicon).

Step 2

Scheduler finds matching silicon

The coordinator picks an opted-in provider with enough free VRAM, matching vendor, and reachable network. Apple Silicon uses Apple's Virtualization framework + MLX bindings; NVIDIA uses the NVIDIA Container Toolkit.

Step 3

Pre-flight benchmark, then charge

The container runs a short benchmark to confirm performance against the spec — if it fails (overheating, contention), the job is rescheduled at no charge. After pass, per-second billing starts.

Pricing

Consumer NVIDIA (24 GB VRAM, e.g. 4090)	$0.20 / GPU-hour
Apple Silicon MLX (M3 Max, M4)	$0.20 / GPU-hour
Pro / data-center class	Up to $2.00 / GPU-hour
Billing granularity	Per-second after first minute
Pre-flight benchmark	Free (only billed if pass)
$GRID discount	20% off list price

Reference range: $0.20 – $2.00 per GPU-hour, tiered by GPU class (consumer / pro / data-center). Provider payout: $0.05 – $0.50 per GPU-hour depending on tier.

What you can do with it

Batch LLM inference

Generate embeddings, summarise long documents, run open-weight models (Llama, Mistral, Qwen). 24 GB VRAM holds a 13B model in 4-bit quantisation comfortably.

Computer vision pipelines

Object detection, segmentation, OCR over image and video archives. Bring a Hugging Face TGI or vLLM template, point it at your S3 bucket, fan out.

Fine-tuning small models

LoRA / QLoRA fine-tunes on consumer cards. Cheaper than reserving a Lambda Labs instance for a few hours, and the pre-flight benchmark protects you from thermally throttled hardware.

Get started

Same SDK as Docker compute — only the workload type and the GPU spec change.

import { IogridClient } from '@iogrid/sdk';

const iogrid = new IogridClient({ apiKey: process.env.IOGRID_API_KEY! });

const w = await iogrid.createWorkload({
  type: 'GPU',
  gpu: {
    image: 'ghcr.io/huggingface/text-generation-inference:latest',
    env: { MODEL_ID: 'mistralai/Mistral-7B-Instruct-v0.3' },
    timeoutSeconds: 3600,
    minVramMib: 24576,
    allowedVendors: ['NVIDIA', 'APPLE'],
  },
});

const { result } = await iogrid.getWorkload(w.id);
console.log(result?.terminalStatus, result?.cost);

Get an API key Read the SDK docs

FAQ

Which GPUs are in the network?: Consumer NVIDIA cards with 24 GB+ VRAM (4090, 5090) and Apple Silicon (M3 Max, M4 Max, M4 Ultra). Pro and data-center class (A100, H100) are available from a smaller pool at proportionally higher rates.
Can I run my own model weights?: Yes. Bring an OCI image that pulls weights from your S3 bucket or Hugging Face — provider hosts never inspect your image. Hugging Face TGI and vLLM are the most common templates.
What is the pre-flight benchmark?: Before charging, the container runs a short workload-shaped probe to verify the chosen GPU is actually performing to spec. If it fails (thermal throttle, VRAM fragmentation, driver issue), the scheduler picks a different provider at no charge.
How does Apple Silicon compare to NVIDIA?: Unified memory means an M4 Max with 128 GB can hold larger models than a 24 GB 4090, but raw throughput on common kernels is lower. MLX-tuned workloads close most of that gap. Pick by model size and target latency.
Is this real-time-suitable?: Not as a primary endpoint — there is variable scheduling latency. Use for batch, fine-tuning, or behind a queue. For latency-sensitive endpoints, pair iogrid with a small datacenter footprint and burst overflow to iogrid.