> ## Documentation Index
> Fetch the complete documentation index at: https://runpod-b18f5ded-docs-366-gpu-param-semantics.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# GPU types

> Available GPU pools and specific GPU types for Flash endpoints. Review setup, configuration, deployment, and usage guidance for Runpod Flash.

Flash provides access to a wide range of NVIDIA GPUs through both pool-based and specific GPU selection. This page lists all available GPU types and explains how to use them.

## GPU selection methods

Flash offers two ways to specify GPU hardware:

1. [GPU pools](/flash/configuration/gpu-types#gpu-pools) (`GpuGroup`): Select from predefined pools of similar GPUs grouped by architecture and VRAM.
2. [Specific GPU types](/flash/configuration/gpu-types#specific-gpu-types) (`GpuType`): Target exact GPU models when you need precise hardware characteristics.

You can use either method or mix both for [advanced fallback strategies](/flash/configuration/gpu-types#advanced-fallback-strategies).

## GPU pools

The `GpuGroup` enum provides access to GPU pools. Each pool contains specific GPU models grouped by architecture and VRAM capacity.

### Available GPU pools

| GpuGroup                 | GPUs Included                                       | VRAM    | Best For                         |
| ------------------------ | --------------------------------------------------- | ------- | -------------------------------- |
| `GpuGroup.ANY`           | Any available GPU                                   | Varies  | Fast provisioning, prototyping   |
| `GpuGroup.AMPERE_16`     | RTX A4000, RTX 4000 Ada, RTX 2000 Ada               | 16GB    | Small models, basic inference    |
| `GpuGroup.AMPERE_24`     | RTX A4500, RTX A5000, RTX 3090                      | 20-24GB | General ML, mid-size models      |
| `GpuGroup.ADA_24`        | L4, RTX 4090                                        | 24GB    | Cost-effective inference         |
| `GpuGroup.ADA_32_PRO`    | RTX 5090                                            | 32GB    | Latest consumer flagship         |
| `GpuGroup.AMPERE_48`     | A40, RTX A6000                                      | 48GB    | Large models, fine-tuning        |
| `GpuGroup.ADA_48_PRO`    | L40S, L40, RTX 6000 Ada                             | 48GB    | Professional inference           |
| `GpuGroup.AMPERE_80`     | A100 80GB PCIe, A100-SXM4-80GB                      | 80GB    | XL models, intensive training    |
| `GpuGroup.ADA_80_PRO`    | H100 80GB HBM3                                      | 80GB    | Cutting-edge inference           |
| `GpuGroup.BLACKWELL_96`  | RTX PRO 6000 Blackwell (Server, Workstation, Max-Q) | 96GB    | Professional Blackwell workloads |
| `GpuGroup.HOPPER_141`    | H200                                                | 141GB   | Largest models, maximum VRAM     |
| `GpuGroup.BLACKWELL_180` | B200                                                | 180GB   | Maximum VRAM, next-gen training  |

### Using GPU pools

```python theme={null}
from runpod_flash import Endpoint, GpuGroup

# Single GPU pool
@Endpoint(name="inference", gpu=GpuGroup.AMPERE_80)
async def infer(data: dict) -> dict:
    ...

# Multiple pools for fallback
@Endpoint(
    name="flexible",
    gpu=[GpuGroup.AMPERE_80, GpuGroup.AMPERE_48, GpuGroup.ADA_24]
)
async def flexible_infer(data: dict) -> dict:
    ...

# Any available GPU (fastest provisioning)
@Endpoint(name="development", gpu=GpuGroup.ANY)
async def dev_infer(data: dict) -> dict:
    ...
```

## Specific GPU types

The `GpuType` enum provides access to specific GPU models. Use these when you need exact hardware characteristics.

A single `GpuType` requests that exact card — this is the only form guaranteed to pin workers to a specific GPU model. If you pass a list of `GpuType` values, placement is advisory at the pool/VRAM-tier level instead. See [GPU selection behavior](/flash/configuration/gpu-types#gpu-selection-behavior) for details.

### Available GPU types

| GpuType                                                           | GPU Model                                               | VRAM   | Architecture |
| ----------------------------------------------------------------- | ------------------------------------------------------- | ------ | ------------ |
| `GpuType.ANY`                                                     | Any available GPU                                       | Varies | Any          |
| `GpuType.NVIDIA_RTX_A4000`                                        | NVIDIA RTX A4000                                        | 16GB   | Ampere       |
| `GpuType.NVIDIA_RTX_A4500`                                        | NVIDIA RTX A4500                                        | 20GB   | Ampere       |
| `GpuType.NVIDIA_RTX_4000_ADA_GENERATION`                          | NVIDIA RTX 4000 Ada                                     | 16GB   | Ada Lovelace |
| `GpuType.NVIDIA_RTX_2000_ADA_GENERATION`                          | NVIDIA RTX 2000 Ada                                     | 16GB   | Ada Lovelace |
| `GpuType.NVIDIA_RTX_A5000`                                        | NVIDIA RTX A5000                                        | 24GB   | Ampere       |
| `GpuType.NVIDIA_L4`                                               | NVIDIA L4                                               | 24GB   | Ada Lovelace |
| `GpuType.NVIDIA_GEFORCE_RTX_3090`                                 | NVIDIA GeForce RTX 3090                                 | 24GB   | Ampere       |
| `GpuType.NVIDIA_GEFORCE_RTX_4090`                                 | NVIDIA GeForce RTX 4090                                 | 24GB   | Ada Lovelace |
| `GpuType.NVIDIA_GEFORCE_RTX_5090`                                 | NVIDIA GeForce RTX 5090                                 | 32GB   | Blackwell    |
| `GpuType.NVIDIA_A40`                                              | NVIDIA A40                                              | 48GB   | Ampere       |
| `GpuType.NVIDIA_RTX_A6000`                                        | NVIDIA RTX A6000                                        | 48GB   | Ampere       |
| `GpuType.NVIDIA_RTX_6000_ADA_GENERATION`                          | NVIDIA RTX 6000 Ada                                     | 48GB   | Ada Lovelace |
| `GpuType.NVIDIA_A100_80GB_PCIe`                                   | NVIDIA A100 80GB PCIe                                   | 80GB   | Ampere       |
| `GpuType.NVIDIA_A100_SXM4_80GB`                                   | NVIDIA A100-SXM4-80GB                                   | 80GB   | Ampere       |
| `GpuType.NVIDIA_H100_80GB_HBM3`                                   | NVIDIA H100 80GB HBM3                                   | 80GB   | Hopper       |
| `GpuType.NVIDIA_RTX_PRO_6000_BLACKWELL_SERVER_EDITION`            | NVIDIA RTX PRO 6000 Blackwell Server Edition            | 96GB   | Blackwell    |
| `GpuType.NVIDIA_RTX_PRO_6000_BLACKWELL_WORKSTATION_EDITION`       | NVIDIA RTX PRO 6000 Blackwell Workstation Edition       | 96GB   | Blackwell    |
| `GpuType.NVIDIA_RTX_PRO_6000_BLACKWELL_MAX_Q_WORKSTATION_EDITION` | NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96GB   | Blackwell    |
| `GpuType.NVIDIA_H200`                                             | NVIDIA H200                                             | 141GB  | Hopper       |
| `GpuType.NVIDIA_B200`                                             | NVIDIA B200                                             | 180GB  | Blackwell    |

### Using specific GPU types

```python theme={null}
from runpod_flash import Endpoint, GpuType

# Single specific GPU (pins the exact card)
@Endpoint(name="inference", gpu=GpuType.NVIDIA_A100_80GB_PCIe)
async def infer(data: dict) -> dict:
    ...

# Multiple specific GPUs (requests any card in this set)
@Endpoint(
    name="flexible",
    gpu=[
        GpuType.NVIDIA_A100_80GB_PCIe,
        GpuType.NVIDIA_A100_SXM4_80GB,
        GpuType.NVIDIA_A40
    ]
)
async def flexible_infer(data: dict) -> dict:
    ...
```

When you pass a list, the Flash SDK converts each `GpuType` to its GPU pool (`GpuGroup`) when it creates the endpoint, excluding any pool members you didn't list. Placement within the selected pools is advisory: workers may be scheduled on a different card with equivalent VRAM rather than one of the exact models you listed, and list order is not preserved. Pass a single `GpuType` when your workload requires an exact GPU model.

## Advanced fallback strategies

Combine `GpuGroup` and `GpuType` for robust availability:

```python theme={null}
from runpod_flash import Endpoint, GpuGroup, GpuType

@Endpoint(
    name="hybrid-selection",
    gpu=[
        GpuType.NVIDIA_A100_80GB_PCIe,  # Specific GPU model
        GpuGroup.AMPERE_48,             # Any card in the 48GB Ampere pool
        GpuGroup.ANY                    # Any GPU: expands to all pools for maximum availability
    ]
)
async def infer(data: dict) -> dict:
    ...
```

## GPU selection behavior

**Single GPU type:**
Passing a single `GpuType` requests that exact GPU model. This is the only form guaranteed to pin workers to a specific card. Flash waits for that model to become available, and jobs stay in queue until capacity is free.

```python theme={null}
gpu=GpuType.NVIDIA_GEFORCE_RTX_4090  # Only the RTX 4090
```

**Single GPU pool:**
Passing a single `GpuGroup` requests any card in that pool.

```python theme={null}
gpu=GpuGroup.AMPERE_80  # Any A100 80GB card
```

**A list of GPU types or pools:**
A list requests any one of the listed options. It is not an ordered preference: the SDK does not preserve list order, and placement within the selected pools is advisory — workers may be scheduled on a different card with equivalent VRAM rather than one of the exact models you listed. Be aware of this when benchmarking or when your workload depends on a specific GPU model.

```python theme={null}
gpu=[GpuGroup.AMPERE_80, GpuGroup.AMPERE_48, GpuGroup.ADA_24]
# Any card from the 80GB, 48GB, or 24GB pools
```

**GpuGroup.ANY:**
Flash selects the first available GPU based on current capacity.

```python theme={null}
gpu=GpuGroup.ANY  # Fastest provisioning, unpredictable GPU type
```

<Note>
  There is currently no API field that reports which GPU model a worker was actually placed on. The endpoint configuration reflects what you requested, not where workers ran, so GPU substitution is invisible in the API. To verify placement, query the GPU from inside the worker (for example, with `nvidia-smi`).
</Note>

<Tip>
  **For production**: Pass a single `GpuType` when you need a specific card for predictable cost and performance.
  **For development**: Use `GpuGroup.ANY` for fastest iteration.
</Tip>

## Multi-GPU workers

Request multiple GPUs per worker using `gpu_count`:

```python theme={null}
@Endpoint(
    name="multi-gpu-training",
    gpu=GpuGroup.AMPERE_80,
    gpu_count=4,  # Each worker gets 4 GPUs
    workers=2     # Maximum 2 workers = 8 GPUs total
)
async def train(data: dict) -> dict:
    ...
```

## Handling unavailability

If requested GPUs are unavailable, jobs stay in queue:

```text theme={null}
Initial job status: IN_QUEUE
[Waiting for capacity...]
```

**Solutions:**

1. **Add fallback options**: Use multiple GPU types.
   ```python theme={null}
   gpu=[GpuGroup.AMPERE_80, GpuGroup.AMPERE_48, GpuGroup.ADA_24]
   ```

2. **Use broader selection**: Switch to `GpuGroup.ANY`.
   ```python theme={null}
   gpu=GpuGroup.ANY
   ```

3. **Contact support**: For capacity guarantees, contact [Runpod support](https://www.runpod.io/contact).