Environment Variables & Configuration
This reference documents the environment variables supported by daisy-vllm, .env, and docker-compose.yml.
1. Authentication & Model Access
| Variable | Default | Description |
|---|---|---|
HF_TOKEN | (empty) | Hugging Face User Access Token. Required for gated or private repositories. |
2. AMD ROCm GPU Configuration
| Variable | Default | Description |
|---|---|---|
ROCR_VISIBLE_DEVICES | 0 | Comma-separated list of GPU indices to expose to the container (e.g. 0,1 for multi-GPU). |
HIP_VISIBLE_DEVICES | 0 | Alternative HIP GPU visibility mask used inside certain PyTorch runtimes. |
GPU_ARCH | Auto-detected | The target AMD GPU architecture (e.g. gfx90a, gfx942, gfx1100, gfx1201). |
DOCC_ROCM_ARCH | ${GPU_ARCH} | ROCm architecture string passed directly to the docc compiler. |
HSA_OVERRIDE_GFX_VERSION | 12.0.1 | Allows running on development/pre-release hardware by aliasing the GFX driver target. |
3. vLLM Engine Memory & Performance
| Variable | Default | Description |
|---|---|---|
GPU_MEMORY_UTILIZATION | 0.85 | Fraction of total VRAM allocated for model weights and KV cache (between 0.0 and 1.0). |
VLLM_TARGET_DEVICE | rocm | Target hardware backend (rocm for AMD GPUs, cpu for fallback testing). |
4. Compiler & Debugging Options
| Variable | Default | Description |
|---|---|---|
DOCC_DEBUG | (empty) | Set to dump to output compiled kernel graphs, fusion traces, and assembly logs to disk. |
DOCC_REUSE_BINARIES | 1 | Cache compiled kernel binaries on disk to skip compilation on subsequent runs. |
Configuration Example (.env)
# Secrets
HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
# Hardware targeting
ROCR_VISIBLE_DEVICES=0
GPU_ARCH=gfx90a
DOCC_ROCM_ARCH=gfx90a
# Engine memory allocation
GPU_MEMORY_UTILIZATION=0.90
MODEL=Qwen/Qwen2.5-1.5B-Instruct