Quickstart

In a few minutes, you'll go from zero to accelerated LLM inference on AMD hardware. The entire workflow is containerized; no host dependencies to manage, no hours spent compiling frameworks from source.

---

Supported Hardware

daisy-vllm runs on AMD ROCm-supported GPUs:

Series	Supported Architectures	Example GPUs
AMD Instinct™	gfx90a, gfx942	MI210, MI250, MI250X, MI300X
Radeon™ Pro & Desktop	gfx1100, gfx1101, gfx1102	Radeon RX 7900 XTX / 7900 XT (RDNA 3)
Next-Gen Hardware	gfx1200, gfx1201	RDNA 4 development targets

Recommendation: For production workloads, AMD Instinct MI210/MI250/MI300X GPUs are recommended. For development, RDNA 3 desktop GPUs work well.

---

Quick Setup

Run the following commands in sequence to go from zero to your first accelerated inference:

# 1. Clone the repository with compiler submodules
git clone --recursive https://github.com/daisytuner/daisy-vllm.git
cd daisy-vllm

# 2. Initialize environment configuration
cp .env.example .env

# 3. Build the docker image
make build-docker-image

# 4. Launch the container environment
make shell

# 5. Install the editable package
pip install -e .

# 6. Compile docc backend
make build-docc

# 7. Run your first accelerated benchmark
python benchmark/qwen2_poem.py -dt bfloat16 -e daisy-vllm -p

---

Expected Output

When the benchmark runs, you'll see daisy-vllm initialize the engine, load the model, capture piecewise CUDA graphs through the docc compiler, and generate an accelerated response:

That's it. You just ran accelerated inference with docc-compiled MLP kernels on an AMD GPU.

---

What Just Happened?

Behind those few commands, a sophisticated compilation pipeline executed:

1. Graph Capture: vLLM's piecewise CUDA graph mode captured the model's execution graph, splitting it at docc_mlp boundaries.
2. MLP Isolation: The SwiGLU MLP layers (Gate/Up projections + SiLU activation + Down projection) were extracted as separate subgraphs.
3. docc Compilation: These MLP subgraphs were lowered through the docc compiler, which generated fused, shape-specialized ROCm kernels tuned for your GPU architecture.
4. Hybrid Execution: The final model runs a hybrid graph. Attention and norms on Torch Inductor, MLPs on compiled docc kernels.

The result? The 83% of GPU time that was spent on generic GEMM library calls is now running through optimized, fused kernels.

Deep Dive: Want to learn why MLP layers bottleneck GPU inference and how docc fuses these operations? Read Architecture & How It Works.

---

What's Next?

• Architecture & How It Works: Deep dive into the MLP bottleneck and compiler integration.
• Workload Benchmarking: Compare performance across vllm, vllm-full, and daisy-vllm.
• Correctness Verification: Validate compiled model numerical fidelity against eager PyTorch.
• Supported Models & Precisions: View validated checkpoints, planned models, and DaisyLLM parameters.
• Frequently Asked Questions (FAQs): Solutions for common ROCm and permission issues.