Skip to main content

Frequently Asked Questions (FAQs)

Find answers to common questions and solutions to runtime errors encountered when building and running daisy-vllm on AMD ROCm hardware. Click on any topic below to expand.

1. Why do I see ModuleNotFoundError: No module named 'daisy_vllm' or 'docc'?

Cause​

In the container base image, vllm is pre-installed, but docc (the custom compiler) and daisy_vllm (the local integration package) are mounted from your host workspace and must be linked to the Python environment.

Solution​

Inside your container shell (make shell), execute:

make build-all

This compiles docc using ROCm LLVM (clang++) and installs daisy-vllm in editable mode (pip install -e .).

Alternatively, launch the developer container from the host with make dev, which runs this automatically.


2. How do I fix HSA_STATUS_ERROR_INVALID_ISA: The instruction set architecture is invalid?

Cause​

This occurs when HSA_OVERRIDE_GFX_VERSION is set to an ISA version incompatible with your physical GPU (for example, setting 12.0.1 on an AMD Instinct MI210 gfx90a card).

Solution​

  • Datacenter GPUs (MI200, MI210, MI250, MI300): Do not set an override. Leave HSA_OVERRIDE_GFX_VERSION unset in your .env file so ROCm detects your native ISA (gfx90a / gfx942).
  • Consumer RDNA 3 GPUs (e.g. RX 7900 XTX): Set HSA_OVERRIDE_GFX_VERSION=11.0.0 in .env.
  • RDNA 4 targets: Set HSA_OVERRIDE_GFX_VERSION=12.0.1 in .env.

3. How do I fix Permission denied on /dev/kfd or /dev/dri?

Cause​

Your Linux user account on the host machine lacks permission to access the AMD Kernel Fusion Driver (/dev/kfd) and DRI direct rendering nodes.

Solution​

Add your user to the render and video system groups on the host:

sudo usermod -a -G render,video $USER

Then log out and log back in, or run newgrp render to refresh your active session. Verify with:

groups | grep -E "render|video"

4. How do I resolve 401 Client Error: Huggingface Model Not Found / Unauthorized?

Cause​

The requested Hugging Face model repository is gated or private, and no valid authentication token was provided.

Solution​

  1. Accept the model license terms on the model's Hugging Face page.
  2. Generate a User Access Token in your Hugging Face settings.
  3. Add your token to .env: 
    HF_TOKEN=hf_yourTokenHere
  4. If invoking docker run manually, include -e HF_TOKEN=$HF_TOKEN.

5. What should I do if I encounter GPU Out of Memory (OOM) during KV Cache allocation?

Cause​

vLLM pre-allocates GPU VRAM for the KV cache based on gpu_memory_utilization. If background processes or the display manager consume GPU memory, allocation fails.

Solution​

For the currently supported DaisyLLM model, max_model_len and gpu_memory_utilization are taken from DoccQwen2ForCausalLM.get_default_config() in daisy_vllm/model_executor/models/qwen2.py. Constructor arguments and .env values do not override these model defaults.

To reduce the memory requirement, lower max_model_len from 4096 (for example, to 2048) in that configuration. If other processes leave too little free VRAM for the configured fraction, also lower gpu_memory_utilization from 0.70, then restart the inference process.


6. What diagnostic logs should I collect when reporting an issue?

Please collect and share the following outputs:

# 1. Host GPU hardware and ROCm driver version
rocm-smi --showdriverversion --showproductname > rocm_info.txt

# 2. Check architecture detection
amdgpu-arch >> rocm_info.txt

# 3. Docker system version
docker version > docker_info.txt