How to install flash attention on HF gradio space

Spaces
1

I tried to put flash-attn in the requirements.txt file to install flash-attention on my space, but it gives error that torch is not installed.

I also tried to put torch above flash-attn but still couldn’t, probably torch is not installed yet.

Please help!

2

hi @nxphi47 ,

One option is to use a custom Dockerfile and install as a build step

Another option if you’re using Gradio/Streamlit SDK is to install at runtime

import subprocess
subprocess.run('pip install flash-attn --no-build-isolation', env={'FLASH_ATTENTION_SKIP_CUDA_BUILD': "TRUE"}, shell=True)