The flag load_in_8bit is used to enable 8-bit quantization with LLM.int8(). LLM.int8 is a lightweight wrapper around CUDA custom functions, so the quantization is only possible in GPU.
You have the required details in offical bitsandbytes github page.
Requirements: Python >=3.8. Linux distribution (Ubuntu, MacOS, etc.) + CUDA > 10.0.