Note: superbeginner here.
Working with Databricks’ dolly-v2-3b.
Installed git 2.40.0.windows.1 and downloaded Dolly
Installed CUDA 12.1.1_531.14
Installed python 3.11.3 and installed the necessary using the following commands:
py.exe -m pip install numpy
py.exe -m pip install accelerate>=0.12.0 transformers[torch]==4.25.1
py.exe -m pip install numpy --pre torch --force-reinstall --index-url https://download.pytorch.org/whl/nightly/cu117 --user
Now, when I run the following commands, everything works ok and Dolly will produce responses.
from transformers import pipeline
generate_text = pipeline(model=“databricks/dolly-v2-3b”, torch_dtype=torch.bfloat16, trust_remote_code=True, device_map=“auto”)
However, I want to not use the trust_remote_code. So I downloaded the instruct_pipeline.py and added it to the C:\Users\xxxxxx\AppData\Local\Programs\Python\Python311\Lib. But when I try to do:
from instruct_pipeline import InstructionTextGenerationPipeline
I get an error as per the screenshot:
Not sure how to proceed?
I’m running everything on windows 11. I’ve also noticed my GPU (1660 super, driver v531.41) is not used at all (0% gpu at task manager), shouldn’t utilization jump to 100% when I run a prompt?