Docker huggingface/transformers-pytorch-gpu and spaCy compatibility

Beginners
1

Hi everyone,

I’m trying to run a Docker container using the huggingface/transformers-pytorch-gpu image for a test project. As part of this project, I need to install spaCy, but I’m running into various installation issues.

I’ve tried several approaches, including installing different versions of spaCy and its dependencies, but every time I attempt this, new issues arise. Some of the problems I’ve encountered include conflicting dependencies, missing packages, or version incompatibilities with the Hugging Face image.

Here’s a summary of what I’ve tried so far:

  1. Basic pip install: I started by running pip install spacy inside the container, but this led to dependency conflicts with some of the Hugging Face libraries.
  2. Specific spaCy version: I tried installing spaCy versions 3.8.2 and earlier, but the installation failed with similar dependency issues.
  3. spaCy transformers: I also attempted to install the spacy-transformers package to ensure compatibility with the Hugging Face transformer models, but it didn’t resolve the conflicts.

I’m hoping someone can guide me on how to properly set up spaCy with the huggingface/transformers-pytorch-gpu Docker image or suggest an alternative approach that works smoothly. If anyone has successfully integrated these tools in a Docker environment, your help would be greatly appreciated!

1 Like
2

Try this.

pip install git+https://github.com/explosion/spaCy

# requirements.txt
git+https://github.com/explosion/spaCy
3

Thank you John6666,

After hours of trying, I finally found the working solution for my project. The tricky part was identifying compatible versions of all the packages I needed. The result of this effort is this Docker image: spacy-intents, which is optimized for intent recognition tasks and fine-tuning spaCy models. Maybe someone will find it useful.

1 Like