I’m trying to contribute to this repository. I’ve cloned the repository down. I ran the install scripts from the main page. I went to the docs folder and ran those scripts in the markdown. I ran the pip install -e docs. In the install script I saw that there was no installer for docs, I believe. I ran the script for doc-builder and that ran completely. I tried generating the markdown script immediately after and that failed.
“WARNING: transformers 4.53.0.dev0 does not provide the extra ‘docs’”
Any advice would be helpful
1 Like
I went to the docs folder
There have been reports of installation errors from Transoformers’ GitHub on Discord, so I think there may be an issue.
However, in this case, there is also the possibility that…
It may need to be from the repository root
rather than inside the docs
folder.
To generate the documentation, you first have to build it. Several packages are necessary to build the doc, you can install them with the following command, at the root of the code repository:
pip install -e “.[docs]”
To trace the problem:
opened 08:43PM - 27 Jan 25 UTC
closed 11:13PM - 27 Jan 25 UTC
It could be that there's just some huge LFS transfer happening silently , but in… stallation of the lighteval is hanging indefinitely for me.
I have tried deleting and recreating my venv, and making sure that I am on python 3.11 and CUDA 12.1.
```
$ pip install -e ".[dev]"
Obtaining file:XXX
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... done
Preparing editable metadata (pyproject.toml) ... done
Collecting transformers@ git+https://github.com/huggingface/transformers.git@main (from open-r1==0.1.0.dev0)
Cloning https://github.com/huggingface/transformers.git (to revision main) to /tmp/pip-install-w8ekpxqk/transformers_12d7dd5f01c446478fcca76df18a0b1c
Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers.git /tmp/pip-install-w8ekpxqk/transformers_12d7dd5f01c446478fcca76df18a0b1c
Resolved https://github.com/huggingface/transformers.git to commit 86d7564611d21731fc004b4e79e472d48c4b0fec
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting trl@ git+https://github.com/huggingface/trl.git@main (from open-r1==0.1.0.dev0)
Cloning https://github.com/huggingface/trl.git (to revision main) to /tmp/pip-install-w8ekpxqk/trl_41176c57d4a640049c4340ff33fe75ec
Running command git clone --filter=blob:none --quiet https://github.com/huggingface/trl.git /tmp/pip-install-w8ekpxqk/trl_41176c57d4a640049c4340ff33fe75ec
Resolved https://github.com/huggingface/trl.git to commit 1123bd0f514164eb297e7d6d48d8d8057c6e7334
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting accelerate>=1.2.1 (from open-r1==0.1.0.dev0)
Using cached accelerate-1.3.0-py3-none-any.whl.metadata (19 kB)
Collecting bitsandbytes>=0.43.0 (from open-r1==0.1.0.dev0)
Using cached bitsandbytes-0.45.1-py3-none-manylinux_2_24_x86_64.whl.metadata (5.8 kB)
Requirement already satisfied: einops>=0.8.0 in ./venv/lib/python3.11/site-packages (from open-r1==0.1.0.dev0) (0.8.0)
Collecting datasets>=3.2.0 (from open-r1==0.1.0.dev0)
Using cached datasets-3.2.0-py3-none-any.whl.metadata (20 kB)
Collecting deepspeed==0.15.4 (from open-r1==0.1.0.dev0)
Using cached deepspeed-0.15.4.tar.gz (1.4 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting hf_transfer>=0.1.4 (from open-r1==0.1.0.dev0)
Using cached hf_transfer-0.1.9-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.7 kB)
Requirement already satisfied: huggingface-hub<1.0,>=0.19.2 in ./venv/lib/python3.11/site-packages (from huggingface-hub[cli]<1.0,>=0.19.2->open-r1==0.1.0.dev0) (0.27.1)
Collecting liger_kernel==0.5.2 (from open-r1==0.1.0.dev0)
Using cached liger_kernel-0.5.2-py3-none-any.whl.metadata (20 kB)
Requirement already satisfied: packaging>=23.0 in ./venv/lib/python3.11/site-packages (from open-r1==0.1.0.dev0) (24.2)
Requirement already satisfied: safetensors>=0.3.3 in ./venv/lib/python3.11/site-packages (from open-r1==0.1.0.dev0) (0.5.2)
Requirement already satisfied: sentencepiece>=0.1.99 in ./venv/lib/python3.11/site-packages (from open-r1==0.1.0.dev0) (0.2.0)
Collecting hjson (from deepspeed==0.15.4->open-r1==0.1.0.dev0)
Using cached hjson-3.1.0-py3-none-any.whl.metadata (2.6 kB)
Requirement already satisfied: msgpack in ./venv/lib/python3.11/site-packages (from deepspeed==0.15.4->open-r1==0.1.0.dev0) (1.1.0)
Collecting ninja (from deepspeed==0.15.4->open-r1==0.1.0.dev0)
Using cached ninja-1.11.1.3-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (5.3 kB)
Requirement already satisfied: numpy in ./venv/lib/python3.11/site-packages (from deepspeed==0.15.4->open-r1==0.1.0.dev0) (1.26.4)
Requirement already satisfied: psutil in ./venv/lib/python3.11/site-packages (from deepspeed==0.15.4->open-r1==0.1.0.dev0) (6.1.1)
Requirement already satisfied: py-cpuinfo in ./venv/lib/python3.11/site-packages (from deepspeed==0.15.4->open-r1==0.1.0.dev0) (9.0.0)
Requirement already satisfied: pydantic>=2.0.0 in ./venv/lib/python3.11/site-packages (from deepspeed==0.15.4->open-r1==0.1.0.dev0) (2.10.6)
Requirement already satisfied: torch in ./venv/lib/python3.11/site-packages (from deepspeed==0.15.4->open-r1==0.1.0.dev0) (2.5.1+cu121)
Requirement already satisfied: tqdm in ./venv/lib/python3.11/site-packages (from deepspeed==0.15.4->open-r1==0.1.0.dev0) (4.67.1)
Requirement already satisfied: triton>=2.3.1 in ./venv/lib/python3.11/site-packages (from liger_kernel==0.5.2->open-r1==0.1.0.dev0) (3.1.0)
DEPRECATION: git+https://github.com/huggingface/lighteval.git@4f381b352c0e467b5870a97d41cb66b487a2c503#egg=lighteval[math] contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce this behaviour change. A possible replacement is to use the req @ url syntax, and remove the egg fragment. Discussion can be found at https://github.com/pypa/pip/issues/11617
Collecting lighteval@ git+https://github.com/huggingface/lighteval.git@4f381b352c0e467b5870a97d41cb66b487a2c503#egg=lighteval[math] (from open-r1==0.1.0.dev0)
Cloning https://github.com/huggingface/lighteval.git (to revision 4f381b352c0e467b5870a97d41cb66b487a2c503) to /tmp/pip-install-w8ekpxqk/lighteval_a5aebb106abb4bb5bf3faea3daefdcb6
Running command git clone --filter=blob:none --quiet https://github.com/huggingface/lighteval.git /tmp/pip-install-w8ekpxqk/lighteval_a5aebb106abb4bb5bf3faea3daefdcb6
Running command git rev-parse -q --verify 'sha^4f381b352c0e467b5870a97d41cb66b487a2c503'
Running command git fetch -q https://github.com/huggingface/lighteval.git 4f381b352c0e467b5870a97d41cb66b487a2c503
Running command git checkout -q 4f381b352c0e467b5870a97d41cb66b487a2c503
```
1 Like
I ran the above command in the root and that is what caused the error :(.
Whenever I run this command: doc-builder build transformers docs/source/en/ --build_dir ~/tmp/test-build
This is the output.
ValueError: There was an error when converting docs\source\en\internal\generation_utils.md to the MDX format.
Unable to find generation.TFGreedySearchEncoderDecoderOutput in transformers. Make sure the path to that object is correct.
I’m just trying to trouble shoot this so I can view my model card that I’m contributing.. Thank you for your help so far.
1 Like
Hmm… Possibly a bug in doc-builder
…?
opened 09:22AM - 28 Mar 22 UTC
closed 10:00AM - 28 Mar 22 UTC
bug
## Describe the bug
Documentation building is failing:
- https://github.com/hu… ggingface/datasets/runs/5716300989?check_suite_focus=true
```
ValueError: There was an error when converting ../datasets/docs/source/package_reference/main_classes.mdx to the MDX format.
Unable to find datasets.filesystems.S3FileSystem in datasets. Make sure the path to that object is correct.
```
opened 09:45AM - 31 May 25 UTC
## Description
When using doc-builder with `convert_notebooks: true`, the links … at the bottom of articles incorrectly point to `.md` files when they should be pointing to `.ipynb` files.
## Steps to reproduce
1. Enable notebook conversion in the workflow file:
```yaml
jobs:
build:
uses: huggingface/doc-builder/.github/workflows/build_main_documentation.yml@main
with:
convert_notebooks: true
# other parameters...
```
2. Build documentation that contains Jupyter notebooks
3. Check the links at the bottom of the generated documentation
## Current behavior
Links at the bottom of articles point to files with `.md` extension, for example:
`[Some notebook title](notebook_name.md)`
## Expected behavior
Links should point to the original `.ipynb` files, for example:
`[Some notebook title](notebook_name.ipynb)`
## Additional context
- This issue affects all notebook files that are converted to MDX
- The incorrect extension makes it harder for users to find and access the original notebook files
- Example repository demonstrating the issue: huggingface/cookbook (see issue #181)
## Environment
- doc-builder version: latest
- Using GitHub Actions workflow for documentation building
On Mac: Cannot build documentation on Mac OS · Issue #32203 · huggingface/transformers · GitHub
1 Like
I’ve made slight progress and now I landed here
Unable to find image_processing_utils_fast.BaseImageProcessorFast in transformers. Make sure the path to that object is correct.
1 Like
Unable to find image_processing_utils_fast.BaseImageProcessorFast in transformers.
This class definitely exists unless you are using a very old version, so the program is probably unable to find the path or has found the wrong path.
Personally, I think that if you are just building the documentation, you should be fine without any dependencies, but maybe in this case you need to install PyTorch and Transformers first…
opened 11:07AM - 26 Sep 23 UTC
closed 08:05AM - 29 Dec 23 UTC
I am trying to build the docs with ```doc-builder ```
using the following co… mmand
```
doc-builder build transformers docs/source/en --build_dir ~/tmp/test-build
```
but got the following error:
```
Traceback (most recent call last):
File "/home/miniconda3/envs/transformers/bin/doc-builder", line 8, in <module>
sys.exit(main())
File "/home/miniconda3/envs/transformers/lib/python3.9/site-packages/doc_builder/commands/doc_builder_cli.py", line 47, in main
args.func(args)
File "/home/miniconda3/envs/transformers/lib/python3.9/site-packages/doc_builder/commands/build.py", line 96, in build_command
build_doc(
File "/home/miniconda3/envs/transformers/lib/python3.9/site-packages/doc_builder/build_doc.py", line 405, in build_doc
sphinx_refs = check_toc_integrity(doc_folder, output_dir)
File "/home/miniconda3/envs/transformers/lib/python3.9/site-packages/doc_builder/build_doc.py", line 460, in check_toc_integrity
raise RuntimeError(
RuntimeError: The following files are not present in the table of contents:
- gpt_neo
- t5
- qdqbert
- videomae
- herbert
- graphormer
- reformer
- mvp
- visual_bert
- dinat
- esm
- ernie
- camembert
- openai-gpt
- bart
- trocr
- informer
- audio-spectrogram-transformer
- whisper
- unispeech-sat
- deberta
- deta
- xlsr_wav2vec2
- xglm
- speecht5
- swin
- gptsan-japanese
- sew
- mobilenet_v1
- layoutlmv3
- mobilevitv2
- align
- git
- perceiver
- xlm-v
- efficientnet
- wavlm
- encoder-decoder
- blenderbot-small
- sew-d
- xls_r
- vision-text-dual-encoder
- conditional_detr
- layoutlmv2
- flava
- trajectory_transformer
- pix2struct
- data2vec
- dinov2
- unispeech
- mctct
- bark
- xlm-prophetnet
- ernie_m
- clipseg
- tvlt
- decision_transformer
- bert-generation
- m2m_100
- ul2
- hubert
- cpm
- clip
- marian
- codegen
- table-transformer
- rembert
- van
- roformer
- opt
- mobilenet_v2
- altclip
- gpt_neox
- mms
- pegasus_x
- big_bird
- nllb-moe
- lxmert
- beit
- deberta-v2
- megatron-bert
- pvt
- dpr
- detr
- rag
- upernet
- bridgetower
- gptj
- levit
- glpn
- xmod
- sam
- llama2
- xlm
- vitdet
- maskformer
- owlvit
- instructblip
- transfo-xl
- pegasus
- longt5
- mbart
- layoutxlm
- barthez
- vision-encoder-decoder
- dit
- flan-ul2
- byt5
- roc_bert
- mega
- encodec
- bert
- phobert
- ibert
- gpt_bigcode
- deformable_detr
- vit_msn
- markuplm
- convnextv2
- yoso
- gpt-sw3
- wav2vec2_phoneme
- focalnet
- donut
- regnet
- jukebox
- mluke
- realm
- autoformer
- bertweet
- chinese_clip
- swiftformer
- umt5
- auto
- t5v1.1
- poolformer
- dialogpt
- vit_hybrid
- xclip
- deplot
- canine
- resnet
- bloom
- groupvit
- fsmt
- bert-japanese
- squeezebert
- mra
- convbert
- biogpt
- xlm-roberta
- nystromformer
- cpmant
- nezha
- bit
- segformer
- rwkv
- bort
- longformer
- time_series_transformer
- blip-2
- mpnet
- pop2piano
- flan-t5
- musicgen
- speech_to_text_2
- distilbert
- tapas
- lilt
- blenderbot
- gpt_neox_japanese
- xlm-roberta-xl
- xlnet
- nllb
- speech_to_text
- vilt
- llama
- roberta-prelayernorm
- yolos
- megatron_gpt2
- plbart
- cvt
- speech-encoder-decoder
- mt5
- vivit
- idefics
- imagegpt
- flaubert
- vit
- bartpho
- clap
- mobilebert
- mgp-str
- efficientformer
- tapex
- switch_transformers
- electra
- mask2former
Add them to docs/source/en/_toctree.yml.
```
Any ideas on why this is happening? Any help will be appreciated. Thanks
opened 10:44AM - 16 Nov 21 UTC
closed 03:36PM - 16 Nov 21 UTC
When I update transformers to the latest commit https://github.com/huggingface/t… ransformers/commit/1cc453d33c5d0be01eaf3050082c125ce87491aa & ran `python src/test_with_transformers.py`, I get the following issue:
```bash
Traceback (most recent call last):
File "src/test_with_transformers.py", line 5, in <module>
build_doc("transformers", "/Users/mishig/Desktop/transformers/docs/source/", "build")
File "/Users/mishig/Desktop/moon/moon-landing/repos/doc-builder/src/doc_builder/build_doc.py", line 254, in build_doc
anchors_mapping = build_mdx_files(package, doc_folder, output_dir, page_info)
File "/Users/mishig/Desktop/moon/moon-landing/repos/doc-builder/src/doc_builder/build_doc.py", line 104, in build_mdx_files
content, new_anchors = resolve_autodoc(content, package, return_anchors=True, page_info=page_info)
File "/Users/mishig/Desktop/moon/moon-landing/repos/doc-builder/src/doc_builder/build_doc.py", line 52, in resolve_autodoc
doc = autodoc(object_name, package, methods=methods, return_anchors=return_anchors, page_info=page_info)
File "/Users/mishig/Desktop/moon/moon-landing/repos/doc-builder/src/doc_builder/autodoc.py", line 199, in autodoc
method_doc = document_object(
File "/Users/mishig/Desktop/moon/moon-landing/repos/doc-builder/src/doc_builder/autodoc.py", line 128, in document_object
name = obj.__name__
AttributeError: 'NoneType' object has no attribute '__name__'
```
1 Like
I might clear out my env and try installing using WSL and see how that works out in a bit.
1 Like
Yeah. If it’s just a bug in the Windows dependencies, it can be avoided with WSL2…
If we don’t need to use a GPU, installing PyTorch itself is straightforward.
It seems like this could happen on Ubuntu or Mac as well, but it helps narrow down the problem.
opened 04:14AM - 05 Jun 25 UTC
bug
### System Info
### System info
OS: Windows 11
Python: 3.13.3 and 3.10
Git: 2.4… 9.0
CMake: 4.0.2
Msys64: Pacman v6.1.0 - libalpm v14.0.0
Pip: 25.1.1
Setuptools: 80.9.0
Visual studio C++ build tools
### NOTE: I followed the steps here [Contribute to 🤗 Transformers](https://huggingface.co/docs/transformers/en/contributing) and for sure system info already existed before following but let me walk through again for additional info.
1- Forked the repo.
2- Cloned it
3- cd transformers (so made sure I am in the right path which is the root for the repo)
3- switched to my own branch
4- made a python virtual environment using python 3.10 then activated it
5- made sure transformers ain't installed inside it
6- installed PyTorch
7- Ran this command `pip install -e ".[dev]"`
### NOTE: I tried making requirements.txt and using this command `pip install -r requirements.txt` but I got no output and I tried installing onnx with pip which happened successfully then Ran this command `pip install -e ".[dev]"` but nothing changed
```
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [11 lines of output]
...\setup.py:36: DeprecationWarning: Use shutil.which instead of find_executable
CMAKE = find_executable('cmake3') or find_executable('cmake')
...\setup.py:37: DeprecationWarning: Use shutil.which instead of find_executable
MAKE = find_executable('make')
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 35, in <module>
File "...\setup.py", line 318, in <module>
raise FileNotFoundError("Unable to find " + requirements_file)
FileNotFoundError: Unable to find requirements.txt
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
```
### Who can help?
_No response_
### Information
- [x] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)
### Reproduction
`pip install -e ".[dev]"`
### Expected behavior
Being able to install transformers for contributing with no issue
1 Like
I had no luck on WSL either
I posted in issue on the GitHub page. Hopefully I can get some additional context there or a workaround to preview my markdown.
Thank you very much
1 Like