I want to do an information retrieval project using Huggingface models.
Is there an API which I can use to fetch all available models? Is there something like this for datasets?
If there is no single way to iterate over all models/datasets, is there at least a way to retrieve model cards/dataset information for a given model/dataset using its name?
Welcome!
For searching over models and datasets (and Spaces), you could use either the available Hugging Face Hub Endpoints or the Hub client library (see list_models and list_datasets).
Then you can specifically download the README.md
file for the dataset or model, which would be the dataset/model card: (Downloading files)
I would suggest using the Hugging Face Hub client library so you can do something like:
import huggingface_hub as hf
models = hf.list_models()
datasets = hf.list_datasets()
# Downloads the file and returns the local path
card = hf.hf_hub_download(models[0].modelId, 'README.md')
3 Likes