Train a model with autotrain on huggingface using the API

[chsafouane]

Hello,

Up to now, I’ve been using the autotrain ui (space) on huggingface to train models.

I would like to know if there is a way to start a model training using autotrain powered bu HF GPUs but only using the API.
I’ve spotted a page in the documentation that talks about this but one has to start the app first: autotrain app --port 8000 --host 127.0.0.1

Is there a way to bypass this and directly train the model on HF ?

Thanks

[abhishek]

there are multiple ways.

1. you can use config files:

example config:

task: llm-sft
base_model: openai-community/gpt2
project_name: autotrain-gpt2-finetuned-guanaco
log: tensorboard
backend: local

data:
  path: timdettmers/openassistant-guanaco
  train_split: train
  valid_split: null
  chat_template: null
  column_mapping:
    text_column: text

params:
  block_size: 1024
  model_max_length: 2048
  max_prompt_length: 512
  epochs: 3
  batch_size: 2
  lr: 3e-5
  padding: right
  optimizer: adamw_torch
  scheduler: linear
  gradient_accumulation: 4
  mixed_precision: fp16

hub:
  username: ${HF_USERNAME}
  token: ${HF_TOKEN}
  push_to_hub: true

change backend from local to one of the spaces backends:

    "spaces-a10g-large": "a10g-large",
    "spaces-a10g-small": "a10g-small",
    "spaces-a100-large": "a100-large",
    "spaces-t4-medium": "t4-medium",
    "spaces-t4-small": "t4-small",
    "spaces-cpu-upgrade": "cpu-upgrade",
    "spaces-cpu-basic": "cpu-basic",
    "spaces-l4x1": "l4x1",
    "spaces-l4x4": "l4x4",
    "spaces-a10g-largex2": "a10g-largex2",
    "spaces-a10g-largex4": "a10g-largex4",

e.g. backend: spaces-a10g-largex2

This will start training directly on spaces.

2. API way:

start the app: autotrain app --port 8000 --host 127.0.0.1
you can see api docs at 127.0.0.1/docs. api endpoint to create project is available at /api/create_project

Note: local datasets are not supported in api mode.

please let me know if you have any questions.

[chsafouane]

@abhishek

"spaces-a10g-large"

It works like a charm.Thank you

I guess the next think I’d like to know is if there is a way to do it using python directly by passing a dict that includes all parameters (as I see that it parses a json containing the configuration under the hood)

[abhishek]

you can also use the autotrain cli, im pushing some improvements there so ill update here when its done