Can one simply calculate loss (given labels) with Inference API?

Beginners
1

Hello!

I’m using T5 with Inference API and different tasks, which works great. However, I couldn’t figure out whether, and if so how, one can calculate the loss with a simple URL call.

In the API, we’d do:

input_ids = tokenizer("my input string", return_tensors="pt").input_ids
labels = tokenizer("a target string", return_tensors="pt").input_ids
loss = model(input_ids=input_ids, labels=labels).loss

What I would like to do, with Inference API, is something like:

import json

import requests

API_TOKEN = "xxxx"
headers = {"Authorization": f"Bearer {API_TOKEN}"}
API_URL = "https://api-inference.huggingface.co/models/t5-small"

def query(payload):
    data = json.dumps(payload)
    response = requests.request("POST", API_URL, headers=headers, data=data)
    return json.loads(response.content.decode("utf-8"))

data = query(
    {
        "inputs": "my input string",
        "options": {
            "labels": "a target string"
        }
    }
)

# Response
loss = data["loss"]

Could something like that be achieved?

Best,

Gregor