Text-embedding-interface on Vertex

Beginners
1

Hi, im having trouble using text-embedding-interface on vertex, I pushed Package text-embeddings-inference · GitHub to my artifact repository and then I deployed it using the python API.

When I try to hit my endpoint I get errors:
inputs = {“instances”: [{“inputs” : “hi”}]}
response = requests.post(url, json=inputs, headers=headers)

Message:
Failed to deserialize the JSON body into the target type: missing field inputs at line 1 column 31

what would be the correct flow to hit the API? I cant seem to see the Vertex endpoint either when I use it locally.

2

Correct Flow to Hit the API:

Prepare the Request Data:
    Create a Python dictionary called inputs with the following structure:
    Python

    inputs = {
        "instances": [
            {"inputs": "your_text_here"}  # Replace "your_text_here" with your actual text
        ]
    }
Ensure that the indentation within the instances list and the dictionary for each instance is correct (four spaces are commonly used).

Set Up Authentication:

To authenticate with your Vertex AI endpoint, you'll need a service account with the necessary permissions. You can create a service account and grant it the required roles (e.g., "Vertex AI Endpoints Invoker") in the Google Cloud Console.
Obtain the service account key and store it securely (e.g., as an environment variable).

Send the Request:

Use a library like requests or the Vertex AI SDK for Python to make a POST request to your Vertex AI endpoint URL. Include the inputs dictionary in the JSON body of the request and set appropriate headers, such as Content-Type: application/json. Here's an example using requests:
Python

import requests

# Replace with your actual endpoint URL
url = "https://REGION-ENDPOINT.endpoints.vertex.ai/projects/PROJECT_ID/locations/REGION/models/MODEL_ID"

# Replace with your service account credentials
headers = {"Authorization": f"Bearer {get_service_account_token()}"}  # Function to get token

response = requests.post(url, json=inputs, headers=headers)
3

This is super helpful! I actually was able to figure out by looking at the source code for TEI and finding the /vertex endpoint, my issue was that I was using the wrong api path that TEI exposes. I didn’t see that there was a /vertex endpoint, which fixed my issue. This should probably be added to the documentation, or I may have missed it.