How to upload documents to the SupabaseVectorStore?

Hi everyone,

I am learning RAG for GAIA, from here: test.ipynb · baixianger/RobotPai at main

However, I was not able to upload documents to Supabase, as shown in screenshots:

I have tried two ways:

# wrap the metadata.jsonl's questions and answers into a list of document
listDict_QA_Doc = []
for dict_RandomQA in listDict_Metadata:
    strQA_Content = f"Question : {dict_RandomQA['Question']}\n\nFinal answer : {dict_RandomQA['Final answer']}"
    dict_QA_Doc = {
        "id": dict_RandomQA['task_id'],
        "content" : strQA_Content,
        "metadata" : {
            "source" : dict_RandomQA['task_id']
        },
        "embedding" : embeddings.embed_query(strQA_Content),
    }
    listDict_QA_Doc.append(dict_QA_Doc)


response = syncClient.table("documents").insert(listDict_QA_Doc).execute()

and

# wrap the metadata.jsonl's questions and answers into a list of document
listDoc_QA_Metadata = []
for dict_Metadata in listDict_Metadata:
    strQA_Content = f"Question : {dict_Metadata['Question']}\n\nFinal answer : {dict_Metadata['Final answer']}"
    doc_QA_Metadata = Document(
        id = dict_Metadata['task_id'],
        page_content = strQA_Content,
        metadata = {"source": dict_Metadata['task_id']},
        embedding = embeddings.embed_query(strQA_Content)
    )
    listDoc_QA_Metadata.append(doc_QA_Metadata)


vector_store = SupabaseVectorStore.from_documents(
    listDoc_QA_Metadata,
    embeddings,
    client=syncClient,
    table_name="documents",
    query_name="match_documents",
)

However, always get the same error:

Error inserting data into Supabase: {'message': 'JSON could not be generated', 'code': 404, 'hint': 'Refer to full message for details', 'details': "b'{}'"}

Could anyone please help? :sob:

1 Like

How about changing the version of pydantic?

pip install pydantic==2.10.6

Just tested, still the same error :sob:

1 Like

Hmm… In that case, could it be that the data you passed is not in the expected JSON structure, as indicated by the error message?

You can verify this by passing extremely simple sample data that is expected to be passed, rather than the actual data.

Solved. :sweat_smile: Need to create a table on supabase before uploading.

1 Like

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.