Inner product on faiss index not seemingly working

🤗Datasets
1

I’m trying ro create the normalized inner product. which would provide scores that in the domain 0, 1, however this appears to not be working.

faiss.normalize_L2(np.array(embedding_c["embeddings"]).astype(dtype=np.float32))
embedding_c.add_faiss_index(column="embeddings", metric_type=faiss.METRIC_INNER_PRODUCT)

leads to scores which are not L2 normalized.

scores, samples = embedding_c.get_nearest_examples(
    "embeddings", np_embedding, k=10
)

scores gives the following

array([41.96955 , 24.942688, 23.119305, 22.931778, 22.90448 , 22.66772 ,
22.657745, 22.409449, 22.363243, 22.353025], dtype=float32)
2

faiss.METRIC_INNER_PRODUCT doesn’t normalize the result.

You should normalize your vectors if you want the result to be between -1 and 1