I’m trying ro create the normalized inner product. which would provide scores that in the domain 0, 1, however this appears to not be working.
faiss.normalize_L2(np.array(embedding_c["embeddings"]).astype(dtype=np.float32))
embedding_c.add_faiss_index(column="embeddings", metric_type=faiss.METRIC_INNER_PRODUCT)
leads to scores which are not L2 normalized.
scores, samples = embedding_c.get_nearest_examples(
"embeddings", np_embedding, k=10
)
scores gives the following
array([41.96955 , 24.942688, 23.119305, 22.931778, 22.90448 , 22.66772 ,
22.657745, 22.409449, 22.363243, 22.353025], dtype=float32)