Hi,
I am using the faiss feature from datasets which is very helpful and easy, except for there is a wired progress bar that I don’t know how to turn off.
The code I am using
datastore = datastore.add_faiss_index(column=self.knn_on, faiss_verbose=False)
query = np.asarray(sample[self.knn_on])
_scores, fewshots_examples = datastore.get_nearest_examples(
index_name=self.knn_on,
query=query,
k=k,
)
And the terminal will keep logging progress bar as the snapshot below:
I am wondering is there any way I can mute it?
1 Like
I think it’s not from the map method but from either the add index or the find nearest method. Because it only show up when I include that logic and map has its own progress bar. Plus I want to keep the map’s tqdm while mute the 1/1 one. Is that possible
1 Like
Plus I want to keep the map’s tqdm while mute the 1/1 one. Is that possible
I think it’s possible, but without modifying the contents of the library, the only option is to manually silence tqdm temporarily like this… Or add_faiss_index_from_external_arrays for better tqdm?https://stackoverflow.com/questions/37091673/silence-tqdms-output-while-running-tests-or-running-the-code-via-cron
Hi, I am trying to know how to use Rag/DPR, but first I want to get familiar with faiss usage.
I checked the official example in
But it seems the snippet code is not self-executable.
So I did some modification, aiming to retrieve similar examples in the sst2 dataset with query ‘I am happy’.
import datasets
from transformers import pipeline
embed = pipeline('sentiment-analysis', model="nlptown/bert-base-multilingual-uncased-sentiment")
ds = datasets.load_dataset('glue', 'sst2', split='test')…
https://stackoverflow.com/questions/62048408/how-to-remove-progressbar-in-tqdm-once-the-iteration-is-complete
