@lewtun thanks for the tips, again!
Based on your pointers, I went ahead and created a kind of semantic search in hungarian! Finally, as a dataset I have used shortened abstracts from wikipedia and calculated the embeddings using a pretrained multilanguage sentence-transformer.
My space is up and running, and the returned results for input queries are more or less relevant! 
It ain’t much, but it’s honest work and it was an interesting project to research and execute!
Cheers,
Endre