Hi @ydshieh , First of all I would like to say thank you for your work on Kosmos-2. I want to using Kosmos to for document AI. I am using trying to convert doclaynet dataset to embeddings. I read your code and research paper and found that first I need to convert bounding box to patch_XXXX thing. I …

Yes, you can specify num_patch_index_tokens when creating Kosmos2Processor. (I think it works when we use from_pretrained and passing num_patch_index_tokens) And while doing process(es), we need to pass num_patches_per_side. (see clean_text_and_extract_entities_with_bboxes) (post_process_generati…

Issue with KOSMOS-2 encoding and decoding

🤗Tokenizers

ydshieh January 24, 2024, 6:09pm 10

Let me know how it goes!

Topic		Replies	Views
Kosmos-2 Fine tuning 🤗Transformers	41	1967	August 19, 2024
Issue on Kosmos-2 model training on new dataset 🤗Transformers	3	443	February 25, 2024
LayoutLMv3 inference - bboxes are incorrect 🤗Transformers	0	121	May 10, 2024
Kosmos-2 batch modality and processing speed 🤗Transformers	0	22	July 18, 2024
LayoutLMv3 processor error Intermediate	4	122	September 27, 2024

Issue with KOSMOS-2 encoding and decoding

Related topics