I trained a model using some content I had lying around (24mb of text). To test I wanted to generate using Command Prompt (cmd) on my Windows machine. I ran the following:
python run_generation.py --model_type=bloom --model_name_or_path=telavir/telavir/SmallCLModelTest --length=360 --temperature=0.7 --p=0.7 --k=100 --repetition_penalty=2.0 --prompt=“This is a blog post on Why Is Visiting A Dentist Every 6 Months So Important”
It returned:
=== GENERATED SEQUENCE 1 ===
This is a blog post on Why Is Visiting A Dentist Every 6 Months So Importantantantantantantantantantantantantantantantantantantantant <-This just kept going after this.
I further ran:
python run_generation_contrastive_search.py --model_name_or_path=“telavir/SmallCLModelTest” --length=1024 --penalty_alpha=0.6 --temperature=0.7 --p=0.7 --repetition_penalty=2.0 --repetition_penalty=2.5 --prompt=“This is a blog post on Why Is Visiting A Dentist Every 6 Months So Important”
And it returned:
=== GENERATED SEQUENCE 1 ===
This is a blog post on Why Is Visiting A Dentist Every 6 Months So Importantantantantantantantantantantantantantantantantantantantantantantantantant <-and it just kept going as well.
I added k=100 and they returned:
=== GENERATED SEQUENCE 1 === ← run_generation.py
This is a blog post on Why Is Visiting A Dentist Every 6 Months So Importantantantant [and so on]
=== GENERATED SEQUENCE 1 === <-python run_generation_contrastive_search.py
This is a blog post on Why Is Visiting A Dentist Every 6 Months So ImportantWG65B8zF� [it stops here]
What am I doing wrong? The model is set to public if anyone wants to test the scripts. I picked them both up here: https://github.com/huggingface/transformers/tree/main/examples/pytorch/text-generation
Update:
I ran it again in jupyter lab using the same prompt. Here’s code I ran:
from transformers import set_seed, AutoModelForCausalLM, AutoModel, BloomForCausalLM, AutoTokenizer
import torch
checkpoint = “telavir/SmallCLModelTest”
if torch.cuda.is_available(): device = torch.device(‘cuda’)
else: device = torch.device(‘cpu’)
model = BloomForCausalLM.from_pretrained(checkpoint).to(device)
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
set_seed(586423)
post_title = “Why Is Visiting A Dentist Every 6 Months So Important”
prompt = f’This is a blog post on {post_title}.’
input_ids = tokenizer(prompt, return_tensors=“pt”).to(device)
*sample = model.generate(*input_ids,max_length=850,min_length=750,num_beams=6,num_beam_groups=3,do_sample=False,top_p=0.7,top_k=100,temperature=0.7,no_repeat_ngram_size=2,repetition_penalty=2.0,diversity_penalty=0.9)
print(tokenizer.decode(sample[0], truncate_before_pattern=[r"\n\n^#“,”^‘’'“,”\n\n\n"]))
After running I got this gobbledygook:
This is a blog post on Why Is Visiting A Dentist Every 6 Months So Important… Excellence Excellence第二轮第二轮 যের যের causar causar flexible flexibleاقياقي forum forum విభజ విభజicediced বিস্ত বিস্ত হৈছিল হৈছিলpora:pora:869869 ذیل ذیل রূপে রূপে_^_^ㄜㄜନୀନୀ இதில் இதில் নিয়ে নিয়েconscons ಆಸ ಆಸ PL PL الله- الله- Natureza Natureza结成结成 3* 3*गरगरटेलटेल miji miji ସିଲ ସିଲ தண தணDiketahuiDiketahui administratiu administratiu事实上事实上DocumentDocument總會總會 MediaWiki MediaWikientoento 707 707 Lợi Lợirfrfepisepisುತ್ವುತ್ವ bebidas bebidasிக்கப்பட்டிக்கப்பட்ட abenduan abenduan counted countedfaafaa المعلم المعلمanderander সত্যি সত্যি مطالب مطالبEEKEEK عَل عَل quelles quelles password: password:LiLi रिटर्न रिटर्नारुारु बैंक बैंक “o “o_VERSION_VERSIONनोटनोटWhat’sWhat’s heroína heroína一级一级GenerGener المهام المهام دیکھنے دیکھنے ഇള ഇള ਨਿਰਦੇਸ਼ਕ ਨਿਰਦੇਸ਼ਕ abdic abdicatakanatakanంతింతి成都成都 Cn Cnlt;lt; ଦୀ ଦୀ দারিদ্র দারিদ্রominomin Artificial Artificial dum dum公约的公约的 harmon harmon MAS MAS深造深造 luzera luzera عہ عہ ബോ ബോ Persib Persibسلمينسلمين Tunggal Tunggalडमिंटनडमिंटनhistoryhistory ഇങ്ങനെ ഇങ്ങനെಿತವಾಗಿದೆಿತವಾಗಿದೆ be be工作人员薪金工作人员薪金 Symbol Symbol chaînes chaînes濕濕 আসন্ন আসন্ন ମହାବ ମହାବ सक्रिय सक्रिय·e·eਯੋਗਯੋਗகாகா gaind gaind élevées élevées রাখেন রাখেনರಾವ್ರಾವ್ datiboa datiboa Thy Thy口的口的েক্সেক্স的生产的生产是一座是一座 réels réelsپردپرد panas panas2F2F2F}{\4cH000000}2F2F2F}{\4cH000000}怎能怎能 explode explode од од style="display: style="display:0x00x0শুরশুর পিস পিস ഉണ്ടാക്കിയ ഉണ്ടാക്കിയ/issues/issuesൃദ്ധൃദ്ധ bear bear الدماء الدماء Administrasi Administrasi ವಶಕ್ಕೆ ವಶಕ್ಕೆ scratch scratch avanza avanzabegbeg്രീകര്രീകരஇருஇருenguaenguaCiCi الْإِمَامُ الْإِمَامُ宗教信仰宗教信仰 195 195 GROUP GROUP dirty dirty idazten idazten引起的引起的 ആവശ്യപ്പെട്ട ആവശ്യപ്പെട്ട azpif azpifَاقَِاقِ falado falado ejemplares ejemplares રોય રોય mengab mengab aliwa aliwa হার্ড হার্ডavanaavana minera mineraவன்வன்在进行在进行കമകമmp3mp3 albums albums域名域名 اشعار اشعارેમ્બરેમ્બર Autriche Autrichee7e7_cmd_cmd variaciones variacionesipiipi भै भै germ germUneUne ল’ ল’这些措施这些措施 applaud applaud विद्युत् विद्युत् deadlock deadlock गरिएको गरिएको والے والےultsults بالحق بالحق विमानस्थल विमानस्थल vitória vitória অনুর অনুর gon gonptpt{{{enllaç}}}{{{enllaç}}}暉暉گئیگئی/10/2015/10/2015