I recently came across neurosama on youtube, I was wondering if he (the creator) uses any ready-made language model that he trained on his data and customized or if he made his own language model? (I am interested in your assumptions) I am a complete 0 in this question, I became interested in this topic (In general machine learning, not neurosama) and I want to study it, but nowhere can I find an answer to this question or at least an answer whether it is worth trying to create your own language model, I am very interested in this topic and I want to at least try, although I do not expect anything, but I would like to at least understand how it all works (Can you recommend any literature on this topic?).
As a fellow newbie in the space, I’m gonna guess that’d be a strong no. There’s many factors to why I don’t think this person made their own Large Language Model (LLM) for the purpose of their channel, but the biggest one has to be cost. Initial training of an LLM typically costs somewhere between hundreds of thousands and millions of dollars. Even though his scope of usage is limited to role-playing which is far less complex than current LLM’s out there, it’s unlikely. Also, his channel seems to have started around the same time that Facebook’s LLaMa 1 was released, so there’s that.
You’d definitely want to start by playing with currently available models. As a primer, go check out ChatGPT to learn how to interact with an LLM; just play around by asking questions and learning what type of responses you get. After that, just look up Meta’s LLaMa 2 model on YouTube, there’s plenty of tutorials on how to download and run it locally or on the cloud.
1 Like