As far as i see there is one remaining problem for wider adoption of LLMs. That being the cryptic and slow to be updated settings (for new LLM models) of Ooba, Silly Tavern etc…
Especially with Mixtral we saw that it had a special Instruct format which they told would give best results. I suspect many users got their bad first experience (with Mixtral especially) because of false settings.
On the other hand new things like Dynamic Temp pop up (with settings again) which are included in ST and koboldcpp, but nobody knows what a good default should look like.
Then there are positively, finetunes like noromaid mixtral, where Undi and Ikari provided Silly Tavern Context, Instruct and Text Completion jsons presets to import on their Huggingface model card.
My suggestion is, that huggingface (or chub) implements a new category is specifically catered to settings of opensource Frontends and integrated with model pages.
Specificaly bound (linked) to huggingface models (one main settings page for 1 specific LLM model with different setting packs).
-
A 5 star ranking vote system.
-
Forum or comments on the specific setting packs.
-
For ST could be Instruct, Text Completion, Context.
-
And even “System Prompts” (most overlooked but opening layers in the LLMs which are only discovered by the use of System Prompts
Centralization of recommended Settings,
-
would increase ease of use for the adaptors of LLM and Roleplayers.
-
It would serve as a discussion point on the different possibilities of different settings for the community to share their experience and ideas, i.e. creativity.
-
having a base for unified settings would help debugging and benchmarking as well. (Benchmark made with settings (A,B,C # id or something like this. (links as well.)
Debugging when someone has problems you could see what settings he used.- a main Complete Setting (versionized template) (for export, import) so that Front Ends like SillyTavern and Ooba could import EVERY Setting with one click for a LLM Model instead of now having to import 3+ different presets at different ST tabs.
ST and other frontend would adopt later on download options of settings probably in their own front ends if a “Complete Settings Template Standard” is created. (ssuf (^_^) Simple Settings u… f… lol)(Standard like the v2 ST Character card standard with github page.)
(Like the gguf single LLM Model file which put all LLM data in a single file making it easier to use and grasp for newcomers)
There are experiments going on with Silly Tavern “Quick replies” for example. What is that you ask? It’s like a for example funny comment (trivia)(rpg stats) at the end of a user input, LLM reply. But the comment is generated by the LLM BUT! isn’t put in context and not eating up tokens. So if we had such a central Settings hub you would have heard probably of such endeavors and interested parties could participate in the experimenting. But without a coordinated settings hub/platform, no coordinated progress on these matters can happen, further weakening the open source community
Especially when so many settings are like “esoteric” Magic which many don’t even know what it is used for. Typical p, k etc… and which are not even documented somewhere afaik.
Especially Mixtral showed us, because it is so instruct specific, the need to have some generalized solution to the setting chaos. We have some rentry sites, personal recommendations etc. total chaos for settings.
In such a chaos not much coordinated progress can be made
Until now people are experimenting with different Mixtral settings to perhaps find a solution to avoid repetition.
But even when someone found it, (i heard from some people they found a good settings “recipe”), they share it on some specific site, where the other LLM users probably won’t find it at all.