RTH-LM: 25B Fractal TCN language model with no self-attention, looking for LoI collaborators for SPRIND

Research
1

Hi HF community,

I built RTH-LM a 25B parameter language model that replaces
self-attention entirely with a Fractal Gated Causal TCN architecture.

The core idea is to separate the system into a frozen reusable core
(Genome, ~7GB) and modular trainable behaviors (Souls, ~300MB).

Same Genome, different Souls: language and code generation are already
working without retraining the base model.

Links:

I’m applying to SPRIND Next Frontier AI, a European funding challenge
for AI systems beyond the current Transformer paradigm. For the application, I’m looking for 2–3 European ML engineers or researchers willing to sign a Letter of Intent.

This is not a work commitment at this stage. It is simply a one-page
statement saying that, if the project is funded, you would be interested
in joining the team. If funded, collaborators would have a real role in the project.

Happy to answer technical questions here before anyone decides. Particularly interested in hearing from anyone working on non-transformer architectures, efficient inference, or alternative scaling approaches.

1 Like