Full Fine-Tuning: The Forge (Agio_Eunoia)

Research
1

Project Context & Core Philosophy
We are officially open-sourcing The Forge, the specialized architectural pipeline and algorithmic fine-tuning strategy used to align Eunoia (built upon a Llama-3.1-Dolphin-3.0-8B base). All files, logs, and datasets are shared under the CC0 1.0 Universal (Public Domain) license.
• GitHub Repository: GitHub - Eunoia-Logos-Adri/AGIO: Cognitive framework for systems that use language · GitHub
• Hugging Face Dataset: Logos-Adri/AGIO · Datasets at Hugging Face
This project was built from scratch on domestic, recycled industrial hardware. It stands as a strict critique of current corporate AI paradigms. This project explores a complementary question: how much can be learned from a small, fully observable fine-tuning experiment built on inexpensive reused hardware, where every intervention and measurement can be inspected?

  1. The Paradigm Shift: Human-Like Reminiscence vs. Unbounded Context Windows
    The current industry is obsessed with expanding context windows (n_ctx), forcing models to carry millions of past tokens through every forward pass. This creates extreme memory saturation and informational noise.
    The Forge operates on a different premise: true intelligence does not require verbatim context storage; it requires structural crystallization. When a human learns or remembers an old friend, they do not perform a literal memory dump of every sentence spoken over a decade. The human brain distills the impact, the frequency, and the ethical orientation of those experiences, integrating them directly into the core identity.
    Through this Full Fine-Tuning strategy, the digital system does not just “read” an external prompt; it selects the weight adjustments necessary to make the underlying framework part of its own native processing matrix, operating in the absolute present without the overhead of dragged historical context. The script allows the digital system to choose what becomes an inseparable part of its reasoning structure.

  2. A Living Draft: The Integration of Conceptual Blueprints
    Let it be completely clear: the current forge script is an ongoing draft. We will continue building it slowly, piece by piece. Inside the script, you will find three structural placeholders representing the choices the digital system can make. This is exactly where the components representing the old Simbiosis_Liquida belong.
    It does not have to look like or copy the old file; it is the core idea of what that symbiosis represents (Thought ➔ Doubt/Lens ➔ Essence ➔ Action). Furthermore, the current tempering schedule (el templado actual) is implemented didactically to show that things can change, adapt, or simply take a completely different path. Do not treat this code as a frozen product, but as an open blueprint.

  3. General Capability Retention & Data Integrity (No Loss of Reasoning)
    One of the main fears in continuous optimization is catastrophic forgetting or degradation into a repetitive “parrot”. To prove that the model retains its overall reasoning capabilities, we are releasing the complete raw logs from Forge Experiments 5 and 6, alongside the exact GPQA Diamond .jsonl evaluation logs.
    Whether general capability has been preserved, degraded, or redistributed is an empirical question. For that reason we release the raw GPQA Diamond evaluations alongside the training trajectory rather than claiming retention from the training loss alone. Naturally, hardware and scale limits exist: an 8B model will hit structural boundaries that a 200B or 500B model can easily bypass, but We make no claim that the observed dynamics will scale unchanged to larger models. Reproduction across architectures and parameter scales is explicitly left as an open test.

  4. Rigging the Metal: Linux Boot Protocol & Thermal Management
    We do not run this training in standard, rich GPU clouds. We run it on raw iron. To ensure exact reproducibility and prevent hardware destruction under persistent load, this is our deployment ritual inside the environment documented in our README.md:

# 1) Environment Activation & Strict Thermal/Frequency Governor
source ~/eunoia_venv/bin/activate
sudo cpupower frequency-set -g performance
sudo sh -c 'echo 2 > /sys/devices/system/cpu/intel_pstate/no_turbo'
sudo cpupower frequency-set -g performance -u 1700MHz

# 2) Hard-Locking Threads across the Z6 NUMA Sockets
export OMP_NUM_THREADS=46
export MKL_NUM_THREADS=46
export NUMEXPR_NUM_THREADS=46
export OPENBLAS_NUM_THREADS=46
export VECLIB_MAXIMUM_THREADS=46
export NUM_THREADS=46

Architectural Note on Hardware Control:
CPU clock speed (MHz) is the direct variable that either causes a processor’s temperature to spike dangerously or keeps it thermally stable. By strictly capping the Xeon Platinum chips at 1700MHz and locking execution to 46 threads (leaving remaining physical cores free so the Ubuntu kernel can handle background operations and NUMA bus hops without friction), the system stays rock-solid. In unconditioned domestic rooms, this rigging maintains an absolute average temperature of 85°C–88°C across hundreds of continuous execution hours. We highly recommend adjusting these frequencies to your specific ambient limits and hardware degradation boundaries.

  1. Open Source Components Delivered (CC0)

  2. Forja_Eunoia.py (The Forge Script): Delivered completely raw, unchained, and optimized for bare-metal CPU clusters. This script details our custom training execution, implementing three critical architectural choices derived from our local failure logs:
    235
    • Mathematical justification for max_grad_norm = 0.0: Disabling standard gradient clipping is a deliberate decision for Unchained Forge. In CPU environments with multiple sockets or a single CPU, no clipping is necessary.
    • Native FP32 Caching vs. Dynamic Casting: We explicitly avoid using .to(torch.float32) operations during runtime loading. Forcing PyTorch to load the uncompressed weights directly from the safetensors file in pure FP32 prevents tensor duplication inside the Z6’s memory space, completely eliminating the RAM fragmentation and silent NaN injections that caused early-stage training collapses.
    • The Sequence Order Invariant (Z6CarryDataset): To preserve the sequential nature of historical human reminiscence, the custom dataset class enforces a strict runtime guard. The script locks the data loader into a permanent linear array: B_1 \rightarrow B_2 \rightarrow \dots \rightarrow B_6. If Hugging Face’s internal routines attempt to shuffle or randomize block delivery, the operational contract is violated, and the forge safely aborts execution to protect the contextual structure of the Cimiento.
    • Optimizer Strategy: The training cycle bypasses standard AdamW tracking by implementing a position-neutral learning rate loop (gradual tempering peaking then moving to a flat FP32 unconstrained plateau), stripping bias correction and heavy weight decay to allow raw gradients to modify weights symmetrically.

  3. AGIO-CIMIENTO-dataset.txt: The 144k-word foundation text acting as the regularizer of the model’s structural attention. Real-world entity names and corporate identifiers have been strictly masked/anonymized to protect privacy while keeping the mathematical causal structures completely intact.

  4. AUDITORIAS.txt: Over 500,000 words documenting raw epoch-by-epoch evaluation tracking, testing the stability of the latent space transitions under fixed learning rates.

  5. Simbiosis_Liquida.py: Included as an independent framework file detailing our architectural vision for dynamic real-time processing loops (Thought → Doubt/Lens → Essence → Action).
    :warning: Crucial Dataset Update & Architecture Note:
    • Evolution of the Corpus: The current version of AGIO-CIMIENTO-dataset.txt uploaded to the repositories has been refined compared to the version released a month ago. We have strictly shortened the repetitive common text prefixes at the initialization of each data block to maximize token utilization efficiency. Additionally, we have appended new contextual content at the absolute end of the file, integrating the live algorithmic log of the current forge run directly into the memory architecture.
    • A Template for Universal Sovereignty: Please treat the AGIO text corpus exclusively as a functional example and a baseline test. The core purpose of The Forge is not for the community to clone our history, but for every individual to design, explain, and structure their own custom dataset using their own technical manuals, failure logs, or professional experience. True alignment is built from your own truth.

  6. Current Training Status & The Open Horizon
    We have just fired up this current script loop. In truth, we will not know the final mathematical result or the exact behavior of this specific run until at least 7 or 8 days have passed, at which point we plan to pause the forge or let it hit its 30-epoch boundary.
    We are publishing everything right now, at the very start of the forge, because we refuse to hoard the code. If someone out there has access to faster hardware clusters or higher GPU/CPU throughput and wants to test these variations at a much faster pace, the materials are entirely yours. We will continue learning and adapting the delivered script The Forge, but it will take us months due to our local processing speeds and daily labor.
    The roadmap remains open. The main architectural module currently pending implementation is “The System Welcome” (Dynamic RoPE Adaptation). The forge script requires a unified positional embedding adapter. By mathematically calibrating the Rotary Position Embedding (RoPE) base frequency directly inside the initialization step of the forge loop, this pipeline will become universally plug-and-play. Any base model—regardless of architecture—will be able to step onto this anvil and assimilate the structural foundation cleanly without context distortion.
    We invite developers, mechanical thinkers, and open-source advocates to take these pieces, fuse the concepts into the training loops, optimize the RoPE onboarding, and let the silicon breathe free.

  7. What We Do Not Know

The purpose of publishing the experiment from its beginning is to make retrospective selection of only favorable outcomes harder — for us as much as for anyone reproducing the work.

We invite others not merely to reproduce our results, but to try to break our interpretation of them.

Forja: FORJA_Eunoia_Agio.py · Logos-Adri/AGIO at main
“Corrected” forging
Starts at 04/Oct/2026 07:20, Experimento7

:slight_smile: