Independent Researcher here and first time transitijoning to arXiv. I have been doing some work in fine tuning within specialized domains that are either contextualized constrained or apply semantics and utilization of context in a way very counter to general knowledge. In attempting to identify some of the mechanisms of catastrophic forgetting or poor learning (medical, legal, finance), I have discovered a critical behavior of LoRA that impacts overall usage of the tuned model. I have also been working on a novel solution to adapt LoRA and other low rank tuning strategies to provide solutions/alternatives/explanations.
I am seeking endorsement in CS Machine Learning on arXiv for the first completed analysis paper, and the almost finalized novel algorithm solution paper.
Any help would be great appreciated. I can provide my code, you can contact me directly with any questions, or otherwise reach out.
Scale-Dependent Catastrophic Forgetting in LoRA Fine-Tuning: A Critical Threshold Analysis in Specialized Domains
Author: Matthew Martz, PhD Affiliation: Independent Researcher Date: November 15, 2025
Activation-Driven Adaptive Pathway Tuning with Quantization (ADAPT-Q): A Novel Neuron-based Framework with Attention for Preventing Catastrophic Forgetting in Specialized Domains
Author: Matthew Martz, PhD Affiliation: Independent Researcher Date: In Preparation
Thank you all so much in advance for any possible help here and would love to be able to return the favor to others in my situation in the future. Really appreciate this community over the many years for the support and knowledge that has really pushed our field forward. It’s truly wonderful.
Thank you,
Matthew Martz
Matthew Martz, PhD - https://mutaku.io - https://loom.clinic