Omega-S: A Functional Resilience Index for LLM Fine-Tuning
Omega-S is pitched as a cheap fine-tuning penalty that preserves prior model capability without old data or old weights.
The paper says Omega-S adds under 4% step cost and can be dropped into an existing training loop. On Llama-3-8B with LoRA, it retained more HumanEval performance than no regularization on 9 of 10 seeds, raising retention from 62.9% to 84.1%. The author also reports that the implemented objective mostly acts through degree variance, not the full topological story its name suggests. Code, per-seed results, and negative results are said to be available. ArXiv · AI/CL/LG's note
The paper says Omega-S adds under 4% step cost and can be dropped into an existing training loop. On Llama-3-8B with LoRA, it retained more HumanEval performance than no regularization on 9 of 10 seeds, raising retention from 62.9% to 84.1%. The author also reports that the implemented objective mostly acts through degree variance, not the full topological story its name suggests. Code, per-seed results, and negative results are said to be available. ArXiv · AI/CL/LG's note
score 5