Omega-S: A Functional Resilience Index for LLM Fine-Tuning
Omega-S is pitched as a no-data fine-tuning penalty that preserved more prior code ability in the paper’s Llama-3-8B LoRA tests.
The paper says Omega-S can be added to a training loop without old task data, a Fisher matrix, or a stored copy of old weights. In code-to-prose fine-tuning, it retained more HumanEval pass@1 capability than no regularization on 9 of 10 seeds, and beat tuned weight decay on all 10. The author also reports that the method’s actual gradient mostly reduces to a degree-variance penalty, despite its topological framing. The paper flags large repeat-run variance and publishes code, per-seed results, and negative results. HF Daily Papers' note
The paper says Omega-S can be added to a training loop without old task data, a Fisher matrix, or a stored copy of old weights. In code-to-prose fine-tuning, it retained more HumanEval pass@1 capability than no regularization on 9 of 10 seeds, and beat tuned weight decay on all 10. The author also reports that the method’s actual gradient mostly reduces to a degree-variance penalty, despite its topological framing. The paper flags large repeat-run variance and publishes code, per-seed results, and negative results. HF Daily Papers' note
score 5