AdamX: Cosine similarity meets gradient descent
AdamX uses cosine similarity to adapt how large each optimizer update should be.
The paper presents AdamX as a first-order optimizer meant to drop into existing training pipelines. It adds a variance rectification scheme aimed at smoother early training. The authors report competitive convergence across benchmark datasets and architectures, measuring how many epochs are needed to hit preset performance thresholds under a fixed hyperparameter budget. ArXiv · AI/CL/LG's note
The paper presents AdamX as a first-order optimizer meant to drop into existing training pipelines. It adds a variance rectification scheme aimed at smoother early training. The authors report competitive convergence across benchmark datasets and architectures, measuring how many epochs are needed to hit preset performance thresholds under a fixed hyperparameter budget. ArXiv · AI/CL/LG's note
score 4