Advanced Modeling Techniques Proposed That Innovatively Improve LLM Training Stability and Multimodal Reasoning and Image Generation Performance
Various Open-Source Frameworks Released Maximizing Agent Memory and Collaboration Efficiency and Enhancing Environmental Adaptability

ROOT: Robust Orthogonalized Optimizer for Neural Network Training

https://arxiv.org/abs/2511.20626

To resolve algorithmic precision issues and training instability occurring during large-scale language model training, a new optimization tool called ROOT with a dual robustness mechanism is proposed. The technique consistently maintains orthogonalization precision through adaptive Newton iteration tailored to matrix size, and suppresses outlier noise effectively by introducing a proximal optimization approach. Experimental results demonstrate that ROOT achieves faster convergence speed and superior final performance even in noisy or non-convex environments compared to existing Muon or Adam-based optimizers, significantly improving the stability of large-scale model training.