Imagine an AI that can not only read and understand text but also interpret visual information like diagrams and graphs to solve complex math problems. That's the ambitious goal of MAVIS, a new project focused on enhancing the mathematical visual reasoning capabilities of Multi-modal Large Language Models (MLLMs). Current MLLMs struggle with visual math problems. They often misinterpret diagrams, fail to align the visual information with the textual question, and can't consistently apply chain-of-thought reasoning. MAVIS aims to tackle these issues head-on. Researchers created an automatic data engine that generates vast quantities of training data. This includes MAVIS-Caption, containing over 550,000 diagram-caption pairs, and MAVIS-Instruct, a collection of 834,000 visual math problems with detailed solutions. This engine can even create problems on its own, without needing human intervention or relying on GPT. The team developed a four-stage training process for MAVIS. First, a specialized vision encoder called CLIP-Math is trained to "see" math diagrams more effectively. Second, the model learns to align CLIP-Math's visual representations with an LLM for better diagram-language understanding. Third, the model is trained on MAVIS-Instruct to learn problem-solving techniques. Finally, it uses Direct Preference Optimization (DPO) to refine its chain-of-thought reasoning skills, resulting in more accurate step-by-step solutions. The results are impressive. The resulting model, MAVIS-7B, surpasses other open-source MLLMs of similar size by a significant margin on benchmark tests like MathVerse. It even outperforms some much larger models, including the second-best LLaVA-NeXT (110B). While these results are exciting, the project is not without its challenges. The generated diagrams, while accurate, sometimes lack the aesthetic polish of human-drawn diagrams. The language generated by the data engine can also be a bit rigid. Future improvements will focus on refining the appearance of diagrams, improving the fluency of the generated text, and broadening the types of mathematical concepts MAVIS can handle. MAVIS represents a significant step forward in AI’s journey towards more generalized intelligence. It demonstrates the potential of combining computer vision with LLMs to tackle challenging multi-modal tasks. As MAVIS and other similar projects evolve, we can expect to see AI systems that are increasingly adept at interpreting and reasoning with visual information, opening up new possibilities in fields like education, research, and engineering.