MathNet Unveiled: A Groundbreaking Dataset to Revolutionize AI’s Mathematical Reasoning and Global Math Education

The International Mathematical Olympiad (IMO) has long been a crucible for the world’s brightest young mathematical minds, a prestigious competition where national delegations present a curated collection of their most ingenious and original problems. For decades, these problem booklets, meticulously crafted and shared amongst peers, have then quietly faded into obscurity, their valuable contents largely inaccessible for systematic study. This trove of challenging, proof-based mathematical puzzles, representing the pinnacle of secondary school mathematics and a diverse range of global problem-solving traditions, remained largely uncatalogued and unavailable, hindering advancements in artificial intelligence research focused on mathematical reasoning and leaving countless aspiring mathematicians worldwide without a centralized, high-quality resource for their training.

This landscape is set to dramatically change with the unveiling of MathNet, a monumental undertaking by researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), King Abdullah University of Science and Technology (KAUST), and the company HUMAIN. MathNet stands as the largest and most comprehensive dataset of proof-based mathematical problems and their solutions ever compiled, boasting over 30,000 expertly authored problems drawn from 47 countries, spanning 17 languages and 143 distinct competitions. This ambitious project, five times larger than the next-biggest dataset of its kind, represents a significant leap forward in both AI development and the democratization of advanced mathematical learning. The findings and the dataset itself are slated for presentation at the International Conference on Learning Representations (ICLR) in Brazil later this month.

The significance of MathNet extends far beyond its sheer scale; its true innovation lies in its unparalleled breadth and depth, meticulously capturing the global tapestry of mathematical thought. Unlike previous datasets, which predominantly drew from the United States and China, MathNet draws its problems from across six continents, encompassing a vast spectrum of linguistic and cultural approaches to mathematics. This inclusive scope ensures that the dataset reflects the rich diversity of mathematical perspectives and problem-solving traditions that exist within the global math community, moving beyond the most prominent and accessible sources.

A Decade-Long Quest for Global Mathematical Treasures

The genesis of MathNet can be traced back to a shared recognition of the untapped potential within the IMO’s problem archives. "Every country brings a booklet of its most novel and most creative problems," explains Shaden Alshammari, an MIT PhD student and the lead author of the paper detailing MathNet. "They share the booklets with each other, but no one had made the effort to collect them, clean them, and upload them online." This sentiment underscores the painstaking, decade-long effort that underpinned the creation of MathNet.

The process of building MathNet was a formidable logistical and archival challenge, requiring the researchers to meticulously track down and digitize 1,595 PDF volumes, totaling over 25,000 pages. These documents ranged from contemporary digital files to decades-old scanned materials, presented in more than a dozen languages. A crucial element of this vast archive originated from an unexpected but invaluable source: Navid Safaei, a dedicated figure within the IMO community and a co-author of the MathNet paper. Safaei had been personally collecting and scanning these national competition booklets by hand since 2006, amassing a personal archive that formed the foundational backbone of the MathNet dataset. His long-standing commitment to preserving these mathematical gems proved instrumental in the project’s success.

Beyond Community Forums: The Value of Expert-Authored Problems

A key differentiator for MathNet is its exclusive reliance on official national competition booklets, in stark contrast to many existing mathematical datasets that primarily source problems from community forums such as Art of Problem Solving (AoPS). The solutions found within these official booklets are not only expert-written but also undergo rigorous peer review. Crucially, these solutions often extend to multiple pages, offering detailed explanations and exploring various approaches to a single problem. This depth provides AI models with a significantly richer learning signal for understanding mathematical reasoning compared to the typically shorter and more informal solutions found in community-sourced datasets.

Furthermore, this meticulous curation makes MathNet an exceptionally valuable resource for students worldwide. Individuals preparing for the IMO or national mathematics competitions now have access to a centralized, searchable repository of high-quality problems and thoroughly worked-out solutions, drawing from a global array of mathematical traditions. "I remember so many students for whom it was an individual effort. No one in their country was training them for this kind of competition," reflects Alshammari, who herself competed in the IMO as a student. "We hope this gives them a centralized place with high-quality problems and solutions to learn from."

The researchers’ deep ties to the IMO community have been integral to the project. Sultan Albarakati, a co-author on the paper, currently holds a position on the IMO board, facilitating collaboration and the potential direct sharing of the dataset with the IMO foundation. To ensure the integrity and accuracy of MathNet, the team assembled a dedicated grading group comprising over 30 human evaluators from a diverse range of countries, including Armenia, Russia, Ukraine, Vietnam, and Poland. This international team collaborated to meticulously verify thousands of the solutions within the dataset.

Tanish Patil, deputy leader of Switzerland’s IMO delegation, commented on the significance of MathNet, stating, "The MathNet database has the potential to be an excellent resource for both students and leaders seeking new problems to work on or looking for the solution to a difficult question. Whilst other archives of Olympiad problems do exist (notably, the Contest Collections forums on AoPS), these resources lack standardized formatting system, verified solutions, and important problem metadata that topics and theory require. It will also be interesting to see how this dataset is used to improve the performance of reasoning models, and if we will soon be able to reliably answer an important issue when creating novel Olympiad questions: determining if a problem is truly original."

AI’s Mathematical Frontier: Uneven Progress and Lingering Weaknesses

Beyond its educational applications, MathNet serves as a critical benchmark for evaluating the performance of artificial intelligence models in mathematical reasoning. The initial results paint a complex picture, revealing a more nuanced reality than recent optimistic headlines about AI’s mathematical prowess might suggest. While state-of-the-art AI models have indeed made remarkable progress, with some reportedly achieving gold-medal performance at the IMO and solving problems that challenge most humans, MathNet highlights the unevenness of this advancement.

Even GPT-5, identified as the top-performing model in the tested cohort, achieved an average score of approximately 69.3 percent on MathNet’s main benchmark, which comprises 6,400 problems. This indicates that the model still struggled with, and failed to solve, nearly one-third of Olympiad-level problems. A particularly striking weakness exposed by MathNet is in visual reasoning. When problems incorporate figures and diagrams, performance across all tested models drops significantly, underscoring that even the most capable AI systems still exhibit a consistent deficit in processing and interpreting visual mathematical information.

Furthermore, the dataset revealed limitations in multilingual capabilities. Several open-source AI models scored a dismal 0 percent on problems presented in Mongolian, a less common language. This finding highlights another critical dimension where current AI systems falter, despite their overall perceived strength. "GPT models are equally good in English and other languages," Alshammari noted. "But many of the open-source models fail completely at less-common languages, such as Mongolian." This observation is particularly important, as the linguistic diversity of MathNet is intentionally designed to address a deeper limitation in how AI models learn mathematics. When training data is disproportionately skewed towards English and Chinese problems, AI models inevitably absorb a narrow slice of mathematical culture. The subtle nuances and unique approaches embedded in problems from different linguistic traditions – such as a Romanian combinatorics problem or a Brazilian number theory problem – can offer fundamentally different perspectives on the same underlying mathematical concepts. Exposure to this broad range, the researchers argue, is crucial for fostering more robust and versatile mathematical thinking in both humans and AI systems.

Beyond Problem-Solving: New Benchmarks for AI Understanding

MathNet introduces novel benchmarks designed to probe AI’s understanding of mathematical structure and problem similarity. One such benchmark focuses on retrieval, assessing whether AI models can accurately identify when two distinct problems share the same underlying mathematical structure. This capability is vital not only for advancing AI development but also for the broader mathematical community itself. The phenomenon of near-duplicate problems appearing in actual IMO exams over the years is a testament to the inherent difficulty in recognizing mathematical equivalences across varied notations, languages, and formats, a challenge that can even stump expert human committees.

In tests involving eight state-of-the-art embedding models, researchers found that even the most proficient models identified the correct structural match only about 5 percent of the time on their initial attempt. These models frequently ranked structurally unrelated problems as more similar than those that were mathematically equivalent, revealing a significant gap in their ability to discern deep structural relationships.

Another benchmark, the retrieval-augmented generation (RAG) test, examines whether providing a model with a structurally related problem before posing a new one enhances its performance. The results indicate that this approach can indeed improve accuracy, but only when the retrieved problem is genuinely relevant. For instance, DeepSeek-V3.2-Speciale demonstrated an improvement of up to 12 percentage points when provided with well-matched retrieved problems. Conversely, irrelevant retrieval led to performance degradation in approximately 22 percent of cases, highlighting the critical importance of relevance in retrieval-augmented learning.

The research team behind MathNet comprises Shaden Alshammari and Navid Safaei, alongside Abrar Zainal, a HUMAIN AI engineer, and Sultan Albarakati, KAUST Academy Director. MIT CSAIL colleagues contributing to this significant work include master’s student Kevin Wen, Microsoft Principal Engineering Manager Mark Hamilton, and professors William Freeman and Antonio Torralba. Funding for this project was generously provided, in part, by the Schwarzman College of Computing Fellowship and the National Science Foundation.

MathNet is now publicly accessible at https://mathnet.csail.mit.edu, offering a vital new resource for researchers, educators, and students globally, poised to accelerate progress in artificial intelligence and enrich the landscape of mathematical education for generations to come.