Fang, Meng
ORCID: 0000-0001-6745-286X, Wan, Xiangpeng, Lu, Fei, Xing, Fei and Zou, Kai
(2025)
MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data
SCIENTIFIC DATA, 12 (1).
1392-.
ISSN 2052-4463, 2052-4463
|
PDF
mathodyssey_sci (8).pdf - Submitted version Available under License Creative Commons Attribution. Download (326kB) | Preview |
Abstract
Large language models (LLMs) have significantly advanced natural language understanding and demonstrated strong problem-solving abilities. Despite these successes, most LLMs still struggle with solving mathematical problems due to the intricate reasoning required. To support rigorous evaluation of mathematical reasoning in LLMs, we introduce the “MathOdyssey” dataset - a curated collection of 387 expert-generated mathematical problems spanning high school, university, and Olympiad-level topics. Each problem is accompanied by a detailed solution and categorized by difficulty level, subject area, and answer type. The dataset was developed through a rigorous multi-stage process involving contributions from subject experts, peer review, and standardized formatting. We provide detailed metadata and a standardized schema to facilitate consistent use in downstream applications. To demonstrate the dataset’s utility, we evaluate several representative LLMs and report their performance across problem types. We release MathOdyssey as an open-access resource to enable reproducible and fine-grained assessment of mathematical capabilities in LLMs and to foster further research in mathematical reasoning and education.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | 46 Information and Computing Sciences, 39 Education, 4602 Artificial Intelligence |
| Divisions: | Faculty of Science & Engineering Faculty of Science & Engineering > School of Computer Science & Informatics Faculty of Science & Engineering > School of Computer Science & Informatics > Artificial Intelligence |
| Depositing User: | Symplectic Admin |
| Date Deposited: | 19 Dec 2025 08:23 |
| Last Modified: | 19 Sep 2026 02:31 |
| DOI: | 10.1038/s41597-025-05283-3 |
| Related Websites: | |
| URI: | https://livrepository.liverpool.ac.uk/id/eprint/3196189 |
| Disclaimer: | The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate. |
Altmetric
Altmetric