MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data



Fang, Meng ORCID: 0000-0001-6745-286X, Wan, Xiangpeng, Lu, Fei, Xing, Fei and Zou, Kai
(2025) MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data SCIENTIFIC DATA, 12 (1). 1392-. ISSN 2052-4463, 2052-4463

[thumbnail of mathodyssey_sci (8).pdf] PDF
mathodyssey_sci (8).pdf - Submitted version
Available under License Creative Commons Attribution.

Download (326kB) | Preview

Abstract

Large language models (LLMs) have significantly advanced natural language understanding and demonstrated strong problem-solving abilities. Despite these successes, most LLMs still struggle with solving mathematical problems due to the intricate reasoning required. To support rigorous evaluation of mathematical reasoning in LLMs, we introduce the “MathOdyssey” dataset - a curated collection of 387 expert-generated mathematical problems spanning high school, university, and Olympiad-level topics. Each problem is accompanied by a detailed solution and categorized by difficulty level, subject area, and answer type. The dataset was developed through a rigorous multi-stage process involving contributions from subject experts, peer review, and standardized formatting. We provide detailed metadata and a standardized schema to facilitate consistent use in downstream applications. To demonstrate the dataset’s utility, we evaluate several representative LLMs and report their performance across problem types. We release MathOdyssey as an open-access resource to enable reproducible and fine-grained assessment of mathematical capabilities in LLMs and to foster further research in mathematical reasoning and education.

Item Type: Article
Uncontrolled Keywords: 46 Information and Computing Sciences, 39 Education, 4602 Artificial Intelligence
Divisions: Faculty of Science & Engineering
Faculty of Science & Engineering > School of Computer Science & Informatics
Faculty of Science & Engineering > School of Computer Science & Informatics > Artificial Intelligence
Depositing User: Symplectic Admin
Date Deposited: 19 Dec 2025 08:23
Last Modified: 19 Sep 2026 02:31
DOI: 10.1038/s41597-025-05283-3
Related Websites:
URI: https://livrepository.liverpool.ac.uk/id/eprint/3196189
Disclaimer: The University of Liverpool is not responsible for content contained on other websites from links within repository metadata. Please contact us if you notice anything that appears incorrect or inappropriate.