Reasoning Models

2026

1. 

   Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs

   Guijin Son, Seungone Kim, Catherine Arnett, and 73 more authors

   arXiv, 09–15 jun 2026

   Website

2. 

   Reasoning over Mathematical Objects: On-Policy Reward Modeling and Test Time Aggregation

   Pranjal Aggarwal, Marjan Ghazvininejad, Seungone Kim, and 18 more authors

   arXiv, 09–15 jun 2026

3. 

   Argument Reconstruction as Supervision for Critical Thinking in LLMs

   Hyun Ryu, Gyouk Chu, Gregor Betz, and 3 more authors

   arXiv, 09–15 jun 2026

4. 

   Scaling Evaluation-Time Compute with Reasoning Models as Evaluators

   Seungone Kim, Ian Wu, Jinu Lee, and 8 more authors

   In Findings of the Association for Computational Linguistics: ACL 2026, 09–15 jun 2026

   ACL 2026 Findings

   Code

5. 

   OptimalThinkingBench: Evaluating Over and Underthinking in LLMs

   Pranjal Aggarwal, Seungone Kim, Jack Lanchantin, and 4 more authors

   09–15 jun 2026

   ICLR 2026

   Code

6. 

   The CoT Encyclopedia: Analyzing, Predicting, and Controlling how a Reasoning Model will Think

   Seongyun Lee, Seungone Kim, Minju Seo, and 9 more authors

   09–15 jun 2026

   ICLR 2026

2025

1. 

   Agentic-R1: Distilled Dual-Strategy Reasoning

   Weihua Du, Pranjal Aggarwal, Sean Welleck, and 1 more author

   arXiv, 09–15 jun 2025

   EMNLP 2025

   Code

2. 

   L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning

   Pranjal Aggarwal, and Sean Welleck

   09–15 jun 2025

   COLM 2025

   Code Website

2023

1. 

   A Survey of Deep Learning for Mathematical Reasoning

   Pan Lu, Liang Qiu, Wenhao Yu, and 2 more authors

   In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2023

   Abs

   Mathematical reasoning is a fundamental aspect of human intelligence and is applicable in various fields, including science, engineering, finance, and everyday life. The development of artificial intelligence (AI) systems capable of solving math problems and proving theorems in language has garnered significant interest in the fields of machine learning and natural language processing. For example, mathematics serves as a testbed for aspects of reasoning that are challenging for powerful deep learning models, driving new algorithmic and modeling advances. On the other hand, recent advances in large-scale neural language models have opened up new benchmarks and opportunities to use deep learning for mathematical reasoning. In this survey paper, we review the key tasks, datasets, and methods at the intersection of mathematical reasoning and deep learning over the past decade. We also evaluate existing benchmarks and methods, and discuss future research directions in this domain.

2022

1. 

   LILA: A Unified Benchmark for Mathematical Reasoning

   Swaroop Mishra, Matthew Finlayson, Pan Lu, and 8 more authors

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Dec 2022

   Abs

   Mathematical reasoning skills are essential for general-purpose intelligentsystems to perform tasks from grocery shopping to climate modeling. Towards evaluating and improving AI systems in this domain, we proposeLILA, a unified mathematical reasoning benchmark consisting of 23 diversetasks along four dimensions:(i) mathematical abilities e.g., arithmetic, calculus (ii) language format e.g., question-answering, fill-in-the-blanks (iii) language diversity e.g., no language, simple language (iv) external knowledge e.g., commonsense, physics. We construct our benchmark by extending 20 datasets benchmark by collecting task instructions and solutions in the form of Python programs,thereby obtaining explainable solutions in addition to the correct answer. We additionally introduce two evaluation datasets to measure out-of-distribution performance and robustness to language perturbation. Finally, we introduce BHASKARA,a general-purpose mathematical reasoning model trained on LILA. Importantly, we find that multi-tasking leads to significant improvements (average relative improvement of 21.83% F1 score vs. single-task models),while the best performing model only obtains 60.40%,indicating the room for improvement in general mathematical reasoning and understanding.

2. 

   NaturalProver: Grounded Mathematical Proof Generation with Language Models

   Sean Welleck, Jiacheng Liu, Ximing Lu, and 2 more authors

   In Advances in Neural Information Processing Systems, Dec 2022

3. 

   Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations

   Jaehun Jung, Lianhui Qin, Sean Welleck, and 4 more authors

   In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Dec 2022

   Abs

   Pre-trained language models (LMs) struggle with consistent reasoning; recently, prompting LMs to generate explanations that self-guide the inference has emerged as a promising direction to amend this. However, these approaches are fundamentally bounded by the correctness of explanations, which themselves are often noisy and inconsistent. In this work, we develop Maieutic Prompting, which aims to infer a correct answer to a question even from the unreliable generations of LM. Maieutic Prompting induces a tree of explanations abductively (e.g. X is true, because ...) and recursively, then frames the inference as a satisfiability problem over these explanations and their logical relations. We test Maieutic Prompting for true/false QA on three challenging benchmarks that require complex commonsense reasoning. Maieutic Prompting achieves up to 20% better accuracy than state-of-the-art prompting methods, and as a fully unsupervised approach, performs competitively with supervised models. We also show that Maieutic Prompting improves robustness in inference while providing interpretable rationales.

2021

1. 

   NaturalProofs: Mathematical Theorem Proving in Natural Language

   Sean Welleck, Jiacheng Liu, Ronan Le Bras, and 3 more authors

   In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), Dec 2021

   NeurIPS 2021 Oral

2. 

   Towards Grounded Natural Language Proof Generation

   Sean Welleck, Jiacheng Liu, Jesse Michael Han, and 1 more author

   In NeurIPS 2021 Workshop on Math AI for Education: Bridging the Gap Between Research and Smart Education, Dec 2021