Alvin Cheung, Berkeley
Abstract
Despite remarkable progress in code generation, frontier AI models still fall well short of human experts on open-ended programming challenges, i.e., problems where the goal is not merely to produce a correct answer, but to discover high-quality algorithms and system designs. In this talk, I will describe three recent work in my group that tackles this challenge. First, I will introduce FrontierCS, a benchmark of 400+ open-ended computer science problems spanning algorithmic optimization and research-grade system design, where we find that state-of-the-art reasoning models lag substantially behind human experts and that simply scaling up reasoning compute does not bridge the divide. Then I will discuss two approaches to address the scarcity of open-ended training data that underlies this gap: first, an automated pipeline that synthesizes new open-ended coding problems by systematically transforming closed-ended competitive programming problems, and a second approach that generates new coding problems using classical code composition as the mechanism.
Bio
Alvin Cheung is a professor in the EECS department at UC Berkeley, where his group works on data management and programming systems research. Work from his group has received a number of best paper / poster / demo awards in different venues. Alvin is a PECASE awardee, a Sloan fellow, and a recipient of early career research awards from the data management research community, the programming languages research community, and various government agencies and companies.