From Specification Understanding to Verification: Formal Specifications as a Bridge Between LLMs and Trustworthy Software
Abstract
Formal program specifications provide a precise semantic foundation for understanding, generating, and verifying software. However, current large language models are still primarily evaluated and used through code-level tasks, while their ability to understand program semantics, produce rigorous specifications, and support end-to-end verification remains insufficiently explored. This talk presents a specification-centric perspective that connects these three challenges into a unified story. We first introduce formal specifications as semantic probes for evaluating code comprehension in LLMs, where specification judgement, selection, infilling, and generation tasks are used to examine whether models can capture program behaviours beyond individual execution traces. Building on this understanding-oriented view, we then present SpecGen, an LLM-based approach that generates formal specifications through conversational prompting, verifier feedback, and mutation-based refinement, aiming to produce specifications that accurately describe program behaviours. Finally, we introduce SpecSyn, which extends specification generation toward real-world program verification by decomposing large programs, synthesising segment-level specifications, and refining their semantic strength through non-equivalent program mutations and variant discrimination. Together, these works form a complete pipeline from specification-based semantic understanding, to automated specification generation, and further to verification-oriented specification synthesis, showing how formal specifications can serve as a bridge between LLM-based software intelligence and trustworthy program verification.
Bio
Lei Bu is currently a professor and the vice dean of Software Institute, Nanjing University. He received his bachelor and PH.D. degree in Computer Science from Nanjing University in 2004 and 2010. He has been visiting scholar in institutes like Carnegie Mellon University, Microsoft Research Asia, and University of Texas at Dallas. His main research interests include formal method, model checking, hybrid system, and cyber-physical system. He has published around 80 papers in leading journals and conferences like TCAD、TC、TSE、TOSEM、TDSC、RTSS、CAV、ICSE、DAC and so on;He has awarded the CCF-IEEE CS Young Scientist Award, The Outstanding Teachers of Computing in Higher Education Award Program, Zhongchuang Software Talent Award, the NASAC Youth Software Innovation Award, the CCF Youth Talent Development Program, and the MSRA Star Track young faculty program, among others.