Speaker: Xin Guo (UC Berkeley)
Xin Guo is the Coleman Fung Chair Professor in Financial Modeling in the Department of Industrial Engineering and Operations Research at the University of California, Berkeley, where she also serves as Department Chair. An internationally recognized leader with fundamental contributions to financial mathematics, stochastic processes, stochastic control, game theory, and machine learning. Xin’s work bridges mathematics, engineering, and data science to address real-world problems in finance, medicine, and engineering. She serves the larger mathematical community in different roles as editorial board member in leading journals and scientific committee member of international events. She is the co-founder of Women in Financial Mathematics.
Registration is free, but mandatory for both in-person and online attendees to ensure the best logistics.
Title: A control-theoretical perspective of continuous-time reinforcement learning
Reinforcement Learning (RL) is a cornerstone of modern machine learning, enabling agents to learn optimal decision-making through interaction with complex environments and other agents. While RL was traditionally developed for discrete-time settings, many real-world physical and financial systems are intrinsically continuous. This series of lectures provides a comprehensive overview of recent advancements in continuous-time RL, analyzed through the rigorous lens of stochastic control theory. It consists of three parts.
Part I. Foundations: Q-Functions and Dynamic Programming
(23/06/2026, 13:00-14:30 plus open discussion time)
We begin by establishing a bridge between RL and classical control theory. By leveraging the Dynamic Programming Principle (DPP) and the Feyman-Kac formula, we explore the similarities between these fields, specifically through the lens of continuous-time Q-functions, and martingale characterisation.
References
[1] Gu, Guo, Wei, Xu (2020). Dynamic programming principle for mean field controls with learning. Operations Research, 2023. Arxiv: 1911.07314.
[2] Gu, X. Guo, Wei, Xu (2021) Mean-Field controls with Q-learning for cooperative MARL: convergence and complexity analysis. SIAM Journal on Mathematics of Data Science. ArXiv:2002.04131.
[3] Cheng, Guo, Zhang (2025). Bridging discrete and continuous RL: stable deterministic policy gradient with martingale characterization. arXiv:2509.23711.
Part II. The Theoretical Divide: Regret Analysis and BSDEs
(24/06/2026, 13:00-14:30 plus open discussion time)
To distinguish RL from classical control, we analyze ``the cost of learning'' through regret analysis. This section delves into the mathematical tools required to quantify performance gaps when the system model is unknown. We show how the regularity of Hamilton-Jacobi-Bellman (HJB) equations and Backward Stochastic Differential Equations (BSDEs) enables establishing regret bounds. In the particular case of Linear-Quadratic (LQ) Models: we present proofs for logarithmic regret in episodic finite-horizon RL via regularities of the Riccati equation.
References
[4] Basei, Guo, Hu, Zhang (2020). Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon. Journal of Machine Learning Research. Arxiv.org/abs/2006.15316.
[5] Guo, Hu, Zhang (2021). Reinforcement learning for linear-convex models with jumps via stability analysis of feedback controls. SIAM J. on Control Optimization (SICON). Arxiv.org/abs/2104.09311.
Part III. Modern Frontiers: Transfer Learning and Rough Paths
(25/06/2026, 13:00-14:30 plus open discussion time)
Finally, we explore how cutting-edge developments from Large Language Models (LLMs), such as transfer learning, can be integrated into RL to significantly boost algorithmic performance. We discuss fast policy learning for LQ control using entropy techniques to balance exploration and exploitation. We will also uncover how the regularity of rough paths provides a framework for analyzing policy transfer in continuous-time RL, offering a robust mathematical foundation for cross-domain adaptation.
References
[6] Guo, Li, Xu (2025). Fast policy learning for linear quadratic control with entropy regularization. SICON. ArXiv:2311.14168.
[7] Cao, Gu., Guo, and Rosenbaum (2024). Transfer learning for portfolio optimization. arXiv:2307.13546.
[8] Guo. and Lyu (2025). Policy transfer for continuous-time reinforcement learning: a (rough) differential equation approach, arxiv2510.15165.
The seminar is in the interest of SURE-AI - Sustainable, Risk-Averse and Ethical AI.
The STAR seminars are for all those in and out of probability, stochastic analysis, control, risk evaluation, statistics, with a view towards applications, in particular to finance, economy, energy markets and production, but also to physics and biology.
This events are held in hybrid format. For the links, you have to register to the mailing list filling the online form
STAR stands for STochastics And Risk! Welcome!