Foundation models have demonstrated unprecedented capabilities through massive training data and sophisticated architectures. However, they face significant challenges in data efficiency and reliability, particularly when data access is limited or sensitive in critical domains such as healthcare and scientific discovery. These challenges create a need for both theoretical insights into model-data interactions and practical solutions for efficient domain adaptation.

In this talk, I will first present my theoretical work on how neural architectures interact with data, where I established conditions under which networks can learn effectively from noisy data through benign overfitting and how mixture-of-experts architectures can efficiently process naturally clustered data. Next, I will discuss approaches to data-efficient alignment for domain adaptation, including a self-play framework that utilizes synthetic data generated by the foundation model itself, thereby reducing the need for real data. Finally, I will outline future directions for advancing foundation models, with an emphasis on scalable synthetic data generation and alignment across diverse data modalities.

Speaker: Mr. Zixiang CHEN
Date: 10 March 2025 (Monday)
Time: 9:30am – 10:30am
Zoom: Link
Poster: Click here