Measuring What Matters: Evaluation as a Driver of Generative AI

Deep generative models are advancing the state of the art in artificial intelligence by effectively capturing and recreating complex patterns in high-dimensional, real-world data. Recent breakthroughs in generating text, images, and video have led to the development of new ecosystems and are already making a tangible impact on the world. Applications in other modalities are expected to soon experience similar growth.

However, the evaluation of generative AI systems presents greater challenges compared to traditional machine-learning evaluation, as there is rarely a single correct output. Multimodal generative models introduce additional complexities, requiring assessment of consistency, complementarity, and coherence across different modalities.

This forum will bring together global experts in deep generative modeling – including both problem owners and model developers from industry, academia, and other sectors – across various domains. The goal is to bring about new, effective evaluation methods that will drive future progress in multimodal generative models, alongside practical tools to promote their adoption within the research community.

This event is a follow-up on the successful Wallenberg Advanced Scientific Forum 2025 under a new name and managed by WARA Media & Language.

Date and time
The workshop starts with lunch on November 3rd and ends after lunch on November 6th, all at Skåvsjöholm located close to Stockholm.

Forum structure
The forum is designed to be highly interactive, consisting nearly exclusively of structured and directed discussion in smaller focus groups, rather than panels, talks, or plenaries. To make the most out of the event, we will use the so-called double-diamond framework for structured problem solving. This organises problem solving into two stages, first selecting the right problem(s) to tackle and then finding solutions for these. Each stage has two phases: first divergence (exploring possibilities), then convergence (narrowing down choices). The in-person forum will also be preceded by one or two short online workshops to kickstart the discussion before we meet up on site.

Programme
A detailed programme will be published closer to the event.

Programme committee
Jonas Beskow, Professor, Department of Speech, Music and Hearing, KTH Royal Institute of Technology, Stockholm, Sweden
Desmond Elliott, Associate Professor, Department of Computing Science, University of Copenhagen, Denmark
Johanna Björklund, Docent and Professor, Department of Computing Science, Umeå University, Sweden
Gustav Eje Henter, Docent and Associate Professor, Department of Speech, Music and Hearing, KTH Royal Institute of Technology, Stockholm, Sweden
Jonas Unger, Professor, Department of Science and Technology, Linköping University, Sweden
Lonni Besançon, Assistant Professor, Department of Science and Technology, Linköping University, Sweden

Questions about event logistics can be sent to [email protected]