Measuring What Matters: Evaluation as a Driver of Generative AI
Deep generative models are advancing the state of the art in artificial intelligence by effectively capturing and recreating complex patterns in high-dimensional, real-world data. Recent breakthroughs in generating text, images, and video have led to the development of new ecosystems and are already making a tangible impact on the world. Applications in other modalities are expected to soon experience similar growth.

However, the evaluation of generative AI systems presents greater challenges compared to traditional machine-learning evaluation, as there is rarely a single correct output. Multimodal generative models introduce additional complexities, requiring assessment of consistency, complementarity, and coherence across different modalities.

This forum will bring together global experts in deep generative modeling – including both problem owners and model developers from industry, academia, and other sectors – across various domains. The goal is to bring about new, effective evaluation methods that will drive future progress in multimodal generative models, alongside practical tools to promote their adoption within the research community.

This event is a follow-up on the successful Wallenberg Advanced Scientific Forum 2025, under a new name and managed by WARA Media & Language.

Date and time
The workshop starts with lunch on November 3rd and ends after lunch on November 6th, all at Skåvsjöholm, located close to Stockholm.

Forum structure
The forum is designed to be highly interactive, consisting nearly exclusively of structured and directed discussion in smaller focus groups, rather than panels, talks, or plenaries. To make the most out of the event, we will use the so-called double-diamond framework for structured problem solving. This organises problem solving into two stages, first selecting the right problem(s) to tackle and then finding solutions for these. Each stage has two phases: first divergence (exploring possibilities), then convergence (narrowing down choices). The in-person forum will also be preceded by one or two short online workshops to kickstart the discussion before we meet up on site.

Program
A detailed program will be published closer to the event.

Questions about event logistics can be sent to [email protected]