MULTIMODAL FOUNDATION MODELS

The recent collaboration with RISE, NVIDIA, and AI Sweden on Language Models (LLMs) has been valuable for both practical and academic insights, particularly around the GPT-SW3 model series (access the model here). The project has deepened understanding of user needs and challenges, with many favoring model-agnostic systems to integrate the best cost-performance LLMs. Some organizations, however, may require private, cloud-based instances to protect intellectual property. Public bodies with sensitive data, such as the Swedish Tax Agency and Swedish Armed Forces, need open models that can be hosted on premises. Looking ahead, the focus is on developing small to medium-sized foundation models, especially for multimodal data like time-series or graphs, where significant scientific and practical gains are expected. Ongoing collaboration with AI Sweden will complement this by providing larger, more versatile models. Key researchers in this area include Love Börjesson (KB Labs) and Marco Kuhlmann (LiU), with an emphasis on attracting international talent.