Wednesdays@DEI: Talks, 07-10-2026

Title: Towards the Next Generation of Long-Context Models
Abstract: Large language models increasingly rely on long contexts, yet standard attention becomes increasingly expensive to compute, and attention distributions can become more diffuse as sequences grow. In this talk, I will present recent work on making sparsity an intrinsic, adaptive property of the attention mechanism, and how this approach can help models focus on relevant information, generalize to contexts longer than those seen during training, and improve the efficiency of both training and inference.
Speaker: Marcos Vinicius Treviso, Assistant Professor at DEI/IST
Bio: Marcos Treviso is an Assistant Professor at Instituto Superior Técnico, University of Lisbon, and Co-founder and Director of AI Research at Gandara AI. He is also a co-PI of the SARDINE Lab and a member of the ELLIS Unit Lisbon. His research focuses on efficient and interpretable machine learning, with recent work spanning efficient GPU algorithms for adaptive sparse attention, long-context generalization, and efficient inference for large language models. His work has received distinctions including ICML Spotlight selections and an ACL Outstanding Paper Award.
Website: https://mtreviso.github.io/
Talk proposed by the AI scientific area.
Important information: At the end of the session, we will reserve a few minutes for Professor Nuno Santos, Coordinator of the PDEIC, to share some information about the Jornadas PDEIC 2026, which will take place in December.
