Título: Towards the Next Generation of Long-Context Models

Resumo: Large language models increasingly rely on long contexts, yet standard attention becomes increasingly expensive to compute, and attention distributions can become more diffuse as sequences grow. In this talk, I will present recent work on making sparsity an intrinsic, adaptive property of the attention mechanism, and how this approach can help models focus on relevant information, generalize to contexts longer than those seen during training, and improve the efficiency of both training and inference.

Orador: Marcos Vinicius Treviso, Professor Auxiliar do DEI/IST

Bio: Marcos Treviso is an Assistant Professor at Instituto Superior Técnico, University of Lisbon, and Co-founder and Director of AI Research at Gandara AI. He is also a co-PI of the SARDINE Lab and a member of the ELLIS Unit Lisbon. His research focuses on efficient and interpretable machine learning, with recent work spanning efficient GPU algorithms for adaptive sparse attention, long-context generalization, and efficient inference for large language models. His work has received distinctions including ICML Spotlight selections and an ACL Outstanding Paper Award. 

Website: https://mtreviso.github.io/

Palestra proposta pela área científica de IA.

Informação importante: No final da sessão, reservaremos alguns minutos para que o Prof. Nuno Santos, Coordenador do PDEIC, possa partilhar algumas informações relativas às Jornadas PDEIC 2026, que se realizarão em dezembro.