Webpages tagged with «Transformers» - Department of Informatics

NeurIPS paper: "Progressive Memory Transformer: Memory-Aware Attention for Time-Series"

Published Sep. 28, 2026 10:46 PM

A hierarchical contrastive learning framework that effectively captures both local and global temporal patterns in time-series analysis by using a progressive memory attention architecture.

The work "Progressive Memory Transformer: Memory-Aware Attention for Time-Series" by Tord Sture Stangeland, Andreas Köhler, Steffen Maeland, and Adín Ramírez Rivera was accepted at NeurIPS 2026!

An image of a man playing banjo with different image partitioning methods. The ViT partitions an image into square blocks, the Voronoi tesselations produce random convex polygons, and the superpixels partition the image adaptively capturing local shapes of the image.

Modular Superpixel Tokenization in Vision Transformers

Published Aug. 19, 2024 12:58 PM

The Vision Transformer (ViT) signifies a new direction in modelling spatial data, representing a shift in focus from local spatial interactions to global contextual relationships. Traditionally, ViTs partition images into square patches to extract tokenized features. But is this necessarily an optimal way of partitioning images for transformers?

Code & preprint links: