Marius Aasan

• Tan, Martine Hjelkrem; Aasan, Marius; Chakraborty, Rwiddhi; Arteaga, Gabriel Yanci; Choi, Changkyu & Rivera, Adín Ramírez (2026). Suppressing Non-Semantic Noise in Masked Image Modeling Representations. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). s. 19319–19329. doi: https:/openaccess.thecvf.com/content/CVPR2026/papers/Hjelkrem-Tan_Suppressing_Non-Semantic_Noise_in_Masked_Image_Modeling_Representations_CVPR_2026_paper.pdf. Fulltekst i vitenarkiv Vis sammendrag

  Masked Image Modeling (MIM) has become a ubiquitous self-supervised vision paradigm. In this work, we show that MIM objectives cause the learned representations to retain non-semantic information, which ultimately hurts performance during inference. We introduce a model-agnostic score for semantic invariance using Principal Component Analysis (PCA) on real and synthetic non-semantic images. Based on this score, we propose a simple method, Semantically Orthogonal Artifact Projection (SOAP), to directly suppress non-semantic information in patch representations, leading to consistent improvements in zero-shot performance across various MIM-based models. SOAP is a post-hoc suppression method, requires zero training, and can be attached to any model as a single linear head.

• Tan, Martine Hjelkrem; Aasan, Marius; Arteaga, Gabriel Yanci & Rivera, Adín Ramírez (2026). SPoT: Subpixel Placement of Tokens in Vision Transformers. Transactions on Machine Learning Research (TMLR). 3(26). Fulltekst i vitenarkiv Vis sammendrag

  Vision Transformers naturally accommodate sparsity, yet standard tokenization methods confine features to discrete patch grids. This constraint prevents models from fully exploiting sparse regimes, forcing awkward compromises. We propose Subpixel Placement of Tokens (SPoT), a novel tokenization strategy that positions tokens continuously within images, effectively sidestepping grid-based limitations. With our proposed oracle-guided search, we uncover substantial performance gains achievable with ideal subpixel token positioning, drastically reducing the number of tokens necessary for accurate predictions during inference. SPoT provides a new direction for flexible, efficient, and interpretable ViT architectures, redefining sparsity as a strategic advantage rather than an imposed limitation.

• Arteaga, Gabriel Yanci; Aasan, Marius; Chakraborty, Rwiddhi; Tan, Martine Hjelkrem; Silva, Thalles & Kampffmeyer, Michael [Vis alle 7 forfattere av denne artikkelen] (2026). Why Prototypes Collapse: Diagnosing and Preventing Partial Collapse in Prototypical Self-Supervised Learning. International Conference on Learning Representations (ICLR). doi: https:/openreview.net/forum?id=fVJEWdwvLO. Fulltekst i vitenarkiv Vis sammendrag

  Prototypical self-supervised learning methods consistently suffer from partial prototype collapse, where multiple prototypes converge to nearly identical representations. This undermines their central purpose—providing diverse and informative targets to guide encoders toward rich representations—and has led practitioners to over-parameterize prototype sets or add ad-hoc regularizers, which mitigate symptoms rather than address the root cause. We empirically trace the collapse to the joint optimization of encoders and prototypes, which encourages a type of shortcut learning: early in training prototypes drift toward redundant representations that minimize loss without necessarily enhancing representation diversity. To break the joint optimization, we introduce a fully decoupled training strategy that learns prototypes and encoders under separate objectives. Concretely, we model prototypes as a Gaussian mixture updated with an online EM-style procedure, independent of the encoder’s loss. This simple yet principled decoupling eliminates prototype collapse without explicit regularization and yields consistently diverse prototypes, which in several settings translate to improved downstream performance.

• Aasan, Marius; Tan, Martine Hjelkrem; Catalano, Nico; Choi, Changkyu & Rivera, Adín Ramírez (2025). Differentiable Hierarchical Visual Tokenization. Advances in Neural Information Processing Systems. ISSN 1049-5258. 38. doi: 10.52202/085713-2930. Fulltekst i vitenarkiv Vis sammendrag

  Vision Transformers rely on fixed patch tokens that ignore the spatial and semantic structure of images. In this work, we introduce an end-to-end differentiable tokenizer that adapts to image content with pixel-level granularity while remaining backward-compatible with existing architectures for retrofitting pretrained models. Our method uses hierarchical model selection with information criteria to provide competitive performance in both image-level classification and dense-prediction tasks, and even supports out-of-the-box raster-to-vector conversion.

• Aasan, Marius & Rivera, Adín Ramírez (2025). Pixel-Level Predictions with Embedded Lookup Tables. I Jenssen, Robert & Bach, Kerstin (Red.), Proceedings of the Symposium of the Norwegian AI Society 2025. CEUR-WS. s. 49–59. Fulltekst i vitenarkiv Vis sammendrag

  Pixel-level prediction tasks inherently face constraints imposed by image resolution and the number of prediction classes. As image resolutions and dimensionalities increase, these constraints lead to significant memory bottlenecks when explicitly modeling high-dimensional embeddings for each individual pixel. Addressing these bottlenecks requires the development of alternative, more efficient representations to facilitate continued progress in the field. In this work, we discuss Embedded Lookup Tables (ELUTs) with indexed segmentation maps as an alternative data structure for more memory efficient representations in image processing. We show that ELUTs are inherently compatible with cost functionals and metrics for pixel-level prediction tasks with a significant reduction in memory overhead.

• Aasan, Marius; Kolbjørnsen, Odd; Solberg, Anne H Schistad & Rivera, Adín Ramírez (2025). A Spitting Image: Modular Superpixel Tokenization in Vision Transformers. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. 15641, s. 124–142. doi: 10.1007/978-3-031-93806-1_11. Fulltekst i vitenarkiv Vis sammendrag

  Vision Transformer (ViT) architectures traditionally employ a grid-based approach to tokenization independent of the semantic content of an image. We propose a modular superpixel tokenization strategy which decouples tokenization and feature extraction; a shift from contemporary approaches where these are treated as an undifferentiated whole. Using on-line content-aware tokenization and scale- and shape-invariant positional embeddings, we perform experiments and ablations that contrast our approach with patch-based tokenization and randomized partitions as baselines. We show that our method significantly improves the faithfulness of attributions, gives pixel-level granularity on zero-shot unsupervised dense prediction tasks, while maintaining predictive performance in classification tasks. Our approach provides a modular tokenization framework commensurable with standard architectures, extending the space of ViTs to a larger class of semantically-rich models.

• Souza, Bruno Cesar de Oliveira; Aasan, Marius; Pedrini, Helio & Rivera, Adín Ramírez (2023). SelfGraphVQA: A Self-Supervised Graph Neural Network for Scene-based Question Answering. I Cho, Minsu & Ham, Bumsub (Red.), 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). IEEE (Institute of Electrical and Electronics Engineers). ISSN 9798350307443. s. 4642–4647. doi: 10.1109/ICCVW60793.2023.00499. Fulltekst i vitenarkiv Vis sammendrag

  The intersection of vision and language is of major interest due to the increased focus on seamless integration between recognition and reasoning. Scene graphs (SGs) have emerged as a useful tool for multimodal image analysis, showing impressive performance in tasks such as Visual Question Answering (VQA). In this work, we demonstrate that despite the effectiveness of scene graphs in VQA tasks, current methods that utilize idealized annotated scene graphs struggle to generalize when using predicted scene graphs extracted from images. To address this issue, we introduce the SelfGraphVQA framework. Our approach extracts a scene graph from an input image using a pretrained scene graph generator and employs semantically-preserving augmentation with self-supervised techniques. This method improves the utilization of graph representations in VQA tasks by circumventing the need for costly and potentially biased annotated data. By creating alternative views of the extracted graphs through image augmentations, we can learn joint embeddings by optimizing the informational content in their representations using an un-normalized contrastive approach. As we work with SGs, we experiment with three distinct maximization strategies: node-wise, graph-wise, and permutation-equivariant regularization. We empirically showcase the effectiveness of the extracted scene graph for VQA and demonstrate that these approaches enhance overall performance by highlighting the significance of visual information. This offers a more practical solution for VQA tasks that rely on SGs for complex reasoning questions.

Se alle arbeider i NVA

• Choi, Changkyu; Tan, Martine Hjelkrem; Aasan, Marius & Rivera, Adín Ramírez (2026). AI in Our Daily Lives - Panel Discussion. Fulltekst i vitenarkiv Vis sammendrag

  AURoRA researchers co-hosted a public seminar and panel discussion with the Embassy of the Republic of Korea to Norway, held at Litteraturhuset in Oslo as part of the K-Initiative campaign. The evening opened with the talk "Mirrors of Intelligence" by Adín Ramírez Rivera (University of Oslo). The panel then discussed trust, fairness and the societal impact of AI with the audience.

• Aasan, Marius (2026). Perceptual Grouping and Adaptive Tokenisation. doi: https:/www.youtube.com/watch?v=l6BeD85wRgQ. Fulltekst i vitenarkiv Vis sammendrag

  As vision modelling is pivoting towards transformer based models, the role of tokenization — the way models partition and interpret the data — becomes increasingly important, as it is the central interface between analysis and data, that dictates how AI models synthesise semantic understanding from sensory data. This talk discusses how this interfact can be leveraged for better interpretability, incorporation of prior knowledge, and new opportunities in self-supervised learning signals. We also discuss the challenges involved in how these mechanisms currently work, and what interesting research opportunities that can be derived from tokenization in the broader context of modern representation learning.

• Aasan, Marius; Wetzer, Elisabeth & Chakraborty, Rwiddhi (2026). Visual Intelligence Workshop on Multimodal AI. Fulltekst i vitenarkiv Vis sammendrag

  Visual Intelligence Workshop on Multimodal AI The workshop focuses on three short talks on different aspects of multimodality. The presentations are designed to appeal to a broad target audience and provide an educational and bottom-up approach to the topic. Our goal is to provide understanding and interesting perspectives for Visual Intelligence’s academic and industry partners. The talks seek to explore the following: How multimodality shapes current AI models. ‍ How it affects how we think of data, as well as interact and understand AI models. ‍ How multimodality can help us paint a broader picture of our data by bridging modalities across domains. ‍ What types of benefits this research can bring to the Visual Intelligence consortium as a whole.

• Choi, Changkyu & Aasan, Marius (2024). Data Vaccination: Balancing Data Security and Utility. Lov & Data. ISSN 0800-7853. Fulltekst i vitenarkiv
• Aasan, Marius (2024). On Tokenization in Vision Transformers. Fulltekst i vitenarkiv
• Aasan, Marius; Kolbjørnsen, Odd; Solberg, Anne H Schistad & Rivera, Adín Ramírez (2024). A Spitting Image: Modular Superpixel Tokenization in Vision Transformers. Fulltekst i vitenarkiv
• Aasan, Marius (2024). STMAE: Seismic Tiles and Masked Autoencoders. Fulltekst i vitenarkiv
• Aasan, Marius; Kolbjørnsen, Odd; Solberg, Anne H Schistad & Rivera, Adín Ramírez (2024). A Spitting Image: Modular Tokenization in Vision Transformers. Fulltekst i vitenarkiv
• Aasan, Marius (2024). Modular Superpixel Tokenization in Vision Transformers. Fulltekst i vitenarkiv
• Aasan, Marius; Kolbjørnsen, Odd; Solberg, Anne H Schistad & Rivera, Adín Ramírez (2024). A Spitting Image: Modular Superpixel Tokenization in Vision Transformers. Fulltekst i vitenarkiv
• Souza, Bruno Cesar de Oliveira; Aasan, Marius; Pedrini, Helio & Rivera, Adín Ramírez (2023). SelfGraphVQA: A Self-Supervised Graph Neural Network for Scene-based Question Answering. Fulltekst i vitenarkiv
• Souza, Bruno Cesar de Oliveira; Aasan, Marius; Pedrini, Helio & Rivera, Adín Ramírez (2023). SelfGraphVQA: A Self-Supervised Graph Neural Network for Scene-based Question Answering. Fulltekst i vitenarkiv
• Aasan, Marius (2023). Idealized Representations: A High Level View of Self-Supervised Learning. Fulltekst i vitenarkiv
• Aasan, Marius (2022). Inverse Problems and Invertibility in Deep Learning - Bridging the Gap with Invertible Encoder Models. Fulltekst i vitenarkiv
• Aasan, Marius; Rivera, Adín Ramírez; Kolbjørnsen, Odd & Solberg, Anne H Schistad (2022). An Information Metric for Partitions on Images using Lattice Theoretic Entropy. Fulltekst i vitenarkiv
• Aasan, Marius (2022). Inverse Problems and Invertibility in Deep Learning. Fulltekst i vitenarkiv

Se alle arbeider i NVA