Martine Tan

• Tan, Martine Hjelkrem; Aasan, Marius; Chakraborty, Rwiddhi; Arteaga, Gabriel Yanci; Choi, Changkyu & Rivera, Adín Ramírez (2026). Suppressing Non-Semantic Noise in Masked Image Modeling Representations. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 19319–19329. doi: https:/openaccess.thecvf.com/content/CVPR2026/papers/Hjelkrem-Tan_Suppressing_Non-Semantic_Noise_in_Masked_Image_Modeling_Representations_CVPR_2026_paper.pdf. Full text in Research Archive Show summary

  Masked Image Modeling (MIM) has become a ubiquitous self-supervised vision paradigm. In this work, we show that MIM objectives cause the learned representations to retain non-semantic information, which ultimately hurts performance during inference. We introduce a model-agnostic score for semantic invariance using Principal Component Analysis (PCA) on real and synthetic non-semantic images. Based on this score, we propose a simple method, Semantically Orthogonal Artifact Projection (SOAP), to directly suppress non-semantic information in patch representations, leading to consistent improvements in zero-shot performance across various MIM-based models. SOAP is a post-hoc suppression method, requires zero training, and can be attached to any model as a single linear head.

• Tan, Martine Hjelkrem; Aasan, Marius; Arteaga, Gabriel Yanci & Rivera, Adín Ramírez (2026). SPoT: Subpixel Placement of Tokens in Vision Transformers. Transactions on Machine Learning Research (TMLR). 3(26). Full text in Research Archive Show summary

  Vision Transformers naturally accommodate sparsity, yet standard tokenization methods confine features to discrete patch grids. This constraint prevents models from fully exploiting sparse regimes, forcing awkward compromises. We propose Subpixel Placement of Tokens (SPoT), a novel tokenization strategy that positions tokens continuously within images, effectively sidestepping grid-based limitations. With our proposed oracle-guided search, we uncover substantial performance gains achievable with ideal subpixel token positioning, drastically reducing the number of tokens necessary for accurate predictions during inference. SPoT provides a new direction for flexible, efficient, and interpretable ViT architectures, redefining sparsity as a strategic advantage rather than an imposed limitation.

• Arteaga, Gabriel Yanci; Aasan, Marius; Chakraborty, Rwiddhi; Tan, Martine Hjelkrem; Silva, Thalles & Kampffmeyer, Michael [Show all 7 contributors for this article] (2026). Why Prototypes Collapse: Diagnosing and Preventing Partial Collapse in Prototypical Self-Supervised Learning. International Conference on Learning Representations (ICLR). doi: https:/openreview.net/forum?id=fVJEWdwvLO. Full text in Research Archive Show summary

  Prototypical self-supervised learning methods consistently suffer from partial prototype collapse, where multiple prototypes converge to nearly identical representations. This undermines their central purpose—providing diverse and informative targets to guide encoders toward rich representations—and has led practitioners to over-parameterize prototype sets or add ad-hoc regularizers, which mitigate symptoms rather than address the root cause. We empirically trace the collapse to the joint optimization of encoders and prototypes, which encourages a type of shortcut learning: early in training prototypes drift toward redundant representations that minimize loss without necessarily enhancing representation diversity. To break the joint optimization, we introduce a fully decoupled training strategy that learns prototypes and encoders under separate objectives. Concretely, we model prototypes as a Gaussian mixture updated with an online EM-style procedure, independent of the encoder’s loss. This simple yet principled decoupling eliminates prototype collapse without explicit regularization and yields consistently diverse prototypes, which in several settings translate to improved downstream performance.

• Aasan, Marius; Tan, Martine Hjelkrem; Catalano, Nico; Choi, Changkyu & Rivera, Adín Ramírez (2025). Differentiable Hierarchical Visual Tokenization. Advances in Neural Information Processing Systems. ISSN 1049-5258. 38. doi: 10.52202/085713-2930. Full text in Research Archive Show summary

  Vision Transformers rely on fixed patch tokens that ignore the spatial and semantic structure of images. In this work, we introduce an end-to-end differentiable tokenizer that adapts to image content with pixel-level granularity while remaining backward-compatible with existing architectures for retrofitting pretrained models. Our method uses hierarchical model selection with information criteria to provide competitive performance in both image-level classification and dense-prediction tasks, and even supports out-of-the-box raster-to-vector conversion.

View all works in NVA

• Choi, Changkyu; Tan, Martine Hjelkrem; Aasan, Marius & Rivera, Adín Ramírez (2026). AI in Our Daily Lives - Panel Discussion. Full text in Research Archive Show summary

  AURoRA researchers co-hosted a public seminar and panel discussion with the Embassy of the Republic of Korea to Norway, held at Litteraturhuset in Oslo as part of the K-Initiative campaign. The evening opened with the talk "Mirrors of Intelligence" by Adín Ramírez Rivera (University of Oslo). The panel then discussed trust, fairness and the societal impact of AI with the audience.

• Tan, Martine Hjelkrem (2026). Sparse Tokens and Patch Representations. doi: https:/www.youtube.com/watch?v=3NvHZFei-S0. Full text in Research Archive
• Tan, Martine Hjelkrem; Rivera, Adín Ramírez & Thomas, Sarina (2023). Hierarchical topic discovery in images; probabilistic modelling of visual data. Full text in Research Archive

View all works in NVA