Changkyu Choi

• Tan, Martine Hjelkrem; Aasan, Marius; Chakraborty, Rwiddhi; Arteaga, Gabriel Yanci; Choi, Changkyu & Rivera, Adín Ramírez (2026). Suppressing Non-Semantic Noise in Masked Image Modeling Representations. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 19319–19329. doi: https:/openaccess.thecvf.com/content/CVPR2026/papers/Hjelkrem-Tan_Suppressing_Non-Semantic_Noise_in_Masked_Image_Modeling_Representations_CVPR_2026_paper.pdf. Full text in Research Archive Show summary

  Masked Image Modeling (MIM) has become a ubiquitous self-supervised vision paradigm. In this work, we show that MIM objectives cause the learned representations to retain non-semantic information, which ultimately hurts performance during inference. We introduce a model-agnostic score for semantic invariance using Principal Component Analysis (PCA) on real and synthetic non-semantic images. Based on this score, we propose a simple method, Semantically Orthogonal Artifact Projection (SOAP), to directly suppress non-semantic information in patch representations, leading to consistent improvements in zero-shot performance across various MIM-based models. SOAP is a post-hoc suppression method, requires zero training, and can be attached to any model as a single linear head.

• Kjetil, Indrehus,; Duric, Adrian; Choi, Changkyu & Ramezani-Kebrya, Ali (2026). Towards Self-Explainable Document Visual Question Answering with Chain-of-Explanation Predictions. arXiv. 2605.06058. doi: https:/arxiv.org/pdf/2605.06058. Full text in Research Archive Show summary

  Document Visual Question Answering (DocVQA) requires vision-language models to reason not only about what information in a document is relevant to a question, but also where the answer is grounded on the page. Existing DocVQA models entangle question-relevant evidence and answer localization and operate largely as black boxes, offering limited means to verify how predictions depend on visual evidence. We propose CoExVQA, a self-explainable DocVQA framework with a grounded reasoning process through a chain-of-explanation design. CoExVQA first identifies question-relevant evidence, then explicitly localizes the answer region, and finally decodes the answer exclusively from the grounded region. Prediction via CoExVQA's chain-of-explanation enables direct inspection and verification of the reasoning process across modalities. Empirical results show that restricting decoding to grounded evidence achieves SotA explainable DocVQA performance on PFL-DocVQA, improving ANLS by 12% over the current explainable baselines while providing transparent and verifiable predictions.

• Suyoung, Bae,; Jaehoon, Lee,; Choi, Changkyu; YunSeok, Choi, & Jee-Hyong, Lee, (2026). Remember Your Trace: Memory-Guided Long-Horizon Agentic Framework for Consistent and Hierarchical Repository-Level Code Documentation. Advances in Neural Information Processing Systems. ISSN 1049-5258. doi: https:/arxiv.org/pdf/2605.14563. Full text in Research Archive Show summary

  Automated code documentation is essential for modern software development, providing the contextual grounding that both human developers and coding agents rely on to navigate large codebases. Existing repository-level approaches process components independently, causing redundant retrieval and conflicting descriptions across documents while producing outputs that lack hierarchical structure. Therefore, we propose MemDocAgent, a long-horizon agentic framework that generates documentation within a single, integrated context spanning the entire repository. It combines two components: (i) Dependency-Aware Traversal Guiding that predetermines a traversal order respecting dependency and granularity hierarchies; (ii) Memory-Guided Agentic Interaction, in which the agent interacts with RepoMemory, a shared memory accumulating prior work traces through read, write, and verify operations. Through an in-depth multi-criteria evaluation, MemDocAgent achieves the best performance over both open and closed-source baselines and demonstrates practical applicability in real software development workflows.

• Han, Jiin; Bae, Suyoung; Choi, Yerim; Lee, Sangyun; Kim, Kyutae & Kim, Ryunho [Show all 9 contributors for this article] (2026). PubTables-QA: A Benchmark Toward Cross-Page Table Reasoning in Table Visual Question Answering. ACM Transactions on Knowledge Discovery from Data. ISSN 1556-4681. doi: https:/respmultimodal2026.github.io. Full text in Research Archive Show summary

  Tables are central to scientific communication, yet their interpretation depends critically on layout, continuity, and surrounding context. While existing multi-page and multi-table benchmarks formally require document-level reasoning, models can often succeed through stochastic shortcut search—locating local answer-bearing regions without navigating table structure. To address this gap, we introduce PubTables-QA, a multi-page TableVQA benchmark designed to audit responsible cross-page visual reasoning in academic documents. Built on PubTables-v2’s structural annotations, PubTables-QA provides 2,106 QA pairs that are unanswerable from any single page, compelling models to reconstruct table continuity across pages or integrate evidence across multiple tables. We further introduce a three-level QA taxonomy spanning Document, Table, and Cell/column levels. Experiments on recent MLLMs, including GPT-4o, Gemini-2.5-Pro, and Gemma-4, show that even the strongest model reaches only 32.6% accuracy. Oracle-setting analysis further shows that performance depends strongly on access to the correct page, table region, and cell-level evidence, highlighting persistent failures in faithful cross-page evidence integration.

• Jung, Kangsoo; Souibgui, Mohamed Ali; Choi, Changkyu & Palamidessi, Catuscia (2026). Selective Disclosure: Controlling Information Leakage in DocVQA Explanations. International Conference on Learning Representations (ICLR). doi: https:/iclr.cc/virtual/2026/10019246. Full text in Research Archive Show summary

  Explainable document visual question answering systems improve transparency by visualizing document regions relevant to a query, but they can unintentionally expose sensitive information through explanation outputs, even when textual answers are restricted. We analyze privacy risks arising from coarse explanations and adversarial jailbreaking queries. To mitigate this, we propose a policy-aware visual explanation sanitization framework based on a Role-centric Attribute-Based Access Control (RABAC) model, combining document structure analysis and line-level localization to mask restricted regions. Our method is model-agnostic, supports dynamic policies, and significantly reduces sensitive information leakage while preserving interpretability.

• Wu, Zhiyuan; Choi, Changkyu; Yu, Shujian; Jenssen, Robert & Ramezani-Kebrya, Ali (2026). Mitigating Embedding Leakage via Latent Disruption with Controlled Reconstruction. Transactions on Machine Learning Research (TMLR). doi: https:/openreview.net/forum?id=nZWBrxJyrS. Full text in Research Archive Show summary

  Pre-trained encoders produce semantically rich latent embeddings, which, however, may expose unintended information through malicious inference or exploitation. We propose SEAL, a framework that mitigates embedding leakage by disrupting latent representations based on information-theoretic principles. It reduces the risk of potential misuse while enabling controlled reconstruction for trusted users. SEAL learns to encode controlled perturbations by minimizing the Matrix Norm-based Quadratic Mutual Information (MQMI) functional between original and perturbed embeddings within a hyperspherical latent space. Meanwhile, a private decoder, jointly trained with the SEAL encoder, is trained to reconstruct the original data that is accessible only to authorized users under an access-controlled setting. Extensive experiments on vision and text datasets demonstrate that SEAL reduces latent leakage, weakens the effectiveness of evaluated inference attacks, and preserves reconstruction under the considered setting.

• Aasan, Marius; Tan, Martine Hjelkrem; Catalano, Nico; Choi, Changkyu & Rivera, Adín Ramírez (2025). Differentiable Hierarchical Visual Tokenization. Advances in Neural Information Processing Systems. ISSN 1049-5258. 38. doi: 10.52202/085713-2930. Full text in Research Archive Show summary

  Vision Transformers rely on fixed patch tokens that ignore the spatial and semantic structure of images. In this work, we introduce an end-to-end differentiable tokenizer that adapts to image content with pixel-level granularity while remaining backward-compatible with existing architectures for retrofitting pretrained models. Our method uses hierarchical model selection with information criteria to provide competitive performance in both image-level classification and dense-prediction tasks, and even supports out-of-the-box raster-to-vector conversion.

• Choi, Changkyu; Subramaniam, Arangan; Handegard, Nils Olav; Ramezani-Kebrya, Ali & Jenssen, Robert (2025). Leveraging Foundation Model Adapters to Enable Robust and Semantic Underwater Exploration. CEUR Workshop Proceedings. 3975(1). doi: https:/ceur-ws.org/Vol-3975/. Full text in Research Archive Show summary

  This position paper presents a framework for intelligent underwater exploration by marrying foundation models (FMs) with multi‑frequency echosounder data. Echosounder data capture backscattered acoustic signals across a range of frequencies, providing rich insights into underwater environments by exploiting the frequency‑dependent scattering properties of underwater targets. However, their heterogeneity and complex structure complicate analysis. To address these challenges, the paper introduces four key innovations aimed at improving echosounder data analysis under dynamic ocean conditions: (1) aligning multi‑frequency echosounder data with FMs via lightweight FM adapters, (2) enabling continual adaptation to temporal distribution shifts in dynamic marine environments, (3) designing semantic tokenizers that preserve spatial structures, and (4) effectively leveraging sparse annotations to minimize dependence on costly labeled data. For each research direction, we map recent artificial intelligence (AI) methodologies to marine acoustic challenges and outline concrete pathways for technology transfer. Preliminary experiments demonstrate that a Vision Transformer (ViT), pretrained on natural images in a self-supervised manner, can segment sandeel schools from multi‑frequency echosounder data without task‑specific retraining. These results substantiate the proposed framework and illustrate the potential of cross‑disciplinary AI methods for ecologically informative underwater exploration.

• Souibgui, Mohamed Ali; Choi, Changkyu; Barsky, Andrey; Jung, Kangsoo; Valveny, Ernest & Karatzas, Dimosthenis (2025). DocVXQA: Context-Aware Visual Explanations for Document Question Answering. Proceedings of Machine Learning Research (PMLR). 267, p. 56549–56569. doi: https:/proceedings.mlr.press/v267/souibgui25a.html. Full text in Research Archive Show summary

  We propose DocVXQA, a novel framework for visually self-explainable document question answering, where the goal is not only to produce accurate answers to questions but also to learn visual heatmaps that highlight critical regions, offering interpretable justifications for the model decision. To integrate explanations into the learning process, we quantitatively formulate explainability principles as explicit learning criteria. Unlike conventional relevance map methods that solely emphasize regions relevant to the answer, our context-aware DocVXQA delivers explanations that are contextually sufficient yet representation-efficient. This fosters user trust while achieving a balance between predictive performance and interpretability in document visual question answering applications. Extensive experiments, including human evaluation, provide strong evidence supporting the effectiveness of our method.

• Kim, Hyeongji; Choi, Changkyu; Kampffmeyer, Michael Christian; Berge, Terje; Parviainen, Pekka & Malde, Ketil (2025). ProxyDR: Deep Hyperspherical Metric Learning with Distance Ratio-Based Formulation. Lecture Notes in Computer Science (LNCS). ISSN 0302-9743. p. 434–450. doi: 10.1007/978-3-031-91585-7_26. Full text in Research Archive
• Wu, Zhiyuan; Choi, Changkyu; Cao, Xiangcheng; Cevher, Volkan & Ramezani-Kebrya, Ali (2025). Addressing Label Shift in Distributed Learning via Entropy Regularization​. International Conference on Learning Representations. Full text in Research Archive Show summary

  We address the challenge of minimizing "true risk" in multi-node distributed learning.\footnote{We use the term node to refer to a client, FPGA, APU, CPU, GPU, or worker.} These systems are frequently exposed to both inter-node and intra-node "label shifts", which present a critical obstacle to effectively optimizing model performance while ensuring that data remains confined to each node. To tackle this, we propose the Versatile Robust Label Shift (VRLS) method, which enhances the maximum likelihood estimation of the test-to-train label importance ratio. VRLS incorporates Shannon entropy-based regularization and adjusts the importance ratio during training to better handle label shifts at the test time. In multi-node learning environments, VRLS further extends its capabilities by learning and adapting importance ratios across nodes, effectively mitigating label shifts and improving overall model performance. Experiments conducted on MNIST, Fashion MNIST, and CIFAR-10 demonstrate the effectiveness of VRLS, outperforming baselines by up to 20% in imbalanced settings. These results highlight the significant improvements VRLS offers in addressing label shifts. Our theoretical analysis further supports this by establishing high-probability bounds on estimation errors.

• Choi, Changkyu; Yu, Shujian; Kampffmeyer, Michael Christian; Salberg, Arnt-Børre; Handegard, Nils Olav & Jenssen, Robert (2024). DIB-X: Formulating Explainability Principles for a Self-Explainable Model Through Information Theoretic Learning. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. ISSN 1520-6149. p. 7170–7174. doi: 10.1109/ICASSP48485.2024.10447094. Full text in Research Archive Show summary

  The recent development of self-explainable deep learning approaches has focused on integrating well-defined explainability principles into learning process, with the goal of achieving these principles through optimization. In this work, we propose DIB-X, a self-explainable deep learning approach for image data, which adheres to the principles of minimal, sufficient, and interactive explanations. The minimality and sufficiency principles are rooted from the trade-off relationship within the information bottleneck framework. Distinctly, DIB-X directly quantifies the minimality principle using the recently proposed matrix-based Rényi’s α-order entropy functional, circumventing the need for variational approximation and distributional assumption. The interactivity principle is realized by incorporating existing domain knowledge as prior explanations, fostering explanations that align with established domain understanding. Empirical results on MNIST and two marine environment monitoring datasets with different modalities reveal that our approach primarily provides improved explainability with the added advantage of enhanced classification performance.

• Choi, Changkyu; Kampffmeyer, Michael; Handegard, Nils Olav; Salberg, Arnt-Børre & Jenssen, Robert (2023). Deep Semisupervised Semantic Segmentation in Multifrequency Echosounder Data. IEEE Journal of Oceanic Engineering. ISSN 0364-9059. 48(2), p. 384–400. doi: 10.1109/JOE.2022.3226214. Full text in Research Archive Show summary

  Multi-frequency echosounder data can provide a broad understanding of the underwater environment in a non-invasive manner. The analysis of echosounder data is, hence, a topic of great importance for the marine ecosystem. Semantic segmentation, a deep learning based analysis method predicting the class attribute of each acoustic intensity, has recently been in the spotlight of the fisheries and aquatic industry since its result can be used to estimate the abundance of the marine organisms. However, a fundamental problem with current methods is the massive reliance on the availability of large amounts of annotated training data, which can only be acquired through expensive handcrafted annotation processes, making such approaches unrealistic in practice. As a solution to this challenge, we propose a novel approach, where we leverage a small amount of annotated data (supervised deep learning) and a large amount of readily available unannotated data (unsupervised learning), yielding a new data-efficient and accurate semi-supervised semantic segmentation method, all embodied into a single end-to-end trainable convolutional neural networks architecture. Our method is evaluated on representative data from a sandeel survey in the North Sea conducted by the Norwegian Institute of Marine Research. The rigorous experiments validate that our method achieves comparable results utilizing only 40 percent of the annotated data on which the supervised method is trained, by leveraging unannotated data. The code is available at https://github.com/SFI-Visual-Intelligence/PredKlus-semisup-segmentation.

• Choi, Changkyu; Kampffmeyer, Michael; Handegard, Nils Olav; Salberg, Arnt Børre; Brautaset, Olav & Eikvil, Line [Show all 7 contributors for this article] (2021). Semi-supervised target classification in multi-frequency echosounder data. ICES Journal of Marine Science. ISSN 1054-3139. 78(7), p. 2615–2627. doi: 10.1093/icesjms/fsab140. Full text in Research Archive Show summary

  Acoustic target classification in multi-frequency echosounder data is a major interest for the marine ecosystem and fishery management since it can potentially estimate the abundance or biomass of the species. A key problem of current methods is the heavy dependence on the manual categorization of data samples. As a solution, we propose a novel semi-supervised deep learning method leveraging a few annotated data samples together with vast amounts of unannotated data samples, all in a single model. Specifically, two inter-connected objectives, namely, a clustering objective and a classification objective, optimize one shared convolutional neural network in an alternating manner. The clustering objective exploits the underlying structure of all data, both annotated and unannotated; the classification objective enforces a certain consistency to given classes using the few annotated data samples. We evaluate our classification method using echosounder data from the sandeel case study in the North Sea. In the semi-supervised setting with only a tenth of the training data annotated, our method achieves 67.6% accuracy, outperforming a conventional semi-supervised method by 7.0 percentage points. When applying the proposed method in a fully supervised setup, we achieve 74.7% accuracy, surpassing the standard supervised deep learning method by 4.7 percentage points.

• Choi, Changkyu; Bianchi, Filippo Maria; Kampffmeyer, Michael & Jenssen, Robert (2020). Short-Term Load Forecasting with Missing Data using Dilated Recurrent Attention Networks. Proceedings of the Northern Lights Deep Learning Workshop. 1. doi: 10.7557/18.5136. Full text in Research Archive Show summary

  Forecasting the dynamics of time-varying systems is essential to maintaining the sustainability of the systems. Recent studies have discovered that Recurrent Neural Networks(RNN) applied in the forecasting tasks outperform conventional models that include AutoRegressive Integrated Moving Average(ARIMA). However, due to the structural limitation of vanilla RNN which holds unit-length internal connections, learning the representation of time series with \textit{missing data} can be severely biased. The goal of this paper is to provide a robust RNN architecture against the bias from missing data. We propose Dilated Recurrent Attention Networks(DRAN). The proposed model has a stacked structure of multiple RNNs which layer of each having a different length of internal connections. This structure allows incorporating previous information at different time scales. DRAN updates its state by a weighted average of the layers. In order to focus more on the layer that carries reliable information against bias from missing data, it leverages attention mechanism which learns the distribution of attention weights among the layers. We report that our model outperforms conventional ones with respect to the forecast accuracy from two benchmark datasets, including a real-world electricity load dataset.

View all works in NVA

• Han, Ji-In & Choi, Changkyu (2026). PubTables-QA: a Multi-Page Table Visual Question Answering for Cross-Page Table Reasoning. doi: https:/www.ekc2026.org/page/programme/stdivision/15. Full text in Research Archive Show summary

  Tables are a central medium for communicating quantitative evidence in scientific documents, yet their compact relational form makes robust interpretation depend on layout and context. However, existing multi-page and multi-table benchmarks may still allow stochastic shortcut search, where models locate local answer-bearing regions without systematically navigating the underlying table structure. To address this gap, we introduce PubTables-QA, a multi-page TableVQA benchmark dataset carefully designed to audit cross-page visual reasoning in authentic academic documents. Built upon the extensive structural table annotations of PubTables-v2, PubTables-QA introduces 2,106 table-based QA pairs, where each question is designed to be unanswerable from a single page alone. This design compels models to reconstruct cross-page table continuity across headers, rows, columns, and cells, or to integrate evidence distributed across multiple tables. PubTables-QA further provides a three-level fine-grained QA taxonomy, Document, Table, and Cell/column levels respectively, ranging from relevant evidence localization and table-level structural understanding, to precise cell/column-level value extraction. Experiments on recent MLLMs including GPT-4o, Gemini-2.5-Pro, and Gemma-4 reveal that even the strongest models achieve only 32.6% overall accuracy. Further, our evidence-aware evaluation confirms that, unlike existing multi-page benchmarks susceptible to stochastic shortcuts, PubTables-QA requires genuine cross-page evidence integration, establishing it as a substantial and unsolved challenge.

• Choi, Changkyu; Tan, Martine Hjelkrem; Aasan, Marius & Rivera, Adín Ramírez (2026). AI in Our Daily Lives - Panel Discussion. Full text in Research Archive Show summary

  AURoRA researchers co-hosted a public seminar and panel discussion with the Embassy of the Republic of Korea to Norway, held at Litteraturhuset in Oslo as part of the K-Initiative campaign. The evening opened with the talk "Mirrors of Intelligence" by Adín Ramírez Rivera (University of Oslo). The panel then discussed trust, fairness and the societal impact of AI with the audience.

• Rivera, Adín Ramírez & Choi, Changkyu (2026). Mirrors of Intelligence. Full text in Research Archive Show summary

  The lecture focuses on how AI systems are built, what they reflect from human data, and why their outputs require critical judgment. In addition, it explains the development of LLMs and VLMs through large-scale data collection, pre-training, fine-tuning, and human-feedback-based alignment. It highlights both the strengths of AI, such as summarizing well-established knowledge and generating standard code, and its limitations, including biases, hallucinations, and sycophancy. The main takeaway is to use AI as an augmentative tool rather than a replacement for human judgment, while remaining aware of bias, questioning plausible but false information, and developing AI literacy as an essential skill.

• Choi, Changkyu (2026). Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models. doi: https:/mishmash.no/assets/events/kristiansand2026/mishmash-opening-conference-2026-abstracts.pdf. Full text in Research Archive Show summary

  Dual-encoder vision-language models systematically fail at negation, scoring "a cat without a dog"' nearly identically to "a cat with a dog.'' We show that this failure is inherent to the single-embedding framework. Under contrastive training, single-embedding methods can at best ignore the negated concept (dilution) but never penalize its presence (rejection). We propose Know-Thy-Not (KTN), a training-free method that introduces an image-side rejection dimension to penalize the negated concept directly in the visual representation. We also introduce COCO-Confusion, the first benchmark requiring genuine rejection to succeed. Across 9 backbones, KTN improves COCO-Confusion mAP by up to +25.6 over vanilla VLMs and +9.6 over the strongest existing method.

• Wetzer, Elisabeth; Choi, Changkyu; Jenssen, Robert; Handegard, Nils Olav & Ebbesson, Lars O.E. (2026). Artificial Intelligence for Sustainable Fisheries: Methods, Monitoring, and Practice. doi: https:/virtual.oxfordabstracts.com/event/75716/program?date=%222026-5-27%22&g_submission=129&g_session=215457. Full text in Research Archive Show summary

  Arctic and Subarctic fisheries face increasing pressure as traditional practices struggle to keep pace with shifting ocean conditions. These challenges call for intelligent, adaptive systems to support sustainable monitoring and management. This session explores how Artificial Intelligence (AI) can contribute to the future of fisheries science by bridging algorithmic innovation with practical application and fostering interdisciplinary exchange. We invite papers that demonstrate how AI, ranging from machine learning and computer vision to emerging LLM-based agentic systems, can support fisheries research and regulation. Relevant applications include acoustic data interpretation for species identification, improved stock assessments, vessel activity monitoring, and adaptive regulatory strategies. Submissions addressing challenges such as limited computational resources, sparse data availability, or operational constraints in remote polar environments are particularly encouraged. The session also highlights the importance of engaging Indigenous knowledge holders and coastal communities in AI development. We especially welcome co-designed frameworks that promote fairness, transparency, and local agency in contexts where environmental data and governance intersect. Through this session, we aim to bring together diverse perspectives that advance responsible and locally grounded uses of AI for sustainable marine stewardship.

• Halnes, Geir; Javaid, Anam; Solvang, Michael; Nichele, Stefano; Riegler, Michael & Singstad, Bjørn-Jostein [Show all 82 contributors for this article] (2025). Proceedings of NORA’s annual conference 2025. Nordic Machine Intelligence (NMI). 5(3), p. 1–14. doi: 10.5617/nmi.12551. Full text in Research Archive
• Choi, Changkyu (2025). Formulating Explainability Principles for a Self-Explainable Model Through Information Theoretic Learning. Full text in Research Archive
• Indrehus, Kjetil; Choi, Changkyu & Ramezani-Kebrya, Ali (2025). Information-Theoretic Approach to Learn Causal Relationship Between Question and Answering in DocVQA. Full text in Research Archive
• Subramaniam, Arangan; Choi, Changkyu; Handegard, Nils Olav; Jenssen, Robert & Ramezani-Kebrya, Ali (2025). Marine Intelligence: Innovations to Enhance Underwater Exploration. Full text in Research Archive
• Choi, Changkyu & Aasan, Marius (2024). Data Vaccination: Balancing Data Security and Utility. Lov & Data. ISSN 0800-7853. Full text in Research Archive
• Rivera, Adín Ramírez; Thomas, Sarina & Choi, Changkyu (2024). Unsupervised Learning of visual feature hierarchy by contrasting hierarchical topic assignments. Full text in Research Archive
• Kim, Hyeongji; Choi, Changkyu; Kampffmeyer, Michael Christian; Berge, Terje; Parviainen, Pekka & Malde, Ketil (2024). ProxyDR: Deep Hyperspherical Metric Learning with Distance Ratio-Based Formulation. Full text in Research Archive
• Kim, Hyeongji; Choi, Changkyu; Kampffmeyer, Michael Christian; Berge, Terje; Parviainen, Pekka & Malde, Ketil (2024). ProxyDR: Deep Hyperspherical Metric Learning with Distance Ratio-Based Formulation. Full text in Research Archive
• Choi, Changkyu; Thomas, Sarina & Rivera, Adín Ramírez (2024). Unsupervised learning of visual feature hierarchy by contrasting hierarchical topic assignments. Full text in Research Archive
• Choi, Changkyu (2023). Introduction to generative AI. Full text in Research Archive Show summary

  Generative AI, extending beyond the realm of deep learning, has sparked widespread interest and curiosity about its impact. This presentation explores the evolution of generative AI in image and text generation, and delves into its practical applications in the real world. The lecture was presented to members of the KSSEA General Assembly, an annual gathering of Korean scientists and engineers from Scandinavian countries. This year's event took place at Copenhagen University in Denmark on October 21, 2023.

• Choi, Changkyu (2023). Recent Deep Learning Models and their Applicability in the Marine Domain Now and in the Near Future. Full text in Research Archive
• Choi, Changkyu (2023). Deep learning landscape 4: Transformers and their applicability in the marine domain now and in the near future. Full text in Research Archive Show summary

  Changkyu Choi, a postdoctoral researcher with SFI Visual Intelligence, delivered four lectures on deep learning at Havforskningsintsituttet during his research stay. His presentations covered both the fundamentals of deep learning and its specific applications within the marine sector.

• Choi, Changkyu (2023). Recent Deep Learning Models and their Applicability in the Marine Domain Now and in the Near Future. Full text in Research Archive Show summary

  A Visual Intelligence postdoctoral researcher Changky Choi provided a talk entitled "Recent Deep Learning Models and their Applicability in the Marine Domain Now and in the Near Future" (14 Sep. 2023). Recording available on: https://youtu.be/oSSlL3GQQLA?si=qlFr911qBl_GhB6x Abstract: In the wake of Large Language Models (LLM), the Transformer model, distinguished by its attention-based learning mechanism, has gained prominence. Originally known for its proficiency in natural language processing, its utility has expanded to computer vision. This presentation will discuss how Transformer models function and their applications in the marine domain. We will further explore current implementations and envision future possibilities, emphasizing the profound impact of deep learning on marine research and endeavors.

• Choi, Changkyu (2023). Deep learning landscape 1: Neural networks basic. Full text in Research Archive Show summary

  Changkyu Choi, a postdoctoral researcher with SFI Visual Intelligence, delivered four lectures on deep learning at Havforskningsintsituttet during his research stay. His presentations covered both the fundamentals of deep learning and its specific applications within the marine sector.

• Choi, Changkyu (2023). Deep learning landscape 2: Image tasks and CNNs. Full text in Research Archive Show summary

  Changkyu Choi, a postdoctoral researcher with SFI Visual Intelligence, delivered four lectures on deep learning at Havforskningsintsituttet during his research stay. His presentations covered both the fundamentals of deep learning and its specific applications within the marine sector.

• Choi, Changkyu (2023). DIB-X: Formulating Explainability Principles for a Self-explainable Model through Information Theoretic Learning. Full text in Research Archive Show summary

  Changkyu Choi, a postdoctoral researcher with SFI Visual Intelligence, delivered a presentation of his work at University of Bergen.

• Choi, Changkyu; Kampffmeyer, Michael Christian; Handegard, Nils Olav; Salberg, Arnt-Børre & Jenssen, Robert (2023). Deep Semi-supervised Semantic Segmentation in Multi-frequency Echosounder Data. Full text in Research Archive Show summary

  The fully supervised approaches achieve good performance provided that high-quality training data and an appropriate choice for the prediction model are assured. However, it is highly challenging for the echosounder data to obtain the class annotation for each backscattering intensity pixel because this relies on the manual annotation process, which is expensive and error-prone. As a key step in this direction, we propose a novel deep semi-supervised semantic segmentation method that efficient- ly uses a small amount of manually annotated data by com- bining it with a large amount of readily available unannotated data in the learning process. Poster presentation at VI days 2023. The original work is published in IEEE Journal of Oceanic Engineering DOI: 10.1109/JOE.2022.3226214.

• Choi, Changkyu (2023). Deep learning landscape 3: Explainable deep learning. Full text in Research Archive Show summary

  Changkyu Choi, a postdoctoral researcher with SFI Visual Intelligence, delivered four lectures on deep learning at Havforskningsintsituttet during his research stay. His presentations covered both the fundamentals of deep learning and its specific applications within the marine sector.

• Choi, Changkyu (2022). Information Bottleneck-based Interpretability Method in Marine Acoustic Data. Full text in Research Archive
• Choi, Changkyu; Yu, Shujian; Kampffmeyer, Michael; Salberg, Arnt-Børre; Handegard, Nils Olav & Salahuddin, Suaiba Amina [Show all 7 contributors for this article] (2022). Explaining Marine Acoustic Target Classification in Multi-channel Echosounder Data using Self-attention Mask, Information-Bottleneck, and Mask Prior. Full text in Research Archive
• Choi, Changkyu (2022). Segmenting Multi-frequency Marine Acoustic Data in a Semi-supervised Fashion. Full text in Research Archive
• Handegard, Nils Olav; Brautaset, Olav; Choi, Changkyu; Furmanek, Tomasz; Hestnes, Arne Johan & Johnsen, Espen [Show all 10 contributors for this article] (2022). Developing and deploying machine learning methods for acoustic data. Full text in Research Archive
• Choi, Changkyu (2021). Marine Vision: When Visual Intelligence meets Marine Science. Full text in Research Archive Show summary

  The innovation power of deep learning and computer vision now reaches marine science. This seminar introduces you to the recent achievement of deep learning in marine science, especially in analyzing the echosounder data. Changkyu Choi is a PhD student in UiT Machine Learning Group and SFI Visual Intelligence, working for novel deep learning methods that bridge computer vision to marine science. His work is also closely collaborated with the stakeholders of SFI Visual Intelligence, e.g., Institute of Marine Research(Havforsknings instituttet) and Norwegian Computing Center (Norsk Regnesentral). The presentation is available on SFI Visual Intelligence YouTube channel (https://youtu.be/-gTq0Lz8afE).

• Choi, Changkyu (2021). Semi-supervised Semantic Segmentation in Multi-frequency Echosounder Data. Full text in Research Archive Show summary

  Multi-frequency echosounder data has been considered as a valuable source of information for the investigation of the underwater environment, such as the benthic habitat observation, and the marine organism detection of various species of fish and plankton. Recent studies show that analysis models of the echosounder data based on supervised deep learning outperform the conventional methods by applying semantic segmentation to the echosounder data. However, there is still a severe limitation on these methods since they heavily depend on the annotation of the data, where the annotation process is manual and requires domain knowledge so that it is extremely expensive in time and cost. To tackle the limitation, we propose a novel semi-supervised deep learning model designed for semantic segmentation that leverages a few annotated data samples together with vast amounts of unannotated data samples, all in a single model. Specifically, two inter-connected losses, namely a self-supervised loss and a supervised loss, optimize one shared convolutional neural network in an alternating manner, where the self-supervised loss exploits the underlying structure of the data, while the supervised loss enforces a certain consistency to the given classes using the labeled data samples. We also provide the analysis based on the preliminary results of our method acquired with a down-scaled dataset, and discuss the insight based on the analysis. The presentation is also available online via YouTube channel of UiT Machine Learning Group (https://youtu.be/LxH8qV15AbQ).

• Choi, Changkyu; Kampffmeyer, Michael; Handegard, Nils Olav; Salberg, Arnt Børre; Eikvil, Line & Jenssen, Robert (2021). Semi-supervised Semantic Segmentation in Multi-frequency Echosounder Data. Full text in Research Archive
• Choi, Changkyu (2021). Deep Semi-supervised Target Classification in Multi-frequency Echosounder Data. Full text in Research Archive Show summary

  Acoustic target classification is a field of research that analyzes the marine acoustic data for the marine ecosystem and fishery management, and within this field, the analysis task of multi-frequency echosounder data is a major interest. The goal is to assign an observed acoustic backscattering intensity to a given acoustic category. The results can be used to estimate the abundance or biomass of the species. One common approach for acoustic target classification is manual categorization, where the operators identify and select regions with similar acoustic properties. However, due to its extremely high cost and the vulnerability to operators' bias, the manual process has limited applicability. Automated and scalable analysis methods are required to cope with the multi-frequency data efficiently. We propose a novel deep semi-supervised learning algorithm for acoustic target classification, which operates when the data is partially annotated. The novelty of our work is that the proposed algorithm exploits the underlying structure of the data including both the annotated part and the unannotated part using two interconnected objective functions, namely a clustering objective and a classification objective. The alternating optimization process of the two objective functions is applied to a single convolutional neural network, allowing the unannotated part of the data to contribute to the decision boundaries with respect to the given classes, which is not applicable for a common supervised deep learning. This method can be applied to the echosounder data as well as being potentially generalized to other data sources since it incorporates the generic idea of deep clustering into supervised deep learning. Extensive experiments conducted on the echosounder data validate the robustness of the proposed method for the patch-based classification and semantic segmentation tasks. The multi-frequency echosounder data used in this study have been annually collected at the North Sea by the Norwegian Institute of Marine Research for the case study of classifying lesser sandeel Ammodytes marinus, a small fish without a swim bladder. The contributions of this work are (1) to develop a {novel} semi-supervised deep learning algorithm that is suitable for segmenting and classifying echosounder data without prior information, and (2) to demonstrate the proposed algorithm on a real test case.

• Handegard, Nils Olav & Choi, Changkyu (2021). Arbeidet hans sparer forskerne for store ressurser, og åpner for et selvstyrt fiskeri. [Journal]. https://www.fiskeribladet.no/. Full text in Research Archive Show summary

  I fremtiden kan det bli langt lettere å bruke kunstig intelligens til å finne riktig fangst. Det sørger en helt ny algoritme for.

• Choi, Changkyu; Kampffmeyer, Michael & Jenssen, Robert (2020). A Robustness Analysis of Personalized Propagation of Neural Prediction. Full text in Research Archive
• Choi, Changkyu (2020). Semisupervised target classification in multifrequency echosounder data. Full text in Research Archive
• Choi, Changkyu; Bianchi, Filippo Maria; Kampffmeyer, Michael & Jenssen, Robert (2020). Short-Term Load Forecasting with Missing Data using Dilated Recurrent Attention Networks. Full text in Research Archive Show summary

  Forecasting the dynamics of time-varying systems is essential to maintaining the sustainability of the sys- tems. Recent studies have discovered that Recurrent Neural Networks (RNN) applied in the forecasting tasks outperform conventional models. However, due to the structural limitation of vanilla RNN which holds unit-length internal connections, learning the representation of time series with missing data can be severely biased. We propose Dilated Recurrent Attention Networks (DRAN), a robust architecture against the bias from missing data. This has a stacked structure of mul- tiple RNNs, with each layer leveraging a different length of internal connections to incorporate previ- ous information at different time scales, and updates its output state by a weighted average of the states in the layers. In order to focus more on specific layers that carries reliable information against missing data bias, our model leverages attention mechanism which learns the distribution of attention weights among the layers. The proposed model achieves a higher forecast accuracy than conventional ones from two benchmark time series with missing data that include a real-world electricity load dataset.

• Handegard, Nils Olav; Andersen, Lars Nonboe; Brautaset, Olav; Choi, Changkyu; Eliassen, Inge Kristian & Heggelund, Yngve [Show all 15 contributors for this article] (2021). Fisheries acoustics and Acoustic Target Classification - Report from the COGMAR/CRIMAC workshop on machine learning methods in fisheries acoustics. Havforskningsinstituttet. Full text in Research Archive Show summary

  This report documents a workshop organised by the COGMAR and CRIMAC projects. The objective of the workshop was twofold. The first objective was to give an overview of ongoing work using machine learning for Acoustic Target Classification (ATC). Machine learning methods, and in particular deep learning models, are currently being used across a range of different fields, including ATC. The objective was to give an overview of the status of the work. The second objective was to familiarise participants with machine learning background to fisheries acoustics and to discuss a way forward towards a standard framework for sharing data and code. This includes data standards, standard processing steps and algorithms for efficient access to data for machine learning frameworks. The results from the discussion contributes to the process in ICES for developing a community standard for fisheries acoustics data.

View all works in NVA