Trym Haavardsholm

• Haavardsholm, Trym Vegard; Opsahl, Thomas Olsvik; Skauli, Torbjørn & Stahl, Annette (2022). Compact multimodal multispectral sensor system for tactical reconnaissance. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. doi: 10.1117/12.2632701. Fulltekst i vitenarkiv Vis sammendrag

  Multispectral imaging is an attractive sensing modality for small unmanned aerial vehicles (UAVs) in numerous military and civilian applications such as reconnaissance, target detection, and precision agriculture. Cameras based on patterned filters in the focal plane, such as conventional colour cameras, represent the most compact architecture for spectral imaging, but image reconstruction becomes challenging at higher band counts. We consider a camera configuration where six bandpass filters are arranged in a periodically repeating pattern in the focal plane. In addition, a large unfiltered region permits conventional monochromatic video imaging that can be used for situational awareness (SA), including estimating the camera motion and the 3D structure of the ground surface. By platform movement, the filters are scanned over the scene, capturing an irregular pattern of spectral samples of the ground surface. Through estimation of the camera trajectory and 3D scene structure, it is still possible to assemble a spectral image by fusing all measurements in software. The repeated sampling of bands enables spectral consistency testing, which can improve spectral integrity significantly. The result is a truly multimodal camera sensor system able to produce a range of image products. Here, we investigate its application in tactical reconnaissance by pushing towards on-board real-time spectral reconstruction based on visual odometry (VO) and full 3D reconstruction of the scene. The results are compared with offline processing based on estimates from visual simultaneous localisation and mapping (VSLAM) and indicate that the multimodal sensing concept has a clear potential for use in tactical reconnaissance scenarios.

• Larsen, Martin Vonheim; Haavardsholm, Trym Vegard & Mathiassen, Kim (2020). In-operation calibration of clock-bias and intrinsic parameters for pan-tilt-zoom cameras based on keypoint tracking. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. doi: 10.1117/12.2573611. Fulltekst i vitenarkiv Vis sammendrag

  We propose a method for jointly estimating intrinsic calibration and internal clock synchronisation for a pantilt- zoom (PTZ) camera using only data that can be acquired in the field during normal operation. Results show that this method is a promising starting point towards using software to replace costly timing hardware in such cameras. Through experiments we provide calibration and clock synchronisation for an off-the-shelf low-cost PTZ camera, and observe a greatly improved directional accuracy, even during mild manoeuvres.

• Haavardsholm, Trym Vegard; Skauli, Torbjørn & Stahl, Annette (2020). Multimodal Multispectral Imaging System for Small UAVs. IEEE Robotics and Automation Letters. ISSN 2377-3766. 5(2). doi: 10.1109/LRA.2020.2967301. Fulltekst i vitenarkiv Vis sammendrag

  Multispectral imaging is an attractive sensing modality for small unmanned aerial vehicles (UAVs) in numerous applications. The most compact spectral camera architecture is based on spectral filters in the focal plane. Vehicle movement can be used to scan the scene using multiple bandpass filters arranged perpendicular to the flight direction. With known camera trajectory and scene structure, it is possible to assemble a spectral image in software. In this letter, we demonstrate the feasibility of a novel concept for low-cost wide area multispectral imaging with integrated spectral consistency testing. Six bandpass filters are arranged in a periodically repeating pattern. Since different bands are recorded at different times and in different viewing directions, there is a risk of obtaining spectral artifacts in the image. We exploit the repeated sampling of bands to enable spectral consistency testing, which leads to significantly improved spectral integrity. In addition, an unfiltered region permits conventional 2D video imaging that can be used for image-based navigation and 3D reconstruction. The proposed multimodal imaging system was tested on a UAV in a realistic experiment. The results demonstrate that spectral reconstruction and consistency testing can be performed by image processing alone, based on visual simultaneous localization and mapping (VSLAM).

• Shimoni, Michal; Idrissa, Mahamadou; Borghys, Dirk; Haavardsholm, Trym Vegard; Opsahl, Thomas Olsvik & Perneel, Christiaan (2017). Urban features classification using 3D hyperspectral data. Workshop on Hyperspectral Image and Signal Processing, Evolution in Remote Sensing. ISSN 2158-6276. 2013-June, s. 1–4. doi: 10.1109/WHISPERS.2013.8080665. Fulltekst i vitenarkiv
• Morrison, Aiden; Sokolova, Nadezda; Haavardsholm, Trym Vegard; Hagen, Ove Kent; Opsahl, Thomas Olsvik & Ånonsen, Kjetil Bergh [Vis alle 7 forfattere av denne artikkelen] (2017). Collaborative indoor navigation for emergency services personnel. IEEE Aerospace Conference. Proceedings. ISSN 1095-323X. 2017-June. doi: 10.1109/AERO.2017.7943729. Fulltekst i vitenarkiv Vis sammendrag

  First responders and other emergency services personnel must often enter buildings which prevent the use of GPS or other satellite navigation signals for positioning. Loss of navigation capability combined with the fact that the buildings are often unknown to the personnel in question makes it more difficult for individual team members to coordinate with one another, and difficult or impossible for the team leader to monitor and direct the actions of each team member. While inertial navigation or pedestrian dead reckoning provide for some degree of navigation in GPS signal denied environments, these solutions degrade with time and may require prohibitively large and expensive inertial solutions to navigate over extended periods, while also allowing each individual user to accumulate independent positioning errors and thereby appearing to ‘drift away’ from one another. This paper presents an implementation of a collaborative navigation system utilizing each of user-to-user radio links, Global Navigation Satellite Systems (GNSS) when available, inertial navigation, pedestrian dead reckoning, as well as camera based Simultaneous Location and Mapping (SLAM) to provide a team of users with absolute and relative situational awareness for themselves and their team. The application of collaborative navigation to such a team provides the triple benefits of providing improved absolute navigation accuracy, improved relative navigation accuracy, and greatly enhanced situational awareness for all cooperating team members.

• Axelsson, Maria; Friman, Ola; Haavardsholm, Trym Vegard & Renhorn, Ingmar (2016). Target detection in hyperspectral imagery using forward modeling and in-scene information. ISPRS Journal of Photogrammetry and Remote Sensing (P&RS). ISSN 0924-2716. 119, s. 124–134. doi: 10.1016/j.isprsjprs.2016.05.008. Fulltekst i vitenarkiv
• Haavardsholm, Trym Vegard; Smestad, Ragnar; Larsen, Martin Vonheim; Thoresen, Marius & Dyrdal, Idar (2016). Scene Understanding for Autonomous Steering, STO-MP-IST-127: Intelligence and Autonomy In Robotics. NATO Science and Technology Organization. ISSN 9789283720683. Fulltekst i vitenarkiv
• Torkildsen, Hans Erling; Haavardsholm, Trym Vegard; Opsahl, Thomas Olsvik; Datta, Urmila; Skaugen, Atle & Skauli, Torbjørn (2016). Compact multispectral multi-camera imaging system for small UAVs. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. doi: 10.1117/12.2224495. Fulltekst i vitenarkiv Vis sammendrag

  Cameras with filters in the focal plane provide the most compact solution for multispectral imaging. A small UAV can carry multiple such cameras, providing large area coverage rate at high spatial resolution. We investigate a camera concept where a patterned bandpass filter with six bands provides multiple interspersed recordings of all bands, enabling consistency checks for improved spectral integrity. A compact sensor payload has been built with multiple cameras and a data acquisition computer. Recorded imagery demonstrates the potential for large area coverage with good spectral integrity. © (2016) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.

• Skauli, Torbjørn; Torkildsen, Hans Erling; Nicolas, Stephane; Opsahl, Thomas Olsvik; Haavardsholm, Trym Vegard & Kåsen, Ingebjørg [Vis alle 7 forfattere av denne artikkelen] (2014). Compact camera for multispectral and conventional imaging based on patterned filters. Applied Optics. ISSN 1559-128X. doi: 10.1364/AO.53.000C64. Fulltekst i vitenarkiv Vis sammendrag

  A multispectral camera concept is presented. The concept is based on using a patterned filter in the focal plane, combined with scanning of the field of view. The filter layout has stripes of different bandpass filters extending orthogonally to the scan direction. The pattern of filter stripes is such that all bands are sampled multiple times, while minimizing the total duration of the sampling of a given scene point. As a consequence, the filter needs only a small part of the area of an image sensor. The remaining area can be used for conventional 2D imaging. A demonstrator camera has been built with six bands in the visible and near infrared, as well as a panchromatic 2D imaging capability. Image recording and reconstruction is demonstrated, but the quality of image reconstruction is expected to be a main challenge for systems based on this concept. An important advantage is that the camera can potentially be made very compact, and also low cost. It is shown that under assumptions that are not unreasonable, the proposed camera concept can be much smaller than a conventional imaging spectrometer. In principle, it can be smaller in volume by a factor on the order of several hundred while collecting the same amount of light per multispectral band. This makes the proposed camera concept very interesting for small airborne platforms and other applications requiring compact spectral imagers.

• Torkildsen, Hans Erling; Hovland, Harald; Opsahl, Thomas Olsvik; Haavardsholm, Trym Vegard; Nicolas, Stephane & Skauli, Torbjørn (2014). Characterization of a compact 6-band multifunctional camera based on patterned spectral filters in the focal plane. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. doi: 10.1117/12.2054477. Fulltekst i vitenarkiv Vis sammendrag

  In some applications of multi- or hyperspectral imaging, it is important to have a compact sensor. The most compact spectral imaging sensors are based on spectral filtering in the focal plane. For hyperspectral imaging, it has been proposed to use a "linearly variable" bandpass filter in the focal plane, combined with scanning of the field of view. As the image of a given object in the scene moves across the field of view, it is observed through parts of the filter with varying center wavelength, and a complete spectrum can be assembled. However if the radiance received from the object varies with viewing angle, or with time, then the reconstructed spectrum will be distorted. We describe a camera design where this hyperspectral functionality is traded for multispectral imaging with better spectral integrity. Spectral distortion is minimized by using a patterned filter with 6 bands arranged close together, so that a scene object is seen by each spectral band in rapid succession and with minimal change in viewing angle. The set of 6 bands is repeated 4 times so that the spectral data can be checked for internal consistency. Still the total extent of the filter in the scan direction is small. Therefore the remainder of the image sensor can be used for conventional imaging with potential for using motion tracking and 3D reconstruction to support the spectral imaging function. We show detailed characterization of the point spread function of the camera, demonstrating the importance of such characterization as a basis for image reconstruction. A simplified image reconstruction based on feature-based image coregistration is shown to yield reasonable results. Elimination of spectral artifacts due to scene motion is demonstrated.

• Ringaby, Erik; Friman, Ola; Forssén, Per-Erik; Opsahl, Thomas Olsvik; Haavardsholm, Trym Vegard & Kåsen, Ingebjørg (2014). Anisotropic scattered data interpolation for pushbroom image rectification. IEEE Transactions on Image Processing. ISSN 1057-7149. doi: 10.1109/TIP.2014.2316377. Fulltekst i vitenarkiv Vis sammendrag

  This paper deals with fast and accurate visualization of pushbroom image data from airborne and spaceborne platforms. A pushbroom sensor acquires images in a line-scanning fashion, and this results in scattered input data that need to be resampled onto a uniform grid for geometrically correct visualization. To this end, we model the anisotropic spatial dependence structure caused by the acquisition process. Several methods for scattered data interpolation are then adapted to handle the induced anisotropic metric and compared for the pushbroom image rectification problem. A trick that exploits the semiordered line structure of pushbroom data to improve the computational complexity several orders of magnitude is also presented.

• Opsahl, Thomas Olsvik & Haavardsholm, Trym Vegard (2013). Estimating the pixel footprint distribution for image fusion by ray tracing lines of sight in a Monte Carlo scheme. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. doi: 10.1117/12.2015746. Fulltekst i vitenarkiv Vis sammendrag

  Images from airborne cameras can be a valuable resource for data fusion, but this typically requires them to be georeferenced. This usually implies that the information of every pixel should be accompanied by a single geographical position describing where the center of the pixel is located in the scene. This geospatial information is well suited for tasks like target positioning and orthorectification. But when it comes to fusion, a detailed description of the area on the ground contributing to the pixel signal would be preferable over a single position. In this paper we present a method for estimating these regions. Simple Monte Carlo simulations are used to combine the influences of the main geometrical aspects of the imaging process, such as the point spread function, the camera’s motion and the topography in the scene. Since estimates of the camera motion are uncertain to some degree, this is incorporated in the simulations as well. For every simulation, a pixel’s sampling point in the scene is estimated by intersecting a randomly sampled line of sight with a 3D-model of the scene. Based on the results of numerous simulations, the pixel’s sampling region can be represented by a suitable probability distribution. This will be referred to as the pixel’s footprint distribution (PFD). We present results for high resolution hyperspectral pushbroom images of an urban scene.

• Palm, Hans Christian; Haavardsholm, Trym Vegard; Ajer, Halvor & Jensen, Cathrine V. (2013). Extraction and classification of vehicles in LADAR imagery. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. doi: 10.1117/12.2015363. Fulltekst i vitenarkiv Vis sammendrag

  The work presented in this paper is based on a dataset recorded with an airborne sensor. It comprises targets like M-60, M-47, M-113, bridge layers, tank retrievers, and trucks in various types of scenes. The background-object segmentation consists of first estimating the ground level everywhere in the scene, and then for each sample simply subtracting the measured height and ground level height. No assumptions concerning flat terrain etc. are made. Samples with height above ground level higher than a certain threshold are clustered by utilizing a straightforward agglomerative clustering algorithm. Around each cluster the bounding box with minimum volume is determined. Based on these bounding boxes, too small as well as too large clusters can easily be removed. However, vehicle-sized clutter will not be removed. Clutter detection is based on estimating the normal vector for a plane approximation around each sample. This approach is based on the fact that the surface normals of a vehicle is more “modulo 90°” distributed than clutter. The aim of the classification has been to classify main battle tanks (MBTs) Two types of algorithms have been studied, one based on Dempster Shafer fusion theory, and one model based. Our dataset comprises clusters of 269 vehicles (among them 131 MBTs), and 253 clutter objects (i.e. in practice vehiclesized bushes). The experiments we have carried out show that the segmentation extracts all vehicles, the clutter detection removes 90% of the clutter, and the classification finds more than 95% of the MBTs as well as removes half of the remaining clutter.

• Breivik, Gøril Margrethe; Løkken, Kristin Hammarstrøm; Brattli, Alvin Andreas; Palm, Hans Christian & Haavardsholm, Trym Vegard (2013). An ATR architecture for algorithm development and testing. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. doi: 10.1117/12.2015369. Fulltekst i vitenarkiv Vis sammendrag

  A research platform with four cameras in the infrared and visible spectral domains is under development at the Norwegian Defence Research Establishment (FFI). The platform will be mounted on a high-speed jet aircraft and will primarily be used for image acquisition and for development and test of automatic target recognition (ATR) algorithms. The sensors on board produce large amounts of data, the algorithms can be computationally intensive and the data processing is complex. This puts great demands on the system architecture; it has to run in real-time and at the same time be suitable for algorithm development. In this paper we present an architecture for ATR systems that is designed to be exible, generic and efficient. The architecture is module based so that certain parts, e.g. specific ATR algorithms, can be exchanged without affecting the rest of the system. The modules are generic and can be used in various ATR system configurations. A software framework in C++ that handles large data flows in non-linear pipelines is used for implementation. The framework exploits several levels of parallelism and lets the hardware processing capacity be fully utilised. The ATR system is under development and has reached a first level that can be used for segmentation algorithm development and testing. The implemented system consists of several modules, and although their content is still limited, the segmentation module includes two different segmentation algorithms that can be easily exchanged. We demonstrate the system by applying the two segmentation algorithms to infrared images from sea trial recordings.

• Opsahl, Thomas Olsvik; Haavardsholm, Trym Vegard & Winjum, Ingebrigt (2011). Real-time georeferencing for an airborne hyperspectral imaging system. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. doi: 10.1117/12.885069. Fulltekst i vitenarkiv Vis sammendrag

  The paper describes the georeferencing part of an airborne hyperspectral imaging system based on pushbroom scanning. Using ray-tracing methods from computer graphics and a highly efficient representation of the digital elevation model (DEM), georeferencing of high resolution pushbroom images runs in real time by a large margin. By adapting the georeferencing to match the DEM resolution, the camera field of view and the flight altitude, the method has potential to provide real time georeferencing, even for HD video on a high resolution DEM when a graphics processing unit (GPU) is used for processing.

• Farsund, Øystein; Rustad, Gunnar; Kåsen, Ingebjørg & Haavardsholm, Trym Vegard (2010). Required Spectral Resolution for Bioaerosol Detection Algorithms Using Standoff Laser-Induced Fluorescence Measurements. IEEE Sensors Journal. ISSN 1530-437X. 10(3), s. 655–661. doi: 10.1109/JSEN.2009.2037794. Fulltekst i vitenarkiv Vis sammendrag

  We have developed and tested a standoff biological aerosol detection demonstrator employing ultraviolet laser-induced fluorescence. It is based on commercially available components including a pulsed 355-nm laser and an intensified charge-coupled device camera. Biological warfare simulants and interferents were released and measured in open air field and closed-chamber laboratory tests. We analyzed the experimental data at different spectral resolutions, using statistics-based anomaly detection, and spectral angle mapping algorithms. The results show that less than 20 spectral channels in the 350-700-nm spectral region are sufficient in order to discriminate between the agents released using these methods. This corresponds to sacrificing high spectral resolution for the benefit of more photons in each channel and reduced computation time.

• Skauli, Torbjørn; Haavardsholm, Trym Vegard; Kåsen, Ingebjørg; Arisholm, Gunnar; Kavara, Amela & Opsahl, Thomas Olsvik [Vis alle 7 forfattere av denne artikkelen] (2010). An airborne real-time hyperspectral target detection system. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. 7695. doi: 10.1117/12.850443. Fulltekst i vitenarkiv Vis sammendrag

  An airborne system for hyperspectral target detection is described. The main sensor is a HySpex pushbroom hyperspectral imager for the visible and near-infrared spectral range with 1600 pixels across track, supplemented by a panchromatic line imager. An optional third sensor can be added, either a SWIR hyperspectral camera or a thermal camera. In real time, the system performs radiometric calibration and georeferencing of the images, followed by image processing for target detection and visualization. The current version of the system implements only spectral anomaly detection, based on normal mixture models. Image processing runs on a PC with a multicore Intel processor and an Nvidia graphics processing unit (GPU). The processing runs in a software framework optimized for large sustained data rates. The platform is a Cessna 172 aircraft based close to FFI, modified with a camera port in the floor.

• Jonsson, Per; Elmqvist, Magnus; Gustafsson, Ove; Kullander, Fredrik; Persson, Rolf & Olofsson, Göran [Vis alle 10 forfattere av denne artikkelen] (2009). Evaluation of biological aerosol stand-off detection at a field trial. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. 7484. doi: 10.1117/12.830401. Fulltekst i vitenarkiv Vis sammendrag

  We have performed a field trial to evaluate technologies for stand-off detection of biological aerosols, both in daytime and at night. Several lidar (light detection and ranging) systems were tested in parallel. We present the results from three different lidar systems; one system for detection and localization of aerosol clouds using elastic backscattering at 1.57 μm, and two systems for detection and classification of aerosol using spectral detection of ultraviolet laser-induced fluorescence (UV LIF) excited at 355 nm. The UV lidar systems were utilizing different technologies for the spectral detection, a photomultiplier tube (PMT) array and an intensified charge-coupled device (ICCD), respectively. During the first week of the field trial, the lidar systems were measuring towards a semi-closed chamber at a distance of 230 m. The chamber was built from two docked standard 20-feet containers with air curtains in the short sides to contain the aerosol inside the chamber. Aerosol was generated inside the semi-closed chamber and monitored by reference equipments, e.g. slit sampler and particle counters. Signatures from several biological warfare agent simulants and interferents were measured at different aerosol concentrations. During the second week the aerosol was released in the air and the reference equipments were located in the centre of the test site. The lidar systems were measuring towards the test site centre at distances of either 230 m or approximately 1 km. In this paper we are presenting results and some preliminary signal processing for discrimination between different types of simulants and interference aerosols.

• Tarabalka, Yulija; Haavardsholm, Trym Vegard; Kåsen, Ingebjørg & Skauli, Torbjørn (2009). Real-time anomaly detection in hyperspectral images using multivariate normal mixture models and GPU processing. Journal of Real-Time Image Processing. ISSN 1861-8200. doi: 10.1007/s11554-008-0105-x. Fulltekst i vitenarkiv Vis sammendrag

  Hyperspectral imaging, which records a detailed spectrum of light arriving in each pixel, has many potential uses in remote sensing as well as other application areas. Practical applications will typically require real-time processing of large data volumes recorded by a hyperspectral imager. This paper investigates the use of graphics processing units (GPU) for such real-time processing. In particular, the paper studies a hyperspectral anomaly detection algorithm based on normal mixture modelling of the background spectral distribution, a computationally demanding task relevant to military target detection and numerous other applications. The algorithm parts are analysed with respect to complexity and potential for parallellization. The computationally dominating parts are implemented on an Nvidia GeForce 8800 GPU using the Compute Unified Device Architecture programming interface. GPU computing performance is compared to a multicore central processing unit implementation. Overall, the GPU implementation runs significantly faster, particularly for highly data-parallelizable and arithmetically intensive algorithm parts. For the parts related to covariance computation, the speed gain is less pronounced, probably due to a smaller ratio of arithmetic to memory access. Detection results on an actual data set demonstrate that the total speedup provided by the GPU is sufficient to enable real-time anomaly detection with normal mixture models even for an airborne hyperspectral imager with high spatial and spectral resolution.

• Kåsen, Ingebjørg; Rødningsby, Anders; Haavardsholm, Trym Vegard & Skauli, Torbjørn (2008). Band selection for hyperspectral target-detection based on a multinormal mixture anomaly detection algorithm. Proceedings of SPIE, the International Society for Optical Engineering. ISSN 0277-786X. 6966. doi: 10.1117/12.777758. Fulltekst i vitenarkiv

Se alle arbeider i NVA

• Haavardsholm, Trym Vegard; Opsahl, Thomas Olsvik; Skauli, Torbjørn & Stahl, Annette (2022). Compact multimodal multispectral sensor system for tactical reconnaissance. Fulltekst i vitenarkiv
• Torkildsen, Hans Erling; Haavardsholm, Trym Vegard; Opsahl, Thomas Olsvik; Datta, Urmila; Skaugen, Atle & Skauli, Torbjørn (2018). Compact multispectral multi-camera imaging system for small UAVs. Fulltekst i vitenarkiv
• Morrison, Aiden; Sokolova, Nadezda; Haavardsholm, Trym Vegard; Hagen, Ove Kent; Opsahl, Thomas Olsvik & Ånonsen, Kjetil Bergh [Vis alle 7 forfattere av denne artikkelen] (2017). Collaborative indoor navigation for emergency services personnel. Fulltekst i vitenarkiv
• Morrison, Aiden; Haavardsholm, Trym Vegard; Hagen, Ove Kent; Opsahl, Thomas Olsvik; Ånonsen, Kjetil Bergh & Eriksen, Erik Holthe (2017). Collaborative Indoor Navigation for Emergency Services Personnel. Fulltekst i vitenarkiv
• Thomassen, Jan Brede; Breivik, Gøril Margrethe; Løkken, Kristin Hammarstrøm; Brattli, Alvin Andreas; Palm, Hans Christian & Haavardsholm, Trym Vegard [Vis alle 7 forfattere av denne artikkelen] (2014). The NFU pod: An airborne research platform for algorithm development and testing. Fulltekst i vitenarkiv Vis sammendrag

  In the Norwegian Captive Carry program (Nytt flymåleutstyr, or NFU, in Norwegian) a pod to be carried by a jet aircraft is under construction. The NFU pod is an airborne research platform incorporating infrared cameras, a high quality navigation system, and an onboard real-time processing system. Data from the various sensors can also be recorded for later processing. This system provides us with a realistic environment for hardware conditions for sensors and scenes for analysis. An important activity in the NFU program is to develop and test algorithms for automatic target recognition. In order to achieve this, we have designed a software architecture for image processing, allowing real-time control of the camera platform. The system is built up around a pipeline structure. Each module in the pipeline represents an algorithm, which can easily be exchanged with alternative modules for flexible testing and development. In this way, alternative algorithms can be tested - even in the air. The presentation will focus on the software architecture for image processing. As an example, we will present an algorithm for segmentation of objects in the scene.

• Breivik, Gøril Margrethe; Løkken, Kristin Hammarstrøm; Brattli, Alvin Andreas; Palm, Hans Christian & Haavardsholm, Trym Vegard (2013). An ATR architecture for algorithm development and testing. Fulltekst i vitenarkiv Vis sammendrag

  A research platform with four cameras in the infrared and visible spectral domains is under development at the Norwegian Defence Research Establishment (FFI). The platform will be mounted on a high-speed jet aircraft and will primarily be used for image acquisition and for development and test of automatic target recognition (ATR) algorithms. The sensors on board produce large amounts of data, the algorithms can be computationally intensive and the data processing is complex. This puts great demands on the system architecture; it has to run in real-time and at the same time be suitable for algorithm development. In this paper we present an architecture for ATR systems that is designed to be exible, generic and efficient. The architecture is module based so that certain parts, e.g. specific ATR algorithms, can be exchanged without affecting the rest of the system. The modules are generic and can be used in various ATR system configurations. A software framework in C++ that handles large data fl ows in non-linear pipelines is used for implementation. The framework exploits several levels of parallelism and lets the hardware processing capacity be fully utilised. The ATR system is under development and has reached a first level that can be used for segmentation algorithm development and testing. The implemented system consists of several modules, and although their content is still limited, the segmentation module includes two different segmentation algorithms that can be easily exchanged. We demonstrate the system by applying the two segmentation algorithms to infrared images from sea trial recordings.

• Skauli, Torbjørn; Torkildsen, Hans Erling; Kåsen, Ingebjørg; Nicolas, Stephane; Haavardsholm, Trym Vegard & Opsahl, Thomas Olsvik (2013). Multispectral and conventional imaging combined in a compact camera by using patterned filters in the focal plane. Fulltekst i vitenarkiv
• Haavardsholm, Trym Vegard; Kavara, Amela; Kåsen, Ingebjørg & Skauli, Torbjørn (2012). Improving anomaly detection with Multinormal Mixture Models in shadow. Fulltekst i vitenarkiv
• Resta, Salvatore; Acito, Nicola; Diani, Marco; Corsini, Giovanni; Opsahl, Thomas Olsvik & Haavardsholm, Trym Vegard (2011). Detection of Small Changes in Airborne Hyperspectral Imagery: Experimental Results over Urban Areas. Fulltekst i vitenarkiv
• Skauli, Torbjørn; Haavardsholm, Trym Vegard; Kåsen, Ingebjørg; Opsahl, Thomas Olsvik; Kavara, Amela & Skaugen, Atle (2011). Hyperspectral imaging technology and systems, exemplified by airborne real-time target detection. Fulltekst i vitenarkiv
• Denstedt, Martin Anders F.; Haavardsholm, Trym Vegard; Skauli, Torbjørn & Randeberg, Lise Lyngsnes (2011). A hyperspectral imaging system for real-time analysis of skin – capabilities and prospects. Fulltekst i vitenarkiv
• Haavardsholm, Trym Vegard; Arisholm, Gunnar; Kavara, Amela & Skauli, Torbjørn (2010). Architecture of the real-time target detection processing in an airborne hyperspectral demonstrator system. Fulltekst i vitenarkiv Vis sammendrag

  An airborne demonstrator for real-time hyperspectral target detection has been developed at FFI. The real-time image processing is challenging, not only due to the computational complexity of the algorithms, but also due to the sustained high data rate. A software framework has been designed in C++ to handle large data flows in a nonlinear pipeline architecture. The cross-platform framework enables full exploitation of multicore processors and graphics processing units (GPU), and even distribution among multiple computers. Object oriented design enables flexible reconfiguration of the pipeline. Tests demonstrate sustained real-time performance of complex anomaly detection processing.

• Opsahl, Thomas Olsvik & Haavardsholm, Trym Vegard (2010). Real time direct georeferencing and orthorectification of images from airborne pushbroom cameras. Fulltekst i vitenarkiv
• Haavardsholm, Trym Vegard; Winjum, Ingebrigt & Opsahl, Thomas Olsvik (2010). Real-time direct georeferencing of images from airborne line scan cameras. Fulltekst i vitenarkiv Vis sammendrag

  The Norwegian Defense Research Establishment (FFI) is developing a technology demonstrator for airborne real-time hyperspectral target detection. The system includes two nadir-pointing line scan cameras. The line scanned images are georeferenced in real-time by intersecting rays cast from the cameras with a 3D model of the terrain underneath. The georeferenced images may then easily be ortho-rectified (e.g by using texture mapping in OpenGL) and overlaid digital maps. This poster presents the performance of a cuda implementation of the georeferencing method.

• Randeberg, Lise Lyngsnes; Haavardsholm, Trym Vegard; Arisholm, Gunnar; Kavara, Amela; Løke, Trond & Skauli, Torbjørn (2010). Rapid hyperspectral characterization of skin. Fulltekst i vitenarkiv
• Bakstad, Lorns Harald; Baksaas, Magnus; Mygland, Katrine; Eikanger, Brage Gerdssønn; Gusland, Daniel & Hauge, Mariann [Vis alle 23 forfattere av denne artikkelen] (2025). LandX21 – teknologi i framtidas stridsfelt. Forsvarets forskningsinstitutt . Fulltekst i vitenarkiv Vis sammendrag

  Etter suksessen med LandX i 2020 besluttet FFI å holde LandX som et årlig eksperiment. To uker hver høst skal forskere, brukere og industri samles på Rena for å eksperimentere med teknologi og nye løsninger for Forsvaret. Eksperimentukene avsluttes med en demonstrasjon for inviterte fra hele sektoren. Målet med demonstrasjonen er å formidle forskning til relevante brukere i Forsvaret og demonstrere hvordan ny teknologi kan bidra til å endre framtidas stridsfelt i landdomenet. I 2021 var temaet for LandX situasjonsforståelse, og forskningsprosjekter fra fem forskningsprogrammer ved FFI deltok. Eksperimenteringen dekket alt fra ubemannede og autonome systemer på bakken og i lufta til sensorfusjon, målfølging og robust og sikker kommunikasjon. I tillegg markerte LandX21 åpningen av FFIs nye innovasjonsarena, ICE worx Rena. I denne rapporten oppsummerer vi innholdet og resultatene fra eksperimentene helt overordnet og peker på relevante rapporter som kan utdype teknologien ytterligere. LandX21 var en suksess. Forskningsprogrammene rapporterte i ettertid at de hadde kommet lenger og lært mer enn de hadde gjort om de ikke hadde testet i felt. Tilbakemeldingene fra tilskuerne på demonstrasjonsdagen var at forskningsarbeidet hadde høy relevans, og at det ble formidlet på en god måte. LandX skal arrangeres igjen i 2022. Da skal vi utvide omfanget, og målsetningen er å samarbeide tettere med brukere for å få enda bedre kobling mellom forskningen og operativ bruk.

Se alle arbeider i NVA