Multispectral imaging is an attractive sensing modality for small unmanned aerial vehicles (UAVs) in numerous military and civilian applications such as reconnaissance, target detection, and precision agriculture. Cameras based on patterned filters in the focal plane, such as conventional colour cameras, represent the most compact architecture for spectral imaging, but image reconstruction becomes challenging at higher band counts. We consider a camera configuration where six bandpass filters are arranged in a periodically repeating pattern in the focal plane. In addition, a large unfiltered region permits conventional monochromatic video imaging that can be used for situational awareness (SA), including estimating the camera motion and the 3D structure of the ground surface. By platform movement, the filters are scanned over the scene, capturing an irregular pattern of spectral samples of the ground surface. Through estimation of the camera trajectory and 3D scene structure, it is still possible to assemble a spectral image by fusing all measurements in software. The repeated sampling of bands enables spectral consistency testing, which can improve spectral integrity significantly. The result is a truly multimodal camera sensor system able to produce a range of image products. Here, we investigate its application in tactical reconnaissance by pushing towards on-board real-time spectral reconstruction based on visual odometry (VO) and full 3D reconstruction of the scene. The results are compared with offline processing based on estimates from visual simultaneous localisation and mapping (VSLAM) and indicate that the multimodal sensing concept has a clear potential for use in tactical reconnaissance scenarios.
Morrison, Aiden; Sokolova, Nadezda; Haavardsholm, Trym Vegard; Hagen, Ove Kent; Opsahl, Thomas Olsvik & Ånonsen, Kjetil Bergh
(2017).
Collaborative indoor navigation for emergency services personnel.
IEEE Aerospace Conference. Proceedings.
ISSN 1095-323X.
2017-June.
doi: 10.1109/AERO.2017.7943729.
Vis sammendrag
First responders and other emergency services personnel must often enter buildings which prevent the use of GPS or other satellite navigation signals for positioning. Loss of navigation capability combined with the fact that the buildings are often unknown to the personnel in question makes it more difficult for individual team members to coordinate with one another, and difficult or impossible for the team leader to monitor and direct the actions of each team member. While inertial navigation or pedestrian dead reckoning provide for some degree of navigation in GPS signal denied environments, these solutions degrade with time and may require prohibitively large and expensive inertial solutions to navigate over extended periods, while also allowing each individual user to accumulate independent positioning errors and thereby appearing to ‘drift away’ from one another. This paper presents an implementation of a collaborative navigation system utilizing each of user-to-user radio links, Global Navigation Satellite Systems (GNSS) when available, inertial navigation, pedestrian dead reckoning, as well as camera based Simultaneous Location and Mapping (SLAM) to provide a team of users with absolute and relative situational awareness for themselves and their team. The application of collaborative navigation to such a team provides the triple benefits of providing improved absolute navigation accuracy, improved relative navigation accuracy, and greatly enhanced situational awareness for all cooperating team members.
Ringaby, Erik; Friman, Ola; Forssén, Per-Erik; Opsahl, Thomas Olsvik; Haavardsholm, Trym Vegard & Kåsen, Ingebjørg
(2014).
Anisotropic scattered data interpolation for pushbroom image rectification.
IEEE Transactions on Image Processing.
ISSN 1057-7149.
23(5),
s. 2302–2314.
doi: 10.1109/TIP.2014.2316377.
(2014).
Compact camera for multispectral and conventional imaging based on patterned filters.
Applied Optics.
ISSN 1559-128X.s. C64–C71.
doi: 10.1364/AO.53.000C64.
The paper describes the georeferencing part of an airborne hyperspectral imaging system based on pushbroom scanning. Using ray-tracing methods from computer graphics and a highly efficient representation of the digital elevation model (DEM), georeferencing of high resolution pushbroom images runs in real time by a large margin. By adapting the georeferencing to match the DEM resolution, the camera field of view and the flight altitude, the method has potential to provide real time georeferencing, even for HD video on a high resolution DEM when a graphics processing unit (GPU) is used for processing.
An airborne system for hyperspectral target detection is described. The main sensor is a HySpex pushbroom hyperspectral imager for the visible and near-infrared spectral range with 1600 pixels across track, supplemented by a panchromatic line imager. An optional third sensor can be added, either a SWIR hyperspectral camera or a thermal camera. In real time, the system performs radiometric calibration and georeferencing of the images, followed by image processing for target detection and visualization. The current version of the system implements only spectral anomaly detection, based on normal mixture models. Image processing runs on a PC with a multicore Intel processor and an Nvidia graphics processing unit (GPU). The processing runs in a software framework optimized for large sustained data rates. The platform is a Cessna 172 aircraft based close to FFI, modified with a camera port in the floor.
The Norwegian Defense Research Establishment (FFI) is developing a technology demonstrator for airborne real-time hyperspectral target detection. The system includes two nadir-pointing line scan cameras. The line scanned images are georeferenced in real-time by intersecting rays cast from the cameras with a 3D model of the terrain underneath. The georeferenced images may then easily be ortho-rectified (e.g by using texture mapping in OpenGL) and overlaid digital maps. This poster presents the performance of a cuda implementation of the georeferencing method.
(2010).
Real time direct georeferencing and orthorectification of images from airborne pushbroom cameras.