DATA SET 3: Bristol Eden Project Multi-Sensor Data Set

EXPERIMENT DESCRIPTION
The current study is part of a larger project on multi-sensor image fusion – that is the fusion of complementary inputs such as visible light and infrared radiation images. We have been concentrating on the human and computational assessment aspect of this fusion process, and are using participant scanpaths as a new and untapped source of information. This experiment involved participants watching videos of a man dressed in camouflage walking through thick foliage (see Fig 1) whilst performing a secondary button-press task.

Fig 1: The ‘soldier’ began walking near to the camera (see Fig 1) and to the left of centre, walks down a path away from camera, then left-to-right across a clearing of trees, and finally right-to-left back across the clearing. The video sequence was filmed in both visible light (Viz) and infrared (IR) at the Eden Project Biome in Cornwall, UK, and more details of this can be found in [1].

Fig 2: In the experiment, participants were shown the registered original input videos (Viz and IR) separately (see Fig 2), the two videos together side-by-side, as well as three fused versions of the videos. The first fusion method was simply averaging between pixel values of IR and the values of each colour plane of Viz. The second used a basic Discrete Wavelet Transform (DWT) method [2], and the third utilised the Dual-Tree Complex Wavelet Transform (DT-CWT) [3]. The participants were shown each of the videos once, in three separate sessions. In the first session half the participants were shown the order: Viz, IR, Viz+IR, AVE, DWT, DT-CWT; and half were shown the order: DT-CWT, DWT, AVE, Viz+IR, IR, Viz. In the second session, both sets of participants swapped order, and in the third session participants viewed the original order from Session 1.

The actual tasks the participants were instructed to carry out were as follows. They were told to primarily concentrate on visually tracking the soldier as well as possible throughout the video sequence. In addition, they were asked to press the ‘SPACE’ bar once whenever the soldier began or finished walking past an electric vehicle positioned in the clearing. That is, when the solder was adjacent to either end of the vehicle.

A Tobii™ x50 remote eye tracker [4] was used to collect eye movement data. This is a table-mounted eye tracker that works at 50 Hz with an approximate accuracy of 0.5 deg. This was run using the ClearView 2.5.1 software package, on a 2.8 GHz Pentium IV dual processor PC, with 3 GB RAM, and twin SCSI hard drives. Stimuli were presented on a 19” flat screen CRT monitor running at 85 Hz, with screen resolution set to 800 by 600 pixels. Participants were required to use a chin-rest positioned 57cm from the monitor screen.

More details of this experiment can be found in [5].

References

SCENE/DISPLAY/STIMULI IMAGES/VIDEOS

SCANPATH (EYE-TRACKER) DATA
The scanpath (eye-movement) data from the experiment in native (Tobii), txt and Excel formats can be found below. The data for the 10 participants in each format is provided as a separate zip file.

Fig 3: The video stimuli plus fixations (3 sessions for only the first 3 participants) plus target rectangles are listed below and are provided for visualisation purposes. An example of a 'hit' (left), i.e. when a fixation is within the target box, and 'miss', i.e. when a fixation is out of the target box are shown in Fig 3.