DATA SET 8: Mike Land's Jigsaw Puzzle Data Set

Description
In jigsaw puzzles the objective is to copy a model – the picture on the box – by assembling pieces drawn from a random source. They require pattern fitting at three levels. The pieces have to be recognised by pattern and colour as belonging to a part of the overall source pattern; they must be chosen so that their pattern fits in with some part of the puzzle already completed; and the outlines of suitable new pieces have to be examined and oriented so as to produce a mechanical fit with other existing pieces in the puzzle. In this study we examined the fixation patterns of two subjects as they completed a 50 piece puzzle of a cartoon scene. The overall cycle loosely followed the model-source-copy pattern of the block copying task of Ballard et al. (1993), with the pick-up and place movements of pieces occurring near the end of the fixations of the source and the beginning of those on the copy. Not surprisingly, however, the timings were very different because of the need to search the source for suitable pieces, and to manipulate each new piece in the area around the copy to get a suitable fit. In our trial the average times spent fixating the model, source and copy were 1.3 ± 0.9 (sd), 2.8 ± 1.7 and 3.2 ± 2.6 s. The ‘cycle time’ between the fetching of new pieces from the source was 15.5 ± 7.2 s, which contrasts with the cycle time in the block copying task of just over 2 s.

Figure1 illustrates an interesting 30 s episode during which the player has two loose pieces (a & b) which he eventually joins and fits to the completed part of the puzzle. The pattern of fixations allows us to reconstruct his thought patterns with some confidence. In the first few seconds, between 1 and 2 piece a is rotated anti-clockwise in three stages (i). While this is happening the glances to the completed part of the puzzle are all to region x, indicating that the player is trying to fit a to this region. However by 2 it is clear that this will not work. He consults the picture on the lid and thereafter directs gaze on the puzzle itself to region y. About a second later (ii) he moves piece a to the right of piece b, probably having also noticed that the right hand side is a straight edge. Interestingly, this move is completed while the eyes are looking at the picture about 20º above, so presumably its trajectory was set up during the previous fixation. Between 3 and 4 the player concentrates on piece b. He moves a and b to the left (iii, iv) and just before 4 he rotates b (v) so that its new right hand profile matches the left profile of a. Just after 4 piece b is lifted and joined to a (vi). After 5 there is quite long period when both pieces, now joined, and the profile of the completed section are fixated and presumably appraised for goodness of fit. The two pieces are lifted together and finally joined to the rest of the puzzle (vii).

This episode illustrates a number of general points. First, the eyes only fixate the parts of the field that are important – the two pieces, the relevant regions of the part-completed puzzle, and on two occasions the relevant region of the picture. When looking at the individual pieces a and b fixation is accurate: the mean fixation distance from the centre of each piece (approximately 2 by 3º) was 1.4º. Second, during the various movements of the pieces they are either fixated during the move, or during the half second before the move. This seems to bear out both the ‘do it where I’m looking’ and the ‘just in time’ rules of Ballard et al. Third, comparisons between patterns and outlines are mostly made by looking from one element to the next and back again, rather than sizing up the situation from a single gaze location. This is particularly clear between 1 and 2 where gaze moves repeatedly from a to the completed part of the puzzle and back again as a is rotated. This leads to the rejection of the hypothesis that a can fit to x, and the development of the new idea that it might fit near y, which is then checked by consulting the picture. Similar ‘aha’ moments precede the move at (vi), again accompanied by much

Fig 1: Gaze movements during a 30 s except from a jigsaw puzzle. Upper part shows the gaze shifts between two pieces (a & b), the part completed puzzle, and the picture on the box. Numbers refer to the sketches below, and roman numerals to the seven movements of the pieces. All the gaze shifts are made with single saccades, and other saccades within each area are shown by short vertical lines. Hand contacts resulting in movement of the pieces are shown by upward (contact) and downward (release) arrows. The lower part of the figure shows details of the movements of the pieces. The movements shown refer to movements made between each illustration and the next.

Method
Eye movement recordings were made with a head-mounted camera that produced a split image in which the top two-thirds showed the scene ahead and the lower third the eye in its socket, imaged via a concave mirror. The location and ellipticity of the iris were used to obtain the coordinates of eye direction, by matching the iris outline to a computer- generated eye model. This was done by hand, frame-by-frame, at 50 f.p.s. The coordinates were used to position a 1º dot on the upper scene view, and each frame re-recorded. Head movements could also be obtained by tracking distant background objects in the scene view. The resulting video contains numerical values (in degrees) of the direction of view of the fovea, a frame counter, and a clock. The videos are reversed left to right as a result of the mirror optical system. to-and-fro checking, and also before move (vii), although here the relevant profiles are now within 2º of each other, and there is a lull in the overt cross-checking seen elsewhere. Overall, this sequence, and many others like it, demonstrate the information-gathering function of eye movements and their close moment-by-moment relationship with the thought processes involved.

References
There are no publications associated with this clip. The study was performed in 1998 with the help of two University of Sussex Students, Lynette Owen and Sam Walker.

See also:
Ballard DH, Hayhoe MM, Li L, Whitehead SD (1992) Hand eye coordination during sequential tasks. Phil Trans R Soc Lond B 337: 331-339

SCENE/DISPLAY/STIMULI IMAGES/VIDEOS

SCANPATH (EYE-TRACKER) DATA

Jigsaw Puzzle Preview (12656 KB, DIVX compression)

Jigsaw Puzzle Full Data Set (uncompressed avi, zipped: 942586 KB)