7. THE SEARCH PROCESS
7.1 Completing the task
Completion refers to the time logged from the moment the observer asked the user to proceed with the task to when the user indicated that they wished to stop. To complete the task does not necessarily mean that the user found the information they were looking for: NoVA was not designed to investigate this issue. For example, some of the users were unable to find any part of the information required for a specific task but still went through the search process, thus simulating a real life search situation.
Some users were forced to abandon tasks because of software compatibility problems. For example, one visually impaired user was unable to find the required information for any of the tasks and felt this was because a recent computer upgrade was no longer fully compatible with the access technology used. Another user experienced problems with the talking web browser reading out the Online Shopping site, and one user had problems getting the screen reader to read out graphical links. Other users experienced problems related to the sites themselves, for example three sighted users were forced to abandon the Online Shopping task because the site had suspended the online shopping facility that day.
7.2 Time taken to complete the task
The time taken indicates how long people spent on each task. Comparisons made between the times taken for sighted and visually impaired users to "complete" each task are described below and depicted graphically in Appendix Three.
7.2.1 Search Engine
Sighted users took between 1.5 minutes and 15 minutes to complete the task. Ten users took 10 minutes or less. Two took between 11 and 20 minutes and none took over twenty minutes. The mean time taken was 6.4 minutes.
Visually impaired users took between 2 and 45 minutes to complete the task. Eight took 10 minutes or less. Seven took between 11 and 20 minutes, and five took over 20 minutes. The mean time taken was 16.5 minutes.
7.2.2 OPAC
All the sighted users took between 1 and 5 minutes to complete the task. The mean time taken was 2.8 minutes.
The visually impaired users took between 5 and 25 minutes to complete the task. Ten users took 10 minutes or less, nine took between 11 and 20 minutes and one user took over 20 minutes. The mean time taken was 11.3 minutes.
7.2.3 Directory
The sighted users took between 50 seconds and 19 minutes to complete the task. Eighteen took 10 minutes or less and two took between 11 and 20 minutes. The mean time taken was 5.2 minutes.
The visually impaired users took between 2 and 20 minutes. Seventeen took 10 minutes or less, three took between 11 and 20 minutes and none took over 20 minutes. The mean time taken was 7.4 minutes.
7.2.4 Online Shopping
The sighted users took between 3 and 18 minutes to complete the task. Seventeen took 10 minutes or less and three took between 11 and 20 minutes. None took over 20 minutes. The mean time taken was 7.2 minutes.
The visually impaired users took between 2 and 30 minutes to complete the task. Fourteen took 10 minutes or less, three took between 11 and 20 minutes and three took over 20 minutes. The mean time taken was 10.2 minutes.
7.3 Surveying: Look At (LA) and Read Out (RO) codes
The action of scanning a web page for options before taking any further action is described by Hawk and Wang as "surveying" (Hawk and Wang 1999). Surveying during the tasks included looking at a page or listening to the details of a page read out (by a screen reader). Codes used to describe both of these surveying actions are LA (Look At) and RO (Read Out). Comparisons made between the instances of surveying by sighted and visually impaired users are described below and depicted graphically in Appendices Four and Five.
7.3.1 Search Engine
During the Search Engine task the sighted users LA pages on a total of 180 different occasions in order to decipher either what the page was about or what to do next. The maximum number of times was 16 and the minimum 3. The mean for the number of times sighted users had to LA pages was on 9 different occasions. None of the sighted users listened to pages RO to them.
The visually impaired users had text RO to them in total 223 different occasions (max 28, min 1) and LA pages a total of 45 different times (max 23, min 1). The mean number of times users LA pages was on 2.3 occasions and had text RO to them on 11.2 occasions during the task. Of the twenty, thirteen just listened to text RO and three users only LA pages (using magnification or looking at the screen up close). Four used a combination of RO and LA (although this tended to be a quick scan once or twice with more emphasis on listening to text read out). Only one made use of RO and LA in fairly equal measures (LA on 8 occasions, RO pages on 11 occasions).
7.3.2 Directory
For the Directory task, sighted users LA pages a total of 155 different times (max 24, min 2). The mean number of occasions the sighted users LA pages was 7.8. Again, none of the sighted users listened to pages RO to them.
The visually impaired users LA pages in total on 26 different occasions (max 10, min 2) and listened to text RO in total on 109 different occasions. The mean number of times users LA pages was on 1.3 occasions and listened to text RO on 5.5 occasions during the task. Of the twenty visually impaired users, fifteen just listened to the text RO to them and two just LA pages. Three used a combination of LA and RO.
7.3.3 Online Shopping
The sighted users LA pages in total on 206 different occasions during the Online Shopping task (max 20, min 4). The mean number of times sighted users LA pages during the Online Shopping task was 10.3. Again, none of the sighted users listened to pages RO to them.
Visually impaired users LA pages on 32 different occasions (max 10, min 1) and listened to text RO on 157 occasions (max 23, min 3). The mean number of times they LA pages was on 1.6 occasions and listened to text RO on 7.9 occasions. Of the twenty, three did not listen to text RO but just LA pages (using magnification or looking at the screen up close) and seventeen just had text RO to them. Three of the users used a combination of LA and RO.
7.3.4 OPAC
Sighted users LA pages during the OPAC task on 153 occasions (max 23, min 5). The mean number of occasions sighted users LA pages was 7.7. Again, none of the sighted users listened to pages RO to them.
Visually impaired users LA pages during the OPAC task on 49 occasions (max 17, min 4) and listened to text RO on 163 occasions (max 27 and min 6). The mean number of times the visually impaired users LA pages was on 2.5 occasions and they listened to text RO or LA pages on 8.2 different occasions. Three users LA pages rather than have them RO (using magnification or looking at the screen up close) and seventeen just had text RO to them. Two of the users used a combination of LA and RO.
[ Previous Section: User Profile Characteristics ] [ Next Section: Serial & Parallel Steps ]