12. CONCLUSIONS
NoVA's observations and other analyses of users searching for information on the web reveal that, unsurprisingly, people who are sighted find searching the web much easier than visually impaired people. Furthermore, people who are visually impaired, but possess enough sight to be able to see part of the screen (either up close or using magnification), find searching the web easier than those whose sight is severely impaired (i.e. those who are either totally blind or have very limited sight). These results confirm the findings of a parallel study conducted by Coyne and Nielsen, which estimated that "the Web is about three times easier to use for sighted users than it is for users who are blind or who have low vision" (Coyne and Nielsen 2001 p 5), although we would not make such explicit claims. Again, as with NoVA, findings from this study also revealed that people using screen magnification appeared to have a higher success rate than those using a screen reader, although the difference was not statistically significant.
Comparisons of time spent searching during the NoVA usability tests showed that visually impaired people have to spend more time searching for information than sighted people (again this concurs with findings of Coyne and Nielsen, 2001). However, times can vary considerably. The design of a site can considerably reduce the time needed if a number of simple design features are included. These would include a logical and meaningful menu, a search facility and a strict limit on the number of links per page (as well as other accessibility design features such as those recommended by the W3C/WAI).
The acceptable (to the user) length of time spent searching for information may have an impact on the success of a search, as users tended to stop after an average (mean) of about 16 minutes whether they had found all, some or none of the information they were looking for. Logging the number of keystrokes (or the number of steps) people have to take whilst undertaking a search shows that visually impaired people take more than twice the number of steps and have to interact with more than twice the number of keystrokes than sighted users. The steps taken for the NoVA tests were broken into "serial" and "parallel", enabling comparisons to be made between the number of times people moved from page to page (or site to site) with the number of times they have to move around a page. The studies show that visually impaired users have to spend much more time navigating around a page than sighted users, and have to use a greater variety of keystrokes to do so. Even if the page is accessible it may have been designed in a parallel way (i.e. with frames and tables), so that a visually impaired user (particularly someone using a screen reader) may have to spend more time navigating around each page. This can lead to frustration and fatigue, and may cause them to stop altogether, which may in turn result in users missing out on potentially useful information. At the extreme it could discourage them from using the web altogether and thus contribute to social exclusion.
It should be noted that users in the NoVA usability tests were looking for information prescribed by the task. Clearly this could have influenced their motivation to stop (i.e. the importance of finding the information may not have been as great as when information was needed for other, personal reasons). However, it is clear that, in a real information searching situation, pages that cause people to stop searching because they are complex or cumbersome to navigate will still be a serious problem. One example could be a user in higher education researching a topic for a piece of coursework using a screen reader to access web-based resources. The user may find a site with resources of a highly academic nature, relevant to their topic and of sound quality, but then miss important documents or references because they give up searching. The site may not necessarily have been inaccessible, but simply designed in an inflexible way. Web designers need to be aware that people will not carry on searching or browsing a site if they have to spend too long or find it too complicated - they will simply go elsewhere.
The type of assistive technology used had a bearing on the search process. Visually impaired users who were able to read a standard size screen up close were better able to browse through non-serial information because they could scan the page rather than try to navigate it line by line. A similar comment applies to those using magnification packages as they could interact with the page in a similar way to someone using a standard size screen, with an element of control over which section to look at - although clearly there were more problems where it was necessary to scan between frames. The main problem observed with magnification packages was that the text on screen could appear very broken or "pixelated" when enlarged, and in some cases this rendered text completely unreadable. Users sometimes had to activate speech (if that was possible) in order to proceed, or to highlight the text to enhance its appearance (although this could be viewed as more of a problem relating to inaccessible web design than a problem relating to user behaviour). While a possible solution is for users to override font/size settings, in reality most users do not do this (and it can have unpredictable effects with all too many pages).
Observations indicated that screen readers forced users to navigate pages in a serial way, which was not always appropriate for the design of the page. For example, pages with a large number of links are designed for quick scanning, but a screen reader is not able to do this and will read out each link one by one. It is also difficult to remember the contents of a page containing a large number of links, and users often have to go back and listen to all the links again in order to find (or not find) a suitable one. This makes the navigation process considerably longer and users expressed feelings of frustration - especially if they were looking for something fairly simple. Similarly, pages that contain duplicate menus (i.e. one at the top and one at the side) force the user to listen to links being read out more than once, which again is time consuming.
For successful browsing of sites the observations identified that hypertext links need to be placed in a logical order and given meaningful titles. Ideally the page should not contain a long list of links, but if this is absolutely necessary then it should at least be in alphabetical order (or offer an alternative list in alphabetical order). The number of links also had a bearing on success, as the greater the number of links per page the longer it took someone to browse, particularly if using a screen reader. Ideally pages should not be cluttered with links (except perhaps for something like a site map) and for reasons explained it is better to adopt the shallow and deep approach which has been identified in previous research as being the preferred approach for someone who is blind or visually impaired. For web sites where it is not possible or practical to offer this approach, for example with very large sites, it is important to provide a search facility as an alternative to browsing through links.
For users who prefer to search a page rather than browse through it, a search box should be placed in a prominent place (such as near to the top of the screen). This is preferred to providing a link to a separate search page as users were sometimes confused by this, assuming "search" would be a search box and not a hypertext link. The search box should be simple and not too small (an option to go to an advanced search may also be offered, along with tips on formulating search terms and queries) and if the search facility is available on all pages it should always be placed in the same position.
Even if a web page passes all the automated accessibility checks and/or adheres to recommendations by bodies such as the W3C, this does not always mean it will be problem-free for users. An example is given by the user who tried to tab quickly through links but missed important information because the screen reader paused before reading out some of the text (e.g.. "Link, Image" (pause) "perform a Title search") and the user had moved on before the link was fully read out. Therefore, the importance of using people for accessibility checking cannot be stressed too much.
Observations during the NoVA usability testing highlighted the fact that expertise with the assistive technology used also had a big impact on the success of the task. Users who were aware of the different options offered by some of the more advanced screen reading technology were able to have a greater degree of control in the way they interacted with a page. For example, they could search within the page for specific information, sort alphabetically, navigate through tables and frames and set preferences to avoid repeated reading out of navigational links at the top of the page. This may have implications for training programmes for people with visual impairments.
It is also important to check web pages using different types of assistive technology. Observations of the NoVA usability testing showed that performance varied between assistive technologies and, as mentioned previously, not everyone uses the most up-to-date versions, perhaps because they cannot afford to keep upgrading their software, or because they feel comfortable using a particular version (much like someone who drives the same car or has been using the same camera for years). It may also be that they are using a public access terminal where the assistive technology has been chosen for them.
Different types of assistive technology present different solutions and problems to accessing a web page. Observations identified problems with screen magnification such as pixelated text (the main problem in the usability tests), but enabled users to search a page that had been designed in a parallel way with relative ease. Screen readers presented many problems for users, such as not always reading out every link (for no particular reason it seems, although this needs further investigation), therefore potentially missing important information. Similarly, partially minimised pages or pages that opened as a new window could also be confusing or misleading because the screen reader would just read what is on screen.
Older screen readers did not work well with frames. Whilst trying to navigate between two frames, one user got stuck in the contents frame and took some time to get back to the main frame. Compatibility can also be a problem. One user, for example, had problems with the screen reader reading out what seemed to be the HTML of the page rather than the page itself. The user had recently upgraded his/her PC and felt that this could have been the cause of the problem rather than the screen reading software.
Some of the more advanced screen readers do allow a degree of user control regarding navigation. However success in using some of the more advanced features of a screen reader seemed to depend on awareness and experience. Those who were quite experienced and made use of elements such as the Links List, Sort Alphabetically and Search in Page options were able to reduce the time spent browsing by eliminating the need to tab back and forth through each link. Use of the Virtual PC key (or the Braille Bar) enabled users to move around the page and pinpoint areas of text or spellings that were not immediately clear to them.
[ Previous Section: Summary of Findings ] [ Next Section: Recommendations ]