10. USABILITY QUESTIONS
Usability questions referred to the users' experience of undertaking the search process. Questions related to overall ease of navigation during each task, executing the search, locating hypertext links, finding the required resources and general feelings during the task, such as satisfaction levels.
10.1 How easy did you find navigation of the Interface to be?
Users were asked how easy they found navigating around the resource.
10.1.1 Search Engine
Of the forty users, just over half commented in a positive way to the question "how easy did you find navigating around the Search Engine?". Comparisons between sighted and visually impaired users reveal that of the twenty-one who responded in a positive way, fourteen were sighted and seven were visually impaired. Comments included:
It was quite easy for straightforward searching.
Pretty easy really.
I find (The Search Engine) one of the better search engines.
Easier because I saw the link to Weather (sighted user).
I use the web that much that I don't really find it much of a problem (visually impaired user).
Of the forty respondents, fourteen expressed negative comments about the ease of navigation around the Search Engine. Negative responses were made by two of the sighted users and by twelve of the visually impaired users. Comments included:
Unpleasant, too many adverts and outside services (visually impaired user).
It's quite difficult because it is quite cluttered. There is too much information there (sighted user).
If the print had been better it would have been easier, but the print was awful (visually impaired user).
What choice language would you like me to use?! I don't like web pages with too much information.... (visually impaired user).
Five of the respondents expressed having neither a positive nor a negative experience - in other words they were neutral to the question. Of this, four were sighted and one visually impaired. Typical comments in this category were:
I use search engines a lot so I'm used to it. They get good search results, but the opening page of (the search engine) is a bit bland (visually impaired user).
Well, you tend to get into a habit of using certain search engines - I use Google which is very clear. In the Search Engine I did find that when I got the list of results I couldn't see them immediately - maybe that's just the size of the screen - you have to scroll down (sighted user).
10.1.2 Directory
Of the forty users in the sample, thirty-nine provided a response to the question how easy did you find navigating around the Directory? Twenty-one users commented in a positive way.
Of the twenty-one who responded, there was a fairly even split between the two samples (ten sighted and eleven visually impaired). Comments from the sighted sample included:
Yes, that was very slick. You have got three different ways of doing it (sighted user).
A lot easier than I thought it would be (sighted user).
Quite easy, has categories and links. Only the search box at the bottom is quite small (sighted user).
Quite good actually, really good (sighted user).
Comments from the visually impaired sample included:
Fairly easy to navigate... brought up some results fairly quickly once it was clear what to do...(visually impaired user).
Really easy, the easiest journal site I have come across (visually impaired user).
Very easy, it was good. Anything with text only is good for people with no sight (visually impaired user).
It was cool actually (visually impaired user).
Seven users responded in a negative way to questions about ease of navigation (four sighted and three visually impaired). Negative comments from the sighted users included:
Not very easy when you are looking for an area and you are not sure what it comes under (sighted user).
Not very easy at all ..... it wasn't clear what it was all about (sighted user).
Not really that helpful ... it wasn't that clear what the A-Z list was, just a mish-mash of titles not in order (visually impaired user).
Negative comments from the visually impaired users included:
Not too good, a lot of it I couldn't read .... I was guessing really (visually impaired user).
... some of the text was hard to read .... a pretty nasty colour (visually impaired user).
Eleven users were fairly neutral about navigation of the Directory interface. This broke down into six of the sighted users and five of the visually impaired users. Neutral comments included:
It wasn't too bad .... Instructions given were not that clear (sighted user).
I found it a bit more difficult because I am not used to it, but eventually found my way around (sighted user).
Brought up some results fairly quickly and once it was clear what to do I managed to get information.... (visually impaired user).
Not too bad (visually impaired user).
Easier than the search engine (visually impaired user).
10.1.3 Online Shopping
Of the forty users in the sample, fifteen responded in a positive way to the question about navigating around the Online Shopping site. Of the fifteen, eight were sighted and seven visually impaired. Positive comments included:
Easy once I had found the links, the links weren't very clear at the beginning (sighted user).
It was easy to navigate. You just followed links (sighted user).
The navigating part wasn't difficult, it is quite nicely laid out, it was logical (sighted user).
Easy, not a problem at all. All the links had labels on them, just goes to show what I use the web for... (visually impaired user).
Absolutely easy (visually impaired user).
Easier than I first thought. You don't get all that information at the top, like with (the search engine) (visually impaired user).
Quite good actually. It's quite well structured. When it first came up it was a bit confusing, there were only about three links on the page, then it said 'new browser' so I knew I was in another window (visually impaired user).
Negative comments were made by thirteen of the users, four sighted and nine visually impaired. Comments included:
Not very clear at all, not sure where to click or where to go (sighted user).
Quite difficult with all the problems with not loading properly, also I couldn't find the page I was originally on (sighted user).
That was awful, I didn't like that at all. I clicked on suits and it gave me the jacket and I wanted the trousers too and I wasn't sure where to go to get the trousers (sighted user).
Not something I would do for fun (visually impaired user)
Not that good. Too many different categories, but if you were shopping you would want all that information but it can be a bit, you've no idea what it is going to be made out of (visually impaired user).
Well, I don't normally bother to search for so long - there is so much to read and so many duplicates. I get tired and it gets on my nerves (visually impaired user).
Awful. The text was unreadable and there are that many graphics (visually impaired user).
Neutral comments were made by eight of the sighted users and four of the visually impaired users. Comments included:
Once you go into the links it was ok, but the order of the suits wasn't helpful at all (sighted user).
I found it easy at first, but I didn't quite understand about the display of the clothes (sighted user).
A bit easier, you can tell that the text is children's wear, women's wear etc but links are not specific, they are mixed up and repeating (visually impaired user).
10.1.4 OPAC
Of the forty users, one of the visually impaired sample did not respond to this question. Eighteen of the remaining thirty-nine expressed positive comments about the OPAC website: twelve sighted and six visually impaired. Positive comments included:
Entirely intuitive and transparent. Tries to do what it says it will (sighted user).
It was a lot easier that I thought - I was looking for problems that weren't there. It is very well laid out and not cluttered and you can see quite quickly where to go (sighted user).
Very simple - yes, it is quite straightforward. The reference list was something I hadn't come across before, had to make a bit of a leap there (sighted user).
I think it is well designed and quite easy to use (sighted user).
I didn't think the info would be there - but found site easy to navigate around (visually impaired user).
This site is much better [than the search engine], it is so easy. The information was there but at first my impatience led me to miss things (visually impaired user).
Pretty easy, pretty self explanatory. Not as many links [as the search engine] and they are pretty clear - title details (visually impaired user).
Eight of the users commented negatively on navigation of the OPAC website (one sighted user and seven visually impaired). Comments included:
I have always found it confusing - what actually links to what (sighted user).
Quite confusing, it had that table that looked quite simple but what was underneath it was a load of jumble (visually impaired user).
It was giving me clues on how to do it, for example where to put the title and that it has an index, but unfortunately I couldn't do it (visually impaired user).
That was not easy - a lot of people would have given up. That first page was a killer! Once you got to other pages it still wasn't easy, but not impossible, first page was like a blank wall, nothing on it. Had to use a lot of guesswork (visually impaired user).
Not very easy, I was looking to find out where to put in the search (visually impaired user).
Thirteen of the users commented in a neutral way on navigation around the OPAC website (seven sighted and six visually impaired). Neutral comments included:
Not too bad - it was easy to find the journal but less easy to find the former title (sighted user).
I didn't exactly know where to look for the former title, I just thought this link might work (sighted user).
It was ok with the bits I am familiar with but with the former title I would probably have looked in title details but I am not sure (sighted user).
I was just guessing in parts but it is a shame because the links were fine, they stood out. I suppose if I used it regularly I might get used to it, but it was so hard to read (visually impaired user).
There is probably a shorter way to find out the publisher. I prefer a search box on the page, like a search engine, rather than having to follow links to search, but basically it was ok (visually impaired user).
The first bit was really good, there were so few links and it was fairly self explanatory, some text and easy to select a type of search. But, when I got to the next stage [the search] it was less easy, especially when I got to the 'Alls well that ends well' - that was a bit distracting (visually impaired user).
10.2 Could you easily locate where to enter search terms?
Users were asked if they could easily locate the search box and where to enter search terms.
10.2.1 Search Engine
Twenty-two of the users said that it was obvious to them where to enter search terms. The majority in this case were sighted users (sixteen) with comments such as:
It was easy to see where to enter search terms (sighted user).
The search facility was one of the first things you saw (sighted user).
It is probably the clearest thing on the page, and it's at the top of the page so you don't have to scroll down (sighted user).
One sighted user said they had to guess, commenting that:
Couldn't see a category so assumed I had to enter search terms (sighted user).
Six of the visually impaired users said it was obvious where to enter search terms, commenting that:
Entered terms where the flashing cursor is, it is quite good, a good size (visually impaired user).
It's got contrast so it's not too bad (visually impaired user).
Four of the visually impaired users used keystrokes to find the search box, three mentioned listening to the screen reader for "edit box" or "find this" to be read out. Four of the visually impaired users said that locating the search box was unclear and one user said that they didn't really get anything at all. Comments from visually impaired users included:
Originally I found it, but there was too much other stuff, it was complicated (visually impaired user).
I thought I knew where to enter terms but it didn't seem to work (visually impaired user).
Even when it was clear where to enter search terms it was not always clear how to activate the search, as one visually impaired user commented:
Sometimes you can just press return, not always though, but don't always know what to press so it is good if you can just press enter (visually impaired user)
10.2.2 Directory
Fourteen of the users (nine sighted, five visually impaired) said that it was obvious to them where to enter search terms. Comments included:
... it was easy to see where to enter terms (sighted user).
it was obvious where to enter the terms but I wasn't convinced that I would get anything just typing it in there (sighted user).
Yes, there was an edit box to enter a simple search - it's good when it says that because 'simple' indicates that there is an advanced alternative (visually impaired user).
I had to scroll down to the bottom but I knew that (the search) would either be at the top or the bottom, search boxes are rarely in the middle (visually impaired user).
Six users (all visually impaired) found the search box when it was read out to them by the screen reader, although it was not always obvious to them where to enter their terms:
Typically, once you got into the search it was by no means clear where to enter your search term - this may be the fault of the screen reader though, it didn't put you right into the place you wanted to be (visually impaired user).
One visually impaired user typed in search terms but was not in the edit box so when they activated the search, no results were retrieved. The user then changed strategy to follow links and was more successful.
Four of the users (all sighted) commented that they did not realise there was a search facility at all:
I didn't see anywhere to enter search terms.
It wasn't obvious where to enter search terms, I don't remember seeing that.
Didn't notice the search facility.
Six of the users (three sighted and three visually impaired) said that it was unclear where to enter search terms. Comments included:
No, not really clear where to enter terms (sighted user).
You couldn't input your own terms... (sighted user)
I just came across the search box [at the bottom of the page] by accident (visually impaired user).
Two users (both sighted) commented that the search box at the bottom of the page was very small:
you don't immediately see where the search box is, you could very easily miss it (sighted user).
One visually impaired user commented that the placing of the search box at the bottom of the page:
was in the wrong place, you had to go all the way down reading everything to find it (visually impaired user).
10.2.3 Online Shopping
Nine of the users (four sighted, five visually impaired) said that it was obvious where to enter search terms. Comments included:
Entering terms was obvious, but the rest was a bit different, with the little boxes to select from rather than typing things in (sighted user).
You could type in the terms, but I don't think it would give you anything that you couldn't get on their list (sighted user).
The edit box was as usual with combo boxes after that (visually impaired user).
Search terms were ok, I am used to this type of searching but there was no real explanation of what to do (visually impaired user).
Six users (four sighted, two visually impaired) said they were confused as to where the search box was. Comments included:
Search was useless. I could select items from the drop-down menu ok, but wasn't sure if I had to type suits into the dialog box and then select other options - so I had to take the trouble to read them (sighted user).
I didn't really know what I was doing and it didn't really come up with what I expected (sighted user).
I was looking for an edit box not a link on a page, it wasn't obvious (visually impaired user).
Two users (both sighted) did not notice that there was a search facility at all, and one (sighted) user commented that "the search box was a bit small".
10.2.4 OPAC
Twenty-one of the users (thirteen sighted, eight visually impaired) said that it was obvious where to enter search terms, comments included:
Entering search terms was ok, but it was a bit awkward having to click in the search box before typing anything in (sighted user).
Where to type search terms was clear, it gave me choices to do keyword or title searches (sighted user).
There is a search box, and it is the only box you can type something into (sighted user).
It was clear where to enter search terms, but you have got to have a bit of experience to get more out of the site (visually impaired user).
I had to do a bit more digging, but once I found it (the search box), it wasn't too bad (visually impaired user).
The screen reader has a keystroke that will take you straight to the first edit field on a page and tell you what it is (visually impaired user).
The first edit box was fine, but on some of the pages there are other bits which can throw you (visually impaired user).
Of the visually impaired sample, two users said that they were confused as to where to enter search terms, two said they guessed and one user said that he/she "did not know where to enter search terms". Other comments included:
It was not obvious where to enter terms, the language used by the OPAC was not familiar to me and it was not clear whether it was just text or a dialog box to enter text (visually impaired user).
It was not obvious where to put search terms, I just guessed (visually impaired user).
I was looking for where to enter terms, this is the trouble with websites - there's nothing standard about them, in particular the beginning of them... you really do have to search around to find where you are supposed to input something (visually impaired user).
10.3 Could you easily locate where the hypertext links are?
Users were asked if they could easily locate the hypertext links.
10.3.1 Search Engine
For the Search Engine task, six sighted users and two visually impaired users said that text in a different colour indicated a hypertext link to them and underlined text was cited by seven of the sighted users. A visually impaired user mentioned that when using a Braille bar, a row of dots along the bottom of the bar indicates a link, and a sighted user mentioned that "links have buttons, like GO or SEARCH or FIND MORE". Other comments included:
I can tell they are links because they look different to the other text - I think they are blue and perhaps underlined (sighted user).
They are in a different colour aren't they? (sighted user).
I know that links are often in blue (visually impaired user).
No problem with links, but there were too many (visually impaired user).
The mouse pointer (or cursor) turning into a hand was mentioned by six sighted and two visually impaired users:
When the cursor changed to a hand I know that's where a link was (sighted user).
If it turns into a hand I tend to think that's a link to somewhere (sighted user).
You can tell links because you get the hand - I look for the hand (visually impaired user).
It says "link" and also "visited link" (visually impaired user).
(The screen reader) says Link first and then reads out the link (visually impaired user).
Using information from the status bar was also mentioned:
Usually links give you something at the bottom - an address that tells me it is a link (visually impaired user).
Two users (both visually impaired) had problems with links, one found the amount of information on the page confusing and one said that they found a link but "it wouldn't let me go to it". Some of the users said they just assumed (or guessed) which text was a hypertext link:
Often I just assume it is a link (sighted user).
I often just click on text because I think it will be a link (sighted user).
10.3.2 Directory
Seven users (six sighted, one visually impaired) looked at the colour of the text. Four users (three sighted, one visually impaired) mentioned underlined text. Two visually impaired users mentioned the mouse pointer or cursor turning into a hand. Other comments relating to hypertext links in the Directory task included:
It was obvious where the links were, but it wasn't obvious where they were going to lead. It was a bit vague (sighted user).
Hypertext links were obvious, they are green not blue, but they are underlined and you still get a sense that they are links (sighted user).
Links were much clearer, it just gave the link with the menu option, not a lot of rubbish (visually impaired user).
Links were ok, but it didn't matter so much because it was a flat structure - one title or country at a time (visually impaired user).
Four users (three sighted, one visually impaired) said locating links were not obvious to them and were sometimes confusing. Comments included:
Links were not obvious no, but I had a feeling that each country would have a link to it (visually impaired user).
Others assumed or guessed which were links, or where the links would lead:
Hypertext links were clear, although some terms were not - what does A-Z mean? A-Z of what? You have to assume. You have to rely on the voice synthesiser and have to retain a lot of information (visually impaired user).
... it was more guesswork than usual really (sighted user).
10.3.3 Online Shopping
Four users (three sighted, one visually impaired) mentioned the mouse pointer or cursor turning into a hand. Three (two sighted, one visually impaired) mentioned the colour of the text being different when it was a link. One (visually impaired) user said that underlined text were links. Eight visually impaired users mentioned the screen reader reading out any links to them and one sighted user said that the status bar told them it was a link. Using the alternative text to locate links in the online shopping task was cited by one of the visually impaired users.
Locating links was sometimes done without thinking:
I didn't think about it really, but I don't think I clicked anywhere expecting to go somewhere and didn't - I assume places are links, I don't know why (sighted user).
Users experienced various problems with links in the online shopping task:
links were ok at first, but finding links to suits was hard, I was struggling there, had to really hunt (visually impaired user).
I could tell it was a link but I was not sure where I was going (visually impaired user).
Links are blue, it's ok, but having them all higgledy-piggledy, all over the place is not good (visually impaired user).
Links aren't that obvious, if you look at the top of the screen they are really small and not separated by anything (sighted user).
Links were not clear on the Home Page, but when you got to the selection of suits they had underlined links - they should stick to one thing, either underline them all or not (sighted user).
10.3.4 OPAC
Eight users (seven sighted, one visually impaired) said that the colour of the text in the OPAC task indicated a hypertext link. Seven (all sighted) said that underlined text indicated links, and six users (three sighted, one visually impaired) said that "the hand" indicated where the links where. Comments included:
Links are blue and underlined; on the left hand menu the colour changes when you move your cursor over the links (sighted user).
I know it is a link because the mouse pointer turns into a hand (sighted user).
Links were obvious and they were labelled with meaningful words, which is good, but I was not always sure what was in them (visually impaired user).
Links were easy, the finger points to them (visually impaired user).
Four of the users (three sighted, one visually impaired) said that they assumed or guessed where the links were:
Links weren't always obvious, just assumed sometimes (sighted user).
Links were not at all clear, it was mixed with the text - it didn't tell you where links were, I just had to guess (visually impaired user).
10.4 Where the results pages easy to "read"?
Users were asked if the results pages displayed easy to "read".
10.4.1 Search Engine
Fourteen users (eight sighted, six visually impaired) said that they found the results pages in the Search Engine task easy to read and understand.
Positive comments about the results pages included:
Gave you a lot of detail which was good, I have been on some where they only give you a title and that's it (sighted user).
Usually give a better description than other search engines (sighted user).
It gives a bit of an abstract so you can tell what type of site it is and it gives the address (sighted user).
Quite liked the descriptions given, didn't find much though, but I only looked at the first page - seemed to be a lot of junk (visually impaired user).
Thirteen users (one sighted, thirteen visually impaired) said they found the results pages hard to read. Comments included:
It reads out a lot of link images, you don't really know how far you have to go to get something sensible, you have to check every option out (visually impaired user).
I can't read the broken text, I have to have it read out or highlight it (visually impaired user).
No, I didn't know what I was doing then (sighted user).
Negative comments relating to the Search Engine results pages were that they were confusing (two sighted, four visually impaired users), there were too many of them (three sighted, two visually impaired) and that they were of poor quality (three sighted, two visually impaired). Comments included:
I find the screen rather too busy, too much going on (sighted user).
Sometimes when I clicked the link I wasn't sure if it was really about the weather... wasn't always sure how results related to my keywords (sighted user).
Seemed to be a lot of junk (visually impaired user).
I was surprised that I couldn't see how many hits I had, so couldn't tell how far I should go (sighted user).
Each link said more or less the same thing with something different below - that put me off a bit (visually impaired user).
Two users (one sighted, one visually impaired) had problems locating the results, commenting that:
I don't like the hits, they are way off down the bottom somewhere and before you get there there's loads of rubbish on screen (visually impaired user).
It wasn't immediately obvious where the results were, it said how many results were there but you had to scroll down the page to view them (sighted user).
10.4.2 Directory
Twenty-three of the users (twelve sighted, eleven visually impaired) said that they found the Directory results pages easy to read and understand. Comments included:
Very clear and uncluttered page (sighted user).
The results were detailed but it was not clear what kind of an audience they were suited for (sighted user).
Results were quite easy to listen to and I can use navigation mode to read line by line (visually impaired user).
Easy to read, brought up plenty with various subjects relating to the Solomon Islands (visually impaired user).
Quite comprehensive - useful information. It was very effective really (visually impaired user).
Three users (all visually impaired) found the results pages hard to read and understand, and six (three sighted, three visually impaired) said they found them rather confusing. Two users (one sighted, one visually impaired) commented that there were too many results. Comments included:
There is a lot of detail, possible a bit too much at one stage (sighted user).
I was not sure what the information was, I had to listen to it (visually impaired user).
I couldn't really see them properly, it would have been extremely frustrating if it was something I really wanted to know about (visually impaired user).
Six users (four sighted, two visually impaired) said that they just read the titles and followed the links rather than reading results descriptions.
To be honest, I didn't really bother to read it, just clicked on links to see what it is (sighted user).
I recognised the sources so didn't even read the description (sighted user).
I didn't really think about it, I just went into them (visually impaired user).
10.4.3 Online Shopping
Thirteen users (four sighted, nine visually impaired) said that they found the Online Shopping Results pages easy to read. Comments included:
Yes, once you got the hang of it - would normally expect to see the price of a suit though, not suit pieces (sighted user).
Results were easy to read, but you really need to know what you are looking for (visually impaired user).
Yes, easy to understand and gave me important information, like it was one of three pages (visually impaired user).
Not sure I'd buy a suit, but I knew what it was wittering on about (visually impaired user).
Ten users (eight sighted, two visually impaired) found the results confusing and eight (six sighted, two visually impaired) thought the results given were of poor quality. Comments included:
The letters underneath the pictures were a bit small, yes they were very small, I think I missed some things (sighted user).
No, they kept going on about suit pieces and I didn't really understand that (sighted user).
These ones (suit descriptions) are tragic because it took me quite some time to realise what they had done, and then it was irritating and in no order (sighted user).
The results were initially a bit confusing.... and I didn't realise at first that I could click on each picture (sighted user).
The suits descriptions were not clear, I made a guess at what it meant (visually impaired user).
10.4.4 OPAC
Twenty-six users (seventeen sighted, nine visually impaired) said that the OPAC results were easy to read and understand. Comments included:
Yes, I think it is a great OPAC actually (sighted user).
The results were arranged in quite a systematic way that you quite quickly get used to (sighted user).
Yes, absolutely easy, (the screen reader) has a very good speech output and it will read out everything to you uninterrupted if you wish (visually impaired user).
Three (visually impaired) users said they found the results hard to read and understand, and seven (three sighted, four visually impaired) said the results were confusing. Comments included:
Some of the results were easy but some (recent issues received) were a bit confusing (sighted user).
I don't think it is obvious what the publisher is - you wouldn't know, it could have been a bit clearer really (sighted user).
The language was not obvious to me, wasn't sure what Serial Holdings meant, but if I used it a bit more I would get used to it (visually impaired user).
The links were clear, but what was behind them was difficult to understand (visually impaired user).
There was not enough information (visually impaired user).
When I hit "begin search" it said that I had one entry, but it listed other entries - the design of the table was unclear (visually impaired user).
The text was broken and horrible, I couldn't highlight either to improve it - although I wouldn't want them to think that I can just highlight it, I would prefer better text (visually impaired user).
10.5 Was it clear where you were in the task?
Users were asked if it was clear where they where during each task.
10.5.1 Search Engine
Twenty-one users (sixteen sighted, five visually impaired) said that it was clear where they were during the Search Engine task. Comments included:
Quite straightforward to use, no problems. Did not get lost at all (sighted user).
I knew where I was in the sites and could go back if necessary, so I didn't feel lost at all (sighted user).
Did not get lost, quite happy. Could press Back to get me back through the layers of results (visually impaired user).
I tried to do it in a methodical way, so no, I didn't really feel lost (visually impaired user).
Six users said that they got lost (all visually impaired) and six (one sighted, five visually impaired) were sometimes confused as to where they were during the task. Comments included:
Oh I got lost! But that doesn't bother me at all, I just go back and try again - if I get lost because there are so many links then I might say "oh sod it" and give up (visually impaired user).
Yes, I got lost, you saw me! Basically I go back and start again (visually impaired user).
I got lost at several points, not just one.... I don't think that was very clear, it is not an easy search engine (visually impaired user).
I felt pretty clear about the path I wanted to go down, it was just that it took me to different places (visually impaired user).
I only got lost when things didn't happen when I expected them to - then I didn't know what to do next (visually impaired user).
Seven of the users (four sighted, three visually impaired) expressed mixed feelings, saying that at times they felt lost, at other times it was clear where they were. Comments included:
When I hit F3 I got lost, that can happen a lot - you just click on something and don't know where you've gone. Also, with someone watching you, you feel stupid and take longer to recover, but I didn't feel it was worse than any other site (visually impaired user).
It wasn't always as clear as it could have been really (sighted user).
I got lost a little bit with the number of times I had to go into different pages to find what I wanted - too many hyperlinks (visually impaired user).
I didn't feel lost, although I did feel frustrated that I was on sites that I thought should be weather forecasts but gave lots of other information, all sorts of stuff (visually impaired user).
10.5.2 Directory
During the Directory task, twenty-five of the users (eleven sighted, fourteen visually impaired) said that it was clear at all times where they were. Comments included:
Yes, it was ok, once you got used to using the site you would be able to navigate around quite quickly (sighted user).
Didn't get lost because I used the Back button and could see where I had been before (sighted user).
I think I knew where I was up to - it was better defined at the top of the page and the links were better defined too (visually impaired user).
Clear at every stage (visually impaired user).
Can't remember having got lost in that one (visually impaired user).
Three users (two sighted, one visually impaired) said that they got lost and one user (sighted) said they felt confusing about where they were during the task. Comments included:
It wasn't clear what category I had to go into, I felt lost right from the very start (sighted user).
I didn't get lost, but I wasn't sure what the categories were (sighted user).
I felt a bit lost and anxious, it wasn't easy (visually impaired user).
Eight of the users (four sighted, four visually impaired) expressed mixed feelings - saying that at times they felt lost and at other time it was clear where they were.
At the beginning I got lost because I didn't know where to find the information but then I found the alphabetical list (sighted user).
When I clicked on General References it took me to a page where there were too many choices - I felt then that I could get lost and I might have to go back and start again (sighted user).
I got a little bit lost in what I was looking for, because for some items, it was not clear what the link was for (visually impaired user).
I didn't go too far because most of it I couldn't read (because the text was broken) (visually impaired user).
10.5.3 Online Shopping
Twenty-one of the users (thirteen sighted, eight visually impaired) said it was clear to them where they were at each stage of the task. Comments included:
It was ok, and there was always a point where you could go back to the Home Page (sighted user).
Yes, pretty obvious where I was, one or two pages took a long time to load though (sighted user).
I didn't feel lost, I just felt the information wasn't very forthcoming, it wasn't very good (sighted user).
It was clear where I was up to and I could find the actual pages, like the next page and previous page (visually impaired user).
It was very clear, it was quite good (visually impaired user).
I knew where I was on that one - you can quickly get used to that page, I've only been onto it once before (visually impaired user).
Only two said that they felt completely lost (both sighted) and three said they felt confused at times (one sighted, two visually impaired). Ten users (four sighted, six visually impaired) expressed mixed feelings about the task, saying that at times they felt lost and at other time it was clear where they were. Comments included:
Definitely got lost! I ended up in haberdashery or something. I got stuck in a loop and couldn't get out, so that's not good (sighted user).
No, I didn't have a clue, I didn't know what to do with the suits and jackets (sighted user).
Not lost, I just didn't know where I was going, I didn't feel I could go beyond suits, my confidence ran out (visually impaired user).
More lost than any other others - pages seemed to take a long time to load as well (visually impaired user).
10.5.4 OPAC
Twenty users (fourteen sighted, six visually impaired) said that it was clear to them where they were at each stage of the OPAC task. Comments included:
It was clear to me and I didn't get lost - but I know the interface (sighted user).
You just go step by step, it is very hierarchical (sighted user).
Fairly clear, also I am familiar with the interface (visually impaired user).
It was quite obvious I had got the right title etc (visually impaired user).
I thought that was a pretty straightforward search (visually impaired user).
Only three users (all visually impaired) said they got lost and three (one sighted, two visually impaired) said that they felt confused. Comments included:
Needed to use a bit of ingenuity, couldn't find anything using the links so tried it in navigation mode (visually impaired user).
When I first did the search and brought up the first page of results, I thought that was all the information, I didn't realise that there would be more levels such as a Title Details link (visually impaired user).
I definitely got lost, it's one of those sites which you need to keep using to get familiar with (visually impaired user).
The distinction between what information would be displayed in Recent Issues and Title Details was not clear (sighted user).
When I clicked on things I sort of expected all the information to come up, not have to follow more links (visually impaired user).
Twelve users (five sighted, seven visually impaired) expressed mixed feelings - saying that at times they felt lost and at other time it was clear where they were. Comments included:
Not lost, just lost faith! The interface is not untidy, but I couldn't be sure what I was hearing was what was actually going on (visually impaired user).
I got a bit lost when I was looking for what it was called before (former title) but with a bit of thinking you realise that there has got to be a link somewhere (sighted user).
A little lost, it was the last ditch effort to go for the Title Details, it wasn't telling me what I was going into (visually impaired user).
10.6 How satisfied are you with the task performed?
Users were asked how satisfied they were with the task performed. Based on the comments given, answers were grouped into positive comments, neutral comments and negative comments.
10.6.1 Search Engine
Ten users responded in a positive way to the Search Engine task (eight sighted, two visually impaired). Comments from both samples included:
Yes, satisfied, although I would have liked the Met Office link to be on top, but it wasn't really far away. (Sighted user)
I view it as a sense of achievement, but I wouldn't have done it at all given the choice. (Visually impaired user)
Eighteen of the users (eleven sighted, seven visually impaired) responded neutrally to the Search Engine task, comments included:
Quite satisfied, I think if I had spent more time I could have found more general forecast, with temperatures etc. (Sighted user)
Fairly satisfied. If the connection had been quicker it would have been better. (Sighted user)
OK, but it took a lot longer that I would have wanted it to. (Visually impaired user)
50%, well I found the Manchester forecast, I didn't expect to do that. So, reasonably successful as I have never used (this search engine) before. (Visually impaired user)
Fairly. I wouldn't use (this search engine). (Visually impaired user)
Negative comments were made by twelve of the users (one sighted, eleven visually impaired). Comments included:
Not very (satisfied), I couldn't find the information and some of the sites seem to have some problems. (Sighted user)
Well, for me it was a lot of action for something very simple! UK national weather - I should have found more easily really.(Visually impaired user).
Not very (satisfied), it was spoilt by the fact that I couldn't find the UK weather (Visually impaired user).
No, frustrated (visually impaired user).
10.6.2 Directory
Twenty-six of the users made a positive response to the Directory task (twelve sighted and fourteen visually impaired). Comments included:
100% (satisfied) (Sighted user)
Very satisfied. (Sighted user)
Yes, more than average. (Sighted user)
Very happy, it was easy and there were no "accidents". (Visually impaired user)
I was satisfied in the end with what I got. (Sighted user)
Yes, I think that was a good one. (Sighted user)
Yes, fine, no problem.(Visually impaired user)
That was good. (Visually impaired user)
Neutral comments were made by eleven of the users (seven sighted, four visually impaired). Comments included:
I am happy with the result, but getting there was a bit hit and miss. (Sighted user)
Well, I found some information relating to the Solomon Islands so I could have used that as a grounding to look for further information. (Sighted user)
The task itself wasn't too bad, but I was guessing all the way through really. There was nothing I could really say yes, that's what I want. (Visually impaired user)
Only two of the users made negative comments (one sighted and one visually impaired). A comment by one user was:
It just takes so long - sighted people just go click, click, click, and there's the answer, what else do you want to know? While I'm still looking for the first bloody link, it's very frustrating! (visually impaired user)
10.6.3 Online Shopping
Thirteen users made positive responses to the Online Shopping task (five sighted, nine visually impaired) Comments included:
Yes, that was ok really. Some of it was irritating, but with the Search I was able to find what I wanted. (Sighted user)
Yes, happy. (Visually impaired user)
Fine, although I won't be coming back to the site! (Visually impaired user)
Neutral comments were made by nine users (six sighted, three visually impaired). Comments included:
Task was alright, but not happy with the information they gave me. (Sighted user).
It was ok, it wasn't too hard. (Sighted user)
Medium - I got what I wanted but it took time and most was unreadable. The nice clear text didn't have the information that I wanted. (Visually impaired user).
Seventeen of the users made negative responses (nine sighted, seven visually impaired). Comments included:
I would have been happier with a search option rather than me doing all the work. (Sighted user)
I'm not satisfied, I'm disappointed. In a way I'm more disappointed because it started off so well. (Visually impaired user).
No, I'm not happy with that one. (Sighted user)
Very unsatisfactory - I'm dissatisfied. (Visually impaired user).
10.6.4 OPAC
Positive responses were made by twenty-one of the users (fifteen sighted and six visually impaired. Comments included:
Definitely satisfied! (Sighted user)
Completely (Visually impaired user)
I wish I was a student here! (Visually impaired user)
Neutral comments were made by twelve of the users (four sighted and eight visually impaired). Comments included:
That was ok, it was quite quick and there were no errors with the software, so it was quite good. (Sighted user)
With a bit of help it was alright. (Visually impaired user)
It'll do! (Visually impaired user)
Quite satisfied ironically - the main reason is that I found what I wanted. (Visually impaired user)
Six users made negative responses to the OPAC task (one sighted, five visually impaired). Comments included:
Not very at all (sighted user)
I can't really say that I was satisfied because I didn't complete it (visually impaired user).
10.7 Were any error messages or pop-up windows explained?
10.7.1 Search Engine
During the Search Engine task, two users experienced error messages (one sighted, one visually impaired). Both users said that they just ignored them because they didn't know what they meant. Both users backed out of that page and tried again.
10.7.2 Directory
One user (sighted) had a pop-up window appear during the Directory task, the user just "clicked on x" to close it.
10.7.3 Online Shopping
Six users (four sighted, four visually impaired) experienced error messages or pop-up message windows during the Online Shopping task. Comments about error messages included:
Didn't know what they meant, I have seen them before but don't understand them (sighted user).
I just leave alone messages such as the proxy server message - I might just turn it off and wait for a bit, it means nothing to me (visually impaired user).
Comments about pop-up message windows included:
Pop-up menus are informative, but not always easy to close (visually impaired user).
I just closed it, didn't read it! (sighted user).
10.7.4 OPAC
No error messages or pop-up message windows occurred during the OPAC task.
[ Previous Section: General Questions ] [ Next Section: Summary of Findings ]