Presentation of Results

Goal achievement:
Task type Task 1 Task 2 Task 3 Task 4 Task 5
Participant
100 100 0 100 100
1
Participant
100 100 100 100 100
2
Participant
Participants 100 100 100 0 0
3
Participant
100 100 100 100 100
4
Participant
100 100 100 100 100
5
Mean 100 100 80 80 80
Median 100 100 100 100 100
Goal Achievement (%) 100- 100-
Range 80-100 80-100 80-100
100 100
SD 0.00 0.00 44.72 44.72 44.72

In the above table, we used Goal Achievement as a metric to measure the percentage of the
participants who finish the whole task or part of it. Some of them did all tasks, but there are
others who did not finish all tasks for different reasons. For example, a participant asked to
leave the test. Another one could not finish the work because it is hard for him. From the data
collected, the range of all tasks goal achievement is between 80 and 100. That mean most of
the participants have finished all of the tasks

Time on task:
Task type Task 1 Task 2 Task 3 Task 4 Task 5
Participant 1 6:34 10:12 5:30 4:40 5:45
Participant 2 3:10 6:47 4:12 9:09 4:24
Participants Participant 3 3:17 6:44 2:25 4:58 6:10
Participant 4 2:56 8:23 3:29 5:47 4:58
Participant 5 4:00 4:10 2:43 2:59 4:04
Mean 4:23 7:07 3:47 5:22 5:28
Median 3.58 7.07 3.38 4.09 5.33
Time on task
Range 3.78 6.02 3.05 6.5 2.06
SD 1.49 2.24 1.26 2.39 0.86

In this table, we calculated all times for each participant in each task; some users took more
time because some technical issues, such as slow internet connection. Moreover, most of the
participants took an extended period because many fields need to be filled in posting a job
task. In contrast, task 3 which is looking for best ISP took less time for all the participants.
 Error Rating:
Task Error description Number
Number rate of issues

1 p1 ● The user could not reach the job Board section even 1
with direct links on the homepage.

p2 ● Results shown after searching for the word “JOBS” 2

are just on the discussion board; no result direct the
user to Job Board Section.
● The user could not reach the job Board section even
with direct links on the homepage.

P3 ● The User clicked on a random link. Maybe the page 1

layout and text density cause this action.

p4 ● Results shown after searching for the word “JOBS” 1

are just on the discussion board; no result direct the
user to Job Board Section

p5 ● The User clicked on a random link. Maybe the page 1

layout and text density cause this action.

2 p1 ● Error while trying to post a job without login 1

p2 ● Error while trying to post a job without login 3

● Error the website did not check for date validity.
● The location field is confusing; the users try to enter a
complete address while the required is only the
postcode.

p3 ● Error while trying to post a job without login. 2

● The user cannot publish more than one job post
every time.(may be website terms and conditions)
ask rosemary

p4 ● Error while trying to post a job without login 2

● The location field is confusing; the users try to enter a
complete address while the required is only the
postcode.

p5 ● Error while trying to post a job without login 2

● Error the website did not check for date validity.

3 p4 ● When the user tries to create a new thread without 1

login, the website asks for credentials, however,
accidentally typing the wrong password will not
notify the user of that mistake and it will show him
the page again.
4 p2 ● Error when tries to edit without login. However, the 1
message is not clear and doesn’t ask for login.

p5 ● Error when tries to edit without login. However, the 1

message is not clear and doesn’t ask for login.

5 p1 ● Error when tries to click on feedback without login. 1

However, the message is not clear and doesn’t ask
for login.

The table above shows the error rates that the participants faced while attempted the usability
test. It is evident on the table that many errors happened for each participant. In many cases,
the website notification system was not helpful regarding directing participants. It happened
due to the poor notification messages. In task two, four users try to post a job without login.
They all faced an error page that says “Access Denied” with no further instructions. Moreover,
task three, if a user tries to login into the discussion board with wrong username and password
combination, nothing will happen, and the page will show no effect. Finally, an error occurs in
the job posting application. If a user entered the wrong date for application due date, the
system would not notify him, and the post process will be done with wrong info.

Completion rate:
Task type Task
Mean 1.8
Median 2
Range 3
Standard Deviation 1.30384
Assisted Completion Rate 2/2 2/2 1/2 2/2 2/2
Unassisted Completion Rate 0/5 0/5 0/5 0/5 0/5
Completion Rate 5/5 5/5 4/5 4/5 4/5

Success in Benchmark:
Task type Task Task type
Success in benchmark Mean 86.2
Media 76
n
 Range 44
SD 18.17
Value 7 9 9 8 88
5 1 4 7

In the success in benchmark usability metric, the median for the mean time that has been taken
from each task was calculated. After that, it was set to the reference value. The reason that lies
behind using it in such a way is that some users spent longer time than usual due to the lack of
the internet connection speed. Therefore, using the mean will not give a correct estimation.

The calculated values show that task one and four has got the least success rate. For task one,
some users try the search tool to look for job board section. They all ended up with discussion
board results that contain all the threads having the word job in the title or body. There were
not a single result that leads to the job board section. As a result, more time was spent on this
task.

Finally, Task four is considered hard due to the vague method for changing the knowledge base.
The user cannot change the first two wikis on the page; this makes a great confusion and some
users get back to the main page for assistance.

Assisted Completion rate:

Task type Task 1 Task 2 Task 3 Task 4 Task 5

One Assistance 1 1 1 1 1
Assists
Two Assistance 0 1 0 1 1

Three Assistance 0 0 0 0 1
Mean 33% 67% 33% 67% 100%
Assisted rate
Median 0 1 0 1 1
Range 1 1 1 1 0
SD 0.57 0.57 0.57 0.57 0

In the Assisted rate metric, we calculate assists show that all tasks have got at least one assisted.
The reason of that is how to login the website in the test because the participant did not register
on this site before looking for the section on the website quickly due to confused with losing
the right section.Therefore, the facilitator guides them to the right place that contains correct
section. Finally, in the section talk with the mod, there was no clear way to write a feedback
because that the fifth user has tried to find the correct link

Satisfaction:
1. How do you rate your overall satisfaction of the site?
This figure show the overall satisfaction on the website, 2 out of 5 said the website is very bad
after their experience through all the tasks.

2. How easy do you think the website is?

This figure shows if the website is useful or not, 2 participants said the website is easy and
another 2 participants said is little bit hard to use the website.

3. How easy do you think the site was to achieve the tasks you were given?
This pie chart shows how the participant feel about doing the tasks, 80 percent said it's easy
to achieve but only 20% said kind of easy.