Final Report

Group A3

Julia Dashevska
Pavel Okopnyi
Andreas Runeson
Laura Vetter
Laura Wiegand

Human-Computer Interaction
Uppsala University
March 2016

Translation tool

Group A3

Introduction
The goal of our project is to improve a computer assisted translation tool (CAT-tool) used by
translators of a ticket selling company. The translators work part time with both customer support
and the translation. We only focus on the translators work with the current system and the process
of translating. The project went through several phases (milestones) such as planning, data
gathering, specifying the problem and designing a solution. After understanding the translation
processes and the related problems we designed and created a prototype, which is the end result
of this project.
First, we present each milestone and go through them step by step. In the third milestone we
introduce the prototype and explain its features. The fourth milestone is a second iteration of the
prototype with further improvements and adjustments based on testing results with the users. The
fifth and last milestone includes additional feedback and the small changes that were made to
finalise the prototype. Finally, we conclude the project and discuss both the project and the
methods used.

Milestone 1
The main issue of the first phase of the project was to define the project goal and develop a first
project plan including the different phases of the project. The goal of the project, as you can see
from the introduction above, was to understand and improve the process of translating articles for a
ticket selling company. In order to reach this goal we had to go through the following main phases:
understanding, problem identification and prototype development.
Understanding
In order to understand the work processes and the sequence of the tasks during the translation
work, we conducted a contextual inquiry. During the observation, two translators were observed
and interviewed while working with the translation tool. The users we interviewed were two
students that work part time as translators, translating from English to their native language which
is Portuguese and Italian in this case. The translators often work in different environments. In our
study we interviewed them in both, the office and the home environment. In summary, we wanted
to observe how new articles were translated today into and how that process could be optimised.
Identify Problems
To identify the existing problems of the current translation tool we conducted group modelling
sessions in order to develop the following models:
Flow Model: Interaction between people (translator and the content provider)
Sequence Model: Actions performed by the translator
Artefact Model: What elements are used? What are the different settings?

Translation tool

Group A3

Furthermore, we aimed to identify user goals and tasks and processes that were conducted during
translation. We tried to find existing problems (within the process / with the tool which is used for
translation)
Prototype
After identifying the problems we planed to create a prototype to improve and solve problems with
the current translation tool. The gaol was to solve the main problems identified during the
observations using the following approach:
First, we created a paper and pen prototype that included improvements of the tool according to
the breakdowns we observed during the contextual inquiry. Second, we tested the prototype with
two users to find out the discoverability of new features and their usefulness. Last, we updated the
prototype based on the given feedback, which is then became the final product of this project.

Milestone 2
In this phase, we conducted contextual inquiries including interviews with two employees of the
company. The contextual inquiry took place at their personal workplace, i.e. at home/in the office,
and lasted about an hour and a half each. Both participants were observed while performing their
usual translation tasks and were asked questions. During the observation we focused on the
workflow of the translation process. This included the communication between the employees, the
work process of a single translation assignment, as well as the artefacts used. Furthermore we
looked at how the translators used the current tool, what tasks were performed in what order and
which technical problems were observed. We summarised the work process and created models
representing the workflow, artefacts used and sequences of actions taken during the translation.
Description of work - Translation
The translation processes we observed consisted of several articles which the translator had to
translate. There are several steps to the translation process. First, the original English text is
written by the copy-writer and passed to the project manager in a form of separate variables, for
instance: title, main text body, summary, etcetera. The project manager organises the variables into
a translation job, which is being assigned to each translator. The translator uses a CAT-tool
(Computer Assisted Translation), the application we are trying to improve, to translate the text into
its native language. In the application, the user interface is divided into two columns, containing the
original text and text fields to insert a translation. The translator writes the translation in the text
fields and ticks checkboxes to indicate that the translation is completed. During the translation
process several tools were used to communicate and verify the context of the translated text.
When the translator has finished translating all (or a part) of the text, he/she presses the Submit
button, which triggers an email to be sent to the project manager.

Translation tool

Group A3

Image 1: Translation tool

Models

Figure 1: Flow Model

Translation tool

Group A3

Figure 2: Artefacts Model

During the observation, we found several breakdowns in the work process, illustrated in the
following sequence model.

Figure 3: Sequence Model

Translation tool

Group A3

Figure 4: Breakdown 1

Figure 5: Breakdown 2

Figure 6: Breakdown 3

Figure 7: Breakdown 4

Translation tool

Group A3

Milestone 3
Based on the observations, the modelling session and the resulting models we came up with
various major breakdowns that we want to improve with our prototype. As interaction and the
sequence of tasks play an important role we decided to create an interactive prototype which can
also directly be used for evaluation. In this document we present the main breakdowns and
problems as well as the solutions which are implemented in the prototype.
The prototype can be found here http://ucsd.nosoc.io/
User Interface
During the observation we noticed that the translators were looking for workarounds to maximise
their efficiency when translating. As the texts are often very similar the translators were copying
existing texts and pasting them into variables, where they later adjusted the texts. Due to missing
variable names they had to look for variables with similar texts by scanning through the tool.
Both translators didnt know exactly where one article ends and a new one starts. As some
variables have a limited amount of characters, translators were forced to use a Word document to
count the characters as the tool doesnt do it for them. Translators were complaining about the
page navigation being located only on the top of the tool, which made them scroll up the entire
page in order to use it. We decided to solve those issues in order to minimise the workarounds.
First of all, we added names to the variables and grouped them in a meaningful way so that
variables with similar texts are placed together to make the copy-pasting easier. The variable
names correspond those names used in the entire company. Each page presents one article, so
that the order of the variable remains always the same and the translator just by looking at the
page knows where to find each variable. We also added the page navigation to the bottom of the
page. Lastly, we added a character count into the tool.

Figure 8: Improvements in the UI of the translation tool

Translation tool

Group A3

Process progress
The tool today includes check boxes. When the translation job is done the translator needs to click
all check boxes before submitting the job. The problem with check boxes is that they first of all
dont get saved once the window was closed, secondly they were more bothering than helping.
The only useful purpose that we could observe was to not use the check box when to mark an
unfinished variable. Therefore we decided to remove the check boxes and to add a red flag
function, that enables to flag a non-finished variable.
Furthermore we want to highlight in the overview, on which page a variable is flagged as a job
cannot be submitted if any variable is flagged.

Figure 9: Red flag function

Communication
One of the observed breakdowns is that there is no direct and instant communication between
translator and copywriter or project manager. Current problems and questions regarding the
original text or the translation are collected in a word document and send to the project manager
when the job is done. This results in various problems for all participants. Furthermore it is possible
that different translators find the same problem in the original text and report the same problem
several times to the project manager. All communication is currently done by email. We decided to
add a communication feature which allows direct communication between translator and related
people. Questions can clarified directly during the translation process.
The communication can still happen via email for the copy writer and the project manager, but the
translator will send the message and receive the answer inside of the translation tool. This allows
to write context-related messages without the need of taking screenshots or copying text. It is also
possible to show all questions that have been asked / answered per variable for all translators.
This helps the translator to know which problems have not been reported yet.

Translation tool

Group A3

Figure 10: The new communication function

Common phrases (originally called Dictionary)

One of the most time consuming breakdowns during translation was that one of the translators
often looked up existing translations in order to find text which could be reused in other places.
Translators often know which phrases have already been translated before and want to have a
consistent translation for all of them.
In order to improve efficiency a dictionary (later renamed to Common phrases) was added to the
prototype. The idea is to have a user-generated list of the most used phrases. The translation tool
automatically uses this dictionary to search in the current article for matching phrases and highlight
these matches directly in the original text. The translator can then see all possible matches and
chooses which one to directly add to the translation text field. It is also possible to directly access
the dictionary from the tool and manually search for common phrases. This reduced the time
spend searching other external sources for existing translations. One major point is that it needs to
be very simple and fast to add new phrases to the dictionary. Furthermore it is possible to add the
translation to the dictionary by marking the original or translated text and then pressing a button.

Figure 11: The dictionary page for searching and adding phrases to the translation

Translation tool

Group A3

Figure 12: The new dictionary function

Context Preview
As one article is split into several variable, it is difficult to understand where a translation is used
and in which context. This is especially true for new translators who are not used to the system and
the current structure of articles. We observed that finding the context for an article was one of the
reasons for leaving the tool and using Google or the original website to get a better understanding
of the context.
As the original english text has been finished already and is available we decided to add a function,
for each variable, which shows the original text of the current article. The translator can see the
selected variable, which he/she is working on, in the context of the specific article. The variable
therefore will be highlighted in the original text. If the variable is used in several places the user will
see these different options as well.

Figure 13: Preview of text embedded in the original website

Translation tool

Group A3

10

Text Preview
We observed that the translators didnt have the time to control the quality of their work. In order to
add a non time consuming help to review the text we decided to add a text preview function. This
function displays the text and hides all the other parts of the tool, enabling the users to only focus
on the text. With this we believe that spelling mistakes and/or other formatting issues can be
identified and resolved.

Figure 14: Preview of text embedded in the original website

Milestone 4
Prototype Evaluation
The goal of the 4th milestone was to evaluate the visibility and usefulness of the new and changed
functions of the translation tool. Therefore we conducted a combination of user tests based on the
prototype and interviews.
Since we only had a limited amount of working functions, little time and only two participants to
test, we decided to choose the following approach:
1. For each of the main features the participant is given a task to solve (Visibility)
2. After the task the users got a detailed explanation about the feature which they were
currently testing by one of the interviewers
3. Finally, we did an interview and discussion about this feature (Usefulness)

Translation tool

Group A3

11

With that combination of testing methods we found out how the translator interacted with the
system, if the examined features are discoverable and - after making sure that the translators
understand the new features - if they are useful.
The following features and areas were evaluated in each of the two tests:

Communication
Process (Flags)
Context and text preview
Common Phrases
General Feedback

Tests
Our first test was conducted at the translators office and took approximately 50 minutes. One
member of our team was present with the translation and the other team member (who took notes)
was connected via Skype and a shared screen. The translator interacted with the prototype sent to
him via a link on his own computer. The test session went very well, the translator was comfortable
and relaxed and we gained relevant insights.
The second test was conducted via Skype using screen sharing. It took about one hour and
consisted of one participant and two members of our team. The participant interacted with the
prototype directly (on his own computer) while the team members observed via Skype and took
notes. In general the test went well with the exception of one time when the Skype call was
dropped but this was quickly solved. No other problems occurred during the test and the data
gathered was of high quality and very helpful for future improvements of the prototype.
Main Findings
A summarised list of all the main finding with both pros and cons of our new features. The findings
are later addressed and further improvements were made.
Feature

Feedback

Comments

General UI

Pro: New style is good, new features

save much time.

Overall the new Ui is very good. The new

features seem to be very useful. Some of
the new features are hard to find and even
harder to understand at first glance. An
explanation would be great.

Con: Some of the new features might

be hard to find and understand.
Communication

Pro: Useful, will save a lot of time.

The comments feature will be very helpful

and is a much better solution to e-mailing
back and forth all the time.

Process (Flag)

Pro: Useful, better than

checkboxes.number in pagination is
very helpful

The addition of flags is much better than the

previous check boxes. Being able to see
how many flags are up in the navigation is
very useful and provides great overview of
what is unfinished.

Con: Not obvious from the very start

Translation tool

Group A3

12

Feature

Feedback

Comments

Common
Phrases

Pro: Saves a lot of time since the need

to search for phrases is eliminated.
Easy to find existing translations. A
very appreciated feature.

The participants would use it alot but the

way each participant would use it varies.
Some wants to use it for long text while
others only use it for smaller phrases. When
a well established list of phrases is
established the feature becomes highly
valuable for the translators work.

Con: Box for adding new phrases is to

small and might be distracting.
Text Preview

Pro: Useful to see the current text and

to find unfinished work and how much
is left to translate.
Con: In the current version the
formatting is breaking. Would be better
to preview the whole article instead of
each variable separately.

Context Preview

Pro: Useful for both novice and

experienced users.
Con: Not clear how to access the
feature.

The participants thought that having the

ability to view the text they are working on
with out all the cluttering of the UI and other
text areas was greater. This made it easier
to read each variable and to find errors.
However they would like to be able to keep
the same formatting as the translated
version (bold and or italic text) and also to
be able to read the entire article this way not
just each variable.
Would provide a great overview where
certain text is located and how it would look
like in the end. The button /link to access the
context preview was unclear and some
participants had a hard time finding it.

Changes
Tutorial
We are implementing a very simple tutorial that shows novice and experienced users where our
new features are located and how they work. The tutorial would work as an overlay which indicated
the location of each feature and a short text describing it. We do this since there were comments
about where to find certain features and how they worked. Some of the advanced features need
more explanation.
General UI
According to the feedback from the tests, we had another look at the general layout and UI of the
translation tool. One issue we faced after the feedback is the difference between pages and
articles. The users want to have a better overview over the article and the context. As we decided
to put one article on one page, we decided to replace the titles page x by the real headline of the
article. This will help the translators to get a better understanding of what they currently translate
and which articles they still have to work on. Furthermore, the goal is to make the identification of
the jobs easier. Therefore we added a header on top of each page where we can include the article
title, the amount of variables, the amount of open flags and comments as well as a link to the
original article to have an good overview at the beginning what they are going to translate (see
image 1). A second addition is a more detailed navigation. The idea is a slide out navigation when
clicking the translation button. This navigation shows the title of each article (on each page) and to

Translation tool

Group A3

13

also include the amount of flagged variables, the amount of comments, new / answered comments
and also to directly show which article has been submitted already (see image 2).
The goal of both ideas is to provide more context and to help translators by providing a better
orientation in the tool and to see various articles on different pages.

Figure 15: Header

Figure 16: Navigation

Communication
Based on the results of our test sessions we decided to make the comment feature similar to the
one existing in other word processors e.g. Google Docs. This includes:
1. In order to write a new comment you mark the text. That triggers a hover comment button
to appear. Clicking on that button opens the comment window. When the comment is sent
the marked text stays marked.
2. Comments posted by all translators are visible and well as who wrote each comment.
3. The main comment button visually signalised new comments by changing its colour.
4. The main comment button visualises the amount of comments.
5. The main comment button enables to see all comments or to hide all of them.
Write a new comment

Figure 17: Step 1

Mark text
Hover comment button appears

Translation tool

Figure 18: Step 2

Click "hover comment button"
Comment box appears

Group A3

14

Comment on existing comments

Figure 19: Step 1

Click marked text

Figure 20: Step 2

New comments are colour
marked in the comment field
Click on the comment box to
add a comment

Figure 21: Step 3

After the comment box has
been clicked a field and button
appear

Notifications in the header

Figure 22: Step 1

The notification icon shows amount of new
comments per page

Figure 23: Step 2

A click on a notification brings you to that page

Existing Comments

Figure 24: Step 1

Figure 25: Step 2
When
new
comments
were

All new comments are colour

added the main comment button
highlighted
is colour highlighted
When you click on a comment
It
says
how
many
new
the to it belonging text is

comments were added

highlighted more intensively
Click
on
the
main
comment

button to see all comment per

variable

Translation tool

Group A3

Figure 26: Step 3

Close either each comment
separately
Click on the main comment
button to hide all comments at
once

15

Common Phrases
The general workflow for the common phrases is going to be updated in order to be simpler and
easier to understand. Formatting will be added to the footer and the user will have some more
options (see the following screenshots).
The goal is to make the process of adding new phrases as simple as possible. The general
feedback for the idea of a common phrases list was very positive, therefore it is important to
improve the steps that are needed to add new translations. The idea of selecting and adding text
work for both tests. That is why we decided to keep this approach and improve it by adding an
Add button which appears together with the Comment button when text is selected by the user.
How to add a common phrase to the list with adding a new translation?

Figure 27: Basic translation tool

Figure 28: User selects original text

The Add Phrase button appears next to the
selected text (right side)

Figure 29: User clicks Add Phrase

Figure 30: User adds translation

A footer opens at the bottom of the page. The

selected text is directly copied to the original text
field. The footer can be minimised, closed and
resized.

User adds the translation by typing the text into the

text field (the format can be changed as well).
To add the text to the common phrases list, the user
clicks Add when he / she is done.

Translation tool

Group A3

16

Figure 31: User clicks Add to add the phrase.

Figure 32: Final translation tool

The footer disappears. If the text has been

translated inside the footer (and not copied) the text
will be added to the translation field automatically.
Both texts are highlighted and an icon shows that
the phrase has been added.

Translation has been added to the common phrased

list and the translation field.

How to add a common phrase to the list with an existing translation?

Figure 33

Figure 34

Figure 35

Action 1: User selects original text

but has translated that text
already before.
Result 1: Button appears

Action: User highlights text in the

original translation field.
Result: The text is added
automatically to the translation
field in the footer.

Action: User clicks add

Result: The footer closes. As the
translated text already exists, both
texts are highlighted and an icon
shows that the phrase has been
added to the list.

Action 2: User clicks add phrase

button
Result 2: Footer opens

How add use the common phrases in the translation tool?

Figure 36

Figure 37

Figure 38

A phrase in the translation tool is

recognised in the common
phrases list. The tool highlights
the phrase.

Action: User clicks on the

highlighted text
Result: A popover opens which
shows the suggested translations.

Action: User selects one

suggested translation.
Result: The selected translation is
added to the translation field.

Translation tool

Group A3

17

Text and Content Preview

It became obvious during our tests that these two features can be misleading. This is especially
true when it comes to the naming of these two features. First of all the features are similar in what
they do and how they look. Both provide a sort of preview of the selected variable. One without any
context (just the text and nothing else) and the other with all the surrounding context that
represents how it would look like when it is put online.
The naming of these features were confusing. The text preview was assessed using a button
called preview and that confused the participants. They believed that it would take them to a
complete preview of the entire website with its context. The context preview had a similar issue. In
this case the context preview is not a button but the variable name itself. The variable name is
linked to a representation of the finished website with its context and a small highlighter that shows
which variable you are currently previewing. Other issues related to these two previews were
related to what exactly was shown in the previews. One participant suggested that it would be
helpful to preview the entire translation with all its variables instead of just one variable at the time,
this also applied to the text preview.
We started thinking of new names for the text preview, which is currently only called preview. Some
of the ideas we had where focus or just simply text preview. Focus is an interesting name since
this function only displays the text without any context allowing the translation to focus on the text
which helps with quality control. For the context preview we believe that the current link is a good
way of accessing the feature but that proper introduction to the function and directions to its
location is needed.
For each of these features a more advanced tutorial would be required (especially for the context
preview). The tutorial needs to direct the user towards the features so they are found. When the
user accesses these features another tutorial, very similar to the other tutorial we mentioned,
would appear and explain step by step how the feature works.

Milestone 5
Based on the feedback in the final presentation, we decided to make some smaller adaptations to
the wireframes and the prototype.
General Layout
Based on the given feedback in the presentation we changed the general layout to make functions
more visible and more understandable. We added the Add Comment and Add Common
Phrases buttons to each variable. Furthermore we added a header containing the Article title, the
amount of finished variables, open flags, comments and a button showing the original article in a
new window. We removed the pagination symbol at the top and bottom of each page and replaced
it by previous and next page buttons. Finally, we updated the general layout of the page to clean
the position of the buttons and to get a general better structure.
Translation tool

Group A3

18

Figure 39: Final general layout

Navigation
We decided to add a button to the header which opens the overview of all the existing pages /
articles of the job the translator is currently working on. This overview contains a button, which
leads back to the translation tool, a button which opens the common phrases list, the overview
button and the notification icon.

Figure 40: Overview in the navigation

Variable Buttons
As some variables contain a lot of text and force the user to scroll down we decided to make the
buttons on top of each variable sticky under the current header. So if the user needs to scroll down
and would normally lose the buttons out of his/her view, the buttons will stick under the header in
will therefore not be lost for the user.
Translation tool

Group A3

19

The green area in the following wireframes shows where the buttons should stay even if the user
scrolls further down.

Figure 41: The buttons in the green area stick under the header when the user starts scrolling

Comments
For the comments functionality we added the following functions:

Figure 42: Comments close

Figure 43: Comments open

Existing comments in a variable will be shown by

adding the comment icon to the spot in the variable
where the comment is attached to.

Clicking the comment icon on the right side of the

variable opens the comment box and shows all
previous comments. It also highlights the text that
the comment belongs to.

If the user clicks on the Add Comment button on top of a variable, the following dialogue will be
shown. The original english text will have a transparent overlay, displaying a hint that the user
should select in this area to comment on. Once the user moves the mouse to this text area, the
overlay will disappear. As soon as the user has selected text, this selection will also be shown in
the comment box over the actual text field.

Translation tool

Group A3

20

Figure 42: Help text for new users

When selecting text in the english text field, two buttons will appear. One for adding a new
comment and one for adding a new common phrase. These two buttons have the same
functionality as the buttons on top of the variable but they are faster to reach as they are closer to
the users mouse pointer. The buttons on top of the variable will be greyed out in order to not
confuse the user where to click.

Figure 43: Selecting text in the original text field

Final Evaluation
As most of our final changes are based on assumptions, we need to make some final tests.
Especially the decisions concerning the commenting function needs to be further evaluated before
making final decisions. Unfortunately, those changes needed more time in order to be implemented
in the prototype so we were not able to finish the tests before the deadline for this report.
Nevertheless, we are going to get some more user feedback before the final presentation.

Discussion and Conclusion

As conclusion, we can summarise that the contextual inquiry is a very helpful method that allowed
us to get a good understanding of the work flow and provided us with valuable insights as it is more
powerful than an interview alone. We were able to collect information that the interviewees were
not aware of, as many processes are unconscious or became habits. The method was appropriate
for our project and for the time frame.
However, we also had some general problems with the method that we want to discuss. As we only
interviewed two translators, the question came up about how many participants are actually
needed to collect valid data. The main challenge we faced here was to get a general picture of the
Translation tool

Group A3

21

current work process and to improve it based on the interviews and test results from only two
users. In regards of the feedback that we received from the testing sessions, we successfully
managed to improve and satisfy the needs of those two users. However, we cannot generalise that
feedback to a bigger population. Other users might have another kind of workarounds and other
needs. In the scope of our project there is no possibility to consider this any further, in a bigger
project however we would have needed to make sure that we applied a sufficiently big, random
sample. That would have made it possible to draw a final conclusion regarding the prototype. The
same problems apply for the testing results: even though our users appreciated the changes made
and were pleased with the prototype, we cannot generalise those results to a larger population.
Additionally another challenge came up during the project that regards the user behaviour. The two
observed users applied different ways of solving the same tasks during the work process. A
specific behaviour was observed with one of the interviewees but not at all with the other. When
developing a solution, in our case a prototype, it is crucial to consider those differences and to
create a solution that can be helpful for all kind of users. In addition, we have to consider both
experienced and novice users and to address the needs of those user groups, which causes
difficulties in the design of the system.
One further challenge we faced with using the contextual inquiry was, that it was difficult to
communicate the findings and the results of one observation session with all other team members.
This resulted in a lack of knowledge about the tool and the scenario which causes problems during
the interpretation session. Findings need to be better communicated to team members who were
not doing the observation themselves. This is one problem what we can imagine that also bigger
teams in companies face, as never every team member can take part in the actual observation but
that communication is the key to share the gained insights. In our team we decided to go through
all notes together, which has proven itself as not sufficient enough as words can be simple
misunderstood and it is difficult to figure out what has actually happened during the observation.
Another method would be to record the whole observation by audio and video and to let the other
team members watch it - this is a good way to develop an understanding for the whole process.
Regarding the topic of our project we were confronted with a very simple and minimalistic
translation tool in the beginning. During the observation we could see that in some aspects the tool
caused several problems and we decided that the current design of the tool is not sufficient to face
the challenges occurring in the translating work process. In order to fix those problems we tried to
identify and solve the existing breakdowns. Nevertheless, by adding more functions which should
make the whole work process more efficient, we also added more complexity. It has been a
challenge to introduce these new functions and to make them as self-explainable as possible so
that not much explanations are needed. In total it was an interesting project as it is a real life
project with real problems and a solution used by actual users. It was a valuable experience to
use a method and conduct processes on the basis of that by ourselves. The scope of the project
was sufficient to learn methods that are useful in our later work-life, but not exceeding our possible
workload.

Translation tool

Group A3

22