Introduction to post-editing
1. Introduction: Why post-editing MT outputs?
Bartolom Mesa-Lao
bm.ibc@cbs.dk
Center for Research and Innovation in Translation and Translation Technology
Copenhagen Business School, Denmark
22/05/2013 SEECAT project
This hand-out presents the basic concepts of post-editing in the localization industry.
Aims of the session:
To acquire basic concepts about post-editing.
To reflect on the concept of quality in localization.
To identify different types & levels of post-editing.
To present general post-editing guidelines.
Contents
1. Introduction: Why post-editing MT outputs? ....................................................... 2
2. Machine Translation ................................................................................................ 2
2.1. MT integrated in the localization process......................................................... 3
3. Basic concepts in post-editing .................................................................................. 4
3.1. Defining Post-editing........................................................................................ 4
3.2. Post-editing vs. Translation .............................................................................. 5
3.3. Post-editing vs. Revision .................................................................................. 5
3.4. Post-editor profile ............................................................................................. 6
3.5. Pre-editing and controlled language ................................................................. 7
4. Common MT errors ................................................................................................. 8
5. Quality in Translation .............................................................................................. 9
5.1. Quality concepts in Localization .................................................................... 11
5.2. Quality of post-edited material: assessment ................................................... 12
6. Types of post-editing .............................................................................................. 13
6.1. Fast post-editing ............................................................................................. 14
6.2. Full post-editing.............................................................................................. 14
7. General post-editing guidelines ............................................................................. 14
7.1. Guidelines for fast post-editing ...................................................................... 15
7.2. Guidelines for full post-editing....................................................................... 15
8. Post-editing effort and productivity...................................................................... 16
8.1. Temporal post-editing effort........................................................................... 17
8.2. Cognitive post-editing effort .......................................................................... 17
8.3. Technical post-editing effort .......................................................................... 17
9. References................................................................................................................ 17
1 of 18
Is it really necessary for a translator to acquire post-editing skills? If the machine will replace
the work of a technical translator, why acquiring these new skills? The answer is simple.
Technical translators need to acquire these skills or at least be familiar with the peculiarities of
this task because there is currently an increasing demand in the market to post-edit texts coming
from machine translation (MT) engines in order to attain different levels of quality.
From the industry perspective, there are several reasons for using MT: a) to lower productivity
prices, b) to publish more content, c) to publish into more languages, d) to publish in less time.
In a recent survey carried out by TAUS (2010), 52% of the sixty seven companies in the US,
Europe and Asia declared that they provided post-editing services on a regular basis to their
clients, and that 74% of the resources they used to carry out the task were freelance translators.
As MT is being improved, the role of post-editors might eventually change but there will be a
need for their involvement in the process of creating automatic output either by editing the
output or implementing changes to the corpus or engines. For example, post-editors could be
involved in selecting the adequate corpus and cleaning up the data so the output is more suitable
for a particular customer as well as providing constant feedback to improve the engines
performance.
There is room for translators in this new field but there is also a need to be prepared and
acquire knowledge so translators can be the best capable resource to carry out these tasks as
well as to contribute to the development of MT and post- editing techniques and guidelines.
According to Vasconcellos and Len (1985), who led the first post-editing experience at the
PAHO (an organization with one of the longest traditions on MT implementation and post-
editing), their experience has led to the conclusion that post-editing requires a trained
professional translator because only an experienced translator will be aware of the words
whose variable meanings are dependent on extra linguistic context. Text disambiguation
requires the attention of a translator with training, experience, good knowledge of the subject
matter, vocabulary in both languages, and technical understanding of what is meant by the text.
Also, they explained that the post-editor is the professional best fit to give feedback about the
engine and to suggest improvements.
Moreover, acquiring post-editing skills might be a good practice in translation training. As
Kliffer (2008) concludes, following an experiment where translation students post-edit raw
output, post-editing drove impressed upon our students the importance of a holistic approach to
interpreting the source text and translating the phrase rather than the word. The activity also
provided them with a taste of what to expect if they undertake a career in translation. He also
remarked that the experience was confidence building for students and increased their
motivation.
As a conclusion, training in post-editing does not only serve the purpose of acquiring new skills
for MT related tasks but it also helps to open up a different perspectives in the already known
translation tasks.
2. Machine Translation
The definition of machine translation on the homepage of the European Association of Machine
Translation (EAMT) reads:
Machine translation (MT) is the application of computers to the task of translating
texts from one natural language to another. One of the very earliest pursuits in
computer science, MT has proved to be an elusive goal, but today a number of systems
are available which produce output which, if not perfect, is of sufficient quality to be
useful in a number of specific domains. (EAMT 2008)
2 of 18
Although the definition is broad, since computers are used to translate texts in other forms that
are not called machine translation, such as translation memories, it reflects the use of MT
today. MT should be useful in a number of specific domains but not necessarily a replacement
for human translation. The idea of a fully automatic high quality translation (FAHQT) has been Phase 1: Phase 2: Phase 3:
replaced by a more practical use of human aided machine translation (HAMT) within restricted Translation memories (TMs) Machine Translation (MT) Post-editing by humans
environments.
Machine translation is used in different industries more or less successfully, especially in those
that produce large contents of highly repetitive nature (as is the content in the localization
Hybrid text
industry) that can be easily understood by an engine. MT is frequently associated with Translation 100% translated
Source text Machine but with
controlled language and controlled translation because if technical writers of source texts follow 0% translated
memory
Translatoin MT
(TM)
repetitive syntactical patterns, they will facilitate the implementation of MT solutions in a given (MT) errors
Human Translator
company, thus increasing their translation capacity and saving costs. Even in this case, not Post-editor
everything is automatic in MT; there is a need for human interaction either before or after the
machine has processed the data. The intervention before the machine processes the data is called
pre-editing and it occurs at the source-language level to change language structures so that the Hybrid text
machine-translation engine is not confronted with ambiguous options. The intervention after the (only translated Target text
with retrieved Untranslated 100%
machine processes the data is called post-editing and it occurs at the target-language level to matches) segments?
translated
x % translated
correct frequent errors in the machine-translated output. Post-editing is still essential to produce
an end-quality product, meaning an end-quality product without frequent language mistakes
found in the machine-translated output.

2.1. MT integrated in the localization process
The standard localization workflow consists of a pre-production or analysis phase, a production
phase and a post-production phase. During the pre-production phase, files are analyzed to
establish type of files, subject matter, language combination and volume by means of word-
counts, thus establishing the complexity of the project. This information serves to calculate the
most frequent variables in a localization project: time, cost and quality. The word-counts are
frequently done using a computer-aided tool (CAT), such as SDL Trados, MemoQ, Dj Vu or
a clients proprietary tool. Project Managers or Localization Engineers, depending on the size of
the agencies, carry out word-counts against an existing translation memory (TM) using a
specific language combination. This process determines the level of full and fuzzy matches in
the text. These figures are used in all the financial transactions of a localization project
(quotations, purchase orders and invoices). There are standards already set for different levels of
fuzzy matches and projects are paid and charged according to these standards (even if fuzzy
match payment experiences some variance in the market).
In recent years, however, there has been a change in the workflow of localization projects.
Many of the main software developers have introduced a new variable: machine translation
(MT). The problem arises precisely at this point because as with any new practice, there is the
need to create new processes. These processes, in turn, are based on answers to new questions.
How should MT segments be charged and paid? How much time would a translator take to
complete the task of post-editing? How should this task be scheduled? What is the
corresponding TM fuzzy match value for MT segments? Should the same localizers be used or
is there a new professional profile needed?
Machine translation is not generally used in isolation but it is included in the same workflow as
existing TMs. MT is used in the localization industry as a new form of TM, assumed to be less
perfect because it has not been created entirely by human translators, but it is introduced in the
same workflow. In this way, translators are asked to use a given CAT tool and download (aka
pretranslate) the existing segments in order to modify or post-edit them. A particular segment
could come from a TM or directly from the MT raw output. There is, in fact, a new hybrid
model created using a combination of MT and TM segments.
Figure 1. Current translation workflow for most language service providers (LSPs)
3. Basic concepts in post-editing
In this first section we would look at the basic concepts necessary to understand the nature of
this task as opposed to other already frequent tasks in translation/localization.
It is quite common that students and professional translators are trained (academically) in
translation strategies and theories, but it is rarer to be trained for revision and post-editing.
Therefore, it is advisable to have a clear idea of the tasks involved in post-editing and revision,
as well as in translation itself, as well as to have a basic knowledge of how MT operates.
Looking at different concepts will help us to define the task and focus on its execution.
3.1. Defining Post-editing
Post-editing can be defined as reviewing a pre-translated text generated by a MT
engine against an original source text, correcting possible errors, in order to comply
with a set quality criteria in as few edits as possible (in general).
That is, the post-editor reads the output provided by the MT engine, observes possible errors,
checks the original in case of doubt and corrects the text according to the quality that has been
agreed originally with the customer. It is important to underline that we are speaking of a set
quality criteria and not a personal idea of translation quality. It is also important that the post-
editor performs these changes in as few edits as possible, thus increasing his or her productivity.
Other definitions given by experts in the field of post-editing and revision are: to edit, modify
and/or correct pre-translated text that has been processed by an MT system from a source
language into (a) target language(s) (Allen 2003:296) or revising the output of a machine
translation program, where revising means the process of checking a draft translation for
errors and making appropriate amendments (Mossop 2001: 168-169, italics in original). In this
last definition, we would assume that in post-editing the draft translation is the MT output
and the post-editor has a role similar to the reviser and therefore carries out similar tasks, mainly
checking for errors and correcting them. However, the nature of these errors is different which
makes the post-editor to consider other factors.

3 of 18 4 of 18
Post-editing: examination and correction of the text resulting from an automatic or semi-
automatic machine system (machine translation, translation memory) to ensure it complies with
the natural laws of grammar, punctuation, spelling and meaning according to the Draft of
European Standard for Translation Services (in Joscelyne 2006).
In this last definition, post-editing also refers to the edition of TMs outputs. Although post-
editing MT outputs and TM outputs tend to run in parallel, they require different skills or at
least they require a different focus on different type of errors. We will see that when comparing
post-edition with revision later on. Although it is not mentioned in Joscelynes definition, it is
important to highlight that the task of post-editing is closely related to the set quality
expectations within a project.
3.2. Post-editing vs. Translation
Now that we have a definition of post-editing. How does post-editing differ from translation?
And how is post-editing related to translation?
There are many theories that give different definitions of translation such as the traditional,
functionalist or communicative approach. However, translation is seen in localization as an
individual step in which the source text is given an equivalent target text. The EN-15028 (the
European quality standard for translation services) defines translation as the rendering of the
written text in the source language into the target language. On many occasions this is only one
single string of source text rendered into another string of target text. Translation, as most of us
understand it, is something more sophisticated and broader, that encompasses an in depth
knowledge of each language and culture in order to communicate the same meaning in both
languages. In the localization industry, however, a simpler concept is used.
In technical translation, the standard translation process is: translators translate the source text
using a substantial amount of given reference material (style guides, glossaries, dictionaries,
term banks and TMs). Then, they will or should revise their work and correct any possible
mistakes. And finally, if there is enough money in the budget to afford that, a reviewer will go
over the translation again and check issues to do with language (including specific
terminology), transfer and layout.
The difference at this point is that, during the post-editing task, the translator already has a draft
version of the source text (MT output) and depending on the quality provided by the MT engine,
the output might require a) translating again from scratch (if it is not useful), b) correcting quite
a lot of errors, c) correcting a few errors or d) simply accepting the proposal without any
change. Therefore the post-editor is faced with two source texts (the actual source and the MT
proposal). In this sense, post-editing is closer to reviewing than to translating. During this
process, translators will use known translation and revision strategies and also new strategies
(that would be described later on) for this type of text.
On occasions, post-editing can be done by a native speaker that does not speak the source
language but that simply revises the target text to conform to the language and layout rules of
the target language. This approach can be extremely dangerous as the monolingual reviewer
might try and decipher obscure passages from MT and simply choose the wrong alternative. The
source text does help in clarifying the output provided by the MT, if obscure, and therefore a
bilingual reviser is of essence.
As a conclusion, post-editing would be a task among the many tasks that a translator can
perform and that belong to the realm of translation but it is not actually to translate although
the post-editor might have to translate an entire sentence because the MT proposal has to be
discarded.
3.3. Post-editing vs. Revision
According to Brian Mossop (2007) revising is that function of professional translators in which
they identify features of the draft translation that fall short of what is acceptable and make
appropriate corrections and improvements.
5 of 18
In a commercial setting, revising is carried out in order to improve texts, supervise quality
produced by contractors, as well as revise work done by new employees or contractors.
Sometimes, this step is not carried out at all for time or budget constraints and sometimes
because the process is already defined as such and it is deemed more efficient not to revise.
Although the EN-15038 specifies that the revision needs to be carried out by a third party, not
all translation companies follow this standard. The fall in the price of translation has also
contributed greatly in the elimination of this quality step.
Post-editing also involves revising but the main difference is the source text, while in post-
editing the text comes from a MT engine (output) when revising, the source is a translation done
by a human translator.
As a consequence the resulting target text contains different type of errors than those found in a
human translation. This type of errors will need to be corrected in a different way depending on
the purpose of the text. As Laurian (1984) states post-editing is not revision, nor correction,
nor rewriting. It is a new way of considering a text, a new way of working on it, for a new aim.
Krings (2001), who has carried out the most comprehensive post-editing research to date, also
points out that this task deals with recurring, predictable errors, while revising checks for
mistranslation or omissions. Later on we will see the most frequent errors found in raw output,
but in general terms, the errors done by a human translator are randomly made and
unpredictable while MT follows certain patterns that can be anticipated according to the
language combination, the type of text and the engine used. On some occasions human errors
are more difficult to spot but at the same time the texts are easier to read as they follow a
human logic. Post-editing involves revising a text that might follow an odd syntactical
structure. This type of texts put a strain on the person reviewing that it is quite different to the
effort required to revise human translations. As Krings points out working with three different
texts in the post-editing situation with source text (source text, machine translation, and the
subjects own target text) leads to an additional cognitive load vis--vis normal translation with
only two texts involved. In conclusion, the task of post-editing appears to be a more
demanding task than translation in terms of cognitive effort.
What seems to be clear is that both revising and post-editing require specific skills, and that
translators are key agents in both activities.
3.4. Post-editor profile
After analyzing what post-editing is and the difference between this task and other translation
related tasks, it would be a natural step to look into the profile needed to carry out the task and
the differences from those requirements needed for a translator.
As we saw before, most translation agencies use their regular pool of freelance translators to
post-edit MT outputs. Each company has its own set of pre-requisites used when recruiting
freelance translator such as: a) native speakers, b) degree in translation or relative subjects, c)
certain experience as translators, d) experience on the subject matter, e) experience on a set of
tools and technology related requirements. Once freelancers are selected as possible candidates,
they usually pass a series of tests and fill in questionnaires related to their experience, and
finally they are tested on the job.
Are these the same requirements needed for a post-editor? Not all freelance translators make
efficient post-editing tasks, and not even all efficient freelance translators. Obviously, those
freelance translators that stand out in the performance of their tasks will have more possibilities
of meeting the requirements of a good post-editor than those that already come short or barely
meet the companys expectations in terms of translation quality.
OBrien (2002) describes some of the post-editing skills required, adding to her own view, the
view of other experts on the field. The skills can be summarized as follows:
Degree on Translation and Interpreting or related subjects.
Previous experience on localization and/or technical translation.
Expert in the subject area and target language.
6 of 18
 Proficient knowledge of the source language and contrastive knowledge of source and target Avoid the use of more than three nouns.
languages. Avoid too many adjectives modifying a noun.
Advanced word processing skills; full key proficiency and efficiency in cursor positioning. Effective Use determiners.
use of search and replace functions. Avoid spelling mistakes and make sure punctuation is correct.
Positive, tolerant and open minded predisposition towards MT. Use the active voice.
Confidence in abilities and technical expertise. Use that, in order to and which after verbs that admit omissions.
Recognition of typical or repetitive MT errors. When using phrasal verbs, make sure that the preposition is as close to the verb as possible.
Ability to use macros and coded dictionaries. Repeat prepositions in conjoined constructions.
Advanced terminology management skills. Use parallel structures in coordinated sentences.
Background knowledge of MT technology and history including types of post-editing and different Use always the same term for the same item/product: avoid synonyms.
levels of expected quality. Use general dictionary terms rather than obscure terms.
Pre-editing and Controlled Language skills. Knowledge of controlled authoring tools. Use acronyms and abbreviations that will not cause ambiguity.
Programming skills (for automatically correcting errors).
For example:
Text Linguistics knowledge.
When reading this text, make sure to take notes.
Some of these skills are shared with those of a translator. However, there are additional skills
such as MT technology knowledge and tolerance, pre-editing and controlled language skills or When you are reading this text, make sure that you take notes.
programming skills that are not normally required when looking for translators to take part in a The consistency of the source text guarantees a smooth process when using MT or TMs and
post-editing project. reduces costs for the companies that use it. Additionally, it avoids translators to constantly
query for obscure passages in the text.
3.5. Pre-editing and controlled language However, controlled language is not always performed on the source texts that will then be
There are several pre-editing techniques that allow reducing the post-editing effort. These are: machine translated and eventually post-edited. Although the post-editing time is reduced
following a style guide (technical writers), controlled terminology (using a set of unique terms considerably, the initial investment required in order to apply controlled language is high, and
when writing) and controlled language. therefore companies might avoid this step. Post-editors will find that a vast number of texts that
they will work with would not be written using controlled language nor will they be pre-edited.
Controlled language means that the source language (e.g. a technical text) is written in a
standard way to avoid lexical ambiguity and complex grammatical structures, and thus making
it easier for the user to read and understand it and consequently easier to apply technology to the 4. Common MT errors
text such as TMs or MTs. As a consequence texts have a consistent and direct style, they can be There are several classifications of MT errors. The aim of classifying the errors is not only to
easily reused, they are easier and cheaper to translate, and easier to read. Controlled language improve MT output by providing feedback but also to raise awareness amongst post-editors. If
focuses mainly on Vocabulary and Grammar and it is intended for very specific domains, even they know the type of errors frequently found when performing this task, it is easier to spot
for specific companies. It is indeed useful to create high quality MT output but also to avail them and to know what to change, thus avoiding unnecessary changes.
fully of existing TMs (avoiding fully matches with minor or unnecessary lexical or syntactical It is important to point out that depending on the type of engine, the content and language pair
changes throughout a text). Basically, controlled language will help disambiguation of terms the type of errors might change considerably. These are just examples of errors and of error
and sentences by keeping a very high level of consistency both externally (terms) and internally typology.
(grammatical structure).
Laurian (1984) distinguishes between three types of errors:
In order to use controlled language, writers write following certain rules to avoid correcting
errors at the post-editing phase. In a way, using controlled language means optimizing the 1. Errors on isolated words.
2. Errors on the expression of relations.
whole process as to make a better use of MT.
3. Errors on the structure and on the information display.
It is useful for translators to be familiar with these controlled language rules in order to
understand the possible problems the MT engine will face and therefore spot output errors more These errors are subsequently classified in three tables:
rapidly. 1.1. Vocabulary, terminology
Overview of general rules of controlled language include (Mitamura 1999, Rico and Torrejn, 1.2. Proper names and abbreviations
2004): 1.3. Relators: in nominal groups and in verbal groups,
1.4. Noun determinants, verbal modificators;
Write short sentences. 2.1 Verb forms (tense),
Use simple grammatical structures: for example avoid complex and ambiguous subordinate 2.2 Verb forms (passive/active)
sentences. 2.3 Expression of modality or not,
Use sentences with nouns rather than using pronouns. 2.4 Negation;
3.1. Logical relations, phrase introducers,
For the same process, step or idea, write the same sentence. 3.2. Word order
Write complete sentences with noun, verb and compliments. 3.3. General problems of incidence.
Avoid the use of gerunds and participles. For example, ing after.
When , While, if, Where or participles not introduced by that. Schffer (2003) from SAP offers the following error classification:
1. Lexical errors

7 of 18 8 of 18
 1.1. General vocabulary
1.1.1. Function words (articles, pronouns, conjunctions)
1.1.2. Other categories (verbs, nouns, adjectives)
1.2. Terminology
1.3. Homographs/Polysemic words (words like uses, report and starts)
1.4. Idioms (MT systems will tend to translate them literally)
2. Syntactic errors
2.1. Sentence/Clause analysis (wrong analysis of structures, relative pronouns, use of commas)
2.2. Syntagmatic structures (wrong interpretation of past participle, for example)
2.3. Word order
3. Grammatical mistakes (for example, the translation of the pronoun IT or gender in the romance
languages or phrasal verbs carry out, porter dehors in French instead of excuter).
3.1. Tense
3.2. Number
3.3. Active / passive voice
4. Errors due to defective input text (mistakes in the source language)
Krings (2001), on a similar line, classifies errors from the MT output of this extensive
study as below. The classification is not intended as a general one but to his particular output.
However, it is useful to see how errors were classified in this extensive study.
Lexical: Part of speech recognition error: verbs recognized as nouns or vice versa.
Lexical: Other: wrong use of certain terms in the context.
Morphology: Word formation: wrong formation of words. For example, Drhten des Telefons
instead of Telefondrte.
Morphology: Other: incorrect infinitive form, incorrect plural form.
Syntax: Word order
Syntax: Other: wrong use of infinitives
Stylistic usage norms
Punctuation: incorrect comma usage
Textual coherence: incorrect gender of anaphoric reference form, inconsistent form of address for
text addressees (Du and Sie)
Textual pragmatics: inappropriate form of address for text addressees.
Literal transfer from ST
He rightly points out that several MT errors can overlap; each error can sometimes be assigned
to different categories.
Although all these classifications are valid for their specific purpose of a particular engine or
project, for the sake of simplicity and practicality we would look at examples of errors classified
in four main areas (similar to Schffers classification):
1. Terminological (verbs as nouns, nouns as adjectives, wrong use of term in context, abbreviations)
2. Grammar and Spelling (tense, gender, number, active and passive voice)
3. Syntactical (Syntactic errors/errors on the structure and on the information display, Word order)
4. Punctuation and Style (upper case and lower case, formatting, form of address)
5. Others: Additions, omissions
5. Quality in Translation
Quality in translation studies is a much debated subject. Different definitions are offered
depending on the school of thought. Defining quality is almost as elusive as defining
translation itself. Knowing when a translation is good is not as easy as it may seem to the
professionals or regular people outside the field.
There is an array of translation theories dealing with the basic concept of translation and
defining what a good translation is. As Chesterman (2000) points out the current pool of
translation memes is a highly heterogeneous one. Some might give more importance to the
9 of 18
word for word translation, the source text and equivalence, the target text and the receiving
reader/culture, the communication act and the role of translator as mediator, the purpose
(skopos) of the translation, or even the mental state of the translator and her cognitive processes.
Of course, every theory draws from the previous one and they all seem to live together, not
altogether in harmony, but at least in constant development through these same differences.
It is obvious then that depending on the translation theory a good translation will be classified
differently. What might appear to be good for one theory might not be sufficient, and sometimes
completely wrong, for another theory. Quality is therefore an obscure and elusive concept. In
MT the predominant theory, as Chesterman (2000) reflects, is equivalence in its most pure
form: strict equivalence is a sine qua non. Instead of waffling about mystical energy,
practitioners of machine translation are concerned with practical rules of language use. They
have to believe that rules exist, and that they are as stable as those of gravity. Pym (2004) also
points out that equivalence is the prevailing translation theory behind all processes in
localization.
And it is not only in translation theory that we find divergent points of view, it seems that
professionals in the translation field have their own very particular view of what a good
translation is and sometimes if they are queried about it, (what is quality for you?), it is hard for
them to come up with a definition.
When translation is a transfer of a source string into a target string with the least amount of
changes and at maximum speed, equivalence becomes the prevailing concept, even without
being conscious about the theory behind it, in any translators behavior when translating. I
would add that the skopos theory also plays a very important role, as the purpose of the
translation and all the players involved in the translation activity play a fundamental role in
localization and in machine translation post-editing. In this context, a good translation is the
one that renders an equivalent target text according to the skopos of the project in question.
Therefore, the translation quality should be judged according to these variables and not
according to an abstract notion of linguistic quality.
In the localization industry, quality is frequently seen as a series of procedures carried out in
order to guarantee a linguistic quality that is then again very volatile and that tends to be
simplified by classifying errors in different categories and counting them. The translation will
be a Pass if the overall count reaches a level, or a Fail if the overall count is below a level. In the
first case, the overall quality is deemed to be good enough.
Brian Mossop who has written a complete and intelligent guide on editing and revising for
translators (2001) distinguishes between quality control and quality assessment and
explains that both contribute to quality assurance.
Quality control occurs before delivering the translation to the customer and it involves all the
steps necessary to provide a translation that fits into the customers needs. Quality Assessment
might occur after delivery and it consists in identifying problems in a text to establish if it meets
the professional standards in the translation company.
Therefore, quality control is text oriented and client/reader oriented and quality assessment is
business-oriented.
He also explains that when revising an overall quality levels need to be considered. He
distinguishes then four types of overall quality levels that I found extremely practical. The first
level (A) is Intelligible: a translation that is readable and clear, and roughly accurate. The
second level (B) is Fully accurate: the translation avoids misleading the reader, it is fully
accurate, but it is only fairly readable and fairly clear. The third level (C) is Well written: the
final translation is fully accurate, clear and quite well tailored and smoothed. And finally the last
level (D) is Very well written: the reading experience is in itself and interesting and enjoyable,
quite apart from the content.
He rightly points out that, in general, translators aim at C level, well written texts or even the D
level, when in fact in some cases only level A or B is required by a specific customer.
10 of 18
5.1. Quality concepts in Localization
In Localization, the concept of quality is considered an implicit value provided in all
translations carried out by Language Service Providers (LSPs) or freelance translators. Quality
is in most cases a given value, an assumed service provided to the client. When working in a
translation agency or as a translator, to deliver good quality is a must. Good quality is, then,
variable depending on the customer, its product, audience, style guides, reference material, and
QA group, amongst others. Since Quality is difficult to define, everyone refers to it in very
general and abstract terms.
What customers refer to quality translation is that translation, especially in technical translation
and localization, reflects exactly the content of the source text. What does reflecting exactly the
content mean? As we saw before, translation is perceived as the rendering of an equivalent
text, almost as a word by word exercise. The target text should contain exactly what the source
text contains with minor exceptions, that is, few adaptations to the local markets. On the other
hand, the LSPs use a much more functionalists approach, that is, the quality provided varies
according to the translation brief discussed with the customer, the focus is on the customers
needs and what they pay for. If they do not pay for review, well then the translation is not
reviewed by a third party. Translators, on the other hand, have different approaches. On some
occasions, they will work for a customer oriented purpose and, on other occasions, they might
work towards their idea of quality; an idea that is related to the use of correct grammar and
language style.
The truth is that there is not much time allowed in localization to offer a very well written
translation (in Mossops definition), and we aim at a well written translation in most cases,
while reality more often than not obliges translation providers in general to produce a Fully
accurate and even Intelligible translation.
Most localization agencies, however, will follow procedures to guarantee the quality of the
translated products. These procedures cover everything from correctly selecting the translators
to checking the quality of the translation or offering the right translation brief during the project.
This set of procedures is normally known as Quality Assurance (QA) and it is designed to
assess the quality of products or services provided. QA implies that a series of steps are taken in
order to guarantee quality and that corrective actions are in place in case errors are detected in
the product or service. Normally, companies will use procedures and indicators to monitor this
process.
Wikipedia offers a very clear definition of Quality Assurance: Quality Assurance refers to
planned and systematic production processes that provide confidence in a product's suitability
for its intended purpose. It refers to a set of activities intended to ensure that products (goods
and/or services) satisfy customer requirements in a systematic, reliable fashion. QA cannot
absolutely guarantee the production of quality products, unfortunately, but makes this more
likely.
Two key principles characterize QA: "fit for purpose" (the product should be suitable for the
intended purpose) and "right first time" (mistakes should be eliminated). QA includes regulation
of the quality of raw materials, assemblies, products and components; services related to
production; and management, production and inspection processes. It is important to realize also
that quality is determined by the intended users, clients or customers, not by society in general.
Quality Control (QC) is the application of QA for a particular project and it will happen during
the life-time of the project while QA will normally be part of the quality processes within the
company.
There are different Quality Standards used in the localization industry. The most important ones
are: ISO 9000 Series, ASTM F2575-06 and EN-15038. The advantage of complying with a
Quality Standard is that the company will register all steps and procedures in the company and
have periodical audits that guarantee the quality of the translation. However, an agency can have
processes documented and zero customer complaints without necessarily being certified. It only
serves as an indication that you might be able to produce a quality translation.
11 of 18
If we apply all the concepts that we have seen before we can conclude that quality is not a set of
grammatical rules set on stone or an ideal to try and reach, it is a variable concept that will very
much depend on the characteristics of a given project as defined on many occasions by the
customer and by the translation agency.
More often than not, there will be no clear information about the quality of the MT output.
Depending on who is providing the information about the output, the quality feedback could be
overly enthusiastic or extremely negative. It is rare to receive a serious analysis of the output
with samples and scores. Some MT output providers might send an automatic score (Blue,
Meteor, NIST or TER) that gives information on how close the output is to human quality with
a single number. Unfortunately, this number might mean very little in practical terms.
It is advisable to assess the output for each language combination using different parameters (for
example, Grammar, Terminology, Format) in a randomly selected set of strings extracted from
the overall content (that could be classified according to segment length) where a post-editor
can then classify the quality of the segment (Excellent, Good, Poor, or even from 0 to 4, or any
other classification).
Even though time is required for this assessment, it will give a clear idea of the productivity
savings the team of post-editors might be expected to obtain during the project. If the post-
editor and the translation team do not have this information, they are working pretty much in the
dark in terms of prices and might be overwhelmed by the number of e-mails sent by post-editors
complaining about the quality of the output with little data available to discuss the matter.
The customers quality expectations for the final project need to be very specific as post-editing
can be superficial or thorough depending on the purpose of that translation.
As in general revision terms, there are different types of expected quality levels. Post-editing is
in general classified in two: Full post-editing leading to human quality translation and rapid
post-editing with minimal corrections for text gisting. Between these two options, there is a
wide range of alternatives. Establishing the quality expected by the customer will help
determining the price as well as writing specific instructions to post-editors. If this is not done,
some might correct only major errors thinking that they are obliged to utilize the MT proposal
as much as possible, while others will correct major, minor, and even acceptable proposals
because they feel the text has to be as human as possible. In general terms, customers know
their readers and the type of text they want to produce. Post-editors should have a very clear
idea of the expected quality. Otherwise, they will not be able to start the assignment.
5.2. Quality of post-edited material: assessment
One of the reasons to introduce MT in the localization cycle is to save costs. It would not make
much sense, then, to review the post-edited text. However, on certain occasions revision might
be necessary to obtain either a very high quality or to determine post-editors competence. The
reviewer should receive the same information as the post-editor. Most localization companies
use review forms that comply with LISA, J2450 standards or similar ones created within the
company. LISA defines 7 categories of errors. These are:
Mistranslation
Accuracy
Terminology
Language
Style
Country
Consistency
Format
Mistranslation refers to the incorrect understanding of the source text; Accuracy to omissions,
additions, cross-references, headers and footers and not reflecting the source text properly;
Terminology to glossary adherence, Language to grammar, semantics, spelling, punctuation;
Style to adherence to style guides; Country to country standards and local suitability;
12 of 18
Consistency to coherence in terminology across the project and Format to correct use of tags,
correct character styles, correct footnotes translation, hotkeys not duplicated, correct flagging,
correct resizing, correct use of parser, template or project settings file.
The errors found are then assigned a severity than can be Minor, Major and Critical. All errors
are weighted according to this severity. For example, an error classified as Minor weights 1
point, if classified as Major, 5 points, and finally if it is deemed to be Critical it is worth the
total amount of allowed errors plus 1.
Similarly, the J2450 errors are classified as:
Wrong term
Wrong meaning
Omission
Structural error
Misspelling
Punctuation error
Miscellaneous error
Errors are then divided into Serious and Minor, and each category is assigned different points
according to these two subcategories.
Wrong term is similar to the previous Terminology, and it refers to not adherence to the
customer glossary, wrong term for the domain, inconsistent term translation, and wrong
translation of a particular term throughout. Wrong meaning would be similar to Mistranslation
and Language, and it refers to wrong word order, incorrect syntagmatic structure, and wrong
grammatical category. Omission would be similar to Accuracy and it refers to missing text.
Structural error corresponds to the previous Language, and it refers to wrong word structure
(case, gender, number, tense, prefix, suffix) and agreement error. Misspelling would correspond
to Language and it refers to problems of orthography in the target language. Punctuation error
corresponds to Language, and it refers to the text complying with complying with the target
language punctuation rules. Miscellaneous errors correspond to Mistranslation and other
categories in LISA and it refers to literal translation, register issues and mistranslation issues.
The J2450 does not contain a Style category because it has been mainly designed for the
automotive industry.
Another alternative would be to create specific post-editing categories. The reviewer should
consider at least the following points:
Accuracy: To what extent the post-edited version contains the same information as the
source text?
Language: Is the Language appropriate? There are no Spelling and Grammar mistakes, the
text follows the customers Style Guide and style is idiomatic.
Terminology: The terminology follows the linguistic reference material provided for the
project and it is consistent.
There could be a rating (from 0 to 4) or an error count in order to provide a final result. Post-
editors should receive this feedback to accelerate their learning curve.
These are only samples and other reviewing aspects can be considered (Readability, Clarity,
Transfer, and Logic) as there are multiple ways of assessing quality. It is crucial, nonetheless, to
establish a relation between the speed and the quality of post-editing because there is no point in
having post-editors with a high productivity rate that do not provide the expected level of
quality.
6. Types of post-editing
As we saw before, there are different levels or types of post-editing. This level or type will be
determined by several factors such as:
13 of 18
The engine used
The language pair
The desired quality specified by the customer or purpose of the translation
The volume of documents that needs to be translated
The time available for the translation
The structure of the given text
They type of readers or users for that particular text
The use of the final text
Depending on these factors, there will be different levels ranging from Full post-editing leading
to human quality or rapid post-editing with minimal corrections for text gisting.
In MT and post-editing, it is frequent to differentiate between texts that will be read quickly, for
internal use and perishable, and texts that will be published and are intended for a wider
audience. In the first case, the texts needs to be understandable and accurate, but the style is not
fundamental and it even admits some grammatical and spelling errors. In the second case, the
text needs to be understandable and accurate, but also the style, grammar, spelling and
terminology need to be similar to the one provided by a human translator. The texts are
classified as well as needed for assimilation (roughly understand the text from another language
that is not yours) or dissemination (publish a text for a wide audience from your native language
into several others, depending on the different aims, the level of post-editing will vary.
Lets not forget that we might have MT output directly published in the Internet. This means
that no post-editing is done in the text and it is published as it is. Normally this type of texts will
have a disclaimer explaining the reader that the text has been translated by a machine.
6.1. Fast post-editing
Fast post-editing evidently points out to the fact that very little corrections are necessary to
publish the text. It is also called gist post-editing, rapid post-editing and light post-editing. It is
used in general for texts that are needed urgently and will have an internal, perishable use,
normally emails, reports, meeting agendas, and very specific technical reports for a small
audience. Allen (2003) specifies that rapid post-editing (part of an inbound translation approach,
meaning that MT is used for acquisition or assimilation or gathering of information) is to
provide minimal editing on texts in order to remove blatant and significant errors and therefore
stylistic issues should not be considered. It is important to point out that fast-post-editing is
also meant to be done in the shortest time possible and, thus, with the minimum number of
changes and keystrokes.
6.2. Full post-editing
On the other hand, full post-editing belongs to the outbound (dissemination) approach (Allen
2003) and it is aimed at a much bigger audience.
In full post-editing, the objective is to obtain a text that corresponds to a human translation,
meaning that the reader will not be able to tell if what he or she is reading came from a machine
or a human translator/writer. In this case, the raw output requires maximal editing not only to
remove blatant and significant errors but to correct all errors and style so the final text is
compliant with the language stylistic norms and also with the customers specific terminological
and stylistic rules. As with fast post-editing, the task is meant to be done in less time that
translating from scratch as to not defeat the purpose of using MT output.
7. General post-editing guidelines
There should not be a post-editing projects without specific post-editing guidelines. These
guidelines are not the same as Style Guides, Project Briefs or Localization kits describing
instructions, technical and linguistic, for the project. Post-editors need language specific
guidelines created for the actual post-editing task.
14 of 18
What should these guidelines cover? Obviously it is difficult to answer this question as it will
depend on the quality of the output, language combination, and the usual variables in MT.
Besides, post-editors cannot be burdened with a whole book on post-editing, as time is of
essence and their work needs to be profitable. The guidelines should be short and precise and
they should cover the following areas:
Description of the type of engine used.
Description of the source text (type and structure of source text).
Brief description of the quality of output for that language combination.
Expected quality by the customer (as described above).
Scenarios when to discard a not useful segment (post-editors should have an idea of how much time
to spend in order to recycle a segment or discard it altogether).
Typical type of errors for that language combination that should be corrected (including reference to
tagging and links).
Changes to be avoided (according to customers expectations, for example certain stylistic changes).
How to deal with terminology (according to output analysis and customers expectations. The
terminology provided by MT could be perfect or it could be obsolete, or a mix alternative).
Even though, time is needed to create guidelines, the more it is devoted to create and improve
them, the better the post-editing task will be performed.
OBrien (2009) advises on general post-editing rules:
Retain as much raw translation as possible.
Dont hesitate too long over a problem.
Dont worry if style is repetitive.
Dont embark on time-consuming research.
Make changes only where absolutely necessary, i.e. correct words or phrases are (a) nonsensical, (b)
wrong, and if theres enough time left, (c) ambiguous.
7.1. Guidelines for fast post-editing
There are different types of guidelines according to the different expectations from the
customer. These are only samples that can be useful for post-editors, but when working on the
field, others might be included or eliminated.
In general terms, it is important that the post-editor should read the source segment first to
understand the meaning of the sentence. Then, proceed to read the MT suggestion and make the
necessary changes.
These are some rules that can be useful when doing fast-post-editing:
Make sure the sentence is accurate.
If the terminology in the MT output is incorrect, do not spend too much time researching.
Be careful not to post-edit word order in a sentence if the sentence can be understood even if it
violates language rules.
Do not change style or change any proposal for stylistic preferences.
Avoid replacing a word with a synonym if the original word is correct.
Do not correct grammar mistakes unless the target sentence does not reflect the source.
7.2. Guidelines for full post-editing
The same considerations we had before can be applicable to full post-editing, that is, there are
different types of guidelines according to the different expectations from the customer as we
already mentioned earlier. These are only samples that can be useful for post-editors, but when
working on the field, others might be included or eliminated.
15 of 18
Post-editors should read the source segment first to understand the meaning of the sentence.
Then, proceed to read the MT suggestion, so that they can decide whether it can be recycled in
post-editing. There are some basic pointers to help with this decision:
The suggestions should be applied if:
1. Large pieces of the sentence/term are correct (these can be reused during post-edit).
2. The raw MT quality is very high, although some minor corrections may be needed.
3. Raw MT output contains several errors which might slow down the post-editing task. However, the
post-editor types slowly, so post-editing still proves to be faster than translating from scratch.
4. The MT output has the correct meaning and it is completely understandable.
You should NOT apply the suggestion if:
1. Raw MT does not make any sense and it would take longer to post-edit than to translate from
scratch.
2. The user takes a few minutes trying to figure out what the raw MT is trying to say, but it doesnt
make sense.
3. If there are multiple errors that require rearranging most of the text.
4. Multiple tagging problems between source and the MT match.
5. There are too many changes terms to change and it will take longer than translating from scratch.
Microsoft offers some guidelines in order to make these decisions:
1. The 5-10 second evaluation rule: this is the maximum time that you should spend evaluating the
validity of the MT suggestion. If it is hard to understand at the beginning, do not read the whole
sentence, proceed to translate from scratch instead.
2. The high 5 and low 5 rule Microsoft rule. When you detect a long sentence, do the following:
Read the first 5 words. If its good, read on until its bad, then stop and copy the correct
part and continue to translate and forget about reading on.
If the first 5 or 6 words arent good, skip to read the last 5 or 6 words. If the last part of the
phrase is correct, use it, or just start the whole thing from scratch.
If both the 5 first and 5 last words are incorrect, do not carry on reading through the middle
to try to identify correct MT segments. Just discard the MT suggestion and proceed to
translate from scratch.
Once the post-editor decides to use the segment (this happens quite quickly in a real project), he
or she can follow these guidelines:
If the terminology in the MT output is incorrect, do not spend time researching this, but apply the
term as used in the approved term database.
Often the output from the MT will be repetitive; this can be used to your benefit as the post-edited
output will be more consistent.
Be careful not to post-edit word order in a sentence that does not violate semantic intelligibility
rules.
Be careful not to change grammatically or semantically correct phrases to stylistically preferred
phrases.
Avoid replacing a word with a synonym if the original word is correct.
On occasions MT suggestions might help out with translators block. This might be useful even
for fuzzy segments.
8. Post-editing effort and productivity
Productivity constitutes one of the big unknown factors in projects involving MT and post-
editing. This is partially due to the fact that using MT in localization projects is relatively new
16 of 18
and, therefore, standard metrics do not exist yet, but mainly to the amount of variables to
consider. At any rate, we have little information on productivity of translators work in general.
The industry uses standards (for example, 2000 to 2500 translated words per day) but we all
know these standards are hardly applicable to all translators. Moreover, there are also agreed
metrics on TM editing (percentages paid according to fuzzy match level), but most translators
would agree in saying that these percentages hardly represent the amount of work they need to
perform on each proposed segment. The studies dealing with productivity when post-editing
MT segments (such as Krings 2001, OBrien 2006, Guerberof 2008 and 2009) do not show
pronounced productivity increases when using MT. Frequently, however, MT developers will
claim that their engine dramatically increases the translators productivity without necessarily
making their methodology available.
There is definitely uncertainty about the gains when using MT and post-editing. A figure that is
normally used when discussing productivity in post-editing is 5,000 words per day but the
reality is that each project will have different productivity according to the different variables.
Krings (2001) discusses post-editing effort as the key element in determining if the application
of MT is worthwhile and distinguishes three main concepts necessary in order to understand
post-editing effort:
8.1. Temporal post-editing effort
Constitutes the time needed in order to correct the machine translated text according to the
given quality. If the post-editor saves time in comparison to human translation then using MT
might be a recommended tool. Depending on the type of errors and in reality on the quality of
the raw output the time involved might be more or less.
8.2. Cognitive post-editing effort
Directly related to the previous concept, the cognitive effort describes the brain effort needed
in order to resolve these MT errors. For example, it is not the same effort to have to correct a
very obvious mistake of gender where the post-editor does not need to research or consult the
source text than correcting a complex syntactical structure that renders the text ambiguous and
that requires checking the original to disambiguate, think about the possible solutions, make and
decision and actually making the correction.
8.3. Technical post-editing effort
This concept refers to the actual physical effort to correct a text. For example, if we need to
deleted, reorder, insert or carry out a combination of all of these actions. The more cursor
movements, for example, required to correct an error, the more technical effort necessary to
post-edit.
As we can see, the easiest variable to measure in post-editing is in fact temporal post-editing,
since for cognitive and technical effort we would require special tools (Translog or eye-
tracking tools) or protocols (Think Aloud Protocols) that make it more difficult to use in the
commercial world.
pp 45.
Guerberof, A. 2009. Productivity and Quality in the post-editing of outputs from translation
memories and machine translation. Localisation Focus. The International Journal of
Localisation. Vol. 7 Issue 1.
Joscelyne, A. 2006. Best practices in post-editing. In TAUS. www.translationautomation.com
Kliffer, D. 2008. Post-Editing Machine Translation as an FSL Exercise. In Porta Linguarum.
Number 9. 53-67
Krings, H. 2001. Repairing Texts: Empirical Investigations of Machine Translation Post-
editing Processes. G. S. Koby, ed. Ohio. Kent State University Press.
Mitamura, T. 1999. Controlled Language for Multilingual Machine Translation. In
Proceedings of Machine Translation Summit VII. Singapore. 13-17.
Laurian, A.M. 1984. Machine Translation: What type of post-editing on what type of
documents for what type of users. Proceedings of the 10th International Conference
on Computational Linguistics and 22nd annual meeting on Association for
Computational Linguistics. 236-238.
Mossop, B. 2001. Editing and Revising for Translators. Manchester: St. Jerome Publishing.
Mossop, B. 2007. Editing and Revising for Translators. 2nd Edition. Manchester: St. Jerome
Publishing. 109
OBrien, S. 2002. Teaching post-editing: A proposal for course content. In Proceedings for
the European Association of Machine Translation. Available from <http://mt-
archive.info/EAMT-2002-OBrien.pdf>. Last accessed May 2013.
OBrien, S. 2006. Eye-tracking and Translation Memory Matches Perspectives: Studies in
Translatology. 14 (3): 185-205.
Rico, C. Torrejn, E. 2004. Controlled Translation as a New Translation Scenario:
Training the Future User. ASLIB Conference Translating and the Computer 26.
Pym, A. 2004. The Moving Text. Localization, translation, and distribution. Amsterdam &
Philadelphia. Benjamins.
Schffer, F. 2003. MT post-editing: how to shed light on the "unknown task". Experiences at
SAP. Controlled language translation, EAMT-CLAW-03. Dublin City University.133-
140.
Vasconcellos, M.; Len, M. 1985. SPANAM and ENGSPAN: Machine Translation at the
Pan American Health Organization. In Computational Linguistics. Volume 11,
Numbers 2-3. 125.

9. References
Allen, J. 2003. Post-editing. In Computers and Translation: A Translators Guide. Harold
Somers, ed. Amsterdam & Philadelphia: Benjamins. 297-317.
Bennett, S. Gerber, L. 2003. Inside commercial machine translation. In Computers and
Translation: A Translators Guide. Harold Somers, ed. Amsterdam & Philadelphia:
Benjamins. 175-190.
Chesterman, A. 2000. Memes of Translation. The spread of ideas in translation theory.
Amsterdam & Philadelphia: Benjamins.
Guerberof, A. 2008. Post-editing MT and TM: a Spanish case. Multilingual. Vol .19. Issue 6.

17 of 18 18 of 18