What are evaluators looking for? How do I succeed?

Remember to take a look at our article on common mistakes in evaluations.

When assigning ratings to your task, our evaluators look at three key categories: accuracy, fluency and style. In order to succeed in your evaluations, you need to perform well across all three categories.

Below is a brief rundown of what evaluators look for in each category:

Accuracy

• Is the text accurately translated overall?
• Are there additions to the target translation which were not in the source?
• Are there omissions - elements of the source text that have not made it into your finished version?
• Has a word been mistranslated? This is usually where a choice of word in the source text does not match the word you have submitted in the target language.
• Has a word been left untranslated?
• Has a word been translated that should have been left as it is in the source?

Fluency

• Is the grammar correct? This depends a lot on the language, but some common aspects are: agreements, tenses, moods, aspect, correct conjugation of verbs, correct use of pronouns, etc.
• Is the word order correct? This also varies depending on the language - for languages with fluid word order, evaluators will look at best practice for syntax and general fluency, but for languages with fixed word order you will also be marked on your adherence to syntactical norms.
• Is the punctuation used correct and appropriate?
• Does the translation use whitespaces and hyphens correctly?
• Are words correctly capitalised where necessary?

Style

• Has the translation used the correct register? There are many variables here, but in particular, evaluators are looking to see if the translation fits with the language guidelines and the translation instructions on the individual task, as well as what is the social norm for that language.
• Is the translation written in the correct variety, as set out in the language guidelines? Is it free from interference from dialects or other language varieties?
• Does the translation comply with the glossary?

The task’s compliance with these guidelines is then converted into a star rating from 1* to 5*:

1 star * The text is of bad quality, and has severe accuracy, fluency and style problems and does not convey the meaning of the source text

2 stars ** The translated text is of poor quality, and has major problems that may affect the accuracy, fluency and/or style, making it difficult to read

3 stars *** The translated text is of fair quality, but contains some issues that particularly affect its fluency, causing the text to sound unnatural

4 stars **** The translated text is of good quality. It is accurate and fluent, containing only minor errors that do not have much impact on the comprehension of the text

5 stars ***** The translated text is perfect, with none or very few errors (e.g. an extra whitespace or a small spelling mistake)