New Automated Evaluation Job
A new automated evaluation job can be created by clicking the New Evaluation Job button on the Automated Evaluation page. A modal window will appear where you can provide the relevant details for your job, including Configuration, MT Configuration, Source Language, Target Language, Note, Text Format, and Source, Target and Reference data (as a TXT File or TMX File).
First, you need to define a Configuration which will be used for execution of the automated evaluation job. For more information about configurations and how to create and maintain them in your workspace, please visit the Configurations section under Settings.
Next, if you want to relate the automated evaluation to a specific MT configuration, you can do so by selecting the relevant MT Configuration from the drop down list. This is not a required field, but it is recommended to correlate your automated evaluations with the appropriate MT configuration for improved grouping, sorting and analysis of evaluations.
Once the Configuration is selected, you must select the Source Language and Target Language of the data you are submitting for evaluation. Selecting the correct source and target languages ensures that the correct language-specific settings are used during evaluation.
The Note field is optional, but it is recommended to use this field, as it allows users to easily find and group specific evaluations for improved analysis options.
The Text Format field allows users to specify the format of the content that is being submitted for evaluation. MT Companion can treat the input as XML format or plain TEXT. The difference between these two options is that XML format preserves in-line tags while TEXT format treats the entire segment as text (including tag marks ‘<’ ‘>’). Tag-enriched XML text might be broken and evaluated incorrectly if it is treated as TEXT format. The default setting for this field is XML.
Now you can select from two scenarios:
Evaluate Text File , or
Evaluate TMX File
For evaluation of content in plain text format, select the TXT FILE option in the dialog box. This will allow you to upload your Target (MT) file, Reference file (human or post-edited translation), and Source file. Use the Choose File button to select and upload the corresponding file for evaluation. Note that you can evaluate only one set of corresponding files per job and all three files must be perfectly aligned (their row count must be the same for all of them).
For evaluation of content contained in TMX files, select the TMX FILE option in the dialog box. This will allow you to upload your Target (MT) file and Reference File (human or post-edited translation). In this case, the source text is already contained in the TMX file, so there is no need to upload a separate source file. Use the Choose File button to select and upload the corresponding file for evaluation. Note that you can evaluate only one set of corresponding files per job.
Whether you uploaded TXT files or TMX files, when the Execute button is clicked, the files are uploaded, and the job starts processing immediately. The dialog box disappears and the automated evaluation job list Auto update function is automatically switched on (to see the progress of the job that was just created).
Note: Once the evaluation results are available, you can switch Auto update off to avoid any disruptions.