New Comparative Assessment (CA) Job

The project manager must fill out the Comparative Assessment job type form.

CA job form fields

FIELD NAME	DESCRIPTION
Name	A unique and meaningful name for internal tracking.	Required
Min. evaluators	A job must have a minimum of one evaluator. More evaluators can be added later in the project, if necessary.	Required
End Date	Deadline for the evaluators. The end date can be modified later on.	Required
Scoring System	The Scoring System dropdown will only show scoring systems that are compatible with the selected job type. The scoring system can be changed until one or more evaluators has accepted the job. See more information about HEVAL Input File Naming Convention	Required
Randomize MT	When this box is ticked, the segments are shown to the evaluators out of order to avoid any bias in the evaluation.	Optional
Description	This optional field allows the project manager to add some additional meta data about the project.	Optional

CA input data format

The input data format requirements for Comparative Assessment jobs are as follows:

CA job input data

File format	CSV
Separator	Semicolon (;)
Minimum number of segments	2
Maximum number of segments	150
Column 1: Source	The source text
Column 2: MT1	The output from MT model 1
Column 3: MT2	The output from MT model 2
Column 4: MT1_model_name	Name of MT model 1
Column 5: MT1_model_name	Name of MT model 2
Column 6: metadata	Internal metadata for tracking and data analysis
Column 7: Additional_information	Internal metadata for tracking and data analysis
Optional Column(at any position): Original_Id	Internal metadata for tracking Original data Id. Maximal length 64 characters
Optional Column(at any position): Domain	Internal metadata for tracking domain category. Maximal length 64 characters