Participation & shared task procedure

The WMT shared task on sign language translation follows the procedures established by the shared task on machine translation that has been running since 2007. Here, we are outlining the participation process in order to make it more clear for new participants. 

Please note: The participation is about exchanging ideas, not about winning the competition! We encourage broad participation -- if you feel that your method is interesting but not state-of-the-art, then please participate in order to disseminate it and measure progress. The community may use your idea in a better manner and cite your paper, or even learn something useful from the negative results. 

Step 1: System training

Step 2: Processing of the test data

Step 3: Submission of the system outputs 

Furthermore, submissions should use the WMT XML format. We provide the test sources in this XML format, see the data tab. The exact submission format we require depends on the translation direction:

DSGS-to-German translation

In this case system outputs are German text. We require that these outputs are combined with the XML file containing the test sources, see here for an example.

Text-to-sign translation

The outputs in the sign languages must be one mp4 video for each line of spoken language input text. Participants are free to submit any content they deem suitable as a translation. Examples: avatar animations, videos featuring photo-realistic signers or a pose estimation video.

Output videos should be wrapped in XML as well, but instead of text the <segment> elements in the XML should contain public links.

Important: we will attempt to retrieve public links right after the deadline.  On this day videos must be available to us. If possible, please make sure that the links are downloadable directly via wget command or similar.

<?xml version='1.0' encoding='utf-8'?>

<dataset id="">

  <doc origlang="de" id="srf.0">

    <src lang="de">


        <seg id="0">Guten Abend meine Damen und Herren - willkommen zur "Tagesschau".</seg>



    <hyp system="YOUR SYSTEM NAME" language="dsgs">


        <seg id="0"></seg>





Example: public URL of a DSGS system output video wrapped in the WMT XML format

Step 4: Human evaluation

Step 5: Paper submission