PROMT participates in Covid-19 MLIA Eval – a European project aimed at summarizing multilingual information related to the coronavirus pandemic. The project organizes a community evaluation effort aimed at accelerating the creation of resources and tools for improved MultiLingual Information Access. Machine translation by PROMT got the highest scores in the first round of the evaluation effort.

Covid-19 MLIA Eval is a project promoted by the European Commission, ELRC(European Language Resources Coordination), ELRA(European Language Resources Association), CLARIN(European Research Infrastructure for Language Resources and Technology) and CLEF Initiative. The project website says the goal of Covid-19 MLIA Eval is to make information on COVID-19 readily available in multiple languages so that everyone could access reliable information from a variety of sources. 

The project evaluates machine translation systems presented by developers from different countries. Covid-19 MLIA Eval comprises three rounds; each round evaluates the quality of machine translation for different language pairs. The first round evaluated the quality of translation for English-Spanish, English-German, English-Italian, English-Greek, English-French and English-Swedish language pairs. 

Developers can participate with both trained systems (Constrained), then training data is provided by organizers, and systems that haven’t been trained to translate the provided content (Unconstrained).  Participants use their systems to translate the provided text – information related to the pandemic. Results are measured using BLEU score and ChrF score – automatic evaluation metrics that compare machine translation output and human translation. 

The project participants in the first round were developers of MT systems from all over the world: specialists from large companies and universities, such as Charles University, Tarjama. PROMT has also participated in the project and was ranked top, according to the results of the first round, started on October 23. PROMT translation for English-Spanish, English-Italian, English-Greek, English-French and English-Swedish got the highest scores.


Participation in Covid-19 MLIA Eval is coordinated by Alexander Molchanov, Head of Statistical and Neural Translation Department at PROMT. "We are glad to participate in this important project that will help people quickly get reliable information related to coronavirus – with machine translation they will not only be able to access news from around the world, but also, for example, learn about the properties of medications or updates on new vaccines and treatments for Covid-19. Being ranked top for nearly every language pair in the first round we are not going to stop, we’re preparing for the second round,” Alexander said.