NLLB-Maloba-h1 — french → lingala translation

The project's translation model. 288 human-translated pairs were enough to gain 8.6 BLEU points; adding 4,700 machine-made pairs brought nothing. The lesson holds for the whole community: it is human translation that moves Lingala forward.

Result

BLEU 32,00 contre 23,37 pour la base. Source : E01, E02. Voir la page Résultats.

Mesures automatiques (E01, E02)
Mesure	Valeur
BLEU, test interne	32,00
BLEU, modèle de base	23,37
BLEU, FLORES+	16,39
BLEU FLORES+, modèle de base	15,49
Significativité	p ≤ 0,001

Jugement humain (trois validateurs) : 66 % de sorties correctes à la majorité, 97 % correctes ou récupérables ; accord α 0,19 : indicatif.

The measurements, as charts

Base model against fine-tuned model — Test interne — 79 phrasesThe project's test set, 79 sentences, never seen during training.

Source : E02, bootstrap apparié de 1 000 tirages

Show the chart data

Measure	Base	Fine-tuned	Gap	p
BLEU	23,37	32,00	+8,63	≤ 0,001
chrF	54,38	61,74	+7,36	≤ 0,001

Base model against fine-tuned model — FLORES+ devtest — 1 012 phrasesFLORES+ is an external reference of 1,012 sentences, independent of the project: the gain is not an artefact of the internal test set.

Source : E02, bootstrap apparié de 1 000 tirages

Show the chart data

Measure	Base	Fine-tuned	Gap	p
BLEU	15,49	16,39	+0,90	≤ 0,001
chrF	50,31	52,09	+1,78	≤ 0,001

Training, epoch by epochh1 climbs until epoch 4 then falls back: that is the point that was kept.

Source : F04, courbe de validation · data

Show the chart data

Epoch	Validation chrF
1	45,51
2	56,23
3	60,13
4	62,40
5	61,68

The rejected version: 4,700 extra machine pairsh2 improves on its own validation but falls back significantly on FLORES+. It was set aside. It is the project's most useful result on translation.

Source : F04, courbe de validation h2 · data

Show the chart data

Step	Validation chrF
75	60,64
150	61,65
225	64,31
300	64,24
375	63,86

What it does well, what it does badly

What it does well

BLEU 32.00 against 23.37 for the base model on the in-house test set; significant gain (p ≤ 0.001).
Gain confirmed on FLORES+ (16.39 against 15.49), the external reference of 1,012 sentences.
66 % of translations judged correct, 97 % recoverable.

What it does badly

Human judgement is indicative only: weak agreement between validators (α 0.19).
Input limited to 256 tokens: not suitable for long texts.
CC-BY-NC 4.0 licence inherited from NLLB-200: no commercial use.

How it was built

Socle facebook/nllb-200-distilled-1.3B, adapté par affinage complet sur 288 paires humaines. Données : 288 paires humaines. Détail dans le rapport F04.

Licence

CC-BY-NC-4.0 : non-commercial. Un modèle garde la licence de son modèle de base, quelle que soit celle des données d'entraînement. Gouvernance des données.