When an Arabic Question Meets an English Answer

There is a comfortable story about Arabic and artificial intelligence, and it is wrong in a way that costs money. The story goes: Arabic is a harder language, so models perform worse in Arabic, so accept a discount and move on.

The research says something more specific and considerably more useful. The loss is not caused by Arabic. It is caused by a mismatch between the language of the question and the language of the document holding the answer. Ask in Arabic when the answer only exists in English and you pay a measured penalty. Ask in English when the answer only exists in Arabic and you pay one too.

That distinction changes what you do about it. If Arabic were simply harder, the response would be resignation. If the problem is a mismatch, the response is architectural: make sure the answer exists in the language the question will be asked in. One of those is a strategy. The other is an excuse.

What follows walks through five separate pieces of published research, what each one actually measured, how much, and where the honest limits sit.

The finding that reframes everything

The clearest measurement comes from work presented at ACL ArabicNLP in 2025 by Amiraz and colleagues, examining retrieval bias in retrieval-augmented generation over mixed Arabic and English corpora. The design matters as much as the result: they built corpora where the supporting evidence for a question could sit in either language, then measured what happened when query language and document language diverged.

The numbers are not small.

Embedding model	Metric	Travel domain	Legal domain
BGE-M3	Retrieval accuracy loss	13%	33%
BGE-M3	End-to-end answer accuracy loss	14%	37%
Multilingual-E5	Hit@20 loss	33%	42%
Multilingual-E5	End-to-end answer accuracy loss	37%	40%

Three things to notice, because each one has a practical consequence.

The loss reaches forty-two per cent. Not a rounding error. On a legal-domain corpus with one of the two embedders tested, nearly half the retrieval performance evaporated when the languages did not line up.

Domain matters enormously. Travel loses thirteen per cent on retrieval with one embedder, legal loses thirty-three per cent with the same embedder. The gap is not noise. Legal and technical language is more precise, more terminology-dependent and less redundant, which means a cross-lingual match has fewer alternative routes to the right document. If your category is regulated, technical or procurement-facing, you sit closer to the worse end of that range.

End-to-end loss can exceed retrieval loss. Look at the BGE-M3 legal row: retrieval drops thirty-three per cent, but the answer accuracy drops thirty-seven. Retrieving a slightly worse set of documents produces an answer that is worse than the retrieval degradation alone would predict, because the generation step then reasons over weaker evidence. Errors compound rather than pass through.

And the two metrics in that table are not the same measurement. Retrieval accuracy and Hit@20 answer different questions about the same pipeline, which is why we present them as separate rows rather than averaging everything into one headline percentage. Merging them would produce a tidier number and a false one.

Why a mismatch costs anything at all

The mechanism is worth understanding, because it tells you which fixes can work and which cannot.

Modern retrieval does not match words. It converts text into vectors, positions them in a shared space, and finds documents whose vectors sit near the query's vector. In a multilingual model, the promise is that a sentence in Arabic and its equivalent in English land in roughly the same neighbourhood.

Roughly is doing a lot of work in that sentence.

The alignment between languages in a shared embedding space is learned from data, and it is far better for language pairs with abundant parallel material than for pairs without. Arabic against English is not the worst case, but it is not English against Spanish either. Add the specifics of Arabic and the gap widens.

Arabic morphology is root-and-pattern based. A single root generates a large family of related forms carrying related meanings, which means surface forms of the same concept vary far more than in English. Sub-word tokenisers, built and tuned largely on English, fragment Arabic words into more pieces per word than they do English words. More tokens per unit of meaning dilutes the signal available to the model, spreads attention thinner, and degrades the precision of the resulting vector.

Then there is the supply problem underneath all of it, which we come back to at the end.

The important implication: this is not fixable by writing better English. If your Arabic-speaking buyer asks in Arabic and your answer exists only in English, you are relying on the weakest link in the pipeline to bridge a gap it demonstrably bridges badly. No amount of English content quality changes that. Only Arabic content does.

Models also prefer English sources, and that is a separate finding

The mismatch penalty is one thing. A preference is another, and the second is easier to misquote.

A 2025 preprint titled Linguistic Nepotism examined language preference in multilingual retrieval-augmented generation and found that models show a pronounced preference for citing English-language documents. The effect was measured with English-language queries, and it amplified for lower-resource languages. In some cases models selected an English document over a non-English one that was more relevant, which is the sharpest version of the finding: the choice was not purely driven by which document best answered the question.

Now the careful part, because this is where marketing content routinely overreaches. This research does not establish that models always fall back to English when asked in Arabic. It establishes a preference measured under specific conditions. Separate earlier work in multilingual retrieval found that retrieving in the user's language can improve Arabic results, which points the other way.

We have seen the strong claim, that models ignore Arabic sources whenever an English one exists, presented as settled fact with a precise-looking percentage attached. During research for this article, that specific figure could not be traced to any paper named by more than one source, and we do not repeat it.

What the two findings do jointly support is a more modest and more actionable position. There is a measured tilt toward English source selection, it is stronger for languages with thin source pools, and Arabic has a thin source pool. So Arabic content does not merely compete with other Arabic content. It also has to overcome a systemic lean, which means the quality and structural clarity of Arabic content matters more than the equivalent English content would need to.

Reasoning degrades too, not only retrieval

It would be convenient if the entire problem sat in the retrieval step, because retrieval is the part you can engineer around. It does not.

The XRAG evaluation, published in EMNLP Findings in 2025, assessed cross-lingual retrieval-augmented generation more broadly and found substantial spread between models. GPT-4o scored highest at 57.58 per cent. Command-R+ scored lowest of those tested at 37.17 per cent. That is a twenty-point gap between frontier models on the same task.

Two implications, and they pull in different directions.

The first is that model choice matters, which sounds obvious until you notice that almost nobody optimising for AI visibility asks which model their buyers actually use. A brand performing acceptably in one engine can be substantially worse positioned in another, and a twenty-point spread on cross-lingual capability is a big part of why.

The second is less comfortable. Even the top scorer here is under sixty per cent. Cross-lingual reasoning over retrieved evidence is not a solved problem at any tier. If your content strategy depends on a model correctly synthesising Arabic evidence in response to an English question, or the reverse, you are depending on a capability that the best available systems perform at roughly coin-flip-plus reliability.

Which brings the argument back to the same place. The safest design does not ask the model to bridge languages. It puts the answer in the language of the question.

What the best Arabic retrieval actually scores

If you are going to build for Arabic retrieval, it helps to know what good looks like, and the honest answer is: better than people assume, and not as good as marketing implies.

Work on optimising retrieval pipelines for Arabic tested embedding models against Arabic corpora and found BGE-M3 and multilingual-E5-large the strongest of those evaluated, scoring 70.99 and 70.31 respectively on the pipeline's composite measure. Those two are also the embedders in the cross-lingual study above, which is a useful coincidence: the models paying the thirteen to forty-two per cent mismatch penalty are the best available Arabic performers, not weak ones.

Separately, work by Al-Rasheed and colleagues presented at ABJADNLP in 2025 evaluated lexical retrieval for Arabic and reported Top-5 recall of 0.88 with mean reciprocal rank of 0.48, and downstream performance with GPT-4 reaching an F1 of 0.90 and accuracy of 0.82.

Read those two together and a pattern appears. Recall of 0.88 means the right document usually makes it into the candidate set. A mean reciprocal rank of 0.48 means it frequently is not at the top of that set. Arabic retrieval is reasonably good at finding relevant material and noticeably weaker at ranking it correctly.

For anyone producing content, that translates directly. Being in the candidate pool is achievable. Being the passage the system reaches for first is the harder problem, and it is decided by how clearly a single extractable unit answers a specific question rather than by how much material you have published.

What demonstrably improves it

There is remediation research too, and it points somewhere useful.

Work presented at ICNLSP in 2025 on enhancing Arabic retrieval-augmented generation reported gains of up to eight points in precision at five, an improvement of around eleven per cent in answering accuracy, and correct citation rates reaching as high as ninety-five per cent under the configurations tested.

That last figure deserves emphasis because it cuts against the fatalism. Correct citation in Arabic is not out of reach. It responds to pipeline design and to how content is structured and chunked.

And here the Arabic-specific research converges with the general research on citation. The peer-reviewed study on generative engine optimisation presented at ACM SIGKDD in 2024 tested content-side changes against citation outcomes and found that named expert attribution produced the largest single lift, at just under forty-one per cent. Statistics paired with a named source and inline citations both produced substantial gains. Keyword stuffing reduced citation rates.

Two literatures, two methods, one direction. The things that make a passage citable are editorial rather than technical: a clear claim, a named source, an attributable expert, a self-contained unit of text. Those work in Arabic. They work harder in Arabic, because the field is thinner.

The supply problem underneath all of it

Every finding above sits on top of one structural fact.

Qatar Computing Research Institute at Hamad Bin Khalifa University, the team that built Qatar's own sovereign Arabic model, states in its own technical publication that Arabic accounts for roughly half a per cent of web data, against more than four hundred million native speakers. They state it not as commentary but as a constraint they had to engineer around.

That figure comes from people whose job was to build an Arabic model, which makes it more credible than the estimates in circulation elsewhere. Other sources place the share higher, up to around five per cent, and they disagree with each other. The direction is not in dispute even where the exact number is.

Here is what half a per cent means in practice, and it is not only bad news.

For the models, it means less material to learn alignment from, which is the ultimate cause of the mismatch penalty. For your competitors, it means very few of them have produced high-quality Arabic content that is structured to be cited. And for you, it means the field competing for the position of cited source in Arabic is thin in a way it has not been in English for fifteen years.

That is a genuine asymmetry and worth naming clearly, without overselling it. Thin competition is not the same as guaranteed results. It means the same quality of work returns more room in Arabic than in English, and it means the window is a function of how long that thinness lasts.

What this means for a Qatari site, concretely

Pulling the five studies together, four decisions follow.

Stop treating Arabic as a translation layer. If a question will be asked in Arabic, the answer needs to exist in Arabic as native content, because the alternative is relying on a bridge that measurably loses between thirteen and forty-two per cent depending on your domain. Regulated and technical categories sit at the worse end.

Measure four paths, not one number. Arabic question with Arabic source, Arabic question with English source, and the two English equivalents. A single blended Arabic visibility percentage averages away the only diagnostic that tells you what to publish next. We go into the reporting discipline in our generative engine optimisation practice for Qatar, because the measurement design is the harder half of the problem.

Build for ranking within the candidate set, not just for inclusion in it. Arabic retrieval reaches recall of 0.88 with mean reciprocal rank of 0.48 in the lexical work cited above. The relevant document usually gets found. It frequently is not first. That is a content-structure problem: one claim per unit, the answer up front, sections that survive extraction.

Lead with attribution. Named experts and named sources produced the largest measured citation lifts in the peer-reviewed literature. It is editorial work rather than technical work, and it is the single biggest thing most Arabic content is missing.

None of that is a promise about outcomes. Nobody controls what a model outputs, and any agency offering a guaranteed citation is describing something outside its control. What the research does support is a probability you can move, in a direction the evidence names, with methods someone else can check. If you want the surrounding picture, including the technical and regulatory layers, our team has documented the full scope for this market.

Where the evidence stops

Three limits, stated plainly, because an article making this argument should be held to the standard it is arguing for.

All five studies were run on general-domain corpora and public benchmarks. None of them measured anything inside Qatar. They demonstrate mechanisms, and mechanisms generalise better than magnitudes do, but the specific percentages should not be restated as Qatari market figures.

No independent study measures how often answer engines cite Arabic sources for Arabic questions in Qatar specifically. That gap was checked during research for this article and remains open. It means every Arabic visibility figure in commercial circulation, including any figure we produce, is a vendor measurement that has to arrive with its query set, its dates and its calculation method attached.

And the numbers themselves are snapshots of systems that change. Model versions move, embedders improve, and a measurement from 2025 describes 2025. The mechanism, that a language mismatch costs accuracy, is structural and will outlast the figures. Treat the mechanism as the finding and the percentages as its illustration.

---

Frequently asked questions

Is Arabic simply harder for AI systems than English?

That framing is misleading and it leads to the wrong response. The measured loss in the cross-lingual research comes from a mismatch between the language of the query and the language of the supporting document, not from Arabic as such. When both are Arabic, there is no cross-lingual penalty to pay. Arabic does carry real structural challenges, including root-and-pattern morphology that multiplies surface forms and tokenisers that fragment Arabic words into more pieces than English words, and Arabic source material on the web is scarce. But the actionable finding is about matching, because matching is something you control by deciding what language to publish in.

How much accuracy is actually lost when the languages do not match?

Between roughly thirteen and forty-two per cent in the work presented at ACL ArabicNLP in 2025, depending on which embedding model and which domain. With BGE-M3, retrieval accuracy fell thirteen per cent on a travel corpus and thirty-three per cent on a legal corpus, with end-to-end answer accuracy falling fourteen and thirty-seven per cent respectively. With multilingual-E5, Hit@20 fell thirty-three and forty-two per cent, and end-to-end accuracy fell thirty-seven and forty. Note that retrieval accuracy and Hit@20 are different measures, so those figures should not be averaged together, and the wide domain gap means technical and regulated categories sit closer to the worse end.

Do AI models just default to English sources whenever they can?

Not exactly, and the overstated version of this claim is common enough to be worth separating out. A 2025 preprint on language preference in multilingual retrieval found a pronounced preference for citing English documents, measured with English queries, with the effect amplifying for lower-resource languages, and in some cases an English document being chosen over a more relevant non-English one. That is a real measured tilt. What it does not establish is that models always fall back to English when asked in Arabic, and separate earlier work found that retrieving in the user's language can improve Arabic results. The defensible position is that a systemic lean toward English source selection exists and is stronger where source pools are thin, which makes the structural clarity of Arabic content matter more, not less.

Which engines handle Arabic and English cross-lingual questions best?

The spread is wider than most people expect. The XRAG evaluation published in EMNLP Findings in 2025 found GPT-4o highest at 57.58 per cent and Command-R+ lowest of those tested at 37.17 per cent, a gap of roughly twenty points on the same task. Two things follow. Engine choice is a real strategic variable, so a brand should know which systems its buyers actually use rather than optimising for one default. And even the leading score sits under sixty per cent, which means cross-lingual reasoning over retrieved evidence is unsolved at every tier and should not be relied on as a bridge between languages.

What actually improves the chance of being cited in Arabic?

Publishing the answer in the language of the question removes the mismatch entirely, which no other intervention does. After that, the peer-reviewed generative engine optimisation study presented at ACM SIGKDD in 2024 found the largest single citation lift came from named expert attribution, at just under forty-one per cent, with statistics paired with named sources and inline citations both producing substantial gains, and keyword stuffing reducing citation rates. Arabic-specific remediation work presented at ICNLSP in 2025 reported gains of up to eight points in precision at five, around eleven per cent in answering accuracy, and correct citation rates reaching as high as ninety-five per cent. The pattern across both literatures is editorial rather than technical.

Is there any advantage to producing Arabic content given all these problems?

Yes, and it comes from the same fact that causes the problems. Qatar Computing Research Institute states in its own Fanar technical publication that Arabic accounts for roughly half a per cent of web data against more than four hundred million native speakers, and it states this as an engineering constraint rather than as commentary. Scarce Arabic source material is why models align the languages imperfectly. It is also why very few competitors have published high-quality Arabic content structured to be cited. The field competing for the cited-source position in Arabic is thin in a way English has not been for a long time, which means comparable work returns more room. That is an asymmetry, not a guarantee, and how long it lasts depends on how long the thinness lasts.

---