The art of literary translation exposes the limits of AI

Artificial intelligence appears to be taking humanity one step closer toward that goal.

AI-powered tools are already being widely used by lawyers to translate legal documents from one language to the next. The mass market romance publisher Harlequin has turned to AI to translate its novels for international audiences. And hospitals are deploying AI translation to communicate directly with patients in multiple languages.

The speed and skill with which these AI-powered translation tools operate are certainly impressive.

But there is an important frontier for translation technology, one that it might never be able to breach: the poem.

That’s because translating poetry, thus far, has been a uniquely human experience. It demands intimate knowledge of two languages, which large language models certainly possess. But it also requires a mastery of different cultures and perspectives, what literary critic Gayatri Chakravorty Spivak calls a “worlding” of language and culture.

Pushing the limits of language

When scholars have studied the creativity of chatbots by prompting them to produce poetry, they’ve noticed that the poems tend to be more homogeneous and standardized than those written by humans.

Chatbots’ poetry translations have similar issues.

AI seems to struggle in three main areas: rendering metaphor, decoding complex sentence structure and creatively conveying mood or emotion.

To demonstrate these flaws, I worked closely with Adeeba Shahid Talukder, an award-winning poet and translator, to write this piece and to translate the 1953 poem “Mulāqāt,” or “Meeting,” composed by one of the most famous Urdu poets of the 20th century, Faiz Ahmed Faiz.

Faiz was born in 1911 in Sialkot, in pre-Partition India. In 1947, when that area in the Punjab became part of present-day Pakistan, Faiz became a citizen of the newly founded country, though his relationship with it was marked by both hope and disillusionment. He lived in different parts of the world for long stretches of time, but he returned to Pakistan toward the end of his life. He died there in 1984.

His poetry is celebrated for its marriage of classical tropes like the ache of unfulfilled longing for the beloved — and the lover’s willingness to be consumed or destroyed in his pursuit of her — with the political struggles of his time, like the plight of working people and the dream of a Marxist revolution. His poetry is notoriously difficult to translate because of its highly ornate, extended metaphors, which have few easy equivalents in English. This forces the translator – be it human or machine – to push the limits of language.

From 1951 to 1955, Faiz was imprisoned on allegations of sedition against the Pakistani state. His first three months were spent in solitary confinement.

From Faiz’s letters, scholars learned that despite the isolation of his imprisonment, he felt as if the experience made him more attuned to his senses, as if he were “falling in love again for the first time.” His time in jail was also among his most prolific as a poet, with “Meeting” composed during his sentence.

Here’s our translation of the opening verses of “Meeting”:

When prompted to provide a “lyrical translation into English,” ChatGPT produced the following:

Chatbots fail at translation’s most basic task

When Faiz composed “Meeting,” he had been separated from his wife and two daughters for two years. He was keenly aware of missing parts of their childhoods that would never return.

This biographical information can help translators understand the poem’s exploration of “vaṣl,” or the meeting of lover and beloved. A common trope in classical Urdu poetry, it usually captures the joy and exultation of the lovers’ union.

But in Faiz’s poem, the union with the beloved also contains an acute awareness of mortality and the transience of beauty – a recognition of what has been lost, and the suffering still to come.

That’s why we rendered the opening lines of “Meeting” as “This night is the dark,/lush tree of a grief greater/than you and I.”

ChatGPT’s translation is more literal: “This night is the tree of pain.”

While there is nothing technically wrong with this translation, the chatbot’s version doesn’t capture the nuances of the tree metaphor and the way its dense, expansive branches can encompass the complexity and beauty of the emotions evoked by the night of the lovers’ union.

AI also fails to grasp the poem’s intricate sentence structure. ChatGPT has translated “in its shadow a thousand moons / have wept away all their light” as a nonsensical construction: “Within a thousand moons’ shadows, / And all my light has wept itself away.”

This error appears to have happened because the chatbot translated “apnā” – a reflexive possessive pronoun in Urdu – as “my,” inaccurately parsing it as referring to the speaker instead of the moons.

Finally, and most importantly, AI models lack the ability to express emotion the way a human can. A machine with no bodily experience of being human cannot meaningfully perceive a poem so enmeshed in human experience. Its engagement is merely superficial.

In its attempt to convey the mood of the original piece, ChatGPT offers: “From its dew in quiet, / A few drops rest upon your brow, / And glitter like tiny diamonds.”

It’s clear that ChatGPT is struggling to decode the grammatical structure of the poem and is trying to make the text lyrical enough to convey the awe and wonder of the original. But the model’s contrivances toward the lyrical – for example, describing diamonds as “tiny” or “glittering” – have no relation to the original poem.

“From its dew in quiet” is an incoherent clause. The phrase that seems to have thrown the model off is “isī kī shabnam se khāmoshī ke yeh cand qaṭre,” or “its dew / has rained drops of silence.”

Urdu literary critic Shamsur Rahman Faruqi has explained how Urdu poets “treat metaphor as fact and then go on to create further metaphors from that fact.” In “Meeting,” the metaphor of the night of the reunion as a tree becomes a fact, which allows for the flowering of a new metaphor – that of the dew on its leaves as drops of silence that fall on the lover and their beloved. These silences, heavy with sorrow, then adorn the beloved like precious jewels, conveying the idea that only a profound grief can beget such beauty.

The model has failed at conjuring this sense of wonder because it cannot parse the poem in accordance with the literary conventions of Urdu poetry.

ChatGPT prefaced its translated text by assuring us that it had “crafted a lyrical, poetic English version of Faiz’s ‘Mulāqāt,’ keeping the imagery, rhythm, and emotional flow intact so it reads like a poem rather than a literal translation.”

Yet as we have shown, its translation fails to accomplish the most basic task of literary translation: to convey the heart of the original.

Chatbots, in other words, are a poor substitute for the literary translator, and they bolster the assertion of the late Indian poet, scholar and translator A.K. Ramanujan that “only poems can translate poems.”

Adeeba Shahid Talukder helped with the research and writing of this article, in addition to the translation of Faiz’s poem.

This article is republished from The Conversation, a nonprofit, independent news organization bringing you facts and trustworthy analysis to help you make sense of our complex world. It was written by: Krupa Shandilya, Amherst College

Read more:

• Indian protesters pull from poetic tradition to resist Modi’s Hindu nationalism
• Thousands of AI‑written, edited or ‘polished’ books are being sold – an eerie echo of Orwell’s ‘novel‑writing machines’
• Absorbed in translation: the art – and fun – of literary translation

Krupa Shandilya does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.