Why Speech Recognition Breaks In Real Call Centers

Building bilingual, multi-tone text-to-speech for banking in Uzbekistan

At a glance

• Robotic TTS isn’t just a user experience problem. In banking interactions, it can undermine customer trust.
• Modern speech synthesis systems rely on a multi-stage pipeline that includes text normalization, acoustic modeling, and a neural vocoder.
• Some regions introduce additional complexity because there can be multiple variances in languages, dialects, usages, and so on.
• Banking deployments require tone control and bilingual continuity so the same voice can operate naturally across different tongues—even when speakers switch languages mid-conversation.

Why “good enough” speech synthesis doesn’t work in banking

In many markets, older “unit selection” text-to-speech systems are still widely used. These systems assemble speech by stitching together prerecorded segments and inserting variable data.

They work, but the result often sounds robotic and constrained.

For banking interactions—customer support, account notifications, or payments reminders—that approach quickly breaks down. Customers notice when speech sounds unnatural, and the experience can feel impersonal or even suspicious.

Modern voice systems need to generate open-ended speech. Instead of recombining prerecorded clips, neural TTS models synthesize speech directly from text. That allows systems to pronounce new words, adapt to conversational phrasing, and maintain natural prosody.

For financial institutions, that difference matters. The voice interface often represents the brand, and customers expect it to sound consistent and trustworthy.

The modern TTS pipeline in plain terms

A production text-to-speech system usually consists of three main layers.

• First is text processing or normalization. This stage converts raw text into a form that can be spoken naturally. Numbers, abbreviations, punctuation, and formatting must all be expanded into pronounceable words.
• Second is the acoustic model. This model converts normalized text into a mel spectrogram—a representation of how speech should sound.
• Third is the vocoder. The vocoder converts the spectrogram into an audible waveform that listeners hear as speech.

The first stage often turns out to be the most difficult. In some languages, there are grammatical structures where numerals and suffixes change pronunciation depending on context. Even a simple phrase such as “4%” may require different spoken forms depending on grammatical case and sentence structure.

Bilingual voice continuity is essential

Some bilingual environments find that different languages are widely used in banking interactions.

Customers may begin a conversation in one language and switch to another mid-sentence. If the voice system suddenly changes speakers or accents when the language switches, the illusion of a single operator breaks.

That’s why banking-grade TTS systems demand voice continuity across languages, not just bilingual capability. The same synthetic voice must be able to move naturally between languages, whether Spanish and English, Uzbek and Russian, Dutch and German, or other regional combinations, while preserving a consistent vocal identity, tone, and emotional character.

This means the AI voice remains recognizably the same to the listener, regardless of which language is being used, so institutions can deliver multilingual conversations without breaking trust or brand consistency.

This becomes especially important for enterprise AI agents designed to serve global or multilingual populations, where customers may switch languages mid-conversation and still expect a coherent, natural interaction.

This requirement eliminates many off-the-shelf hosted solutions. It becomes even more restrictive when institutions also require on-premise deployment to protect sensitive financial data.

Tone of voice is a functional requirement

In banking communication, tone affects outcomes.

Different situations require different emotional registers.

Friendly tones are appropriate for customer support interactions and routine account assistance. Neutral tones work for informational messages and confirmations. Strict tones may be necessary for urgent notifications or delinquency reminders.

If every interaction sounds identical, the message loses clarity. A payment reminder and an urgent notice should not sound interchangeable.

That means tone control must be designed into the system. Voice recordings need to be supervised carefully so emotional cues stay distinct and consistent across the synthesized voice.

Recording and evaluation: where most teams underestimate the work

Creating a bilingual, multi-tone TTS voice requires significant recording effort.

Voice actors must record speech in both languages across several tone profiles. Scripts must be carefully designed to capture the phonetic variety needed for natural synthesis.

Let’s look at just one example: In deploying for a digital bank in Uzbekistan, Acclaim had to manage the practical complication that the population is accustomed to using multiple alphabets. Production teams had to maintain duplicate recording scripts in Latin and Cyrillic to accommodate participants in the recording process who use different languages.

This isn’t the only instance where this might occur. In South America, for instance, a banking or CX platform may need to handle Spanish plus Portuguese in Brazil, English in Guyana, Dutch in Suriname, and, in some cases, Indigenous or mixed-language speech patterns. Even just within Spanish, accent, cadence, vocabulary, and code-switching can vary across countries and populations.

The time commitment can be substantial. Roughly 20 hours of clean recorded speech may require over 100 hours in the studio, including setup time, retakes, and supervision.

Evaluation is equally important. Generic testers rarely catch subtle issues that appear in real banking conversations. A practical approach is to recruit experienced call-center professionals to evaluate synthesized speech using structured scoring systems for natural flow and tone appropriateness.

What production impact looks like when voice technology actually works

Once speech recognition and speech synthesis reach sufficient quality, voice automation can move beyond pilot projects and into real operations.

Financial institutions can use voice AI agents to manage high-volume workflows such as payment reminders, account notifications, and customer support calls.

In one example deployment, AI-powered voice agents handled more than 90% of reminder calls for early-stage delinquency loans and delivered a reported 10-fold improvement in operational efficiency. This shows how voice automation begins to scale once speech technology becomes reliable enough for real customer conversations.

How Acclaim helps

Acclaim’s voice-first architecture addresses the challenges of multilingual banking voice systems.

The platform supports bilingual speech environments by combining custom ASR and neural TTS models with domain-specific training data. That allows voice agents to maintain a consistent voice identity across Uzbek and Russian, even when conversations switch languages mid-stream.

Acclaim also supports tone-aware voice synthesis so banks can deliver different communication styles—friendly, neutral, or strict—depending on the interaction.

Because deployments can run on-premise or in a private cloud, institutions retain control of sensitive financial data while still benefiting from modern voice AI automation.

The result? Voice technology that sounds natural, stays consistent across languages, and supports real banking operations rather than scripted demos.