pipette
ENEnglish

Closed-Loop Dual-Reference Guidance for Accent Control in Cross-Lingual Voice Cloning

Yuanzhong Zheng, Wei Huang, Ming Ma, Kailun Jin, Youhai Jiang

PreprintUso en el mundo real

En palabras de los autores

Zero-shot cross-lingual voice cloning often carries the reference speaker's accent into the target language. We propose closed-loop dual-reference guidance, which pairs the original reference with a reusable native anchor constructed by voice conversion (VC). The references initialize separate language-model (LM) states that guide a shared acoustic trajectory; each completed patch then updates both states, keeping their generated histories aligned with the emitted speech. Anchor construction runs once per speaker, and online synthesis uses a frozen TTS model without per-utterance VC. On 20 held-out French speakers, the fixed setting reduces residual French LID mass by 73% versus direct cloning, at an ECAPA similarity cost of only ; relative to anchor-only cloning, it recovers . The method achieves the lowest mean WER () and highest mean GOP () among the held-out test arms. In blinded preference tests, native-English listeners favor it over direct cloning on pronunciation naturalness, and over both anchor-only cloning and a TTSVC cascade on speaker identity. Online RTF is approximately 26% lower than the cascade's ( vs.\ ), and the same setting reduces Spanish LID mass by 56% without re-tuning.

Resultado principalEl resumen no menciona limitaciones.

Apareció: jueves, 22 de octubre. arXiv. Preprint, todavía sin revisión por pares.