Published Paper Details:

Automatic speaker diarization and voice cloning have become essential components in modern speech-driven applications, including intelligent transcription systems, conversational analytics, and personalized audio generation. This paper proposes a unified framework that integrates WhisperX, PyAnnote, and neural Text-to-Speech (TTS) models to achieve highly accurate voice diarization and realistic voice cloning. The system first applies WhisperX for ASR transcription with word-level alignment, ensuring fast and precise text extraction. Speaker segments are detected using PyAnnote's state-of-the-art speaker embedding and clustering techniques, which significantly improve diarization accuracy in multi-speaker environments. Each identified speaker embedding is then mapped to a corresponding voice profile used by a neural TTS model to generate synthetic speech that preserves pitch, timbre, and prosodic characteristics of the original speaker. The integration of these modules produces a seamless pipeline capable of identifying speakers, transcribing speech, and replicating voices with high naturalness. Experiments on dialogue datasets demonstrate notable improvements in diarization error rate, transcription accuracy, and perceptual voice similarity. The proposed system shows potential for applications in personalized virtual assistants, automated content creation, podcast enhancement, call center analytics, and accessibility technologies.