wq2012/aec_single_interfering

AEC-Seq2seq Neural Baseline — Single Interfering Voice (AecSingleInterfering)

This is not an officially supported Google product.

Multi-source attention sequence-to-sequence Acoustic Echo Cancellation (AEC-Seq2seq) neural baseline model trained on 24 kHz LibriTTS user speech mixed at 0 dB SNR with reverberant LJ Speech (single-speaker TTS interference). Uses dual audio encoders (SpeechEncoderV1) on both the microphone mixture and the full reference TTS playback audio (~180–310 KB side input).

• GitHub Repository: https://github.com/wq2012/tec
• PyPI Package: textual-echo-cancellation (v0.2.3)
• Interactive Hugging Face Space Demo: https://huggingface.co/spaces/wq2012/tec
• Paper: Textual Echo Cancellation (IEEE SLT 2021, arXiv:2008.06006v4)
• Audio Demo Page: https://google.github.io/speaker-id/publications/TEC/

Open-Source Reproduction Notice: This pretrained model was trained using the standalone open-source reproduction library (wq2012/tec) built on lingvo and tensorflow. The original IEEE SLT 2021 paper was conducted using Google's internal codebase and 281k-utterance internal LibriTTS + TTS datasets rather than this open-source reproduction.

---

Training Configuration

• Condition: Single interfering voice (LibriTTS + LJ Speech)
• Dataset Splits: 400 training utterances (train), 100 validation utterances (dev), 100 test-clean utterances, and 100 test-other utterances at 24 kHz mixed at 0 dB SNR with synthetic image-method room impulse responses (RT60 = 0.25 s).
• Architecture: Speech encoder (SpeechEncoderV1: 2× strided 3×3 Conv2D with stride (2, 2), 1× Bi-CLSTM with stride (1, 1), and 3× Bi-LSTM layers), text encoder (TtsEncoderV2: 98-symbol ASCII character vocabulary VOCAB_SIZE = 98, 512-dim embedding, 3× Conv1D + 1× Bi-LSTM, for TEC models), and multi-source GMM monotonic attention decoder (MultiSourceFbeDecoderV1 / FbeDecoderV1: 2×256 Pre-Net, 2-layer autoregressive LSTM, frame reduction factor r = 4, and 5-layer 1D convolutional PostEditConvNet).
• Trainable Parameters: 18,496,991 (18,501,730 total model variables; 221.98 MB checkpoint shard; 19.18 MB dynamic-range quantized model.tflite)
• Checkpoint Selection: Validation-loss early stopping (best.ckpt selected on the 100-utterance dev split over 500 training steps with batch size 8 and Adam lr = 1e-3).

---

Model Performance

1. Open-Source Reproduction Evaluation (AecSingleInterfering, N = 100 mixtures per split)

Evaluated on 24 kHz LibriTTS (test-clean: N = 100 mixtures, 1062 reference words; test-other: N = 100 mixtures, 1,012 reference words) mixed at 0 dB SNR under the Single interfering voice (LibriTTS + LJ Speech) condition, using Qwen3-ASR-0.6B-F16 via audio.cpp for ASR WER scoring and 13-MFCC Dynamic Time Warping for MCD (with 10,000-resample non-parametric bootstrap 95% confidence intervals):

Metric	test-clean (N = 100)	test-other (N = 100)	Paper Reference (AEC-Seq2seq)
WER (%) ↓	7.91% (84/1062, 95% CI [5.68, 10.48])	17.29% (175/1012, 95% CI [13.34, 21.76])	8.30% / 24.3%
MCD (dB) ↓	4.97 dB (95% CI [4.60, 5.35])	4.96 dB (95% CI [4.61, 5.31])	6.38 dB / 7.07 dB
Side Input Size (KB) ↓	235.403 KB (padded: 235.403 KB)	236.991 KB (padded: 236.991 KB)	310 KB
Complexity (GFLOPS) ↓	6.36 GFLOPS	6.40 GFLOPS	9.51 GFLOPS (401 frames)

2. Full Comparison Across All Pretrained Models & Baselines (N = 100 per split)

Condition	Method	Hugging Face Model	WER (%) test-clean ↓	WER (%) test-other ↓	MCD (dB) test-clean ↓	MCD (dB) test-other ↓	Side Input test-clean (KB) ↓	Avg Complexity (GFLOPS) ↓
Single interfering voice (LibriTTS + LJSpeech)	GroundTruth	—	3.86 [2.43, 5.49]	6.62 [4.66, 8.78]	0.00	0.00	0.000	0.00
	MicrophoneSignal	—	80.32 [68.49, 93.55]	102.77 [88.25, 119.32]	9.92 [9.56, 10.27]	10.74 [10.32, 11.16]	0.000	0.00
	NlmsAec (AEC-NLMS)	—	58.38 [47.37, 71.05]	85.77 [71.61, 102.19]	9.51 [9.16, 9.86]	10.33 [9.92, 10.75]	235.403	0.24
	NoSideInputSingleInterfering	wq2012/vanilla_seq2seq_single_interfering	39.08 [30.26, 48.91]	54.35 [44.66, 64.93]	7.07 [6.86, 7.29]	7.38 [7.13, 7.64]	0.000	4.17
	AecSingleInterfering	wq2012/aec_single_interfering	7.91 [5.68, 10.48]	17.29 [13.34, 21.76]	4.97 [4.60, 5.35]	4.96 [4.61, 5.31]	235.403	6.36
	TecSingleInterfering	wq2012/tec_single_interfering	29.76 [23.09, 36.97]	50.59 [41.96, 59.66]	5.52 [5.16, 5.89]	5.55 [5.18, 5.93]	0.074	4.62
Multiple interfering voices (LibriTTS + VCTK)	GroundTruth	—	3.06 [1.90, 4.35]	6.85 [4.77, 9.16]	0.00	0.00	0.000	0.00
	MicrophoneSignal	—	50.77 [40.86, 62.15]	67.33 [56.40, 79.77]	7.50 [7.13, 7.87]	7.82 [7.41, 8.24]	0.000	0.00
	NlmsAec (AEC-NLMS)	—	34.22 [25.72, 44.51]	48.58 [39.15, 59.91]	7.01 [6.67, 7.37]	7.43 [7.04, 7.83]	203.138	0.21
	NoSideInputMultiInterfering	wq2012/vanilla_seq2seq_multi_interfering	63.33 [55.27, 71.69]	69.86 [61.89, 78.18]	8.26 [7.76, 8.76]	9.32 [8.77, 9.89]	0.000	3.59
	AecMultiInterfering	wq2012/aec_multi_interfering	6.84 [4.98, 8.85]	14.86 [11.25, 18.79]	4.65 [4.28, 5.02]	4.67 [4.28, 5.06]	203.138	5.48
	TecMultiInterfering	wq2012/tec_multi_interfering	24.31 [18.94, 30.47]	36.25 [30.34, 42.52]	4.74 [4.42, 5.07]	4.91 [4.54, 5.30]	0.050	3.90

3. Text-Content Sensitivity Ablation (N = 100 per split)

Verifies that TtsEncoderV2 actively exploits the semantic and phonetic content of the interfering TTS transcript rather than acting as a content-independent bias:

Split	Side-Input Condition	WER (%) ↓	MCD (dB) ↓	ΔWER vs. matched	Paired p-value (Bootstrap / Wilcoxon)
single_test_clean (TecSingleInterfering)	matched (true TTS transcript)	29.76%	5.52 dB	—	—
	shuffled (mismatched transcript)	70.90%	9.26 dB	+41.15%	p < 0.0001 / p = 2.61e-12
	empty (no text)	71.94%	9.32 dB	+42.18%	p < 0.0001 / p = 4.89e-13
	Vanilla-Seq2seq	39.08%	7.07 dB	+9.32%	p < 0.0001 / p = 6.69e-04
multi_test_clean (TecMultiInterfering)	matched (true TTS transcript)	24.31%	4.74 dB	—	—
	shuffled (mismatched transcript)	48.52%	7.92 dB	+24.21%	p < 0.0001 / p = 9.10e-09
	empty (no text)	47.29%	7.83 dB	+22.98%	p < 0.0001 / p = 1.41e-07
	Vanilla-Seq2seq	63.33%	8.26 dB	+39.02%	p < 0.0001 / p = 1.99e-11

4. Input SNR Ablation on single_test_clean (N = 100 mixtures per SNR)

Input SNR	MicrophoneSignal WER / MCD	NlmsAec WER / MCD	Vanilla-Seq2seq WER / MCD	TecSingleInterfering WER / MCD	AecSingleInterfering WER / MCD
+5 dB	51.88% / 9.03 dB	40.58% / 8.72 dB	9.23% / 6.28 dB	8.47% / 4.69 dB	5.65% / 4.33 dB
0 dB	80.32% / 9.92 dB	58.38% / 9.51 dB	39.08% / 7.07 dB	29.76% / 5.52 dB	7.91% / 4.97 dB
-5 dB	119.02% / 10.73 dB	90.30% / 10.28 dB	96.80% / 7.82 dB	87.76% / 6.34 dB	17.23% / 5.73 dB
-10 dB	124.86% / 11.40 dB	121.28% / 10.95 dB	103.86% / 8.35 dB	101.04% / 6.91 dB	49.25% / 6.33 dB

---

Files in This Repository

• best.ckpt.data-00000-of-00001, best.ckpt.index, best.ckpt.meta, checkpoint: TensorFlow / Lingvo checkpoint for AecSingleInterfering (selected by minimum validation loss on dev).
• model.tflite: Dynamic-range quantized TensorFlow Lite (.tflite) FlatBuffer model for AecSingleInterfering (requires TFLite Flex SELECT_TF_OPS runtime for dynamic RNN TensorList operations).
• evaluation_metrics.json: Verified N = 100 evaluation results (test-clean and test-other), 95% bootstrap confidence intervals, paired significance tests, and ablations for AecSingleInterfering.

---

How to Use

1. Install textual-echo-cancellation

pip3 install textual-echo-cancellation huggingface_hub

2. Download the Model from Hugging Face

from huggingface_hub import snapshot_download

model_dir = snapshot_download(repo_id="wq2012/aec_single_interfering")
print("Downloaded model to:", model_dir)

3. Run Inference via CLI (scripts/inference.py)

python3 -m scripts.inference \
  --model AecSingleInterfering \
  --checkpoint_path "${MODEL_DIR}/best.ckpt" \
  --mixed_wav /path/to/mixed_input.wav \
  --interfering_wav /path/to/reference_tts.wav \
  --output_wav /tmp/enhanced_clean.wav

4. Run Inference via Python API

import os
from huggingface_hub import snapshot_download
from tec import inference

model_dir = snapshot_download(repo_id="wq2012/aec_single_interfering")
ckpt_path = os.path.join(model_dir, "best.ckpt")

result = inference.run_inference_on_wav(
    model_name="AecSingleInterfering",
    mixed_wav_path="/path/to/mixed_input.wav",
    interfering_wav_path="/path/to/reference_tts.wav",
    checkpoint_path=ckpt_path,
    output_wav_path="/tmp/enhanced_clean.wav",
)
print("Enhanced log-Mel spectrogram shape:", result["predicted_mel"].shape)

5. On-Device Inference with Quantized TFLite (model.tflite)

Note: Because the multi-source attention decoder uses dynamic tf.while_loop / TensorList operations, model.tflite is exported with tf.lite.OpsSet.SELECT_TF_OPS (Flex delegate) enabled.

import os
import numpy as np
from huggingface_hub import snapshot_download
from tec import inference

model_dir = snapshot_download(repo_id="wq2012/aec_single_interfering")
tflite_path = os.path.join(model_dir, "model.tflite")

dummy_mel = np.zeros((100, 128), dtype=np.float32)
tflite_out = inference.run_tflite_inference(
    tflite_path=tflite_path,
    mixed_mel=dummy_mel,
    interfering_mel=dummy_mel,
)
print("TFLite predicted log-Mel shape:", tflite_out["predicted_mel"].shape)

Expected console output:

TFLite predicted log-Mel shape: (100, 128)

---

Citation

If you use this model or the textual-echo-cancellation library in your research, please cite the original paper:

@inproceedings{ding2021textual,
  title={Textual Echo Cancellation},
  author={Ding, Shaojin and Jia, Ye and Hu, Ke and Wang, Quan},
  booktitle={2021 IEEE Spoken Language Technology Workshop (SLT)},
  pages={666--673},
  year={2021},
  organization={IEEE}
}