| --- |
| license: other |
| license_name: ltx-2-community |
| tags: |
| - video-generation |
| - audio |
| - lora |
| - temporal-control |
| - text-to-video |
| - arxiv:2609.02367 |
| --- |
| |
| <div align="center"> |
|
|
| <h1>The Missing Temporal Link: Temporal Context Routing<br>for Script-Driven Audio-Video Generation</h1> |
|
|
| <p> |
| Yichen Liu<sup>1</sup>, Quanwei Zhang<sup>2</sup>, Haozhe Wang<sup>3</sup>, Donghao Zhou<sup>4</sup>, Xiaojie Li<br> |
| Yang Shi<sup>2</sup>, Jiaming Liu<sup>2</sup>, Ruihua Huang<sup>2</sup>, Yingtian Zou<sup>5</sup>, Daquan Zhou<sup>1</sup> |
| </p> |
|
|
| <p> |
| <sup>1</sup> Peking University · <sup>2</sup> Qwen Applications<br> |
| <sup>3</sup> HKUST · <sup>4</sup> CUHK · <sup>5</sup> Shanghai Jiao Tong University |
| </p> |
|
|
| [](https://dagroup-pku.github.io/Temporal-Context-Routing.github.io/) |
| [](https://arxiv.org/abs/2609.02367) |
| [](https://huggingface.co/papers/2609.02367) |
| [](https://github.com/DAGroup-PKU/Temporal-Context-Routing) |
| [](https://huggingface.co/starry0929/Temporal-Context-Routing/blob/main/temporal-context-routing.safetensors) |
| [](https://huggingface.co/starry0929/Temporal-Context-Routing/blob/main/test_prompts_200.json) |
|
|
| </div> |
|
|
| **Temporal Context Routing (TCR)** aligns script-specified shot and dialogue timing with the shared video-audio timeline. Timing bypasses the text encoder and enters cross-attention logits as a routing score, separating *what* to generate from *when* it should appear. Built on LTX-2.3 22B, TCR reduces shot-boundary error from **1.11 s to 0.042 s**—about one frame at 24 fps—and raises dialogue timing accuracy from **28.3% to 84.1%**. |
|
|
| This repository hosts the released **LoRA** (`temporal-context-routing.safetensors`, ~2.3 GB) and the **200 held-out test prompts** (`test_prompts_200.json`) used in the paper. Training and inference code is at [DAGroup-PKU/Temporal-Context-Routing](https://github.com/DAGroup-PKU/Temporal-Context-Routing). Demos are available on the [project page](https://dagroup-pku.github.io/Temporal-Context-Routing.github.io/). |
|
|
| ## Files |
|
|
| | File | What it is | |
| | --- | --- | |
| | `temporal-context-routing.safetensors` | TCR LoRA, rank 128, trained on LTX-2.3 22B (paper 7k-step recipe) | |
| | `test_prompts_200.json` | 200 held-out script JSON files with per-shot / per-line `time_range` | |
|
|
| You still need the **LTX-2.3 22B** checkpoint and the **Gemma 3 12B** text encoder locally. One 80 GB GPU is the working setup. |
|
|
| ```bash |
| huggingface-cli download starry0929/Temporal-Context-Routing \ |
| temporal-context-routing.safetensors test_prompts_200.json \ |
| --local-dir ./weights |
| ``` |
|
|
| Then follow the [code README](https://github.com/DAGroup-PKU/Temporal-Context-Routing#inference): |
|
|
| ```bash |
| bash scripts/infer.sh \ |
| --checkpoint /path/to/ltx-2.3-22b-dev.safetensors \ |
| --text-encoder-path /path/to/gemma-3-12b-it \ |
| --lora-path ./weights/temporal-context-routing.safetensors \ |
| --output outputs/tcr_infer.mp4 |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{liu2027missingtemporallink, |
| title = {The Missing Temporal Link: Temporal Context Routing |
| for Script-Driven Audio-Video Generation}, |
| author = {Liu, Yichen and Zhang, Quanwei and Wang, Haozhe and |
| Zhou, Donghao and Li, Xiaojie and Shi, Yang and Liu, Jiaming and |
| Huang, Ruihua and Zou, Yingtian and Zhou, Daquan}, |
| booktitle = {Under review}, |
| year = {2027} |
| } |
| ``` |
|
|
| ## License |
|
|
| The LoRA is trained on LTX-2.3 and is intended to be used with that backbone under the [LTX-2 Community License](https://github.com/DAGroup-PKU/Temporal-Context-Routing/blob/main/LICENSE). |
|
|
