Devotd Github
NewMultimodal

zhikangniu/voxcanvas — VoxCanvas-41M Speech Editing Data Pipeline

zhikangniu/voxcanvas describes a processing pipeline that builds a Chinese-English dataset for instruction-based speech editing training.

Repositoryzhikangniu/voxcanvas ↗

zhikangniu/voxcanvas describes a processing pipeline that builds a Chinese-English dataset for instruction-based speech editing training.

GitHub description: VoxCanvas-41M: A Large-Scale Bilingual Dataset for Instruction-Based Speech Editing and Its Learning Dynamics

stars
★ 20
Forks
0
Language
Python
License
MIT
Read original source ↗View on GitHub ↗

zhikangniu/voxcanvas is a repository for processing the VoxCanvas-41M dataset used to train instruction-based speech editing models. It provides a 5-stage pipeline that takes real recorded speech as input and connects alignment, annotation, synthesis, validation, and export. The repository is implemented in Python and is licensed under MIT. No public release version is listed in FACTS.

The pipeline links editing segments and instructions to the source audio, producing speech that matches the instruction content. It builds the editing segments and instructions after speech recognition and forced alignment, and uses F5-TTS in the synthesis stage. The validation stage checks the speech recognition results and speaker similarity. The speech recognition and forced alignment models used at each stage are included in the code, and the execution scripts are described as supporting per-stage resumption as well as single- or multi-GPU processing. The mini execution example contains 20 Chinese and English speech samples. Links to the paper, dataset, and demo page are published as well, so the pipeline and related materials can be followed.

However, FACTS lists no public release version and presents no evaluation figures for judging dataset distribution or training performance. Therefore, what can be confirmed at present is only the processing stages and the execution method, and this information alone makes it difficult to judge the quality of the dataset or the effect of model training.