Self-training ASR Guided by Unsupervised ASR Teacher
Abstract
Self-training has gained increasing attention due to its no- table performance improvement in speech recognition. How- ever, conventional self-training techniques have two key limi- tations: (1) labeled dataset is required for training a teacher to produce a pseudo-target, and (2) the first teacher trained with the small labeled dataset faces an over-fitting issue, generat- ing noisy pseudo-targets for unseen datasets. Our approach adopts an unsupervised automatic speech recognition model as the teacher, thus solely utilizing the unlabeled dataset. As the proposed model also learns phonetic information from the UASR teacher at the intermediate layer, the pseudo-target at the higher layer contains more ASR-related information than that of Data2vec2. Experimental results on the LibriSpeech show that our model outperforms Data2vec2, the state-of-the-art self- supervised learning model, achieving 8.9% and 4.3% relative word error rate reduction on test-clean and test-other.
- ConferenceInterspeech
- TopicSpeech Recognition
- Year Published2024
Abstract
Self-training has gained increasing attention due to its no- table performance improvement in speech recognition. How- ever, conventional self-training techniques have two key limi- tations: (1) labeled dataset is required for training a teacher to produce a pseudo-target, and (2) the first teacher trained with the small labeled dataset faces an over-fitting issue, generat- ing noisy pseudo-targets for unseen datasets. Our approach adopts an unsupervised automatic speech recognition model as the teacher, thus solely utilizing the unlabeled dataset. As the proposed model also learns phonetic information from the UASR teacher at the intermediate layer, the pseudo-target at the higher layer contains more ASR-related information than that of Data2vec2. Experimental results on the LibriSpeech show that our model outperforms Data2vec2, the state-of-the-art self- supervised learning model, achieving 8.9% and 4.3% relative word error rate reduction on test-clean and test-other.