Ziling Cheng

TL;DR: In this paper, we described our submission to the BabyLM Challenge, and investigated sample-efficient pretraining strategies. On the data side, we focused on improving data utilization, specifically different batching strategies for training. Our findings indicated that the formatting of the input data can significantly impact and improve downstream task... [Read More]

TL;DR: In this paper, we mainly introduce a large-scale headline-generation dataset for 14 Indic languages and English. With the new dataset, we try to answer important questions about indic and multilingual models/research. We first perform headline generation task with Varta, and our results indicate that the Indic headline generation task... [Read More]