Pea-KD: Parameter-efficient and accurate Knowledge Distillation on BERT

Open Access

Peer-reviewed

Research Article

About the Authors

Ikhyun Cho

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft

Affiliation Department of Computer Science and Engineering, Seoul National University, Seoul, Republic of Korea

U Kang

Roles Conceptualization, Supervision, Validation, Writing – review & editing

Affiliation Department of Computer Science and Engineering, Seoul National University, Seoul, Republic of Korea

The authors have declared that no competing interests exist.

Teachers
Is the Subject Area "Teachers" applicable to this article?

Thanks for your feedback.
Distillation
Is the Subject Area "Distillation" applicable to this article?

Thanks for your feedback.
Language
Is the Subject Area "Language" applicable to this article?

Thanks for your feedback.
Human learning
Is the Subject Area "Human learning" applicable to this article?

Thanks for your feedback.
Natural language processing
Is the Subject Area "Natural language processing" applicable to this article?

Thanks for your feedback.
Deep learning
Is the Subject Area "Deep learning" applicable to this article?

Thanks for your feedback.
Learning
Is the Subject Area "Learning" applicable to this article?

Thanks for your feedback.
Natural language
Is the Subject Area "Natural language" applicable to this article?

Thanks for your feedback.