1 code implementation • 17 Dec 2023 • Nikita Starodubcev, Artem Fedorov, Artem Babenko, Dmitry Baranchuk
While several powerful distillation methods were recently proposed, the overall quality of student samples is typically lower compared to the teacher ones, which hinders their practical usage.