Fig 1.
Schematic diagram of the essence of knowledge distillation (KD).
Fig 2.
The framework of the Squeeze-and-Excitation block.
Fig 3.
Feature fusion pyramid of an Nt.
Table 1.
Details of experimental parameter settings.
Table 2.
Top-1 accuracy of student network with ResNet20 on CIFAR-10 test dataset.
Table 3.
Top-1 accuracy of student network WRN_16_2 on CIFAR-10 test dataset.
Table 4.
Top-1 accuracy of student network with VGG8 on CIFAR-100 test set.
Table 5.
Top-1 accuracy of student network with ResNet20 on CIFAR-100 dataset.
Table 6.
The performance of the student network VGG8 on the CIFAR-100 dataset by using different extraction combinations.
Table 7.
The Top-1 accuracy of the student network ResNet20 using different fusion methods where the teacher is ResNet56 on CIFAR-100.
Table 8.
The Top-1 accuracy of the student network VGG8 using different fusion methods where the teacher is VGG13 on CIFAR-100.
Table 9.
Top-1 accuracy of student network VGG8 on CIFAR-100 dataset.