Factorized Layers for Compression

Shedding light on fairness through a video dataset

Articles

Microsoft wrote a very lengthy post around auto-ml and knowledge distillation which accompanies their paper in ICLR. The main promise is to use factorized neural layers to create a low-rank model compression for the original model and use also these layers to compress the information and make an over complete representation for the model to both…