Mixture of Experts in Training
Keras Preprocessing Layers
SAVi’s demo is really cool that can discover various visual entities in videos that I thought you might enjoy!
Articles
Keras Preprocessing Layers aims to combine model building with the feature engineering and representation layers. They aim to provide a flexible and expressive way to build data preprocessing pipelines. These prebuilt layers can be mixed and matched with custom layers and other Tensorflow functions in a seamless manner.
Microsoft releases a new library called Tutel which is tailored towards Mixture of Experts(MoE) deep learning modeling technique to make the training easier for PyTorch. It supports GPU and CPU training options under the hood.
FastMoE which is another library to solve a similar problem for large scale deep learning training.
Microsoft released foundation(pre-trained and large) model called T-NLRv5 in this blog post. It shows better performance than other models in GLUE and SUPERGLUE benchmark tasks.
Datasets
RedCaps a large-scale dataset of 12M image-text pairs collected from Reddit. Images and captions from Reddit depict and describe a wide variety of objects and scenes. The data is collected from a manually curated set of subreddits, which give coarse image labels and allow us to steer the dataset composition without labeling individual instances.
Libraries
Dabs is a library that allows benchmarking for self-supervised tasks in a domain agnostic manner. It used PyTorch, Weights and Biases and Hydra.
NUWA is a unified multimodal pre-trained model that can generate new or manipulate existing visual data (i.e., images and videos) for 8 visual synthesis tasks:
Text to Image
Sketch to Image
Image Completion
Image Manipulation
Text to Video
Sketch to Video
Video Prediction
Video Manipulation
TinyNeuralNetwork is a library that allows end to end training/deployment of PyTorch models(converting to TFLite for inference at the end). The library allows you to use the following features out of the box:
Pruner: OneShot (L1, L2, FPGM), ADMM, NetAdapt, Gradual, End2End
Quantization-aware training: TinyNeuralNetwork uses PyTorch's QAT as the backend and optimizes its usability with automating the fusion of operators and quantization of computational graphs
Books
Mathematics for Machine Learning is a free book that explains core concepts of deep learning in a digestible manner.
Introduction to Probability for Data Science is another book that covers probability concepts for various machine learning algorithms.
Trustworthy Machine Learning is a “beta” book that covers various responsible topics in machine learning.
Workshops
Workshop on Systems Challenges in Reliable and Secure Federated Learning is a workshop on Federated Learning(FL) and how to deploy FL in production securely and reliably.