2021 is coming to an end, and this newsletter is the N-1th of the newsletter for this year! The Nth one will be about best articles out of 2021 which are going to be chosen some 40 odd something newsletters published this year.
For 2022, I will make some changes in the newsletter especially on the order of sections as based on engagement; as libraries section are most useful section for most readers.
I am also planning to create static pages to compile a list of tutorials, classes and conferences for 2022. If you have anything that you want to show and demonstrate, I am all ears outside of these sections!
Bugra
Articles
DeepMind released 280 Billion parameter language model in this blog post called Gopher.
Google wrote an interesting blog post where they outlined how to do dataset distillation.
Neural Network Pruning is an excellent blog post from Nathan Hubbens that go over various different pruning methods when it comes to neural network pruning.
Transactions on Machine Learning Research was announced through a blog post and it is a sister journal to JMLR.
Adi Fuchs has excellent a series of posts on AI Acceleration, it gives a good overview, where we are in terms of landscape and some future predictions/directions. If you are working/using accelerators for your ML workload, check them out!:
Libraries
Flowtorch is PyTorch based library to build various probability distributions and sample them easily.
Iris is an end to end Photos platform, that was built on top of PyTorch.
Theseus is a library for differentiable nonlinear optimization built on PyTorch to support constructing various problems in robotics and vision as end-to-end differentiable architectures.
PromptSource uses a simple templating language to programmatically map an example of a dataset into a text input and a text target.
NeuralProphet bridges the gap between traditional time-series models and deep learning methods.
MQBench open-source model quantization toolkit based on PyTorch fx. It provides:
SOTA Algorithms. With MQBench, the hardware vendors and researchers can benefit from the latest research progress in academia.
Powerful Toolkits. With the toolkit, quantization node can be inserted to the original PyTorch module automatically with respect to the specific hardware. After training, the quantized model can be smoothly converted to the format that can inference on the real device.
DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.
The paper is also available in here.
PyFlow is an open-source tool for modular visual programing in Python. It should be a good teaching tool and if you want a different user interface than the commonly used notebooks, you might want to take a look.
Tutorials
Transformers tutorial covers a variety of concepts in Transformers in an approachable way.
HuggingFace has also a course on how you can leverage Transformers library in a better way. The second part of tutorial is also available in their website.
Conferences
British Machine Computer Vision Conference has the all of the videos available online.
Videos
Google’s ML Community day talks is available in here.
Mathematics for Machine Learning videos are available in here.
Linear Algebra class from CMU is in here.
MIT open-sourced a linear algebra class in here.
MLOps NYC Summit 2021 has a number of good MLOps topics like model monitoring, model deployment and model optimizations.