This is a list of practices I adopted along the way that I think are worth sharing.
I coded this Transformer from scratch for learning. This is based on The Annotated Transformer by Harvard NLP, which uses PyTorch.
It is tested on a toy problem instead of NLP data to make things simpler.
This is a re-implementation of my previous Tensorflow PPO code. I just coded it to try out PyTorch. This is also fully documented and should help someone trying to learn about PPO.
This library helps you organize machine learning experiments. It maintains TensorBoard summaries, checkpoints, produce pretty console outputs, and also adds a header with progress of experiments to python source files. It also has tools to plot custom charts based on TensorBoard summaries.
I coded a DQN agent to play Atari. It is a stand alone implementation. I went through the Open AI DQN so it's very similar to it. I coded this also in a literate fashion similar to my previous PPO implementation.
I implemented a reinforcement learning agent using PPO to play Atari Breakout. It is a standalone implementation on TensorFlow.
I coded in a literate form with all the mathematical formula's etc, embedded in comments so that it can be used as a tutorial if any one wants to - and of course as a reference for my self.
I started working on a bunch of helper classes to use TensorFlow on Jupyter notebooks.
They provide nice diagrams and mathematical formulas as outputs based on the TensorFlow operations. It helps you understand the code better, and also acts as an inline help when coding.
This is a very simple Generative Adversarial Network build with deeplearn.js on ObservableHQ.
I coded this as an experiment to try out ObservableHQ, because I've loved almost all the projects by its creators @jashkenas and @mbostock.
I implemented a Long short-term memory (LSTM) module on numpy from scratch. This is for learning purposes. The network is trained with stochastic gradient descent with a batch size of 1 using AdaGrad algorithm (with momentum).
We are considering moving out codebase from CoffeeScript to TypeScript.