Intro. How I Plan to Teach Myself Deep Learning Using Only Free Resources
Learning Deep Learning Series Part 1: Videos
Learning Deep Learning Part 2: Online Courses
Learning Deep Learning Part 3: Github Repos
This is the third and final part in a series of articles called “Learning Deep Learning” in which the author chronicles his attempt to teach himself the topic of deep learning using only free resources.
I’ve finally arrived at the last leg of my learning deep learning journey: Github repositories. In the previous two parts, I used videos and online courses. For each of the two GitHub repos featured in this article, I cloned them into my local machine and ran the code as opposed just reading the code on the repo’s page.
After the first two parts of this series, I feel like I’m in a relatively good position. I understand the fundamental theories of deep learning, its various forms and uses, and how to train a simple model using the Python deep learning libraries Keras and Tensorflow. The material in this article will be more about practicing my skills as opposed to learning new ones. I feel it’s in my best interest to further improve and solidify the stuff I already I know than to burden myself with some of the more complex deep learning material and topics.
Deep Learning Keras TensorFlow
The first repo in this article comes courtesy of Lerio Maggio. Maggio’s repo entitled “Deep Learning Keras Tensorflow” is an exhaustive and well-annotated collection of Keras and Tensorflow tutorials. There’s enough material in this repo for a college-level course on this subject. It is divided into five parts that includes everything from LSTM to CNNs to AutoEncoders. In this write-up, I will only cover the first two sections: Introduction and Supervised Learning.
While it may be tempting to skip the introduction section, I nevertheless went through those notebooks and found them to be helpful. Maggio does an excellent job of crafting his Jupyter Notebooks as a professional lesson. His descriptions of concepts such as perceptrons, gradient descent, and activation functions are concise yet effective. Even though I was already with those ideas, I did feel as though I gained a better understanding from Maggio’s annotated jupyter notebooks.
In the two sections I reviewed, Maggio’s opts to create neural network model from scratch to teach his lessons. This may seem intimidating to a novice, but I think it’s an excellent decision to unveil the magic behind these so-called “black box” algorithms. When I read his code to build and train perceptron and adaptive linear neuron models, it made me feel a sense of relief because I could understand the purpose of the code just from a single look at it.
The second section of the repo is devoted to the basics of Python deep learning libraries Theano, Tensorflow, and Keras. Up until this point, I had mainly use Keras, a little bit of Tensorflow, and not at all Theano. I found this section particularly helpful because it illuminated the differences among the three modules, especially the differences in the code used to utilize each library. Maggio again deftly describes the basics of each library, his Keras lesson is probably the best one I’ve seen so far in this series. The Keras material, specifically the parts on the layers and activations, cemented my grasp of the library.
This repo validated the idea that it can be beneficial to learn the same thing but from another resource and that viewing, an idea through another lens isn’t wasteful but instead practice. Overall, I very much enjoyed this resource compared to ones I’ve already experienced so far in this series, I definitely plan on finishing the rest of the sections in this repo.
Deep Learning with Python Notebooks
When it came to picking which repositories to use for this article, it was a no-brainer include this repo from Francois Chollet, the author of Keras and an Artificial Intelligence Researcher at Google. I also highly recommend Chollet as a Twitter follow as well, he regularly shares sage advice and tips on machine and deep learning.
Chollet’s repo serves to host the code for his book “Deep Learning with Python” and even though I have not read the book, I did find this code useful, though after going through his repo I am considering buying the book.
What differentiates Chollet’s repo from Maggio’s is that Chollet primarily uses real examples and datasets to teach neural nets. Chollet’s code is used to classify or predict movie reviews, housing prices, and more. That these two repos complement each other is one of the main reasons I chose them.
This repo’s value in this series is that it bridges the gap between theory and practice. So far, I concentrated more on the concepts and the more abstract parts of deep learning and haven’t delved too much to applying the technology to real-world data. Chollet does a great job of showing how to preprocess data and build neural nets to suit a given dataset. In parts 3.5 and 3.6, he goes over how to set up a loss function and an optimizer for algorithms that classify movie reviews and newswires. Another area in which Chollet’s lesson proved to be very helpful is the understanding the idea of loss and how to effectively use minimize in Keras.
The best part of the repo was on overfitting and underfitting. Even though I was already familiar with this subject because I know machine learning, it was a good experience to learn how this applies to deep learning. The lesson focuses on how best to finetune and employ Keras layers, parameters, and regularizers to prevent overfitting, which is a common issue with neural nets. Chollet then goes on to demonstrate to deal with overfitting using the IMDB dataset.
The Finish Line
This marks the end of my “Learning Deep Learning” series but it’s certainly not the end of my deep learning education. Over the course of the three articles in this series, I learned about the foundations of deep learnings, its various applications, and how to train a simple neural net using Keras and TensorFlow. I hope that my running diary of my education was as helpful to those who seek to learn this powerful skill as it was to me. If you’re looking follow my path, then please check out my Github repo of free deep learning resources.
