12 Frequently Asked Questions on Deep Learning (with their answers)!

Introduction

From Facebook’s research to DeepMind’s legendary algorithms, deep learning has climbed its way to the top of the data science world. It has led to amazing innovations, incredible breakthroughs, and we are only just getting started!

However if you are a newcomer to this field, the word “deep” might throw you into doubt. Deep learning is one of the hottest topics of this industry today, but it is unfortunately foreign and cryptic to most people. A lot of people carry an impression that deep learning involves a lot of mathematics and statistical knowledge.

If you had similar questions about deep learning, but were not sure how, when and where to ask them – you are at the right place. This article should answer most of what you would want to know.

By end of this article, we will dispel a few myths about deep learning and answer some widely asked questions about this field. We have also included plenty of resources to get you started.

Here is the exciting part – It isn’t as difficult as most people make it out to be. Read on to find more!

Table of Contents

• What is Deep Learning and why is it so popular?
• Is Deep Learning just a hype or does it have real-life applications?
• What is the difference between Deep Learning and Machine Learning?
• What are the prerequisites for starting out in Deep Learning?
• Is it necessary to have a research background in Deep Learning to start out in it?
• Which Tools/Languages should I prefer to build Deep Learning models?
• Why are GPUs necessary for building Deep Learning models?
• When (and where) to apply Deep Learning?
• Do we need a lot of data to train deep learning models?
• Where can I find some basic project ideas in order to practice deep learning?
• What are the free learning resources for Deep Learning?
• What are Deep Learning interview questions?
• What is the future of Deep Learning?

What is Deep Learning and why is it so popular these days?

Deep Learning is nothing but a paradigm of machine learning which has shown incredible promise in the recent years. This is because of the fact that Deep Learning shows great analogy with the functioning of the human brain. The superiority of the human brain is an evident fact, and it is considered to be the most versatile and efficient self-learning model that has ever been created.

Let us understand the functioning of a deep learning model with an example:

What do you see in the above image?

The most obvious answer would be “a car”, right? Despite the fact, that there is sand, greenery, clouds and a lot of other things, our brain tags this image as one of a car. This is because our brain has learnt to identify the primary subject of an image.

This ability of deriving useful information from a lot of extraneous data is what makes deep learning special. With the amount of data that is being generated these days, we want our models to be better with more of this data being fed into it. While deep learning models get better with the increase in the amount of data. 

Now although Deep Learning has been around for many years, the major breakthroughs from these techniques came just in the recent years. This is because of two main reasons – the first and foremost, as we saw before, is the increase of data generated through various sources. The infographic below succinctly visualizes this trend. The second is the growth in hardware resources required to run these models. GPUs, which are becoming a requirement to run deep learning models, are multiple times faster and they help us build bigger and deeper deep learning models in comparatively less time than we required previously.

This is the reason that Deep Learning has become a major buzz word in the data science industry.

Is Deep Learning just a hype or does it have real-life applications?

Deep Learning has found many practical applications in the recent past. From Netflix’s famous movie recommendation system to Google’s self-driving cars, deep learning is already transforming a lot of businesses and is expected to bring about a revolution in almost all industries. Deep learning models are being used from diagnosing cancer to winning presidential elections, from creating art and writing literature to making real life money. Thus it would be wrong to say that it is just a hyped topic anymore.

Some major applications of deep learning that are being employed by technology companies are:

• Google and Facebook are translating text into hundreds of languages at a time. This is being done through some deep learning models being applied to NLP tasks and is a major success story.
• Conversational agents like Siri, Alexa, Cortana basically work on simplifying the speech recognition techniques through LSTMs and RNNs. Voice commands have added a whole new domain to the possibilities of a machine.
• Deep learning is being used in impactful computer vision applications such as OCR (Optical Character Recognition) and real time language translation
• Multimedia sharing apps like Snapchat and Instagram apply facial feature detection which is another application of deep learning.
• Deep Learning is being used in Healthcare domain to locate malignant cells and other foreign bodies in order to detect complex diseases.

However, some people develop a thinking that deep learning is overhyped because of the fact that labeled data required for training deep learning models is not readily available. Even if the data is available, the computational power required to train such models does not come cheap. Hence, due to these barriers, people are not able to experience the power of deep learning and term it as just hype.

Go through the following blog to build some real life deep learning applications yourself:

• 6 Deep Learning Applications a beginner can build in minutes (using Python)

What is the difference between Deep Learning and Machine Learning?

This is one of the most important questions that most of us need to understand. The comparison can be done mainly on the below three verticals:

Data dependencies

The most important difference between deep learning and traditional machine learning is its performance as the scale of data increases. When the data is small, deep learning algorithms don’t perform that well. This is because deep learning algorithms need a large amount of data to understand it perfectly. On the other hand, traditional machine learning algorithms with their handcrafted rules prevail in this scenario. Below image summarizes this fact.

Feature engineering

Feature engineering is a process of putting domain knowledge into the creation of feature extractors to reduce the complexity of the data and make patterns more visible to learning algorithms to work. This process is difficult and expensive in terms of time and expertise.

In Machine learning, most of the applied features need to be identified by an expert and then hand-coded as per the domain and data type.

For example, features can be pixel values, shape, textures, position and orientation. The performance of most of the Machine Learning algorithm depends on how accurately the features are identified and extracted.

Deep learning algorithms try to learn high-level features from data. This is a very distinctive part of Deep Learning and a major step ahead of traditional Machine Learning. Therefore, deep learning reduces the task of developing new feature extractor for every problem. Like, Convolutional NN will try to learn low-level features such as edges and lines in early layers then parts of faces of people and then high-level representation of a face.

Interpretability

Last but not the least, we have interpretability as a factor for comparison of machine learning and deep learning. This factor is the main reason deep learning is still thought 10 times before its use in industry.

Let’s take an example. Suppose we use deep learning to give automated scoring to essays. The performance it gives in scoring is quite excellent and is near human performance. But there’s is an issue. It does not reveal why it has given that score. Indeed mathematically you can find out which nodes of a deep neural network were activated, but we don’t know what there neurons were supposed to model and what these layers of neurons were doing collectively. So we fail to interpret the results.

On the other hand, machine learning algorithms like decision trees give us crisp rules as to why it chose what it chose, so it is particularly easy to interpret the reasoning behind it. Therefore, algorithms like decision trees and linear/logistic regression are primarily used in industry for interpretability.

If you would like to learn about a more in-depth comparison between machine learning and deep learning, I recommend you go through the following blog:

• Deep Learning vs. Machine Learning – the essential differences you need to know!

What are the prerequisites for starting out in Deep Learning?

Starting out in deep learning is not as difficult as people might make you believe. There are a few elementary basics that you should cover before diving into deep learning. Deep learning requires knowledge of the following topics:

• Mathematics: You should be comfortable with probability, derivatives, linear algebra and a few other basic topics. Khan Academy offers a decent course covering almost all the above topics here.
• Statistics: The basics of statistics are required for going forward with any machine learning problem. Understanding the concepts of statistics are essential because most of the deep learning concepts are derived from assimilating the concepts of statistics. You can check the online courses available here. 
• Tool: A decent level of coding skills are required for implementing deep learning into real life problems. Coursera’s, Introduction to Data Science in Python is a decent course to start off with Python as a tool.
• Machine Learning: Machine learning is the base for deep learning. One can not start learning deep learning without understanding the concepts of machine learning. You could go through Intro to Machine Learning or Andrew Ng’s course Machine Learning for a theoretical base.

For a more detailed understanding about the prerequisites please follow:

• A Complete Guide on Getting Started with Deep Learning in Python

Do I need to do a PhD to make a career in Deep Learning?

No, a PhD is not a mandatory requirement to make a career in deep learning. You can learn, experiment, and build up your work experience portfolio without going to university. The emphasis for any job or role is usually on demonstrating your competence, and not on your degree, per se.

Having said that, a PhD in a specific field (like linguistics for NLP) will definitely accelerate your path if you choose to combine that with deep learning.

Which Tools/Languages should I prefer to build Deep learning models?

I would recommend you use Python, because of its robust ecosystem for machine learning. The python ecosystem comprises of developers and coders who are providing open source libraries and support for the community of python users. This makes the task of writing complex codes for various algorithms much easier and the techniques easier to implement and experiment with.

Also, Python being a more generalized programming language, can be used for both the development and implementation. This greatly simplifies the transition from development to operations. That is, a deep learning product that can predict the price of flight tickets, can not only be developed in python but can also be attached with your website in the same form. This is what makes Python a universal language.

Besides this, I would suggest that beginner’s use high level libraries like Keras. This makes experimentation easier by providing abstraction to the unnecessary information that is hidden under the algorithms. And giving access to the parameters that can be tweaked to enhance the performance of such models. Let us understand this with an example:

When you press the buttons on a television remote, do you need to care about the background processes that are happening inside the remote? Do you need to know about what signal is being sent out for that key, or how is it being amplified?

No, right?

Because maybe an understanding of these processes is required for a physicist but for a lame man sitting in his bedroom, it is just an information overload.

There are also other contenders apart from Python in the deep learning space such as R, Julia, C++, and Java. For alternatives of libraries, you can check out TensorFlow, Pytorch, Caffe2, DL4J, etc. We should stay updated with their developments as well.

If you are not well versed with programming, there are also a few GUI based softwares, that require no coding, to build deep learning models, such as Lobe or Google’s AutoML, among others.

Why are GPUs necessary for building Deep Learning models?

When you train a deep learning model, two main operations are performed:

• Forward Pass
• Backward Pass

In forward pass, input is passed through the neural network and after processing the input, an output is generated. Whereas in backward pass, we update the weights of neural network on the basis of error we get in forward pass.

Both of these operations are essentially matrix multiplications. A simple matrix multiplication can be represented by the image below

Here, we can see that each element in one row of first array is multiplied with one column of second array. So in a neural network, we can consider first array as input to the neural network, and the second array can be considered as weights of the network.

This seems to be a simple task. Now just to give you a sense of what kind of scale deep learning – VGG16 (a convolutional neural network of 16 hidden layers which is frequently used in deep learning applications) has ~140 million parameters; aka weights and biases. Now think of all the matrix multiplications you would have to do to pass just one input to this network! It would take years to train this kind of systems if we take traditional approaches.

We saw that the computationally intensive part of neural network is made up of multiple matrix multiplications. So how can we make it faster?

We can simply do this by performing all the operations at the same time instead of doing it one after the other. This, in a nutshell, is why we use GPU (graphics processing units) instead of a CPU (central processing unit) for training a neural network.

When (and where) to apply Neural Networks ?

Deep Learning have been in the spotlight for quite some time now. Its “deeper” versions are making tremendous breakthroughs in many fields such as image recognition, speech and natural language processing etc.

Now that we know it is so impactful; the main question that arises is when to and when not to apply neural networks? This field is like a gold mine right now, with many discoveries uncovered everyday. And to be a part of this “gold rush”, you have to keep a few things in mind:

• Firstly, deep learning models require clear and informative data (and mostly big data) to train. Try to imagine deep learning model as a child. It first observes how its parent walks. Then it tries to walk on its own, and with its every step, the child learns how to perform a particular task. It may fall a few times, but after few unsuccessful attempts, it learns how to walk. If you don’t let it, it might not ever learn how to walk. The more exposure you can provide to the child, the better it is.
• It is prudent to use Deep Learning for complex problems such as image processing. Deep Learning algorithms belong to a class of algorithms called representation learning algorithms. These algorithms break down complex problems into simpler form so that they become understandable (or “representable”). Think of it as chewing food before you gulp. This would be harder for traditional (non-representation learning) algorithms.
• When you have an appropriate type of deep learning to solve the problem. Each problem has its own twists. So the data decides the way you solve the problem. For example, if the problem is of sequence generation, recurrent neural networks are more suitable. Whereas, if it is image related problem, you would probably be better of taking convolutional neural networks for a change.
• Last but not the least, hardware requirements are essential for running a deep neural network model. Neural nets were “discovered” long ago, but they are shining in the recent years for the main reason that computational resources are better and more powerful. If you want to solve a real life problem with these networks, get ready to buy some high-end hardware!

Do we need a lot of data to train deep learning models?

It is true that we need a large amount of data to train a typical deep learning model. But we can generally overcome this by using something called transfer learning. Let me explain thoroughly.

One of the barrier for using deep learning models for industry applications is where the data is not in huge amount. A few examples of data needed to train some of the popular deep learning models are:

However, a deep learning model trained on a specific task can be reused for different problem in the same domain even if the amount of data is not that huge. This technique is known as Transfer Learning.

For instance, we have a set of 1000 images of cats and dogs labeled as 1 and 0 (1 for cat and 0 for dog) and we have another set of 500 test images that we need to classify. So, instead of training a deep learning model on the data of 1000 images, we can use a pre-trained VGGNet model and retrain it on our data and use it to classify the unlabeled set of images. A pre-trained model may not be 100% accurate in your application, but it saves huge efforts required to reinvent the wheel.

You may have a look at this article to get a better intuition of using a pre-trained model.

Where can I find basic project ideas in order to practice deep learning?

To practice deep learning, ideas alone will not help. We also need labeled data to test our ideas using deep learning.

1. For beginners, getting started with the MNIST data is highly recommended. The dataset contains handwritten digits with their actual labels, i.e., numbers from 0 to 9. You can even compete in the Identify the Digits competition to evaluate your models
2. For intermediate users, this Age Detection challenge is a nice project to work on. The dataset consists of facial images of Indian movie actors. The task is to predict the age of a person from his or her facial attributes. For simplicity, the problem has been converted to a multiclass problem with classes as Young, Middle and Old.

You can also refer this list of exciting deep learning datasets and problems.

What are some of the free learning resources for Deep Learning?

Being a comparatively newer technology, there is not enough content and tutorials available for the beginners. However, free-quality content and resources related to deep learning are steadily increasing. The learning resources can be classified on the different applications of deep learning.

• Image Processing:

1. Convolutional Neural Networks by Andrew Ng
2. Architecture of Convolutional Neural Networks (CNNs) demystified
3. A Beginner’s Guide To Understanding Convolutional Neural Networks
4. Machine Learning is Fun! Part 3: Deep Learning and Convolutional Neural Networks
5. ImageNet Classification with Deep Convolutional Neural Networks
6. R-CNN
7. Going Deeper with Convolutions

• Sequence Prediction/Time Series:

1. Sequence Models by Andrew Ng
2. Essentials of Deep Learning : Introduction to Long Short Term Memory
3. The Unreasonable Effectiveness of Recurrent Neural Networks
4. Understanding LSTM Networks
5. Recurrent Neural Networks & LSTMs

• Natural Language Processing:

1. Deep Learning for Natural Language Processing
2. Distributed Representations of Words and Phrases and their Compositionality
3. Sequence to Sequence Learning with Neural Networks
4. Deep Speech: Scaling up end-to-end speech recognition

Besides this, you can also go through the following blogs for a more extensive list of resources:

• A Complete Guide on Getting Started with Deep Learning in Python
• Awesome Deep Learning Resources

What are some Deep Learning interview questions?

Some common questions that may be asked on deep learning are:

1. How do deep learning models learn?
2. What are some limitations of a deep learning model?
3. What are the differences between feedforward neural networks and recurrent neural networks?
4. What are activation functions and why are they required?
5. What is a CNN and what are its applications?
6. What is pooling? How does it work?
7. What is a dropout layer and why is it used?
8. What is the vanishing gradient problem and how do we overcome that?
9. What are optimization functions? Name a few of the common optimization functions.

Do note that this is not an exhaustive list that will make you completely ready for an interview. You can go through the following skill test to test yourself on important questions on deep learning.

• 30 Questions to test a Data Scientist on Deep Learning
• 40 Questions to test a data scientist on Deep Learning
• 45 Questions to test a data scientist on basics of Deep Learning

What is the future of Deep learning?

Deep learning has come a long way in recent years, but still has a lot of untapped potential. We are still in the nascent stages of this field, with new breakthroughs happening seemingly every day. One of the use-cases that we can definitely see in the suture is of automobile industry, where Deep Learning can revolutionize it by making self-driving cars a reality. While we don’t have a crystal ball to predict the future, we can see deep learning models requiring less and less involvement from human data scientists and researchers.

In the immediate future, we can definitely see a trend where the knowledge of deep learning will be a skill required by every Data Science practitioner. In fact, you must have caught sight of a job position spurn out recently, called a “Deep Learning Engineer”. This person is responsible to deploy and maintain Deep Learning models used by various departments of that company. Needless to say, there will be a huge demand of such people in the industry.

Currently, one of the limitations of DL is that it does what a human asks of it. It requires tons of data to learn it’s target objective, and replicates that. This has induced bias in certain applications. We can see this improving over time such that the bias is eliminated in the training process.

We might even stop differentiating deep learning from the other types of learning, with time. It is primed to become a popular and commonly used field and will not require special branding efforts to market or sell.

There are a lot of cases still where researchers, after training a DL model, are unable to explain the ‘why’ behind it. “It’s producing great results but why did you tune a hyperparameter a certain way?” Hopefully with the rapid advancement in DL, we will see this black box concept becoming history, and we can explain the intuition behind the decision it takes.

End Notes

These are broadly the answers to the most frequently asked questions on our portal or elsewhere by the people who want to jump onto this Deep Learning bandwagon.

Do you have any other questions on deep learning that you need clarification on? Use the comments section below to ask; or hop onto our Discussion portal, and we will help you out!

Learn, compete, hack and get hired!