The Talent500 Blog
Path From Zero To Implementing Deep Learning Research Papers: A Practical Step-by-step Guide 1

Path From Zero To Implementing Deep Learning Research Papers: A Practical Step-by-step Guide

Remember Chappie, the AI based law enforcement robot that accomplished the seemingly impossible feat of puzzling out human behaviour? Now tech companies are trying to make him…well, a version of him a reality. You’ve read the news and along with it jargon like Machine Learning, Artificial Intelligence and Deep Learning.

One particular field that has frequently been in the spotlight in the past few years is Deep Learning. It is a popular branch of Machine Learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. While Machine Learning has been around for over five decades, Deep learning has only taken off in the last eight years or so as a way to solve more complex problems like self-driving cars, which couldn’t be effectively devised by shallow machine learning algorithms.

Simply put, Deep Learning is effective in deciphering patterns from data. Instead of writing programs that solve problems, with machine learning we create programs that learn to solve the problem from examples. It’s this process of learning that allows the algorithm to improve over time and to be more clever than if people wrote down a very rigid set of instructions for them.

This blew my mind, back when I started. I knew Deep Learning was a hot topic, and it still is. I’ve constantly been asked about how to pick up the skills and progress in the field by friends, colleagues and juniors, so I thought I’d write up a short guide about how I got started and some of my learnings through the process. My aim is to provide a learning path to all those who are new to deep learning and also the ones who want to explore it further. 

Let’s go!

1. Study Machine Learning first

If you are new to Data Science and don’t know how gradient descent works, weight decay, regularization, linear algebra, and calculus, start with the basics of Machine Learning. Machine Learning with Andrew Ng on Coursera is an excellent resource to get a grasp of the basics. Andrew Ng is a decorated Stanford CS professor who was among the first to implement Deep Learning algorithms on GPUs. And, of course, complete the assignments. 

2. Understand Basic Deep learning Theory

Next step: there are two options – 

  1. Take up CS231n: Convolutional Neural Networks for Visual Recognition by Stanford on Youtube. The 2016 version taught by Karpathy is more educative and detailed than the newer ones. 
  2. Or study by Andrew Ng. 

Opt for CS231n because it’s more rigorous and covers more topics. Also mind that it has a very steep learning curve. On the other hand, if you are looking for a more simplified version, then would be an ideal choice to start out in the field. 

3. Pick up practical Deep Learning 

If you can stomach it, in parallel to learning DL theory, you can start Part 1: Practical Deep Learning for Coders taught by Jeremy Howard for his mission to democratize Deep Learning. Howard is an entrepreneur, researcher, ex-Kaggle president and an awe-inspiring  teacher. The course will give you a hands on experience with state of the art Deep Learning architectures and will get you started on Kaggle. The best part about this course is that in the very first lesson, you will start training models. This course requires a GPU, and considering they are expensive, i have seen it is one of the main reasons why Deep Learning is difficult to penetrate. So, if you don’t own one, you could use the free ones by Google Colab. 

4. Keep up with latest developments in DL

And finally, keep up with the most important advancements in the field. Supplement Point 3 with Part 2: Cutting Edge Deep Learning for Coders. This will help you develop a deep understanding of neural network foundations and implement cutting edge research in the field. DL has only become mainstream only in the last eight years, and therefore most of the information out there is still in the form of research papers and riddled with Greek alphabets and daunting equations (another reason why aspirants drop the bundle!). But rest assured this course will help you break important information down and get comfortable with reading papers.

End Notes

As with learning anything new, learning Deep Learning will take time. And in my opinion, it is a process best undertaken with utmost dedication and grit. A major problem I see with youngsters is that they are looking for quick results, and often jump from course to course without seeing one through entirely, especially when things get complicated. So as someone who has stuck with it, I would urge you to put in the time and stick with the courses till the end to master it. Depending on how much time you dedicate to studying, each course will take between one to two months.

And the best way to actually learn, not just in theory is to ensure you complete all of the assignments offered by the courses. The assignments are not only for self-assessment purposes, but offer optimum learning experience and help you understand the practical application of the theoretical concept. And, learning is an iterative and a continuous process. So to ensure continued learning, continue to find new papers and implement them. Deep learning is advancing and evolving so quickly that in order to stay on top of state of the art architectures you are required to keep at it and get comfortable with researching and reading.

This article was originally published as a Linkedin Pulse by Naveen Benny on 28 August 2020.

Naveen Benny

Naveen Benny

Expert AI practitioner building Deep Learning systems. Inquisitive about Computer Vision and GANs. Loves yoga and is always-on meditative mode. Leads the data science team at Talent500.

Add comment