Best Resources to Learn Machine Learning
Updated: Feb 15
Machine learning is the study and use of algorithms and statistical techniques to make computers learn from data, without being explicitly programmed. These algorithms are mathematical models built on sample data, from which they learn patterns, to then make inferences about future or unseen data.
Machine learning algorithms can be grouped into 3 main categories, those suitable for supervised learning, those for unsupervised learning and those for reinforcement learning. In supervised learning, the computers use algorithms to learn from data where there is a “label” or a “target” available, or in other words, the outcome is known. Unsupervised learning groups a collection of techniques used to learn patterns from data where no “label” is available, that is, when we do not know the outcome. Reinforcement learning refers to the learning of actions that need to be taken in order to maximise a reward.
Some of the most common and widely used learning algorithms for supervised learning include Linear and Logistic Regression, Classification and Regression Trees, Random Forests, Gradient Boosted Trees, Support Vector Machines, Nearest Neighbours and Neural Networks. Among the algorithms for unsupervised learning we find clustering, like k-means and hierarchical clustering, One-Class Support Vector Machines, Isolation Forests, and univariate and multivariate Gaussian models for anomaly detection.
To become a data scientist, there are a few data science core competencies required, which include the use of tools like Python, R, and in most cases SQL, and certainly Git. In addition, a good and solid understanding of the different algorithms for supervised and unsupervised learning is required. Mastering Machine Learning algorithms takes time, but, what is the rush anyways? The key is to keep up with it, take courses, read books, read articles, practice what we learn, enter data science competitions, and every time we pick up a project, try to do it better, try something new, challenge ourselves. Slow and steady wins the race, or so they say :)
In recent years, a number of courses and online resources have sprung up covering almost every aspect of machine learning. Knowledge is at hand for the committed student to step into and excel in machine learning. The available Machine Learning resources cover everything from the very basics to more in depth knowledge of the field. My personal preference is to dig deep and understand each topic and each algorithm well. This doesn’t mean that we need to know every technique in depth right from the start. Instead, with time and dedication, we revisit the same concepts once and again, and try and grasp new layers of knowledge.
In this post, I describe what I think are the best resources to learn Machine Learning, which will allow a dedicated student to take in the very basic concepts and then dive into more complex information if so desired. With this said, let’s begin…
Disclaimer: Opinions in this article are my own and I do not become any financial compensation from the links included in this blog. The article does not contain affiliate links.
Online Courses to Learn Machine Learning
The Analytics Edge, MIT (edX)
Machine Learning, Stanford University (Coursera)
Machine Learning A-Z™: Hands-on Python and R in Data Science (Udemy)
In Depth Intro to Machine Learning in 15 Hrs Of Experts Video (Dataschool)
Advanced Machine Learning Specialisation, (Coursera)
Artificial Intelligence, Columbia University (edX)
University Classes on Machine Learning.
Complete Collection of Machine Learning Classes, Stanford University, Andrew Ng
Books on Machine Learning.
Introduction to Statistical Learning, by Trevor Hastie, Robert Tibshirani and others
Elements of Statistical Learning, by Trevor Hastie, Robert Tibshirani and Jerome Friedman.
Classification and Regression Trees by Leo Breiman
Scientific Articles of Machine Learning Algorithms
Random Forests, Leo Breiman, Journal of Machine Learning, 2001
Bagging Predictors, Leo Breiman, Machine Learning, 1996
Other Blogs with Resources to Learn Machine Learning
Online Courses to Learn Machine Learning
Online courses provide a variety of material to learn machine learning, including filmed lectures, articles and problems to resolve through programming and machine learning. Many online courses also offer the possibility to interact among students and with the teaching staff. Courses often have quizzes and assignments to test the acquired knowledge and support the learning experience. There are multiple online courses on machine learning that are suitable for complete beginners, even with no programming experience, as well as for those with already some knowledge of programming or machine learning. Let’s explore them…
1. The Analytics Edge, MIT (edX)
The Analytics Edge course on edX, designed and taught by instructors from the MIT, will teach you how to use R for data analysis and machine learning, as it explores a variety of data from real business scenarios. You will learn to use R and to tackle data science problems all in one go. The Analytics Edge covers the basic machine learning algorithms, i.e., Linear and Logistic Regression, Classification Trees and Random Forest. It also teaches clustering, linear optimisation and includes a very good introduction to text analytics. The Analytics Edge includes a variety of real world examples in which different learning algorithms have been successfully implemented, and provides a compelling argument about the value and use of these statistical learning models. It will give you a solid ground to start tackling your own data science projects.
I highly recommend The Analytics Edge if you are a complete beginner both to machine learning and programming. Why is The Analytics Edge such a good course? First, the instructors have gathered a fascinating list of real world examples of how analytics and machine learning have been used to drive business growth. These stories include the famous Netflix competition, IBM Watson, Moneyball and examples from well-known companies like Twitter and eHarmony. These stories are fun to learn but most importantly, they contribute to your breadth of knowledge on how data science can be used to bring business value, and will equip you well to answer interview questions, if you have little-to-no-previous experience. Second, you will be practising and programming in R right from the start. The instructors teach a topic and immediately set out to tackle it with commands, code and approaches implemented in R, and the course is designed so that you can practice programming whilst you are taking the lesson. The course also includes a variety of programming exercises designed to develop and strengthen your programming skills and your approach to resolve these kinds of problems. Third, the course covers all the essential techniques utilised in data science, including Linear and Logistic Regression, Classification Trees and Random Forests, clustering and text analytics. You will develop a very good intuition of when to use each model and how to implement them in R. Fourth, the instructors have included a Kaggle competition at the end of the course, which is extremely motivating for consolidating everything you have learnt. Fifth, Allison O’Hair, one of the main instructors, is extremely clear and engaging in her classes and makes it a real pleasure to follow her along. Finally, this course requires no programming experience and no exhaustive calculus or math knowledge, so everyone can take it and learn a lot from it.
The Analytics Edge is an extensive course (13 weeks) and requires around 10-15 hrs per week of work and dedication to go through the lectures and do the assignments. And this is a good thing, right? No pain, no gain. Almost certainly, you will find out at the end of the course that you have learnt a ton and it was definitely worth your time.
The Analytics Edge course on edX can be taken for free, or you can choose to pay a fee if you want a certification. The only downside of this course is that it runs once a year only, and therefore you need to wait until the next course begins to be able to access to the content. Once you enrol though, you can access it any time, even after the course finishes.
2. Machine Learning, Stanford University (Coursera)
The course Machine Learning from Andrew Ng, available on Coursera, is definitely the most well-known and perhaps, as well, the most thorough online class on Machine Learning. The Machine Learning course has the best mix of intuition and actual mathematical understanding of the different algorithms taught throughout the course. Intuition and mathematical understanding are built step by step, one algorithm at the time, one concept at the time, so that before you notice, you pass from building simple linear models, to developing more complex non-linear solutions.
The course Machine Learning starts teaching the simplest algorithms (Linear and Logistic Regression) and adds complexity to them, lecture by lecture, to show how, by combining variables in a non-linear manner, you can transition from linear to more complex non-linear models. The course Machine Learning also teaches you what the algorithms do behind the scenes to learn their parameters, that is, to learn the patters from the data. Therefore, Machine Learning from Andrew Ng leaves you with a more complete understanding of how the algorithms work, what they are learning and how they are learning it. In the course Machine Learning, Andrew Ng provides a clear explanation of regularisation, why it is important and how to implement it in your algorithms. Then he expands on to cover complex learning models like Neural Networks and the classic Support Vector Machines. The course also includes lectures on unsupervised learning, introducing clustering, recommender systems and anomaly detection using univariate and multivariate Gaussian distributions.
Although some level of calculus and knowledge of probability are required to get the most out of this course, the classes are designed so that you will be able to understand and implement the algorithms even if you are not deeply familiar with these concepts. Curiously, these classes do not cover classification or regression trees, so you will need to learn these concepts from other sources.
The course includes several multiple choice quizzes and code assignments. Code assignments are done in Matlab or its open source version Octave, which may put some people off. I’d suggest you try to implement these exercises in Python as well, to expand your knowledge to this tool.
Machine Learning from Andrew Ng is a must for every self-taught Data Scientist. It can be audited for free, or you can choose to pay a fee if you wish to have a certification.
3. Machine Learning A-Z™: Hands-on Python and R in Data Science (Udemy)
Machine Learning A-Z™ on Udemy is an extensive course, which covers almost every available algorithm for supervised learning, both in the context of regression and classification. The course shows how to use Linear and Logistic Regression, Classification and Regression Trees, Random Forests and Gradient Boosted Machines, Support Vector Machines, Nearest Neighbours and Naïve Bayes. The course also includes a section for each of the following areas: i) clustering for unsupervised learning (k-means and hierarchical clustering), Reinforcement Learning, Natural Language Processing, Deep Learning, Dimensionality Reduction including Principal Component Analysis and Linear Discriminant Analysis, as well as a section on data pre-processing and a dedicated section to the XGBoost implementation of Gradient Boosted Machines.
Machine Learning A-Z™ is very suitable for complete beginners with minimal knowledge of machine learning, data science and programming. The instructors begin each section with an intuition about the algorithm and the problem, and then a demonstration of how to tackle the problem both in Python and in R. Machine Learning A-Z™ capitalises on the applied use of machine learning, and offers therefore a light touch on math or the actual working of the algorithms. It focuses more on getting the students started in the implementation of these models in R or Python. So if you are keen to start building machine learning models without delay, this is the right course for you. You will learn how to do so using the Python industry standard library for machine learning, Scikit-learn, or the appropriate packages in R, like caret. One thing students highlight in their reviews though, is the lack of demonstrations of the use of these algorithms to real life problems.
I recommend that any student wishing to excel in data science ensures that they understand the underlying math and workings of the algorithm, for example, how the algorithms learn the patterns, or how decision trees select the features at each split, among other things. Therefore, it is a good idea to follow up this course by reading perhaps one of the books on Machine Learning that I mention later on in this post.
A big plus of Machine Learning A-Z™ is the number of different machine learning algorithms covered, as well as the amount of code provided to the students, which you can take home and apply in your own projects. The course has also consistent high ratings by the students. Courses on Udemy are not free but you can get them at a reasonable price with discounted coupons, which Udemy advertises regularly.
4. In Depth Intro to Machine Learning in 15 Hrs of Experts Video (Dataschool)
In the in Depth Intro to Machine Learning in 15 hrs of experts’ video on Dataschool, Dr. Hastie and Dr. Tibshirani discuss much of the material they gathered in the book An Introduction to Statistical Learning, with Applications in R, which I cover later on in the post. The course in Depth Intro to Machine Learning in 15 hs of experts’ video covers linear and polynomial regression, logistic regression and linear discriminant analysis. It also covers cross-validation and regularisation methods like ridge and lasso. The course then expands on to discuss, splines models and generalised additive models, tree-based methods like Random Forests and boosted trees, and support vector machines. In Depth Intro to Machine Learning browses also over unsupervised learning methods like principal components and clustering. The course is not heavy on maths, trying to describe the methods without heavy reliance on formulas and it offers good examples of applications in R. This course can be taken for free and it is a good alternatively if you prefer watching videos to reading a book.
5. Advanced Machine Learning Specialisation (Coursera)
Having mastered the fundamentals of Machine Learning, which include, Linear and Logistic Regression, Classification and Regression trees, Random Forests and Gradient boosted Machines, as well as, algorithms for clustering and anomaly detection, you are ready to give the next step in the Machine Learning journey.
The Advanced Machine Learning Specialisation represents this next step, providing an introduction to many follow up areas of machine learning, including: i) Deep Learning, ii) Reinforcement Learning, iii) Computer Vision, iv) Natural Language Processing and, v) Bayesian methods. It also offers advanced implementations of machine learning to enter into and do well in data science competitions, as well as to address scientific problems like the large Hadron Collider Challenges.
On this point, I would say that if it is Deep Learning what you would like to learn, there are better and more extensive courses both in Coursera and other platforms, which I will review in a separate post. This specialisation is a good option for a next step overview on several more advanced concepts in machine learning, without really deep diving into any one of them.
Three of the courses in the Specialisation, Introduction to Deep Learning, Practical Reinforcement Learning and Deep Learning for Computer Vision, provide an introduction to the different uses of Deep Learning. You will first get a basic understanding of modern Neural Networks and their applications to computer vision and Natural Language Processing, with some example applications. Next you will learn the foundations of Reinforcement Learning and implement Reinforcement Learning to teach Neural Networks to play games, and finally, you will learn more about the uses of Deep Learning in computer vision, specifically in the context of face recognition, image classification and annotation, and object recognition, among other topics.
In the course Bayesian Methods for Machine Learning, you will learn the basics of Bayesian methods, from how to define a probabilistic model to how to make the predictions. You will learn to automate this workflow in the context of image analysis and drug discovery.
The course application of Machine Learning to Hadron Collider Challenges promises to apply machine learning to the biggest data generation source to date, where you will have the opportunity to apply your machine learning skills in Experimental Physics.
The course How to Win a Data Science Competition: Learn from Top Kagglers, is perhaps the best follow up course to the previous online courses mentioned earlier in this post: The Analytics Edge and Stanford’s Machine Learning on Coursera. How to Win a Data Science Competition was developed by a group of Kaggle competitors who won several competitions. They will teach you every aspect you need to consider to rank high in a Kaggle competition, including how to pre-process your data, how to create new features from various data sources, including tabular, text and image data, how to implement reliable cross-validation, how to tune hyper-parameters and how to carry on stacking of multiple machine learning models, to extract an extra edge of model performance. Although many of these techniques may not be suitable for the work carried on in businesses and other organisations, the course, and more importantly, entering into Kaggle competitions, will help you consolidate those basic concepts and fundamentals on Machine Learning. In fact, Kaggle competitions are one of the best options available to practice Machine Learning.
Going back to the course, the authors also cover and use the latest implementations of gradient boosting machines, lightGBMs, very popular in Kaggle winning solutions. A small downside, is that the course doesn’t offer extensive code, so you will have to find the way to code the concepts you learn by yourself.
The Advanced Machine Learning Specialisation was developed by a team of 21 lecturers, professors and researchers from the Russian HSE School of Computer science, as well as data scientist from various industries, which are known for being top Kaggle competitors. Note that this is a more advanced course and assumes basic knowledge of machine learning. The Specialisation can be audited for free, however the audit route only gives access to the video lectures and some of the assignments. If you want to access the full content or get a certification, there is a monthly fee attached.
6. Artificial Intelligence, Columbia University (edX)
The most relevant course for Machine Learning is the second one in the specialisation, called not surprisingly Machine Learning. The course covers the fundamentals as well as more advanced topics in Regression, like Linear, Ridge and Bayesian regression, it covers Laplace approximation, kernel methods and Gaussian processes, as well as Support Vector Machines, Classification and Regression Trees, Random Forests and Gradient Boosted Trees, clustering, dimensionality reduction, Markov models and association analysis. The course Machine Learning offers equal emphasis on theory and practice, and it covers more topics, comparatively, than the more well-known Machine Learning from Andrew Ng available on Coursera, therefore providing a more advanced course. The concepts of machine learning are applied to real-life challenges and the instructor, Professor Paisley has received amazing reviews and highlighted for giving clear explanations.
The other courses of the specialisation will help you gain expertise in the design of neural networks and applications of artificial intelligence in robotics and computer vision. The courses of the specialisation can be audited for free, or you can choose to pay a fee for the certification.
University Classes on Machine Learning
1. Complete Collection of Machine Learning Classes, Stanford University
The Complete Collection of Machine Learning Classes from Professor Ng correspond to the course CS229 taught at Stanford University. The course provides a broad introduction to machine learning and statistical pattern recognition, covering in depth supervised and unsupervised learning, learning theory, reinforcement learning and adaptive control. The classes include applications of Machine Learning to robotics, data mining, speech and text recognition, among others. The Complete Collection of Machine Learning Classes from Professor Ng at Stanford University are heavy on maths, and are therefore a natural follow up to the course Machine Learning which he teaches in Coursera. If you are familiar with machine learning algorithms, these classes will help you take your knowledge to the next level.
Books on Machine Learning
1. Introduction to Statistical Learning, by T. Hastie, R. Tibshirani and others
Introduction to Statistical Learning is a highly recommendable book for an introduction to Machine Learning. It covers Linear and Polynomial Regression, cross-validation, Classification and Regression Trees, bagging and boosting, and clustering, among other Machine Learning algorithms. It is not heavy on maths, providing a clear intuition on how algorithms work, and it includes code in R with several examples. Very easy to read, therefore providing a soft and friendly introduction to Machine Learning. A must read for complete beginners as an alternative to online courses. And the good news are, the book is freely available online.
2. Elements of Statistical Learning, by T. Hastie, R. Tibshirani and J. Friedman
Elements of Statistical Learning is a renowned and classical book for an introduction to Machine Learning. This book also covers Linear and Polynomial Regression, cross-validation, Classification and Regression Trees, bagging and boosting, and clustering, among other Machine Learning algorithms. Elements of Statistical Learning offers in addition, a mathematical explanation with full formulas and their derivation, for the optimisation functions of the algorithms, therefore equipping you with a deeper understanding of the models. Elements of Statistical Learning is therefore a very good book for consultation as you progress in your career in Data Science and it is available online for free.
3. Classification and Regression Trees by Leo Breiman
The book Classification and Regression Trees covers in detail the rationale and mathematics behind classification and regression trees and Random Forests. It explains extensively how these models are built and what they are trying to minimise. Classification and Regression Trees includes a thorough explanation of cross-validation and bagging of algorithms. This is the natural follow up consultation book to the online courses, where you will learn a soft touch on these algorithms. If you feel that after taking the online courses, your understanding of classification trees is not deep enough, this is definitely the resource for consultation. This books is unfortunately a bit expensive. If you are lucky enough to be at University, maybe you can borrow it from the library. Alternatively, you can refer to the original scientific papers from Leo Breiman, which I quote later on in this post.
Scientific Articles of Machine Learning Algorithms
1. Random Forests, Leo Breiman, Journal of Machine Learning, 2001
Random Forests by Leo Breinman is a scientific article where the author examines deeply how Random Forests work. The article compares Random Forests against other tree based algorithms and explains its advantages, how they generalise better, and why, in theory, they are not prone to over-fitting. The article is good for those seeking a more clear and mathematical understanding of how the algorithm works.
2. Bagging Predictors, Leo Breiman, Machine Learning, 1996
Bagging Predictors by Leo Breiman is perhaps the precursor theory to the development of Random Forests. It explains how the performance of an algorithm can be improved by generating multiple versions from the same algorithm, each one created from bootstrap replicates of the learning dataset. Good to get familiar with the concept of bootstrapping datasets and model ensemble.
Kaggle Learn, free micro-courses on various aspects of machine learning supported by Kaggle.
Kaggle Competitions, entering any of the available competitions on Kaggle is the best way to practice the acquired concepts. In addition, you can learn from solutions and code implemented by other people. Kaggle also offers simpler educative competitions, like for example the Titanic, or House Prices ones, for those stepping into Data Science and Machine Learning.
Papers with Code, all the latest research papers in ML with code.
Two Minute Papers, short video breakdowns of AI and other research papers.