Machine Learning Algorithm Implementation

What's this about

In order to build powerful models, I need to understand how machine learning algorithms work and how to tune hyperparameters so that I can achieve my goal. The basic idea is to study each algorithm on Introduction to Statistical Learning and implement the algorithm. I will try to implement the algorithm without using sklearn if I can because I think it will help me understand how the model works.

What algorithms to work

I plan to work on algorithms below:

1. Linear Regression (done)
2. Logistic Regression (done)
3. KNN
4. Decision Tree
5. Random Forest
6. SVM
7. K-means
8. Dimension Reduction
9. XGBoost
10. Gradient Boosting
11. Naive Bayes

How I work on them

I will work on this in three steps:

1. Understand the theory behind the algorithm including:

• how to get the formula of the algorithm
• when to use it
• cost function
• pros/ cons
• compare to similar models
• mapreduce computation
• time complexity

2. Implement algorithm in Python.
3. Utilize sklearn document to work on adjust parameters to practice improving the algorithm.

Resources

1. Machine Learning by Andrew Ng from Stanford University
2. An Introduction to Statistical Learning