ML Algorithms Source Code

机器学习算法源码实现。

---

Contributer: datamonday

Github Repo: https://github.com/datamonday/ML-Algorithm-Source-Code

---

Supervised Learning

1. • kNN (k-Nearest-Neighbors)
2. • Logistic Regression
3. • Gaussian Discriminat Analysis
4. • Naive Bayes

5. Decision Tree

   • ID3
   • C4.5
   • CART (Classification and Regression Decision Tree)

6. SVM (Support Vector Machine)

   • SVC (Support Vector Classifier)
   • SVR (Support Vector Regression)

7. Linear Regression

   • PolynomialRegression
   • LassoRegression (L1)
   • RidgeRegression (L2)
   • ElasticNet (L1+L2)

8. Bagging

   • Random Forest

9. Boosting

   • AdaBoost
   • GBDT
   • XGBoost

10. Stacking

11. Perceptron

    • Simple Perceptron
    • Multi-Layer Perceptron

12. LDA (Linear Discriminant Analysis)

    • Simple LDA
    • Multi-Class LDA

Unsupervised Learning

1. • PCA (Principal Component Analysis)
2. K-Means
   • K-Means
   • K-Means ++
   • ISODATA
3. • Fuzzy C-Means
4. • Gaussian Mixed Model
5. • SOFM (Self-Organized Feature Map)
6. • DBSCAN
7. AutoEncoder (AE)

• AE
• VAE

8. GAN (Generative Adversarial Network)
   • DCGAN
   • StyleGAN
9. Restricted Boltzmann Machine (RBM)
10. Anormaly Detection
    • Isolation Forest
    • One-Class SVM

Compute Intelligence

1. • GA (Genetic Algorithm)
2. • PSO (Particle Swarm Optimization)
3. • ACO (Ant Clony Optimization)
4. • SA (Simulated Annealing)
5. • NeuroEvolution

Deep Learning

1. Activation Functions

   • Sigmoid
   • Tanh
   • Softmax
   • ReLU
   • LeakyReLU
   • PReLU
   • ELU
   • SELU
   • SoftPlus

2. Loss Functions

   • HigeLoss
   • SquareLoss
   • CrossEntropy
   • MSE
   • RMSE
   • RMSLE
   • MAE
   • MAPE

3. Optimizers

   • GD (Gradient Descent)
   • SGD (Stochastic Gradient Descent)
   • Mini-GD
   • NAG (Nesterov Accelerated Gradient)
   • SGD + Momentum
   • AdaGrad
   • AdaDelta
   • RMSProp
   • Adam
   • AdaMax
   • Nadam (Adam + NAG)

4. Layers

   • Dense
   • Flatten
   • Reshape
   • Dropout
   • Activation
   • Batch Normalization
   • Layer Normalization
   • Group Normalization
   • RNN
   • LSTM
   • Conv1D
   • Conv2D
   • MaxPooling2D
   • AvgPooling2D

5. Neural Networks

Reinforcement Learning

1. Q-Learning
2. Sara-Learning

Utils

1. Kernels

   • Linear Kernel
   • Polynomial Kernel
   • RBF Kernel

2. Data Pipeline

3. Online Flow Data Process

4. Data Operation

   • shuffle data
   • normalize
   • standardize
   • batch iterator
   • divide on features
   • polynomial features
   • get random subsets
   • normalize
   • standardize
   • train test split
   • k fold cross validation sets
   • bootstrap sample
   • to categorical
   • to nominal
   • make diagonal

5. Evaluation Metrics

   • calculate_entropy
   • mean_squared_error
   • calculate_std_dev
   • calculate_variance
   • accuracy_score
   • recall_score
   • precision_score
   • f1_score
   • calculate_covariance_matrix
   • calculate_correlation_matrix

6. Feature Selection

   • Filter
   • wrapper
   • statistical

7. Fine-Tune Hyperparameters

   • Grid Search
   • Random Search
   • Bayesian Optimization
   • Hyperband

8. PlotFunctions

   • loss curve
   • residual loss curve
   • acc curve
   • roc
   • p-r curve

---

Reference

1. https://github.com/eriklindernoren/ML-From-Scratch