Machine Learning Algorithm Selection

[!question] How to decide which algorithm to use?
In core, its always trial and error. But we can decrease the search range by a few factors

• Explainability
  • K-nearest Neighbor (KNN)
  • Decision Tree
  • Linear Regression
• In-memory or out-of-memory
  • K-nearest Neighbor (KNN) or Support Vector Machine (SVM) keeps every data in-memory
  • Neural Network can be used for out-of-memory
• Number of features or Example
  • For millions of data or features, use Neural Network or Logistic Regression
  • For less, Support Vector Machine (SVM)
• Nonlinearity of Data
  • Linear - Support Vector Machine (SVM), Logistic Regression, Linear Regression
  • Else - Neural Network, Ensemble Learning
• Training Speed
  • Fast - Logistic Regression, Linear Regression, Decision Tree
  • Slow - Neural Network
• Prediction Speed
  • Fast - Support Vector Machine (SVM), Linear Regression, Logistic Regression, SOME Neural Network
  • Slow - K-nearest Neighbor (KNN), Ensemble Learning, Very Deep Neural Network

• Scikit Learn Algorithm Selection Cheatsheet