Demystifying Machine Learning

Summary:

• Parametric models: Models with a fixed set of parameters.
• Non-parametric models: Models are not constrained to be of a specific form or structure. The parameters may vary based on the training dataset.
• Hyperparameters: Variables used to control a model.
• Underfitting: Your model does not capture the structure from the dataset enough.
• Overfitting: Your model has become too sensitive to small changes in your dataset.
• High bias: Model is too constrained, causing underfitting to the dataset.
• High variance: Model is too unconstrained, causing overfitting to the dataset.
• Bias-variance tradeoff: Aim for the ‘sweet spot’ of low bias and low variance!
• Occam’s razor: When more than one model that give similar performance, favour simpler models over complex ones.