Demystifying Machine Learning

Summary:

• Regularization: Controls overfitting by preferring simpler models.
• Use a hyperparameter $\lambda$ to control the regularization.
• Different types:
  • $L_1$ regularization: encourages fewer parameters
  • $L_2$ regularization: encourages smaller parameter values (spread out)