Demystifying Machine Learning

Summary:

• An artificial neuron is a basic unit for a Neural Network
• An artificial neuron is a linear equation followed by an activation function
• Example activation functions:
  • Step function: Used by the perceptron. No longer used.
  • Logistic
  • Hyperbolic tangent (tanh)
  • Rectified Linear Unit (ReLU)
  • Variants of ReLU:
    • Leaky ReLU
    • Exponential Linear Unit
• Artificial neurons $\neq$ biological neurons