Last updated
What Is a Derivative?
The derivative of a function measures its instantaneous rate of change at any point. Geometrically, it's the slope of the tangent line to the function's graph at that point. Derivatives are the foundation of calculus and appear throughout physics (velocity, acceleration), economics (marginal cost, marginal revenue), machine learning (gradient descent), and engineering.
The derivative of f(x) is written as f'(x), df/dx, or Df(x). It's defined as the limit: f'(x) = limh→0 [f(x+h) − f(x)] / h.
Basic Differentiation Rules
| Rule | Formula | Example |
|---|---|---|
| Power rule | d/dx[xⁿ] = nxⁿ⁻¹ | d/dx[x³] = 3x² |
| Constant rule | d/dx[c] = 0 | d/dx[5] = 0 |
| Constant multiple | d/dx[cf(x)] = cf'(x) | d/dx[3x²] = 6x |
| Sum rule | d/dx[f+g] = f'+g' | d/dx[x²+x] = 2x+1 |
| Product rule | d/dx[fg] = f'g + fg' | d/dx[x·sin x] = sin x + x·cos x |
| Chain rule | d/dx[f(g(x))] = f'(g(x))·g'(x) | d/dx[sin(x²)] = cos(x²)·2x |
| Quotient rule | d/dx[f/g] = (f'g − fg') / g² | d/dx[x/sin x] = (sin x − x·cos x) / sin²x |
Common Derivatives
| Function | Derivative |
|---|---|
| sin(x) | cos(x) |
| cos(x) | −sin(x) |
| tan(x) | sec²(x) |
| eˣ | eˣ |
| ln(x) | 1/x |
| aˣ | aˣ · ln(a) |
| arcsin(x) | 1/√(1−x²) |
| arctan(x) | 1/(1+x²) |
Derivatives in Machine Learning
Gradient descent — the algorithm that trains neural networks — relies entirely on derivatives. The gradient of the loss function with respect to each weight tells the optimizer which direction to adjust the weights to reduce the loss. Backpropagation is just the chain rule applied repeatedly through the layers of a neural network.