Traditional programming writes explicit rules: input plus code yields output. Machine learning (ML) inverts this — you supply examples of inputs and desired outputs, and the algorithm learns the rule (a model) itself. ML is a subfield of artificial intelligence. Its main paradigms:

• Supervised learning — learn from labelled examples (input → known answer). Includes regression (predict a number) and classification (predict a category).
• Unsupervised learning — find structure in unlabelled data, e.g. clustering similar items.
• Reinforcement learning — an agent learns by trial and error, guided by rewards.

Learning is an optimisation: define a loss function measuring how wrong the model is, then adjust the model's parameters to minimise it. The workhorse method is gradient descent — compute the slope (gradient) of the loss with respect to each parameter and step in the downhill direction. A neural network stacks many simple weighted-sum-plus-nonlinearity units into layers, and deep learning uses many such layers to learn rich representations.

Picture the loss as a landscape of hills and valleys, where each horizontal coordinate is a parameter setting and height is the error. Learning is walking downhill to the lowest valley. The gradient points uphill fastest, so we step the opposite way; the learning rate sets the step size — too small and training crawls, too large and it overshoots and diverges. Repeat over the data and the model's error falls.

The central peril is generalisation. A model that merely memorises its training data — overfitting — performs perfectly in practice and uselessly on anything new, like a student who memorised the answer key without understanding. We guard against this by holding out a test set the model never trains on, and by regularisation. The whole point of ML is to perform well on unseen data; training accuracy alone is a vanity metric.

We implement linear regression by gradient descent from scratch — the "hello world" of ML. The model learns the line y = w·x + b that best fits data generated from y ≈ 2x + 1, purely by minimising squared error.

# Training data roughly following y = 2x + 1
X = [1, 2, 3, 4, 5]
Y = [3, 5, 7, 9, 11]

w, b = 0.0, 0.0          # parameters, start ignorant
learning_rate = 0.01
n = len(X)

for epoch in range(1000):                 # 1000 passes over the data
    # 1. Predict and measure error (mean squared error)
    preds  = [w * x + b for x in X]
    errors = [preds[i] - Y[i] for i in range(n)]

    # 2. Gradients of the loss w.r.t. w and b (calculus of MSE)
    grad_w = (2 / n) * sum(errors[i] * X[i] for i in range(n))
    grad_b = (2 / n) * sum(errors[i]          for i in range(n))

    # 3. Step downhill: adjust parameters against the gradient
    w -= learning_rate * grad_w
    b -= learning_rate * grad_b

print("learned: y = " + str(round(w, 3)) + "x + " + str(round(b, 3)))
# -> learned: y = 2.0x + 1.0   (recovered the underlying rule!)

# Predict an unseen input -> generalisation
print("predict x=6:", round(w * 6 + b, 2))   # ~13.0

No rule for the line was ever written — only a loss to minimise and the downhill rule of gradient descent. After a thousand tiny steps the parameters converge to w≈2, b≈1, and the model correctly predicts an input it never saw. This exact loop, scaled to billions of parameters, underlies modern deep learning.