A coding-cost explanation of entropy, cross entropy, and KL divergence: what is unavoidable uncertainty, what is the cost of coding with the wrong distribution, and why cross entropy naturally appears in classification and language-model training.

A practical guide to loss functions: when to use MSE, MAE, Huber, binary cross entropy, cross entropy, KL divergence, hinge loss, contrastive loss, and triplet loss.