A
| Term | Definition |
| Antiderivative | A function whose derivative is the given function. Defined only up to an additive constant, which is where the constant of integration comes from. |
| Asymptote | A line a curve approaches without meeting. Vertical asymptotes mark where a function is undefined, horizontal ones describe its behaviour at infinity. |
| Augmented Matrix | The coefficient matrix of a linear system with the constant column attached, the form row reduction operates on. |
| Average Rate of Change | The change in output divided by the change in input over an interval, the slope of the secant line. The derivative is its limit as the interval shrinks to zero. |
B
| Term | Definition |
| Basis | A set of vectors that is linearly independent and spans the space, so every vector has exactly one representation in it. The number of basis vectors is the dimension. |
C
| Term | Definition |
| Chain Rule | The rule for differentiating a composition: the derivative of f(g(x)) is f'(g(x)) times g'(x). Applied repeatedly through a network’s layers, it is exactly what backpropagation computes. |
| Change of Basis | Re-expressing vectors and transformations in a different coordinate system, through multiplication by a change of basis matrix. Diagonalisation is the special case where the new basis makes the transformation as simple as possible. |
| Characteristic Polynomial | The polynomial det(A minus lambda I), whose roots are the eigenvalues of A. |
| Column Space | The span of a matrix’s columns, that is all vectors reachable as Ax. A system Ax equals b has a solution exactly when b lies in it. |
| Concavity | Whether a curve bends upward or downward, given by the sign of the second derivative. Concave up means the derivative is increasing. |
| Constant of Integration | The arbitrary constant added to an indefinite integral, since differentiation destroys constant terms. |
| Continuity | The property that a function has no jumps, holes or blowups: the limit at a point exists and equals the function’s value there. |
| Critical Point | A point where the derivative is zero or undefined. Candidate location for a maximum or minimum, and not every critical point is either. |
| Cross Product | An operation on two vectors in three dimensions producing a vector perpendicular to both, with magnitude equal to the area of the parallelogram they span. Defined only in three dimensions. |
D
| Term | Definition |
| Definite Integral | The signed area under a curve between two limits, the limit of Riemann sums. |
| Derivative | The instantaneous rate of change of a function, the limit of the difference quotient. Geometrically the slope of the tangent line. |
| Determinant | A scalar summarising how a square matrix scales volume, and whether it is invertible. A zero determinant means the transformation collapses space into a lower dimension, so information is destroyed and no inverse exists. |
| Diagonal Matrix | A matrix whose non-diagonal entries are all zero. Multiplication by one simply scales each coordinate independently, which is why diagonalising a problem simplifies it. |
| Diagonalisation | Writing a matrix as P D P inverse, with D diagonal and P holding the eigenvectors. It turns repeated matrix multiplication into repeated scalar powers, which is what makes matrix powers and exponentials tractable. |
| Differentiability | The existence of a derivative at a point. Differentiability implies continuity; continuity does not imply differentiability, as a corner shows. |
| Differential Equation | An equation relating a function to its derivatives, describing systems by how they change rather than by their state. |
| Dimension | The number of vectors in a basis of a space, the number of independent directions available. |
| Directional Derivative | The rate of change of a multivariable function along a specified direction, computed as the dot product of the gradient with a unit vector in that direction. |
| Divergence | For a vector field, the scalar measuring net outward flow at a point. |
| Domain | The set of inputs for which a function is defined. |
| Dot Product | The sum of elementwise products of two vectors, equal to the product of their magnitudes times the cosine of the angle between them. Zero exactly when the vectors are orthogonal, which is why it is the workhorse of similarity measures. |
E
| Term | Definition |
| Eigendecomposition | The factorisation of a matrix into its eigenvectors and eigenvalues. Only available when enough independent eigenvectors exist, which is why the singular value decomposition is the more general tool. |
| Eigenvalue | The scalar lambda for which Av equals lambda v, the factor by which the transformation stretches its eigenvector. |
| Eigenvector | A non-zero vector whose direction is unchanged by a linear transformation, only scaled. These directions are the natural axes of the transformation, which is why principal components are eigenvectors of a covariance matrix. |
| Elementary Row Operation | Swapping rows, scaling a row, or adding a multiple of one row to another. These preserve the solution set, which is what makes Gaussian elimination valid. |
| Euler’s Number | The constant e, approximately 2.71828, the base for which the exponential function equals its own derivative. That self-derivative property is why it appears throughout growth, decay and probability. |
| Extremum | A maximum or minimum of a function, either local or global. |
F
| Term | Definition |
| First Derivative Test | Classifying a critical point by the sign change of the first derivative around it: positive to negative gives a local maximum, negative to positive a local minimum. |
| Function | A rule assigning exactly one output to each input in the domain. |
| Fundamental Theorem of Calculus | The result linking the two branches: differentiation and integration are inverse operations, and a definite integral can be evaluated as the difference of an antiderivative at the endpoints. |
G
| Term | Definition |
| Gaussian Elimination | Reducing a matrix to row echelon form through elementary row operations, to solve linear systems, find rank, or compute an inverse. |
| Gradient | The vector of partial derivatives of a scalar multivariable function. It points in the direction of steepest increase, and its magnitude is the rate of that increase, which is the entire basis of gradient descent. |
| Gram-Schmidt | The procedure converting a set of independent vectors into an orthonormal set spanning the same space, by successively subtracting projections. Numerically unstable in its naive form, so implementations use a modified version. |
H
| Term | Definition |
| Hadamard Product | Elementwise multiplication of two matrices of the same shape. Distinct from matrix multiplication, and the operation most numerical libraries perform when you use the ordinary multiplication symbol on arrays. |
| Hessian | The matrix of second partial derivatives of a scalar function. Its definiteness classifies critical points, and its size is why second-order optimisation is impractical for large models. |
I
| Term | Definition |
| Identity Matrix | The square matrix with ones on the diagonal and zeros elsewhere, leaving any conformable vector or matrix unchanged. |
| Implicit Differentiation | Differentiating an equation that defines a relationship without solving for one variable, applying the chain rule to the dependent variable throughout. |
| Improper Integral | An integral with an infinite limit or an unbounded integrand, defined as a limit. It either converges to a finite value or diverges. |
| Indefinite Integral | The family of antiderivatives of a function, written with a constant of integration. |
| Inflection Point | A point where concavity changes, so the second derivative changes sign. |
| Inner Product | A generalisation of the dot product to abstract vector spaces, defining length and angle. |
| Integral | The accumulation of a quantity, geometrically an area or volume. |
| Integration by Parts | The technique derived from the product rule: the integral of u dv equals uv minus the integral of v du. The standard tool for products of unlike functions. |
| Intermediate Value Theorem | The result that a continuous function on an interval takes every value between its endpoint values. The justification behind bisection root-finding. |
| Inverse Matrix | The matrix A inverse satisfying A times A inverse equals the identity. Exists only for square matrices with non-zero determinant, and in numerical work you solve the system rather than forming the inverse explicitly. |
J
| Term | Definition |
| Jacobian | The matrix of first partial derivatives of a vector-valued function, describing its best linear approximation at a point. The multivariable generalisation of the derivative, and the object the chain rule composes in backpropagation. |
K
| Term | Definition |
| Kernel | In linear algebra, the null space of a matrix, the set of vectors mapped to zero. A non-trivial kernel means the transformation is not invertible. |
L
| Term | Definition |
| LU Decomposition | Factorising a matrix into lower and upper triangular factors, so systems can be solved efficiently by forward and back substitution. |
| L’Hôpital’s Rule | For an indeterminate limit of the form zero over zero or infinity over infinity, the limit of the ratio equals the limit of the ratio of derivatives. Check the form applies before using it, since it gives wrong answers otherwise. |
| Lagrange Multiplier | The method for optimising a function subject to constraints, by setting the gradient of the objective proportional to the gradient of the constraint. The multiplier itself measures how much the optimum would improve if the constraint were relaxed. |
| Least Squares Solution | The vector minimising the squared error when a system has no exact solution, obtained from the normal equations A transpose A x equals A transpose b. The geometric statement is that the residual is orthogonal to the column space. |
| Limit | The value a function approaches as its input approaches a point. The foundation both derivatives and integrals are defined on. |
| Linear Approximation | Approximating a function near a point by its tangent line or plane, the first-order Taylor expansion. |
| Linear Combination | A sum of vectors each scaled by a coefficient. |
| Linear Dependence | A set of vectors in which at least one is a linear combination of the others, so it contributes no new direction. |
| Linear Independence | A set of vectors in which none is a linear combination of the others. |
| Linear Transformation | A map preserving addition and scalar multiplication. Every such map on finite-dimensional spaces is a matrix multiplication once a basis is chosen, which is why matrices and transformations are used interchangeably. |
M
| Term | Definition |
| Maclaurin Series | A Taylor series centred at zero. |
| Matrix | A rectangular array of numbers representing a linear transformation or a dataset. |
| Matrix Multiplication | The operation composing two linear transformations, with entry i,j the dot product of row i and column j. Not commutative, and the inner dimensions must match, which is where most shape errors originate. |
| Mean Value Theorem | The result that a differentiable function on an interval has some interior point where the instantaneous rate of change equals the average rate of change over the interval. |
| Monotonic | Consistently non-decreasing or non-increasing, equivalently having a derivative that does not change sign. |
N
| Term | Definition |
| Newton’s Method | Root finding by repeatedly following the tangent line to its intercept. Converges very fast near a root and can diverge from a poor starting point. |
| Norm | A measure of a vector’s length. The L2 norm is the Euclidean length, the L1 norm the sum of absolute values, and the choice between them is exactly the difference between ridge and lasso regularisation. |
| Null Space | The set of vectors a matrix maps to zero. Its dimension is the nullity, and its size measures how much information the transformation destroys. |
| Numerical Integration | Approximating a definite integral by evaluating the function at sample points, through the trapezoidal rule, Simpson’s rule or Monte Carlo methods. |
O
| Term | Definition |
| Orthogonal | Perpendicular, meaning the inner product is zero. Orthogonal directions carry independent information, which is why so many decompositions aim for them. |
| Orthogonal Matrix | A square matrix whose columns are orthonormal, so its transpose is its inverse. It preserves lengths and angles, making it numerically well behaved. |
| Orthonormal | Mutually orthogonal and each of unit length. |
| Outer Product | The product of a column vector with a row vector, producing a matrix of rank one. |
P
| Term | Definition |
| Partial Derivative | The derivative of a multivariable function with respect to one variable, holding the others fixed. |
| Positive Definite | A symmetric matrix for which x transpose A x is positive for every non-zero x, equivalently all eigenvalues positive. It identifies a genuine minimum in optimisation and a valid covariance structure in statistics. |
| Power Rule | The derivative of x to the n is n times x to the n minus 1. |
| Product Rule | The derivative of a product: the derivative of f times g is f’ g plus f g’. |
| Projection | The component of one vector lying along another, or onto a subspace. Least squares regression is exactly the projection of the outcome onto the column space of the predictors. |
| Pseudoinverse | The Moore-Penrose generalisation of the inverse to non-square or singular matrices, giving the least squares solution. Computed from the singular value decomposition. |
Q
| Term | Definition |
| QR Decomposition | Factorising a matrix into an orthogonal factor Q and an upper triangular factor R. The numerically stable route to least squares, preferred over forming the normal equations directly. |
| Quotient Rule | The derivative of f over g is (f’ g minus f g’) over g squared. |
R
| Term | Definition |
| Rank | The number of linearly independent columns of a matrix, equivalently the dimension of its column space. Full rank means no redundancy among the columns, and rank deficiency is what makes a regression’s coefficients non-identifiable. |
| Rank-Nullity Theorem | The result that rank plus nullity equals the number of columns. Every dimension either survives the transformation or is collapsed into the null space. |
| Riemann Sum | The sum of areas of rectangles approximating the area under a curve, whose limit as widths go to zero defines the definite integral. |
| Row Echelon Form | The staircase form produced by Gaussian elimination, with leading entries moving right as you descend. Reduced row echelon form additionally has leading ones with zeros above and below. |
| Row Space | The span of a matrix’s rows. Its dimension equals the rank, the same as the column space, which is not obvious and is one of the central results of the subject. |
S
| Term | Definition |
| SVD | See Singular Value Decomposition. |
| Saddle Point | A critical point that is a maximum in one direction and a minimum in another. In high-dimensional optimisation these vastly outnumber local minima, which changed how the field thinks about training difficulty. |
| Scalar | A single number, as opposed to a vector or matrix. |
| Second Derivative Test | Classifying a critical point by the sign of the second derivative, or in several variables by the definiteness of the Hessian. Inconclusive when the second derivative is zero. |
| Series | The sum of the terms of a sequence, which converges or diverges. |
| Singular Matrix | A square matrix with zero determinant, therefore not invertible. |
| Singular Value Decomposition | The factorisation of any matrix into U Sigma V transpose, with orthogonal factors and non-negative singular values. It always exists, it reveals rank and the best low-rank approximation, and it underlies principal component analysis and the pseudoinverse. |
| Span | The set of all linear combinations of a collection of vectors. |
| Sparse Matrix | A matrix that is mostly zeros, stored and multiplied in formats that skip them. The difference between feasible and infeasible for text and graph data. |
| Spectral Theorem | The result that a real symmetric matrix has real eigenvalues and an orthonormal eigenvector basis. It is why covariance matrices always decompose cleanly, which is what makes principal component analysis work. |
| Squeeze Theorem | If a function is bounded between two others that converge to the same limit, it converges to that limit too. |
| Stationary Point | A point where the derivative or gradient is zero. |
| Subspace | A subset of a vector space that is itself a vector space, closed under addition and scalar multiplication and containing the zero vector. |
| Symmetric Matrix | A matrix equal to its own transpose. |
T
| Term | Definition |
| Tangent Line | The line touching a curve at a point with the same slope, the best linear approximation there. |
| Taylor Series | The representation of a function as an infinite sum of terms built from its derivatives at a point. Truncating it is where linear and quadratic approximations in optimisation come from. |
| Trace | The sum of the diagonal entries of a square matrix, equal to the sum of its eigenvalues. |
| Transpose | The matrix obtained by exchanging rows and columns. |
| Triangular Matrix | A matrix with all entries above or below the diagonal equal to zero. Systems involving one solve directly by substitution, which is the point of LU and QR factorisations. |
U
| Term | Definition |
| U-Substitution | Integration by change of variable, the reverse of the chain rule. |
| Unit Vector | A vector of length one, used to specify direction alone. |
V
| Term | Definition |
| Vector | An ordered list of numbers, interpretable as a point, a direction with magnitude, or a feature representation. |
| Vector Field | An assignment of a vector to every point in space, such as a gradient field. |
| Vector Space | A set closed under addition and scalar multiplication and obeying the vector space axioms. |
Z
| Term | Definition |
| Zero Vector | The vector with all components zero, the additive identity and a member of every subspace. |
[…] Glossary: calculus and linear algebra […]
[…] Glossary: calculus and linear algebra […]