mathlib — Optimization

Simplex LP, line search backtracking, PSO, L-BFGS-B. ← Back to hub

LP

1 Simplex (LP)

Minimize c′x s.t. Ax = b, x ≥ 0.

—
Line search

2 Backtracking

Minimize f(x) along direction d; Armijo backtracking chooses α.

—
f(α) curve (colored by α); vertical lines = trial steps; dot = accepted α.
PSO

3 Particle swarm optimization

Minimize cost function in 2D with PSO.

Cost surface (blue = low, red = high); white line = best-so-far trajectory; cross = final best.
—
Levenberg-Marquardt

4a Nonlinear least squares

Fit curve to data: minimize ||r(x)||². Example: fit y = a*x + b to points.

Data points and fitted line.
—
L-BFGS-B

5 L-BFGS-B (bound-constrained)

Minimize smooth function with gradient and box bounds [lower, upper].

Cost surface; cross = solution. Bounds shown as box when not [-5,5]².
—