mathcore::ml — Optimization and Autodiff Module

The mathcore::ml module provides symbolic and numerical optimization utilities through the Optimization struct. It builds on mathcore::calculus::Calculus for symbolic differentiation, enabling exact gradient and Hessian computation, gradient-descent parameter optimization, forward-mode automatic differentiation, Taylor series expansion, and Jacobian matrix evaluation at a point. All methods are free associated functions on the unit struct Optimization.

`pub struct Optimization`

pub struct Optimization;

A zero-sized namespace struct. Import it with:

use mathcore::ml::Optimization;

`gradient`

Computes the gradient of a scalar-valued expression with respect to each variable in vars, returning a vector of partial-derivative expressions in the same order as the input variable slice. Each partial derivative is obtained by calling Calculus::differentiate(expr, var).

pub fn gradient(expr: &Expr, vars: &[String]) -> Result<Vec<Expr>, MathError>

expr

&Expr

required

The scalar expression to differentiate. Typically a loss function or potential energy that depends on one or more of the listed variables.

vars

&[String]

required

Ordered list of variable names. The output gradient vector has the same length and ordering: result[i] is ∂expr/∂vars[i].

Returns Ok(Vec<Expr>) of length vars.len(), or Err(MathError) if any partial differentiation fails.

use mathcore::ml::Optimization;
use mathcore::parser::Parser;

// ∇(x^2 + y^2) = [2x, 2y]
let expr = Parser::parse("x^2 + y^2").unwrap();
let vars = vec!["x".to_string(), "y".to_string()];
let grad = Optimization::gradient(&expr, &vars).unwrap();

println!("∂/∂x = {}", grad[0]); // 2 * x
println!("∂/∂y = {}", grad[1]); // 2 * y

`hessian`

Computes the Hessian matrix of second-order partial derivatives for expr. The result is an n × n matrix (as a Vec<Vec<Expr>>) where entry [i][j] equals ∂²expr / (∂vars[i] ∂vars[j]).

pub fn hessian(expr: &Expr, vars: &[String]) -> Result<Vec<Vec<Expr>>, MathError>

expr

&Expr

required

The scalar-valued expression to differentiate twice.

vars

&[String]

required

Variable names. The Hessian is len(vars) × len(vars).

Returns Ok(Vec<Vec<Expr>>) where the outer vector indexes rows and the inner vector indexes columns.

use mathcore::ml::Optimization;
use mathcore::parser::Parser;

// H(x^2 + 2*x*y + y^2) = [[2, 2], [2, 2]]
let expr = Parser::parse("x^2 + 2*x*y + y^2").unwrap();
let vars = vec!["x".to_string(), "y".to_string()];
let h = Optimization::hessian(&expr, &vars).unwrap();

println!("H[0][0] = {}", h[0][0]); // ∂²/∂x² = 2
println!("H[0][1] = {}", h[0][1]); // ∂²/∂x∂y = 2

`gradient_descent`

Minimises loss_fn using vanilla (full-batch) gradient descent. Gradients are computed once symbolically before the loop, then numerically evaluated at the current parameter values on each iteration.

params ← params − learning_rate × ∇loss_fn(params)

pub fn gradient_descent(
    loss_fn: &Expr,
    initial_params: HashMap<String, f64>,
    learning_rate: f64,
    iterations: usize,
) -> Result<HashMap<String, f64>, MathError>

loss_fn

&Expr

required

The scalar loss expression to minimise. Must evaluate to a numeric value at every parameter point visited during descent.

initial_params

HashMap<String, f64>

required

Starting values for every variable that appears in loss_fn. The keys of this map also define which variables are treated as learnable parameters.

learning_rate

f64

required

Step size multiplier applied to the gradient at each iteration. Typical values are in the range 0.001 – 0.1.

iterations

usize

required

Number of gradient-descent steps to perform. There is no early-stopping criterion; the method always runs for the full iteration count.

Returns Ok(HashMap<String, f64>) with the parameter values after all iterations, or Err(MathError) if any gradient evaluation produces a non-numeric result.

use mathcore::ml::Optimization;
use mathcore::parser::Parser;
use std::collections::HashMap;

// Minimise (x - 3)^2  →  converges toward x = 3
let loss = Parser::parse("(x - 3)^2").unwrap();
let mut init = HashMap::new();
init.insert("x".to_string(), 0.0);

let result = Optimization::gradient_descent(&loss, init, 0.1, 100).unwrap();
println!("x* ≈ {:.4}", result["x"]); // ≈ 3.0

`automatic_differentiation`

Evaluates both the function value and its derivative at a specific numeric point using symbolic differentiation followed by numeric evaluation. This is equivalent to forward-mode automatic differentiation for single-variable expressions.

pub fn automatic_differentiation(
    expr: &Expr,
    var: &str,
    value: f64,
) -> Result<(f64, f64), MathError>

expr

&Expr

required

The expression to evaluate and differentiate.

var

&str

required

The variable with respect to which the derivative is computed.

value

f64

required

The numeric value at which both expr and its derivative are evaluated.

Returns Ok((f, f')) where f is the function value and f' is the derivative value at value, or Err(MathError) if either evaluation produces a non-numeric result.

use mathcore::ml::Optimization;
use mathcore::parser::Parser;

// f(x) = sin(x) at x = π/4
let expr = Parser::parse("sin(x)").unwrap();
let pi_over_4 = std::f64::consts::PI / 4.0;
let (f_val, df_val) = Optimization::automatic_differentiation(&expr, "x", pi_over_4).unwrap();

println!("f(π/4)  = {:.6}", f_val);   // ≈ 0.707107
println!("f'(π/4) = {:.6}", df_val);  // ≈ 0.707107  (cos(π/4))

`taylor_series`

Computes the Taylor series expansion of expr about center up to and including terms of degree order. Coefficients are obtained by repeated symbolic differentiation evaluated numerically at the center point:

T(x) = Σ_{n=0}^{order}  [f^(n)(center) / n!] * (x - center)^n

Terms with a zero coefficient are still included in the sum (as 0.0 * (x − center)^n) but do not affect evaluation.

pub fn taylor_series(
    expr: &Expr,
    var: &str,
    center: f64,
    order: usize,
) -> Result<Expr, MathError>

expr

&Expr

required

The function to expand.

var

&str

required

The expansion variable.

center

f64

required

The point about which the Taylor expansion is computed. Use 0.0 for a Maclaurin series.

order

usize

required

The highest power to include. An order of 5 yields terms up to (x − center)^5.

Returns Ok(Expr) with the Taylor polynomial as a symbolic expression, or Err(MathError) if differentiation or evaluation fails.

use mathcore::ml::Optimization;
use mathcore::parser::Parser;

// Taylor series of sin(x) about 0 up to order 5
// ≈ x - x^3/6 + x^5/120
let expr = Parser::parse("sin(x)").unwrap();
let taylor = Optimization::taylor_series(&expr, "x", 0.0, 5).unwrap();
println!("{}", taylor);

`jacobian`

Computes the Jacobian matrix of a vector-valued function at a specific numeric point. Entry J[i][j] equals ∂functions[i] / ∂vars[j] evaluated at point.

pub fn jacobian(
    functions: &[Expr],
    vars: &[String],
    point: &HashMap<String, f64>,
) -> Result<DMatrix<f64>, MathError>

functions

&[Expr]

required

The component functions f₁, f₂, ..., fₘ. The Jacobian has m rows, one per function.

vars

&[String]

required

The variables x₁, x₂, ..., xₙ with respect to which each partial derivative is taken. The Jacobian has n columns.

point

&HashMap<String, f64>

required

The numeric point at which all partial derivatives are evaluated. Must contain an entry for every variable in vars.

Returns Ok(DMatrix<f64>) of shape m × n, or Err(MathError) if any partial derivative evaluates to a non-numeric result.

use mathcore::ml::Optimization;
use mathcore::parser::Parser;
use std::collections::HashMap;

// J of [x^2 + y, x*y] at (x=1, y=2)
// Row 0: [2x, 1]  → [2, 1]
// Row 1: [y,  x]  → [2, 1]
let f1 = Parser::parse("x^2 + y").unwrap();
let f2 = Parser::parse("x*y").unwrap();

let vars = vec!["x".to_string(), "y".to_string()];
let mut point = HashMap::new();
point.insert("x".to_string(), 1.0);
point.insert("y".to_string(), 2.0);

let j = Optimization::jacobian(&[f1, f2], &vars, &point).unwrap();
println!("J =\n{}", j);

`optimize_newton`

Finds a local minimum (or maximum) of func with respect to var using Newton’s method for optimisation. At each iteration the current point is updated by x ← x - f'(x) / f''(x), converging to a critical point where f'(x) = 0. Both the first and second derivatives are computed symbolically.

pub fn optimize_newton(
    func: &Expr,
    var: &str,
    initial: f64,
    tolerance: f64,
    max_iterations: usize,
) -> Result<f64, MathError>

func

&Expr

required

The scalar function to optimise. Must be differentiable at least twice with respect to var.

var

&str

required

The optimisation variable.

initial

f64

required

The starting point for Newton iteration.

tolerance

f64

required

Convergence threshold on |f'(x)|. Iteration stops when the first derivative falls below this value.

max_iterations

usize

required

Maximum number of Newton steps before returning the current estimate.

Returns Ok(f64) with the critical point found, or Err(MathError) if the second derivative is too small to divide by or if evaluation produces a non-numeric result.

use mathcore::ml::Optimization;
use mathcore::parser::Parser;

// Find the minimum of (x - 2)^2  →  x* = 2
let func = Parser::parse("(x - 2)^2").unwrap();
let x_star = Optimization::optimize_newton(&func, "x", 0.0, 1e-10, 100).unwrap();
println!("x* = {:.6}", x_star); // ≈ 2.0

`lagrange_multipliers`

Constructs the Lagrangian for constrained optimisation and returns its gradient with respect to all original variables plus one Lagrange multiplier per constraint. Given an objective f and constraints g₁, g₂, ..., gₘ, the Lagrangian is:

L = f + λ₁·g₁ + λ₂·g₂ + ... + λₘ·gₘ

The gradient components ∂L/∂xᵢ and ∂L/∂λⱼ form the KKT stationarity conditions. Each λⱼ variable is named λ0, λ1, etc.

pub fn lagrange_multipliers(
    objective: &Expr,
    constraints: &[Expr],
    vars: &[String],
) -> Result<Vec<Expr>, MathError>

objective

&Expr

required

The objective function f(x₁, ..., xₙ) to optimise.

constraints

&[Expr]

required

Equality constraints written in the form g(x) = 0. Each constraint contributes one Lagrange multiplier to the Lagrangian.

vars

&[String]

required

The original decision variable names. The returned gradient has vars.len() + constraints.len() components.

Returns Ok(Vec<Expr>) containing the partial derivatives of the Lagrangian with respect to each variable and each multiplier, or Err(MathError) if differentiation fails.

use mathcore::ml::Optimization;
use mathcore::parser::Parser;

// Optimise f = x + y  subject to  g = x^2 + y^2 - 1 = 0
let objective = Parser::parse("x + y").unwrap();
let constraint = Parser::parse("x^2 + y^2 - 1").unwrap();
let vars = vec!["x".to_string(), "y".to_string()];

let kkt = Optimization::lagrange_multipliers(
    &objective,
    &[constraint],
    &vars,
).unwrap();

// kkt[0] = ∂L/∂x, kkt[1] = ∂L/∂y, kkt[2] = ∂L/∂λ0
println!("∂L/∂x  = {}", kkt[0]);
println!("∂L/∂y  = {}", kkt[1]);
println!("∂L/∂λ0 = {}", kkt[2]);

Core API

Modules

mathcore::ml — Optimization and Autodiff Module

`pub struct Optimization`

`gradient`

`hessian`

`gradient_descent`

`automatic_differentiation`

`taylor_series`

`jacobian`

`optimize_newton`

`lagrange_multipliers`

Build docs developers (and LLMs) love

Core API

Modules

Documentation Index

​pub struct Optimization

​gradient

​hessian

​gradient_descent

​automatic_differentiation

​taylor_series

​jacobian

​optimize_newton

​lagrange_multipliers

Build docs developers (and LLMs) love

`pub struct Optimization`

`gradient`

`hessian`

`gradient_descent`

`automatic_differentiation`

`taylor_series`

`jacobian`

`optimize_newton`

`lagrange_multipliers`