Shunyaya Symbolic Mathematics — Logistic/sigmoid, softplus, softsign (2.43)

Abstract
We lift three common smooth nonlinearities—logistic/sigmoid, softplus, and softsign—to symbolic numerals (m, a) as unary magnitude maps with alignment carry. We state domains, ranges, monotonicity/curvature, stable numerics, and order effects. Under collapse phi(m,a) = m, all formulas reduce to the classical ones.

Definitions (alignment carry)

For x = (m, a):

sigmoid*(m, a)  = ( 1 / (1 + exp(-m)) , a )         # also called logistic
softplus*(m, a) = ( log(1 + exp(m))   , a )
softsign*(m, a) = ( m / (1 + |m|)     , a )
  • Domain: all real m.
  • Alignment: carried unchanged (a passes through).
  • Collapse: phi( f*(m,a) ) = f(m) for each f.

Ranges, monotonicity, curvature

Logistic / sigmoid σ(m)

Range: (0, 1),  strictly increasing on R
σ'(m)  = σ(m) * (1 - σ(m))
σ''(m) = σ(m) * (1 - σ(m)) * (1 - 2σ(m))
  • Curvature: convex on (-inf, 0), concave on (0, inf), inflection at m = 0 (σ(0)=1/2).

Softplus sp(m) = log(1 + exp(m))

Range: (0, inf),  strictly increasing on R
sp'(m)  = σ(m)
sp''(m) = σ(m) * (1 - σ(m))  > 0      # globally convex
  • Asymptotics: sp(m) ~ m for m >> 0; sp(m) ~ exp(m) for m << 0.

Softsign ss(m) = m/(1+|m|)

Range: (-1, 1),  strictly increasing on R
For m >= 0: ss'(m) = 1/(1+m)^2,  ss''(m) = -2/(1+m)^3   (concave)
For m < 0 : ss'(m) = 1/(1-|m|)^2 = 1/(1-m)^2,  ss''(m) =  2/(1-m)^3   (convex)
  • Curvature: convex on (-inf, 0), concave on (0, inf), inflection at m = 0.

Order and collapse

  • Magnitude order: all three are strictly increasing on R, so (m1,a1) <=_m (m2,a2) ⇒ f*(m1,a1) <=_m f*(m2,a2) for f in {sigmoid, softplus, softsign}.
  • Strength order S_beta (optional): with equal alignments (a1=a2) the same preservation holds; with differing a the strength order may differ from magnitude order (see 2.37–2.38).
  • Collapse check: setting a = +1 everywhere returns the classical curves exactly.

Stable numerics (recommended formulas)

To avoid overflow/underflow:

sigmoid(m) = 0.5 * (1 + tanh(m/2))             # stable on R
softplus(m) = { m + log1p(exp(-m))   if m > 0
              { log1p(exp(m))        otherwise
erfc-style guards not needed here; use expm1/log1p where available

Manifest knobs:

sigmoid.impl   = "tanh_half" | "1_over_1p_exp"
softplus.split = m_threshold (default 0)
precision_targets = {abs_tol: 1e-12, rel_tol: 1e-12}

Worked examples

A) Logistic and curvature point

x1 = (-2, 0.3)  → sigmoid*(x1) ≈ ( 0.1192 , 0.3 )
x2 = (  2, 0.6)  → sigmoid*(x2) ≈ ( 0.8808 , 0.6 )
x0 = (  0, 0.1)  → sigmoid*(x0) = ( 0.5    , 0.1 )

Order preserved on magnitudes; alignment carried.

B) Softplus with split implementation

y1 = (-3, 0.4)  → softplus*(y1) = ( log1p(exp(-3)) , 0.4 ) ≈ ( 0.0486 , 0.4 )
y2 = (  2, 0.9)  → softplus*(y2) = ( 2 + log1p(exp(-2)) , 0.9 ) ≈ ( 2.1269 , 0.9 )

C) Softsign saturation

z1 = (-3, -0.2) → softsign*(z1) = ( -3 / (1+3) , -0.2 ) = ( -0.75 , -0.2 )
z2 = ( 10,  0.2) → softsign*(z2) = ( 10 / 11    ,  0.2 ) ≈ ( 0.9091 , 0.2 )

Softsign smoothly saturates toward ±1 while carrying alignment.

Use with selectors and thresholds

  • Smooth step vs Heaviside: sigmoid(k*m) approximates H(m) as k -> +inf. Pair with 2.35 policies if replacing a hard threshold with a smooth one; alignment is carried, so thresholding decisions remain audit-ready.
  • Clamp alternatives: softplus is a smooth max(m,0); softsign is a smooth tanh-like squashing into (-1,1) without exponentials.

Implementation notes (manifest)

function_set = ["sigmoid","softplus","softsign"]
alignment_policy = "carry"           # unary
sigmoid.impl = "tanh_half"           # or "1_over_1p_exp"
softplus.split_threshold = 0         # use piecewise formula shown above
domain = R
order_metric = "magnitude"           # for order claims; S_beta optional with declared beta
zero_canonicalize = true             # if result magnitude is exactly 0, return (0, +1)

Takeaway

Logistic, softplus, and softsign lift cleanly as increasing magnitude maps with alignment carry. They provide smooth alternatives to steps and clamps, preserve magnitude order, and admit stable implementations across R. Under collapse (a = +1), they match the classical curves exactly.

Navigation
Previous → Special functions II — erf, Bessel, polylog, Lambert W (2.42)
Next → Multivariate lifts & aggregators (2.44)

Disclaimer
Observation only. Results reproduce mathematically; domain claims require independent peer review. Defaults: mult_mode = M2, clamp_eps = 1e-6, |a| < 1. All formulas are presented in plain text. Collapse uses phi(m,a) = m.