A hands-on guide to TorchCurves, the PyTorch library that makes B-spline and Legendre curves trainable layers. You will install it, fit a curve, choose a basis and knot count, build a monotone calibration curve and an additive model on tabular data, compare them with scikit-learn, SciPy, a KAN and an MLP, export the coefficients so they can be audited without PyTorch, and test the lot. Every output below comes from our own run.
torchcurves (0.3.4, Apache-2.0, Python 3.12+, PyTorch 2.9+) gives you BSplineCurve, LegendreCurve, BSplineBasis and Sum. Inputs of shape (batch, curves) become points of shape (batch, curves, dim), with gradients to both inputs and coefficients. Use B-splines with 8 to 12 coefficients and a second-difference penalty as your default; enforce monotone shapes by construction; standardise inputs and pick the input map deliberately; export knots, coefficients and the map to JSON and check them against SciPy. For a single standalone curve SciPy is faster and fits better; TorchCurves earns its place when the curve lives inside a PyTorch model.
This technote accompanies our blog post Differentiable Curves in PyTorch: Small, Auditable Models with TorchCurves, which covers the why for a wider audience. API details are from the torchcurves source at version 0.3.4 and the paper by its author, Alex Shtoff [1][2]. Facts are as of 16 November 2026.
Who this is for, what you will build
Who. Python developers and data scientists who know basic PyTorch (modules, optimisers, backward()) and want models whose per-input behaviour can be plotted, constrained and reviewed. No spline theory is assumed; we explain what you need as we go.
What you will build, in six short scripts and one test file:
- A hello-world that shows the tensor shapes.
- A single fitted curve, comparing knot counts, B-spline against Legendre, and SciPy's smoothing spline.
- A monotone probability-calibration curve, against an unconstrained spline and isotonic regression.
- An additive model (one curve per feature) and a small KAN on tabular data, against linear regression, a scikit-learn spline GAM, gradient boosting and an MLP.
- An export of the trained curves to JSON, re-evaluated in SciPy, plus a
torch.exportprogram for serving. - A pytest file that checks shapes, gradients, SciPy agreement and monotonicity.
Prerequisites. Python 3.12 or newer, about 1 GB of disk for PyTorch, and any recent laptop. Everything here runs on CPU in seconds to a few minutes. We used Python 3.13, PyTorch 2.15.0, torchcurves 0.3.4, SciPy 1.18.1, scikit-learn 1.9.1 and NumPy 2.5.4 on an Apple-silicon Mac.
1. Install
Use a fresh virtual environment. On Linux, install the CPU-only PyTorch wheel first unless you need CUDA; it is several times smaller. On macOS the default wheel is already CPU (with Apple's MPS backend).
python3 -m venv .venv && source .venv/bin/activate
# Linux, CPU only (skip on macOS, or use the CUDA index for GPUs)
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install torchcurves scipy scikit-learn pytest
python -c "import torch, torchcurves; print(torch.__version__, torchcurves.__version__)"
torchcurves has one dependency, torch>=2.9.0, and declares requires_python >= 3.12 [3]. If pip says no matching distribution, check your Python version first.
2. Hello, curve: the shapes you must remember
Every curve module takes a 2-D input of shape (B, C): B samples, one column per curve. It returns (B, C, D): one D-dimensional point per sample per curve. The curves' coefficients live in control_points with shape (C, K, D), where K is the number of coefficients per curve.
# s1_hello.py
import torch
import torchcurves as tc
torch.manual_seed(0)
u = torch.rand(4, 3) # (batch B, curves C)
curve = tc.BSplineCurve(num_curves=3, dim=2, knots_config=8)
y = curve(u) # (B, C, D)
print(u.shape, "->", y.shape)
print(curve)
print("control_points:", tuple(curve.control_points.shape))
print("knots:", [round(k, 3) for k in curve.knots.tolist()])
y.sum().backward()
print("grad on control points:", curve.control_points.grad is not None)
Output:
torch.Size([4, 3]) -> torch.Size([4, 3, 2])
BSplineCurve(num_curves=3, n_control_points_per_curve=8, dim=2, degree=3, knots_shape=torch.Size([12]))
control_points: (3, 8, 2)
knots: [-1.0, -1.0, -1.0, -1.0, -0.6, -0.2, 0.2, 0.6, 1.0, 1.0, 1.0, 1.0]
grad on control points: True
Three things to notice. knots_config=8 means eight control points (coefficients) per curve, not eight knots: a cubic spline with eight coefficients has 12 knots, with four repeated at each end so the curve is well behaved at the boundaries. The knots span -1 to 1 by default. And gradients flow to the coefficients through an ordinary backward() call, because the library implements a custom autograd function for B-splines [2].
If you have seen BSplineCurve(num_control_points=8, degree=3, dim=1) in a blog post (including an earlier version of ours), it is wrong for every published version: there is no num_control_points argument and num_curves is required. The real signature is BSplineCurve(num_curves, dim, degree=3, knots_config=10, parameter_range=None, input_map="real.rational"). LegendreCurve(num_curves, dim, degree, input_map="real.rational", checkpoint_segments=None) takes a degree instead of knots; note that it stores its coefficients as (K, C, D), with the coefficient axis first.
3. Fit one curve, and choose a basis and knot count
The first real job: fit a noisy 1-D signal with a smooth trend and a sharp change of slope at x = 0.7. We try B-splines with 6 to 48 coefficients, one heavily over-sized spline with a roughness penalty, Legendre polynomials of matching size, and SciPy's make_smoothing_spline, which picks its own smoothing level by generalised cross-validation [9].
# s2_fit.py
import numpy as np
import torch
import torchcurves as tc
from scipy.interpolate import make_smoothing_spline
torch.manual_seed(0)
rng = np.random.default_rng(0)
def truth(x):
# smooth trend plus a sharp change at x = 0.7
return np.sin(2 * np.pi * x) + 1.5 * (x > 0.7) * (x - 0.7) * 10
x = rng.uniform(0, 1, 600)
y = truth(x) + rng.normal(0, 0.2, x.size)
idx = rng.permutation(x.size)
tr, va = idx[:400], idx[400:]
grid = np.linspace(0, 1, 501)
def to_t(a):
return torch.as_tensor(a, dtype=torch.float32).reshape(-1, 1) # (B, 1 curve)
def make_curve(kind, size):
if kind == "bspline":
# knots spread uniformly over [0, 1]; "real.clamp" passes x through unchanged
return tc.BSplineCurve(1, 1, knots_config=size, parameter_range=(0, 1), input_map="real.clamp")
# Legendre lives on [-1, 1]; clamp(2x - 1) maps [0, 1] onto it
return tc.LegendreCurve(1, 1, degree=size, input_map="real.clamp")
def prep(kind, x):
return to_t(x) if kind == "bspline" else to_t(2 * x - 1)
def second_diff(c):
# roughness penalty on neighbouring coefficients (P-spline style)
return (c[:, 2:] - 2 * c[:, 1:-1] + c[:, :-2]).pow(2).sum()
def fit(kind, size, lam=0.0, steps=1500):
curve = make_curve(kind, size)
opt = torch.optim.Adam(curve.parameters(), lr=0.05)
xt, yt = prep(kind, x[tr]), to_t(y[tr])
for _ in range(steps):
opt.zero_grad()
pred = curve(xt).squeeze(-1)
loss = (pred - yt).pow(2).mean()
if kind == "bspline" and lam > 0:
loss = loss + lam * second_diff(curve.control_points)
loss.backward()
opt.step()
with torch.no_grad():
va_rmse = (curve(prep(kind, x[va])).squeeze(-1) - to_t(y[va])).pow(2).mean().sqrt().item()
g = curve(prep(kind, grid)).squeeze().numpy()
true_rmse = float(np.sqrt(np.mean((g - truth(grid)) ** 2)))
return curve, va_rmse, true_rmse
print(f"{'basis':9} {'size':>4} {'lambda':>7} {'params':>6} {'val RMSE':>9} {'vs truth':>9}")
for kind, size, lam in [("bspline", 6, 0), ("bspline", 12, 0), ("bspline", 24, 0),
("bspline", 48, 0), ("bspline", 48, 1e-3),
("legendre", 5, 0), ("legendre", 11, 0), ("legendre", 23, 0)]:
curve, va_rmse, true_rmse = fit(kind, size, lam)
n = sum(p.numel() for p in curve.parameters())
print(f"{kind:9} {size:>4} {lam:>7g} {n:>6} {va_rmse:>9.3f} {true_rmse:>9.3f}")
o = np.argsort(x[tr])
sp = make_smoothing_spline(x[tr][o], y[tr][o]) # GCV picks the smoothing level
print(f"{'scipy':9} {'':>4} {'GCV':>7} {'':>6} {np.sqrt(np.mean((sp(x[va]) - y[va])**2)):>9.3f} "
f"{np.sqrt(np.mean((sp(grid) - truth(grid))**2)):>9.3f}")
print("noise std (floor for val RMSE): 0.200")
Output (val RMSE is error against held-out noisy points, so 0.200 is the floor; "vs truth" is error against the noise-free function):
basis size lambda params val RMSE vs truth
bspline 6 0 6 0.226 0.110
bspline 12 0 12 0.207 0.046
bspline 24 0 24 0.209 0.044
bspline 48 0 48 0.218 0.071
bspline 48 0.001 48 0.208 0.044
legendre 5 0 6 0.228 0.110
legendre 11 0 12 0.211 0.047
legendre 23 0 24 0.209 0.047
scipy GCV 0.206 0.027
noise std (floor for val RMSE): 0.200
How to read it:
- Too few coefficients underfit (6 coefficients, error against truth 0.110). Twelve to 24 is the sweet spot here.
- Too many overfit (48 coefficients, 0.071), and a small penalty on second differences of neighbouring coefficients fixes it (48 with penalty, 0.044). This is the P-spline recipe of Eilers and Marx: use generous knots and let a penalty control smoothness [4]. It makes the knot count much less critical.
- B-spline and Legendre are close at equal size on this signal. The difference shows up in behaviour, not the score: a B-spline coefficient moves the curve locally, a Legendre coefficient moves it everywhere (figure below), so B-splines handle kinks and constraints better and Legendre series are smoother and more compact for genuinely smooth functions.
- SciPy wins as a standalone smoother (0.027). It solves the penalised least-squares problem exactly and picks the penalty automatically; gradient descent with a fixed learning rate gets close but not all the way. If one curve is your whole model, use SciPy. Use TorchCurves when the curve is a layer in something bigger.
Rules of thumb that held up in our runs: start with cubic B-splines (the default degree=3) and 8 to 12 coefficients per input; if you go above 20, add a roughness penalty; choose the final size on validation data. Reach for Legendre when the function is very smooth and you want the fewest parameters, and for large degrees use checkpoint_segments to trade compute for memory in the backward pass [2].
4. Input maps: where your knots actually land
Curves live on a bounded interval (by default -1 to 1), but features do not. Every module therefore applies an input map first. The built-in options are "real.rational" (the module default in 0.3.4, x / sqrt(s² + x²)), "real.arctan", "real.clamp" (clip x / s to the interval), and "nonneg.rational" and "nonneg.arctan" for non-negative inputs. For a custom scale use objects such as tc.maps.Real.clamp(scale=3.0), or pass any callable f(x, out_min, out_max) [2].
Practical guidance:
- Standardise first, then choose. For roughly normal features,
tc.maps.Real.clamp(scale=3.0)gives even knots over ±3 standard deviations and a flat extrapolation beyond, which is easy to explain. - Use the rational or arctan maps for heavy tails, where you want some resolution far out and accept that it is coarse.
- For inputs already in [0, 1], set
parameter_range=(0, 1)with"real.clamp"so the knots sit exactly on your data range, as in the scripts above. - The 0.3.0 release replaced the old
normalize_fnandnormalization_scalearguments withinput_map[2]. Code written for 0.2.x will fail.
5. A monotone calibration curve
A common production need: turn a raw model score into a probability that never decreases as the score rises. B-splines make this easy, because a spline whose coefficients are increasing is itself increasing [2]. We parameterise the coefficients as a first value plus a cumulative sum of positive steps (softplus keeps them positive), and pass them to a BSplineBasis. The comparison is an unconstrained spline of the same size and scikit-learn's isotonic regression.
# s3_monotone.py
import numpy as np
import torch
import torch.nn.functional as F
from torch import nn
import torchcurves as tc
from sklearn.isotonic import IsotonicRegression
from sklearn.metrics import log_loss
torch.manual_seed(0)
rng = np.random.default_rng(1)
# Synthetic calibration data: a raw score in [0, 1] and a 0/1 outcome whose
# true probability rises with the score (slowly, then steeply).
n = 3000
score = rng.beta(2, 2, n)
p_true = 1 / (1 + np.exp(-(8 * score**2 - 4)))
y = (rng.uniform(size=n) < p_true).astype(np.float32)
tr, te = np.arange(0, 2000), np.arange(2000, n)
class MonotoneCurve(nn.Module):
"""A 1-D cubic B-spline whose coefficients are forced to increase,
so the curve itself is non-decreasing (a property of B-splines)."""
def __init__(self, n_coef=10):
super().__init__()
self.basis = tc.BSplineBasis(degree=3, knots_config=n_coef,
parameter_range=(0, 1), input_map="real.clamp")
self.raw = nn.Parameter(torch.zeros(n_coef))
def coefficients(self):
first, rest = self.raw[:1], F.softplus(self.raw[1:]) # increments > 0
return torch.cumsum(torch.cat([first, rest]), 0)
def forward(self, x): # x: (B,)
c = self.coefficients().view(1, -1, 1) # (1 curve, C, D=1)
return self.basis(x.view(-1, 1), c).view(-1) # logits, (B,)
class FreeCurve(nn.Module):
"""The same spline with unconstrained coefficients."""
def __init__(self, n_coef=10):
super().__init__()
self.curve = tc.BSplineCurve(1, 1, knots_config=n_coef,
parameter_range=(0, 1), input_map="real.clamp")
def coefficients(self):
return self.curve.control_points.view(-1)
def forward(self, x):
return self.curve(x.view(-1, 1)).view(-1)
def roughness(c):
return (c[2:] - 2 * c[1:-1] + c[:-2]).pow(2).sum()
def train(model, steps=2000, lam=0.0):
opt = torch.optim.Adam(model.parameters(), lr=0.05)
xt = torch.as_tensor(score[tr], dtype=torch.float32)
yt = torch.as_tensor(y[tr])
for _ in range(steps):
opt.zero_grad()
loss = F.binary_cross_entropy_with_logits(model(xt), yt)
if lam > 0:
loss = loss + lam * roughness(model.coefficients())
loss.backward()
opt.step()
return model
grid = torch.linspace(0, 1, 1001)
for name, model, lam in [("free B-spline", FreeCurve(20), 0.0),
("monotone B-spline", MonotoneCurve(20), 0.0),
("monotone + smooth (1e-3)", MonotoneCurve(20), 1e-3)]:
train(model, lam=lam)
with torch.no_grad():
p_te = torch.sigmoid(model(torch.as_tensor(score[te], dtype=torch.float32))).numpy()
g = torch.sigmoid(model(grid)).numpy()
drops = int((np.diff(g) < -1e-6).sum())
print(f"{name:28} test log-loss {log_loss(y[te], p_te):.4f} grid points where curve falls: {drops}")
iso = IsotonicRegression(out_of_bounds="clip", y_min=1e-3, y_max=1 - 1e-3).fit(score[tr], y[tr])
print(f"{'isotonic regression (sklearn)':28} test log-loss {log_loss(y[te], iso.predict(score[te])):.4f}"
f" distinct output levels: {len(np.unique(iso.predict(grid.numpy())))}")
print(f"{'true probability':28} test log-loss {log_loss(y[te], p_true[te]):.4f}")
for lam in (0.0, 1e-3):
torch.manual_seed(0)
m = train(MonotoneCurve(20), lam=lam)
c = [round(float(v), 2) for v in m.coefficients().detach()]
print(f"lambda={lam:g} last 4 coefficients: {c[-4:]}")
Output:
free B-spline test log-loss 0.3318 grid points where curve falls: 140
monotone B-spline test log-loss 0.3295 grid points where curve falls: 0
monotone + smooth (1e-3) test log-loss 0.3296 grid points where curve falls: 0
isotonic regression (sklearn) test log-loss 0.3311 distinct output levels: 37
true probability test log-loss 0.3268
lambda=0 last 4 coefficients: [2.63, 2.89, 3.61, 32.61]
lambda=0.001 last 4 coefficients: [2.14, 3.1, 4.09, 5.09]
The unconstrained spline wiggles: it falls in 140 of the 1,000 grid steps, which you cannot ship as a price or risk curve. The monotone one never falls, by construction, and is also slightly better on held-out data. Isotonic regression is monotone too, but it is a step function with 37 levels here; the spline is smooth and has 20 numbers you can print.
The last two lines show a pitfall worth knowing. Without a penalty, the final coefficient drifted to 32.6 on the logit scale: few training points sit near a score of 1, and nothing stops the edge coefficient running away. A roughness penalty of 0.001 brought it to 5.1 with no loss of accuracy. Always inspect the edge coefficients.
The library's README and example notebook use the same trick in a richer form: an encoder network maps auction features to increasing coefficients, and each sample gets its own curve of win probability against bid [2]. That is the third pattern in the tensor-shapes figure.
6. An additive model and a KAN on tabular data
Now a tabular problem. We generate five standardised features with known effects (a saturating tanh, a U-shape, a step, a wave, and one feature with no effect) plus a hidden interaction between the first two, and Gaussian noise with standard deviation 0.3. The additive model is one BSplineCurve with five curves and 12 coefficients each, followed by tc.Sum: prediction = bias + f0(x0) + ... + f4(x4). This is a generalised additive model (GAM), trained by gradient descent; the neural additive model of Agarwal et al. is the same idea with small networks instead of splines [7].
The KAN stacks two curve layers with a sum after each, as in the library README; Kolmogorov-Arnold networks put a learnable 1-D function on every edge of the network [5].
# s4_additive.py
import numpy as np
import torch
from torch import nn
import torchcurves as tc
from sklearn.linear_model import LinearRegression, RidgeCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import SplineTransformer, StandardScaler
from sklearn.ensemble import HistGradientBoostingRegressor
torch.manual_seed(0)
rng = np.random.default_rng(0)
# --- synthetic tabular data with known shape functions -----------------
n, F_ = 6000, 5
X = rng.normal(size=(n, F_))
shapes = [
lambda v: np.tanh(2 * v), # saturating
lambda v: 0.5 * v**2, # U-shape
lambda v: np.where(v > 0.5, 1.0, 0.0), # threshold
lambda v: np.sin(2 * v), # wave
lambda v: 0 * v, # irrelevant feature
]
interaction = 0.5 * X[:, 0] * X[:, 1] # something an additive model cannot see
y = sum(f(X[:, j]) for j, f in enumerate(shapes)) + interaction + rng.normal(0, 0.3, n)
tr, va, te = np.split(rng.permutation(n), [4000, 5000])
scaler = StandardScaler().fit(X[tr])
Xs = scaler.transform(X).astype(np.float32)
y32 = y.astype(np.float32)
class AdditiveCurves(nn.Module):
"""y = bias + sum_j f_j(x_j); each f_j is one learnable B-spline."""
def __init__(self, n_features, n_coef=12):
super().__init__()
self.curves = tc.BSplineCurve(n_features, 1, knots_config=n_coef,
input_map=tc.maps.Real.clamp(scale=3.0)) # covers +/- 3 sd
self.sum = tc.Sum(dim=-2)
self.bias = nn.Parameter(torch.zeros(1))
def forward(self, x): # (B, F)
return self.sum(self.curves(x)).squeeze(-1) + self.bias # (B,)
def roughness(self):
c = self.curves.control_points # (F, C, 1)
return (c[:, 2:] - 2 * c[:, 1:-1] + c[:, :-2]).pow(2).sum()
def kan(n_features, hidden=8, n_coef=10):
m = tc.maps.Real.clamp(scale=3.0)
return nn.Sequential(
tc.BSplineCurve(n_features, hidden, knots_config=n_coef, input_map=m), tc.Sum(),
tc.BSplineCurve(hidden, 1, knots_config=n_coef, input_map=m), tc.Sum(),
nn.Flatten(0),
)
def mlp(n_features, hidden=64):
return nn.Sequential(nn.Linear(n_features, hidden), nn.ReLU(), nn.Linear(hidden, hidden),
nn.ReLU(), nn.Linear(hidden, 1), nn.Flatten(0))
def fit_torch(model, lam=0.0, epochs=200, lr=1e-2, batch=256):
opt = torch.optim.Adam(model.parameters(), lr=lr)
Xt, yt = torch.from_numpy(Xs[tr]), torch.from_numpy(y32[tr])
Xv, yv = torch.from_numpy(Xs[va]), torch.from_numpy(y32[va])
best, best_state = float("inf"), None
for _ in range(epochs):
perm = torch.randperm(len(Xt))
for i in range(0, len(Xt), batch):
b = perm[i:i + batch]
opt.zero_grad()
loss = (model(Xt[b]) - yt[b]).pow(2).mean()
if lam > 0:
loss = loss + lam * model.roughness()
loss.backward()
opt.step()
with torch.no_grad():
v = (model(Xv) - yv).pow(2).mean().item()
if v < best: # keep the best epoch on validation
best, best_state = v, {k: t.clone() for k, t in model.state_dict().items()}
model.load_state_dict(best_state)
return model
def rmse(pred):
return float(np.sqrt(np.mean((pred - y[te]) ** 2)))
def torch_pred(model):
with torch.no_grad():
return model(torch.from_numpy(Xs[te])).numpy()
rows = []
lin = LinearRegression().fit(Xs[tr], y[tr])
rows.append(("linear regression", F_ + 1, rmse(lin.predict(Xs[te]))))
gam = make_pipeline(SplineTransformer(n_knots=10, degree=3), RidgeCV(alphas=np.logspace(-3, 3, 13))).fit(Xs[tr], y[tr])
rows.append(("SplineTransformer + RidgeCV", gam[-1].coef_.size + 1, rmse(gam.predict(Xs[te]))))
add = fit_torch(AdditiveCurves(F_), lam=1e-3)
rows.append(("torchcurves additive", sum(p.numel() for p in add.parameters()), rmse(torch_pred(add))))
k = fit_torch(kan(F_))
rows.append(("torchcurves KAN (5-8-1)", sum(p.numel() for p in k.parameters()), rmse(torch_pred(k))))
torch.manual_seed(0)
m = fit_torch(mlp(F_), lr=3e-3)
rows.append(("MLP (64-64)", sum(p.numel() for p in m.parameters()), rmse(torch_pred(m))))
hgb = HistGradientBoostingRegressor(random_state=0).fit(Xs[tr], y[tr])
rows.append(("HistGradientBoosting", None, rmse(hgb.predict(Xs[te]))))
print(f"{'model':30} {'params':>7} {'test RMSE':>9}")
for name, p, r in rows:
print(f"{name:30} {p if p is not None else '-':>7} {r:>9.3f}")
print(f"{'noise floor':30} {'':>7} {0.3:>9.3f}")
# --- inspect the learned shape functions --------------------------------
torch.save(add.state_dict(), "additive.pt")
grid_raw = np.linspace(-2, 2, 5)
with torch.no_grad():
G = np.tile(scaler.mean_, (len(grid_raw), 1))
out = []
for j in range(F_):
Gj = G.copy(); Gj[:, j] = grid_raw
fj = add.curves(torch.from_numpy(scaler.transform(Gj).astype(np.float32)))[:, j, 0].numpy()
out.append(fj - fj[len(grid_raw) // 2]) # centre at x = 0 for reading
print("\nlearned f_j(x) - f_j(0) at x =", grid_raw.tolist())
names = ["tanh(2x)", "0.5x^2", "step>0.5", "sin(2x)", "none"]
for j in range(F_):
true = shapes[j](grid_raw) - shapes[j](0.0)
print(f" x{j} ({names[j]:9}) learned " + " ".join(f"{v:6.2f}" for v in out[j]))
print(f" {'':13} true " + " ".join(f"{v:6.2f}" for v in true))
Output:
model params test RMSE
linear regression 6 1.191
SplineTransformer + RidgeCV 61 0.603
torchcurves additive 61 0.600
torchcurves KAN (5-8-1) 480 0.471
MLP (64-64) 4609 0.341
HistGradientBoosting - 0.393
noise floor 0.300
learned f_j(x) - f_j(0) at x = [-2.0, -1.0, 0.0, 1.0, 2.0]
x0 (tanh(2x) ) learned -1.00 -1.01 0.00 0.97 1.01
true -1.00 -0.96 0.00 0.96 1.00
x1 (0.5x^2 ) learned 2.05 0.51 0.00 0.54 1.98
true 2.00 0.50 0.00 0.50 2.00
x2 (step>0.5 ) learned -0.01 0.01 0.00 1.06 0.89
true 0.00 0.00 0.00 1.00 1.00
x3 (sin(2x) ) learned 0.68 -0.88 0.00 0.95 -0.61
true 0.76 -0.91 0.00 0.91 -0.76
x4 (none ) learned 0.02 0.01 0.00 0.04 -0.01
true -0.00 -0.00 0.00 0.00 0.00
What the numbers say:
- The additive TorchCurves model matches the scikit-learn spline GAM (0.600 against 0.603, both 61 parameters). Same model class, different trainer. If your model is purely additive and tabular, scikit-learn's
SplineTransformerplus a ridge regression is simpler and faster [8]. - Both additive models miss the interaction and sit well above the noise floor of 0.30. That is the honest cost of one readable curve per feature. If the interaction matters, add it explicitly (for example a curve of the product, or a 2-D term) rather than abandoning the structure.
- The KAN finds some of the interaction (0.471) with 480 parameters across 48 curves; it is more expressive but no longer reads as one curve per feature.
- The MLP and gradient boosting win on accuracy (0.341 and 0.393) and lose on readability. Your decision is which side of that line the use case needs.
- The shape table recovers the truth: for example f1 at x = ±2 is 2.05 and 1.98 against a true 2.00, and the irrelevant x4 stays within 0.04 of zero.
Why train a GAM in PyTorch at all? Because it can then be one part of a larger model: an additive "explainable" head next to an embedding model, curves whose coefficients are produced by another network, a monotone curve in a pricing model, or continuous embeddings of numeric features inside a transformer or factorisation machine, which is the use case the author's earlier research targeted [6].
7. Export and inspect the coefficients
An auditable model is one somebody else can re-evaluate. The script below rebuilds the curve layer, loads the trained weights, writes knots, degree, input map and coefficients to JSON, re-evaluates them with SciPy's BSpline from the JSON alone, and captures the layer with torch.export for serving.
# s5_export.py
import json
import numpy as np
import torch
import torchcurves as tc
from scipy.interpolate import BSpline
# Rebuild the additive model's curve layer and load the trained weights.
curves = tc.BSplineCurve(5, 1, knots_config=12, input_map=tc.maps.Real.clamp(scale=3.0))
state = torch.load("additive.pt")
curves.load_state_dict({k.removeprefix("curves."): v for k, v in state.items()
if k.startswith("curves.")})
# 1. Export everything needed to re-evaluate the curves without PyTorch.
spec = {
"library": f"torchcurves {tc.__version__}",
"degree": curves.degree,
"knots": curves.knots.tolist(),
"input_map": {"type": "clamp", "scale": 3.0, "range": list(curves.parameter_range)},
"bias": state["bias"].item(),
"coefficients": curves.control_points[..., 0].detach().tolist(), # (features, C)
}
with open("curves.json", "w") as f:
json.dump(spec, f, indent=1)
print("exported", len(spec["coefficients"]), "curves x", len(spec["coefficients"][0]), "coefficients")
# 2. Re-evaluate with SciPy from the JSON alone and compare.
x = torch.linspace(-4, 4, 801).unsqueeze(1).repeat(1, 5) # includes out-of-range values
with torch.no_grad():
y_torch = curves(x)[..., 0].numpy()
lo, hi = spec["input_map"]["range"]
u = np.clip(x.numpy() / spec["input_map"]["scale"], lo, hi)
y_scipy = np.stack([BSpline(np.array(spec["knots"]), np.array(c), spec["degree"])(u[:, j])
for j, c in enumerate(spec["coefficients"])], axis=1)
print("max |torch - scipy| =", float(np.abs(y_torch - y_scipy).max()))
# 3. Can the layer be captured for serving?
try:
batch = torch.export.Dim("batch") # without this the batch size is fixed at 8
ep = torch.export.export(curves, (torch.zeros(8, 5),), dynamic_shapes={"u": {0: batch}})
print("torch.export: ok,", sum(1 for n in ep.graph.nodes if n.op == "call_function"), "ops")
except Exception as e:
print("torch.export failed:", type(e).__name__, str(e).splitlines()[0][:120])
with torch.no_grad():
diff = (ep.module()(x) - curves(x)).abs().max().item()
print("exported program matches eager:", diff == 0.0)
torch.export.save(ep, "curves.pt2") # load with torch.export.load(...).module()
Output:
exported 5 curves x 12 coefficients
max |torch - scipy| = 6.234304983365746e-07
torch.export: ok, 80 ops
exported program matches eager: True
The SciPy re-evaluation agrees to within 6.2 × 10-7 (float32 rounding), including inputs beyond ±3 standard deviations, where the clamp map flattens the curve. That JSON file is now the model of record: small enough to review, diff in Git and attach to a model-risk document, and evaluable in any language with a B-spline routine.
Two export details. First, torch.export specialises on the example input's shapes: our first attempt without dynamic_shapes failed at runtime with Guard failed: u.size()[0] == 8 for any other batch size. Mark the batch dimension dynamic, as above. Second, torch.jit.trace still works in PyTorch 2.15 but prints a deprecation warning recommending torch.export. We did not test ONNX export.
8. Test it
Four tests cover what usually goes wrong: shapes and input validation, gradients (checked numerically with torch.autograd.gradcheck in float64), agreement with SciPy, and the monotonicity guarantee.
# test_curves.py
import numpy as np
import pytest
import torch
import torchcurves as tc
from scipy.interpolate import BSpline
def test_shapes():
c = tc.BSplineCurve(num_curves=3, dim=2, knots_config=8)
assert c(torch.rand(4, 3)).shape == (4, 3, 2)
with pytest.raises(ValueError):
c(torch.rand(4, 2)) # wrong number of curves
def test_gradients_bspline_and_legendre():
torch.manual_seed(0)
for curve in (tc.BSplineCurve(2, 1, knots_config=6),
tc.LegendreCurve(2, 1, degree=4)):
curve = curve.double()
u = (torch.rand(5, 2, dtype=torch.float64) * 1.8 - 0.9).requires_grad_()
assert torch.autograd.gradcheck(curve, (u,))
def test_matches_scipy():
c = tc.BSplineCurve(1, 1, knots_config=9, parameter_range=(0, 1), input_map="real.clamp")
x = torch.linspace(0, 1, 101).unsqueeze(1)
ref = BSpline(c.knots.numpy(), c.control_points[0, :, 0].detach().numpy(), c.degree)(x.numpy())
np.testing.assert_allclose(c(x).detach().numpy()[:, 0, 0], ref[:, 0], atol=1e-5)
def test_increasing_coefficients_give_increasing_curve():
basis = tc.BSplineBasis(knots_config=10, parameter_range=(0, 1), input_map="real.clamp")
coef = torch.cumsum(torch.rand(10), 0).view(1, 10, 1)
y = basis(torch.linspace(0, 1, 2001).unsqueeze(0), coef.expand(2001, -1, -1).contiguous())
assert torch.all(torch.diff(y.view(-1)) >= -1e-6)
.... [100%]
4 passed in 1.03s
Add tests for your own model on top: the expected sign or direction of each feature's curve, bounds on edge coefficients, and a golden JSON export that must not change without review.
9. Running it in production
- Package. Pin
torchcurves,torchand Python in your lock file; this is a young 0.x library and 0.3.0 already had breaking changes. Store the library version in the exported JSON, as our script does. - Containers. A CPU-only PyTorch wheel keeps images small, and curve models rarely need a GPU for inference. For the pure-additive case you can skip PyTorch at serving time entirely and evaluate the JSON with SciPy or a few lines of NumPy.
- CPU or GPU. The author's benchmarks show the library is slower than SciPy or NumPy for small batches and few curves, because of Python overhead, roughly level with EfficientKAN for KAN training, and faster at larger batch sizes and dimensions [1]. Measure on your own shapes before choosing hardware.
- Checkpoints. Use
state_dict(). The knot vector is a buffer, so it is saved with the weights; to change knots, load a state dict rather than editing the tensor, so the cached parameter range is refreshed [2]. - Monitoring. Watch the input distribution against the knot range: the share of inputs beyond the clamp limits is a direct measure of how often the model is extrapolating flat. Track calibration of monotone curves over time, and alert if retraining moves an edge coefficient sharply.
- Review. Plot every curve on each retrain and diff it against the previous release. A shape change is a model change, even if the accuracy metric is flat.
Pitfalls and debugging
ValueError: Input u must be a 2D tensor of shape (N, num_curves=...): you passed a 1-D tensor or the wrong number of columns. Usex.view(-1, 1)for a single curve.TypeError: ... unexpected keyword argument 'num_control_points'or'normalize_fn': invented or pre-0.3.0 arguments. Useknots_configandinput_map.- Number of control points must exceed the degree: a cubic needs at least four coefficients.
- Flat curve where you expected detail: your data sits outside the knot range, or the rational map has bunched the knots near zero. Plot the knots in raw units (section 4).
- Wild values at the edges: few data points near the boundary. Add a roughness penalty or move the range in.
- Shape mix-ups with
LegendreCurve.coefficients: it is (K, C, D), not (C, K, D) likeBSplineCurve.control_points. - Slow training on tiny batches: expected from Python overhead; use larger batches or SciPy for single curves.
How it compares
| Tool | What it is | Trainable inside PyTorch | Use it when |
|---|---|---|---|
| torchcurves | B-spline and Legendre curve layers, basis modules, functional API | Yes, inputs and coefficients | Curves are part of a larger PyTorch model, or coefficients come from another network |
SciPy (make_smoothing_spline, BSpline) | Classic spline fitting and evaluation | No | One curve, fitted once; reference evaluation of exported curves |
scikit-learn SplineTransformer + linear model | Spline features for a GAM-style model | No | Purely additive tabular models with standard tooling |
IsotonicRegression (scikit-learn) | Monotone step-function fit | No | Quick monotone calibration where steps are acceptable |
| pykan, efficient-kan | KAN implementations | Yes | KAN research; efficient-kan was the paper's speed baseline [1][10] |
xitorch Interp1D, torch-interpol, torch-cubic-spline-grids | Differentiable interpolation or spline grids | Partly | Interpolating data or learning dense grids, per the torchcurves README's own comparison |
| MLP, gradient boosting | General-purpose learners | MLP yes | Accuracy first, interactions matter, explanation can be post hoc |
The original version of our blog compared TorchCurves with NVIDIA Kaolin Wisp and Torch Lens Maker. They are differentiable libraries for neural fields and geometric optics respectively, not curve-fitting alternatives, so we have left them out here.
FAQ
Is this a replacement for gradient boosting? No. On our tabular test boosting and an MLP were more accurate. Curves are for when readability or a shape rule matters more than the last few points of accuracy, or when the curve is a component of a neural model.
Can a curve output a vector? Yes: dim sets the output dimension. A curve from a scalar to a 16-dimensional vector is a continuous embedding of that scalar.
Can I set my own knots? Yes: pass a 1-D tensor as knots_config and the number of control points is inferred. You cannot combine that with parameter_range.
Does it run on GPU? Yes, it is pure PyTorch, so .to("cuda") works; the paper's benchmarks include GPU KAN training [1]. Whether it is faster than CPU depends on batch size and number of curves.
How many coefficients should I use? Start at 8 to 12 per input with a small roughness penalty and tune on validation data. With a penalty, being generous costs little.
Is the paper's API the same as the package? Mostly. The paper (v1) names the default input map as real.arctan and mentions a num_knots argument; the 0.3.4 code defaults to "real.rational" and uses knots_config. Trust the installed source and the documentation for your version [11].
If you are building interpretable models into a production pipeline and want help with versioning, testing and monitoring them, see our MLOps setup service.
References
- A. Shtoff, "TorchCurves: Differentiable Parametric Curves", arXiv:2611.14516v1, 10 November 2026
- alexshtf/torchcurves on GitHub: README, source (v0.3.4), example notebooks and release notes
- torchcurves on PyPI
- P. H. C. Eilers and B. D. Marx, "Flexible smoothing with B-splines and penalties", Statistical Science 11(2), 1996
- Z. Liu et al., "KAN: Kolmogorov-Arnold Networks", ICLR 2025
- A. Shtoff, E. Abboud, R. Stram, O. Somekh, "Function Basis Encoding of Numerical Features in Factorization Machines", TMLR 2024
- R. Agarwal et al., "Neural Additive Models: Interpretable Machine Learning with Neural Nets", NeurIPS 2021
- scikit-learn: SplineTransformer
- SciPy: make_smoothing_spline
- efficient-kan: a pure-PyTorch KAN implementation
- torchcurves documentation
PyTorch is a trademark of The Linux Foundation. Other names are the property of their respective owners. This article is independent and not endorsed by the authors of the libraries discussed. Code in this technote is provided as an example; check it against the current library documentation before relying on it.