Your video games, your GPS dot and every bridge you've crossed lean on a trick about 350 years old, or about 3,700, depending on who you ask: guess wrong, slide down a tangent, guess again. This is the field manual for Newton-Raphson and its gang of cousins, and for what to do when the curve fights back.
In 1999 a video game shipped with a few lines of code that, for years, almost nobody could explain. Quake III Arena had to work out 1/√x millions of times a second to light its levels, and square roots were slow. So the function took the bits of a decimal number, pretended they were a whole number, shifted them one place, and subtracted the result from a magic constant: 0x5F3759DF. Next to that line, a programmer had left a two-word review.
The source code went public in 2005 and the internet lost its mind over the magic number. But the magic number only gets you close, within a few percent. Look at the line after it, the one marked 1st iteration. That's one step of Newton's method, and it takes the error from a few percent to a fraction of a percent. Somebody wrote a second step, then commented it out. They didn't need it: one step was enough.
That's the whole trick of this manual. Start with a guess that's wrong. Fix it with a tangent. Repeat until wrong stops mattering. The same move draws your game's lighting, puts the dot on your map, trains the AI that wrote half your inbox, and, pointed at a steel frame, tells you how far it sags before it gives up.
the same code calls itself "evil floating point bit level hacking." i've never felt so seen. evil, spelled backwards, is how i hack.
You have a curve, and you want the spot where it crosses zero. You can't solve it on paper, but you can work out where the curve is and how steep it is at any point you pick.
So you stand at a guess. You draw the tangent there, the straight line that just kisses the curve. Then you slide down the tangent to where it hits zero. That's your next guess. Do it again.
Near the answer the curve and its tangent are almost the same line, so every slide lands very close. Wrong by 0.1 becomes wrong by 0.01, then 0.0001, then 0.00000001. The number of correct digits roughly doubles every step. Mathematicians call that quadratic convergence. We call it the snowball.
| STEP | GUESS FOR √2 | RIGHT DIGITS |
|---|---|---|
| 0 | 1 | 1 |
| 1 | 1.5 | 1 |
| 2 | 1.41666666… | 3 |
| 3 | 1.41421568… | 6 |
| 4 | 1.41421356237469 | 12 |
↑ four slides, a dozen digits. a calculator shows 10.
YBC 7289 is a clay tablet about the size of a cookie, now at Yale. A student scratched a square on it with its diagonals and wrote √2 in base 60: 1;24,51,10. That works out to 1.41421296, right to about six significant figures, around 3,700 years ago.
Nobody knows exactly how they got it. The leading theory is guess, divide, average: guess 1.5, divide 2 by it, average the two, repeat. Heron of Alexandria wrote that recipe down in the 1st century AD. Do the algebra and it's exactly Newton's method on x² − 2. The Babylonians were riding tangents before tangents existed.
Isaac Newton described the method around 1669 in a manuscript called De analysi. His showcase example was x³ − 2x − 5 = 0, and he squeezed it down to 2.09455147. But his version had no tangents and no derivatives. He rewrote the whole polynomial around each new guess, which is a lot of algebra per step. It reached print in 1685, inside a book by John Wallis.
In 1690 Joseph Raphson published a cleaner version that kept reusing the original equation. Then in 1740 Thomas Simpson wrote it with calculus, for any equation, and even for two equations at once. That's the version everyone actually uses. So the "Newton-Raphson method" could fairly be called the Simpson method. Nobody does.
Newton's x³ − 2x − 5 went on to become the standard test equation for root-finding methods for centuries. If you ever invent a solver, it's tradition to try it on Newton's cubic first. (Start at 2. You'll have 2.1 after one step and 2.0945514 after three.)
newton did it. raphson posted it. simpson made it make sense. history gave it to the guy with the apple. sounds like the internet.
yep, that's the solver. you're probably wondering how it ended up at x = 4.7 × 10¹⁸. to understand that, we have to go back to one innocent little guess…
Newton is a sniper with no peripheral vision. Close to the answer, it's untouchable. Far from it, the tangent can point anywhere. Four ways it goes wrong:
1. The ping-pong. Try x³ − 2x + 2 = 0 starting at 0. The tangent sends you to 1. From 1, it sends you back to 0. Then 1. Then 0. Forever. Every step looks reasonable, and you never get anywhere.
2. The launch. If the curve is nearly flat where you stand, f′ is close to zero, and dividing by it fires your next guess off to the edge of the universe. That's the 4.7 × 10¹⁸.
3. The runaway. For the cube root, f(x) = ∛x, the formula simplifies to xnew = −2x. Every step doubles your distance from the answer, which is zero. It's the only answer, and you can't reach it from anywhere except 0.
4. The fractal. In 1879 the mathematician Arthur Cayley asked an innocent question: run Newton on the complex numbers, where z³ = 1 has three answers. Which starting points end up at which answer? He solved the easy version with two answers, then wrote that the three-answer version "appears to present considerable difficulty" and stopped there. It took computer graphics a century later to show why.
The lesson for everything that follows: Newton is perfect once you're close and a gamble when you're not. Everything else in this manual is about getting it close, and keeping it there.
start in the wrong neighbourhood and you end up in a different galaxy. not a math thing. a life thing.
A structure isn't one equation. A computer model of a frame is a mesh of points, and each one can move and rotate, so there's one equation per way of moving: thousands of them, sometimes millions. And they all say the same thing: at every point, the push back equals the push.
That's equilibrium. The structure's internal forces (what the steel pushes back with) must balance the loads. Whatever's left over is the residual, the out-of-balance force, and the job is to drive it to zero.
In a linear analysis, push-back is just stiffness × movement, and you solve it once. Real life isn't linear:
• Materials give up. Steel yields, concrete cracks, and the stiffness drops.
• Shapes change. When parts rotate a lot, the loads act on a different geometry than you started with.
• Things touch. Gaps close, bolts bear, cables go slack.
So the structure becomes a curve, and Newton comes back. The "slope" f′ becomes a giant table, the tangent stiffness matrix: how hard every point resists every small movement right now. Each Newton step means building that matrix and solving one huge linear system. A computer does that in seconds or hours, depending on the size of the model.
You also don't throw the whole load on at once. You walk up the curve in small increments, and at each one Newton snaps you back onto the true curve. This is called path following, and it's how almost every serious nonlinear engineering program works.
| THE MATH WORD | THE ENGINEER WORD |
|---|---|
| x, the unknown | Displacements of every point in the model |
| f(x), how wrong you are | Residual: out-of-balance force |
| f′(x), the slope | Tangent stiffness matrix Kt |
| One Newton step | Assemble Kt, factorize it, solve Kt Δu = residual |
| The root | Equilibrium |
| "Converged" | Residual small enough, by a test you picked (see 07) |
| "Diverged" | Throw the step away, cut it in half, pray |
equilibrium: when everything pushes back exactly as hard as it gets pushed. me and my landlord. me and the group chat.
The most expensive thing a nonlinear solver does is build and factorize that giant stiffness matrix. Every cousin below is a different answer to one question: how often do you pay for that?
Rebuilds the tangent stiffness on every single iteration. When it's close, nothing is faster in number of iterations: the digits double. Each iteration is the expensive kind, though, and far from the answer it can still launch itself into space.
Builds the matrix once, at the start of the step (or once, ever), and keeps reusing it. Each iteration costs pennies. The catch is speed: it cuts the error by a fixed fraction per iteration instead of doubling the digits, so it can need dozens of iterations where Newton needs four.
It has one sneaky virtue. When the structure is softening (steel yielding), the old matrix is too stiff, so each correction falls short and it creeps up on the answer from one side. That's slow, but it doesn't overshoot. When the structure is stiffening (cables tightening, gaps closing), the old matrix is too soft and it overshoots, sometimes wildly.
Never rebuilds the matrix. After every iteration it compares how far it moved with how much the residual changed. That ratio is a slope, so it patches its picture of the stiffness with a cheap correction. It's like learning someone's personality from their reactions instead of reading their diary.
Keeps the cheapskate's old matrix, but remembers the last handful of corrections and blends them to aim the next one better. Structural engineers took it up for nonlinear frame models around 2010. You get close to Newton's speed for close to the cheapskate's price.
Newton picks a direction and a distance. Line search keeps the direction but questions the distance: go 40% of the way? 160%? It tries a few and keeps whichever leaves the smallest residual. In 1966 Larry Armijo wrote down the rule most versions still use: only accept a step if it makes things sufficiently better. It turns the sniper into a hiker who taps the ground before every step.
If a step won't converge, throw it away, halve it and try again. Every commercial program does it. It's cheap insurance, with one warning: if the solver keeps halving at the same spot, the algorithm usually isn't the problem. The physics is telling you something: a peak, a buckle, a mechanism. That's the next section.
Here's the part that trips people up. The algorithm (section 05) decides how to fix each step. The control decides what you hold fixed while you take the step. Mix them however you like: Newton with any of three controls, the cheapskate with any of three. And the control is often what decides whether you make it over the hill.
Most structures don't just bend more and more. They climb to a peak load (a limit point) and then get weaker as they move. Some do worse: they snap through, jumping suddenly to a new shape. And a few snap back, where the load drops and the deflection briefly runs backwards.
The most natural one. At the peak there is no balance at a higher load, the tangent goes flat (the launch from 03), and the solver dies. The real structure jumps: that's snap-through.
DIES AT THE PEAKThis is what a lab testing machine does. It rides over the peak and down the softening side, no problem. But at a snap-back the controlled point would have to move backwards, and it never does.
DIES AT SNAP-BACKLets load and displacement float, and fixes the length of the step instead. It follows peaks, snap-through and snap-back. The catch: each step has two possible answers, and choosing the wrong one walks you back down the hill.
SURVIVES (MOSTLY)Arc length came out of aerospace and shell buckling: Wempner in 1971, Riks in 1972 and 1979, then Crisfield and Ramm in 1981, who made it practical. A cousin, generalized displacement control (Yang and Shieh, 1990), measures how stiff the structure currently is and resizes each step to match. It's another way over the hump.
The Venus flytrap shuts in about a tenth of a second, with no muscles. A 2005 study in Nature showed how: each lobe is a curved shell, bowed outward. When a fly trips the hairs, the plant changes the lobe's curvature until it hits a limit point, and snap-through does the rest. Same physics as the rubber jumping popper toy you turn inside out and set on a table.
Hours after about 5,000 fans left a basketball game, the space-frame roof of the Hartford Civic Center in Connecticut fell into the empty arena under snow and ice. Nobody was hurt. The roof had been designed with a computer analysis that was cutting-edge for its day. Investigators found that bars in the roof's compressed top layer had buckled, a stability failure the model hadn't captured. During construction the frame had sagged about twice as much as predicted, and the warnings were waved off because the computer said it was fine.
A solver that never meets the peak will tell you there isn't one.
snap-through is when you stop being nice all at once. snap-back is when you take it back. arc length is therapy.
You never hit zero exactly. You stop when the residual is small. Small compared to what? That choice is called the convergence test, and it quietly decides how much your answer can be trusted.
Force test. How much out-of-balance force is left? It's the honest one, but watch the units. One list holds forces in kN and moments in kN·m (or kN·mm), and whichever has the bigger numbers takes over the total. The test can end up checking only moments and ignoring forces, or the other way round.
Displacement test. How far did the last iteration move? Cheap and unit-safe, but it has a trap: the cheapskate moves tiny amounts even when it's far from the answer. Slow can look exactly like converged.
Energy test. Force × movement, in one number with consistent units. A popular middle ground.
Relative or absolute? "Relative" means compared with something else: the applied load, or how wrong the first iteration was. Loosen the tolerance and every step gets accepted a little out of balance. Those leftovers pile up step after step, and the curve drifts off equilibrium while still looking perfectly smooth. Smooth isn't the same as right.
The fix is the same one engineers use for mesh size: tighten it until the answer stops changing. If halving the tolerance moves your curve, the old curve was showing you the tolerance, not the structure.
every hole in this manual, in one place. 15 doors. pick one. we're not responsible for where you end up.
New manuals drop on the 13th. Get them first, read every one free, or slap a sticker somewhere to show you've been here.
found this printed somewhere? read it, then leave it somewhere weirder. field manuals are supposed to travel.