Watershed · Benchmark study
One model. Two ways to execute it.
An insurer might need to update its portfolio value after one policy changes, then run thousands of market scenarios to see how the whole portfolio responds. Both jobs use the same formulas. They ask the computer to do very different amounts of work.
Watershed lets you define those formulas once and record what each one depends on. It can use that dependency graph to update just the affected results on a CPU. For a batch of scenarios, you compile the same graph into a plan and run it across CPU threads or on an NVIDIA GPU.
- Incremental CPU
- Change one input and recalculate the results it affects.
- Batched GPU
- Run a compiled plan across many independent sets of inputs.
The insurance problem
The model is a savings policy with guarantees. It follows savings.CashValue_ME_EX2 from lifelib, an open-source library of actuarial reference models, and Watershed’s results are checked against it.
A policyholder pays in, either once at the start or every month. The insurer keeps a share of each premium, the load, and puts the rest into an investment fund in the policyholder’s name.
Each month the insurer takes a fee from the fund, 1% a year here, and the fund then rises or falls with the market.
The policy ends in one of three ways:
- The term ends. The policyholder is paid the fund, or the sum assured if that is more. The sum assured is an amount fixed when the policy was sold.
- The policyholder dies. The same promise applies: the larger of the fund and the sum assured.
- The policyholder cashes in early, which actuaries call a lapse. They get the fund, less a surrender charge on some policies in their first ten years. There is no guarantee on a lapse.
So the insurer earns fees and loads, pays expenses and commission, and carries one real risk. If the fund is below the sum assured when a policy ends by death or at term, the insurer makes up the difference from its own money. That shortfall is the cost of the guarantee, and it exists only when markets have done badly.
How many policies reach each ending is assumed, not known. Deaths come from a mortality table by age. Lapses start at 10% a year and fall to 2%. In this model they also follow the fund: policyholders whose fund is well above the guarantee cash in more often, and those whose fund has fallen below it tend to stay, because the guarantee is now worth having. That makes the guarantee cost more just when it bites.
One group of policies, four markets
Take 100 identical policies. Each holder paid a single premium five years ago into a ten-year policy with a sum assured of 100,000. Each fund stands at 105,000 today, and there are five years left.
Here is that group projected along four market paths, with the model’s own formulas and lifelib’s tables.
| Market | Fund at end | In force | Top-up | Guarantee cost | Fees |
|---|---|---|---|---|---|
| Up 6% a year | 133,658 | 80.4 | 0 | 0 | 498,000 |
| Up 2% a year | 110,381 | 81.5 | 0 | 0 | 457,000 |
| Down 25%, then partly back | 98,587 | 83.7 | 1,413 | 107,000 | 398,000 |
| Down 25%, then up 2% a year | 81,083 | 84.8 | 18,917 | 1,452,000 | 357,000 |
“Partly back” is a fall of 25% in the first year, then rises of 15%, 10%, 2% and 2%. “In force” is how many of the 100 policies are expected to reach the end of the term.
Three things are worth noticing.
The guarantee costs nothing on the first two paths and 1.45 million on the last. A projection along one “expected” path would price it at zero.
The path matters, not just where it ends. The third path recovers most of its fall, and the guarantee still costs 107,000.
Fewer policyholders leave when the fund is down: 84.8 of the 100 are still there on the worst path against 80.4 on the best. Those are expected numbers, which is why they aren’t whole. More policies in force means more guarantees to pay, while the fees earned on smaller funds fall.
The figures come from running this group through the benchmark’s model. The group itself is made up for the example, as are the four paths.
What the insurer calculates
The insurer wants to know what its promises are worth today. For every policy it projects each future month’s cash flows: premiums in, and claims, surrender payments, expenses and commission out. A payment due in the future is worth less than the same payment now, so each is discounted back to today, at about 2% a year here. Added up, that is a present value.
- Policies and assumptionsEach group’s premium, fund, term and sum assured; mortality, lapses, fees, expenses
- Monthly projectionThe fund after premium, fee and the month’s return; deaths, lapses and policies left
- Cash flows and guaranteesWhat is paid on death, on lapse and at the end, and how much of it is above the fund
- Discounted valuesEach month’s cash flow brought back to today and added up
- Portfolio resultsFour totals across every group
The model produces four present values for the whole book: the net cash flow to the insurer, the cost of the guarantee on death, the cost of the guarantee at the end of the term, and the fees.
None of these has a formula, because each depends on the path the market takes. So the insurer draws many possible paths and values the book along each. One path is a market scenario: 240 monthly returns, twenty years, applied to every fund. Here they are drawn at random around 2% growth a year with 10% yearly volatility.
The average over the scenarios is the expected cost. The spread shows how exposed the insurer is, and the bad tail is what it must hold capital against. On this synthetic book a single steady path prices the end-of-term guarantee at 0.8 billion; 10,000 paths price it at 8.0 billion. A model that runs too few scenarios misses most of the cost.
This is the technique insurers use, applied to made-up data. The policies, the funds’ volatility and the scenarios are synthetic, the book represents no real insurer, and the run is not a regulatory valuation. The models page lists what is taken from lifelib and what is invented.
From a million policies to 19.9 million calculations
Projecting a million policies one at a time would repeat a lot of work, so actuaries group them. A model point is a group of identical policies: same age, term, premium, sum assured and fund. It is projected once and its results are multiplied by the number of policies in it. The group of 100 in the example above is one model point.
The benchmark’s book has 20,000 model points standing for 1,009,152 policies. They are generated at random: terms of 10, 15 or 20 years, entry ages from 20 to 59, single or monthly premiums, three levels of load and surrender charge, and 1 to 100 policies in each.
Each model point is projected for every month it has left, 95 on average and 240 at most. A month is about ten small calculations: the fund after the premium, after the fee, its growth, its mid-month value, deaths, lapses, policies left, next month’s fund, and three running present values.
Watershed records each of those as a node in a graph, with an edge to every value it reads. That is 1.9 million policy-months and 19,889,057 nodes. A month reads the month before, so the graph is 967 steps deep.
One market scenario evaluates every node once. Ten thousand scenarios is 2 × 1011 evaluations in double precision.
Why the model is run again and again
A valuation is not run once. The same model is rerun whenever something it depends on moves:
- A policy changes. A policyholder alters the cover, new business is written, a data error is corrected. One model point changes and the totals must follow.
- An assumption changes. Mortality, lapses, expenses or the discount curve are revised, or deliberately stressed to see what a worse world would cost.
- The market moves. Every fund has a new starting value and the scenarios are drawn afresh.
These ask for very different amounts of work. The first touches one group’s projection. The last touches everything, thousands of times over. A system that recalculates everything for each is slow for the first; one built only for single updates cannot do the last.
The most demanding case is capital: what the insurer would lose in the worst year in two hundred. Done directly it is a simulation inside a simulation, and it is the calculation the industry most often approximates. The worst year in two hundred describes it and what we ran.
How the two paths work
Go back to the group of 100 policies, now one of 20,000 model points feeding the portfolio totals.
Suppose that group’s sum assured changes. Its own projection is out of date, a few hundred of the 19.9 million nodes, and so are the totals it feeds. Nothing else is. Watershed follows the graph’s edges from the changed input, recalculates what it reaches and reuses the rest. That is incremental recomputation, and it runs on a CPU.
Now suppose the market moves instead. A new scenario changes the return in every month, which every model point reads, so every node is out of date. Following edges saves nothing. For thousands of scenarios Watershed compiles a plan from the same formulas: it sorts the nodes so that independent ones run together, and a GPU works them out for many scenarios at once. Each scenario gets its own results.
The third case combines them. A batch of scenarios has been run and then one model point changes. Every scenario’s result is stale, but only through that one group. If some intermediate values were kept from the first run, only the affected part needs redoing for each scenario. Combining the two paths measures that.
What we measured
We tested both paths on a synthetic savings-policy portfolio with a graph of 19.9 million calculation nodes. A model point represents a group of identical policies. A market scenario is a path of 240 monthly returns for the whole portfolio.
- 58 µs to update one model point on a Ryzen 9 5950X. Each change recalculated 570 nodes on average. The figure is the median of five runs, each averaging 1,000 changes.
- 10.2 s to evaluate 10,000 market scenarios on one RTX 3090. The figure is the median of five runs and includes data transfers and GPU kernel compilation. Building the graph and compiling the scenario plan took another 22.7 s and 17.7 s respectively.
- 6.2 s for specialised JAX code on the same GPU. This was the fastest measured implementation of the insurance batch. It used a hand-written version of the model, sorted by remaining policy term. The timing excludes tracing, compilation and preparing constants.
The CPU update and GPU batch timings describe different jobs. For the same 10,000-scenario batch, the estimate on 32 CPU threads was about 7.5 minutes, scaled from a 128-scenario sample.
Combining the two paths
Suppose the 10,000-scenario batch has been run and then one model point changes. Every scenario’s result is now out of date. An experimental feature keeps some intermediate values from the first run, uses the graph to find what the change can reach, and recalculates only that for every scenario.
- 0.14 to 0.22 s to bring all 10,000 scenarios up to date. We tried five different model points; each figure is the median of five runs. Between 5,771 and 9,327 nodes were recalculated, on CPU threads.
- 11.2 s to run the whole batch again on the GPU, or 29 s including compiling the plan again, which a changed model point otherwise needs.
The updated results were identical, to the last bit, to compiling and running everything again. The run kept 11,436 values per scenario, 915 MB in all. This only helps a change that reaches a small part of the model, and the caller chooses which values to keep. The same shortcut could be written by hand for one model in JAX; here it comes from the model’s dependencies. On a second model, a book of 20,000 loans, the engine chose what to keep and each update took about 30 ms against 7.2 s. See the measurements.
The worst year in two hundred
A European insurer has to hold enough capital to survive the worst year in two hundred (Solvency II, Article 101). That is a question about the future value of the book, and it is the hardest thing this model is asked.
Why it is a run inside a run
To know what the book would be worth after a bad year, it has to be valued as it would stand then. One valuation already takes thousands of market paths. So the direct calculation is a simulation inside a simulation:
- draw many ways the next year could go, the outer scenarios;
- from each, value the whole book over many market paths, the inner scenarios;
- sort the losses and read off the one that only one year in two hundred exceeds.
With 1,000 outer scenarios and 1,000 inner, that is a million valuations of the whole book.
What is usually done instead
A Milliman paper says of the direct calculation: “This brute force approach is too time-consuming to be used in a real insurance business case” (Boumezoued, Cherchali and Truc, 2022).
The capital is typically worked out by the standard formula, which adds up a set of prescribed stresses. Insurers with models of their own, more than half the market by size (IMF, 2025), commonly value the book at far fewer points and fit a curve through them, which is “less numerically burdensome than a brute-force nested stochastic approach” (Milliman, 2013).
Running the direct calculation on GPUs is not new either. Commercial actuarial platforms do it: Aon’s PathWise is built on GPUs, and FIS publishes a case study of its Prophet platform on them.
What we ran
The engine’s capital example does the direct calculation on the insurance model above, on one desktop GPU.
| What | Value |
|---|---|
| Book | 4,000 model points standing for 200,278 policies, a fifth of the book used above. 3,210,292 nodes |
| Outer scenarios | 1,000. Each is a rise or fall in every fund over the year (5% expected growth, 15% volatility) and a move in interest rates (0.7 of a point, typically) |
| Inner scenarios | 1,000 market paths from each, the same random draws every time, so that two outer scenarios differ only by the scenario |
| Valuations of the book | 1,000,000 |
| Time on the GPU | 3 minutes, in each of three runs |
| lifelib, in Python on one core | About 98 hours, scaled from its measured rate. Not run |
An outer scenario changes the fund values and the discount curve. Those are shared inputs of the plan, so each one is a change of a few numbers and another run. The plan and the inner paths stay on the GPU throughout.
The result was the same figure in all three runs: a loss of 6.1 billion in the worst year in two hundred.
What it showed
On this book, the shortcut overshoots. Two prescribed stresses, a 39% fall in funds and a one-point move in rates, added the way the standard formula adds half-correlated market risks, give 9.8 billion. The direct calculation gives 6.1 billion.
That is a statement about a made-up book and a simplified version of the formula. It shows the kind of difference the direct calculation can expose, not the size of it for any real insurer.
What it is not
- The year is a shock, not a projection. Funds and rates move and the book is valued as it stands today. A year of ageing, premiums and claims is left out, as the prescribed stresses leave it out too.
- Only two risks move. Fund values and the level of interest rates, drawn independently. Mortality, lapses and expenses stay at their best estimates.
- Five scenarios decide the answer. The worst year in two hundred of 1,000 is the fifth worst. We have not measured how far the figure moves with more outer scenarios.
- It is not a capital calculation. The book is synthetic, with one product, no assets and no management actions.
What is different here
Not the method, and not the use of a GPU. Two things about how the model is held.
The model is ordinary formulas in a general engine. Nobody wrote GPU code for it: the plan the GPU runs is derived from the same graph that is checked against lifelib.
And that graph also does incremental updates. After a batch has run, a change to one model point brings every scenario up to date in a fraction of a second (above). A fitted curve has to be fitted again when the book changes. We have not yet combined the two, an incremental update of a nested run, so that is a direction and not a result.
The nested run has also not been done at the scale of the eight-GPU study, which ran two million flat scenarios of the full 20,000-model-point book in 70 seconds. Doing so needs each lot of work to carry its own outer scenario, which the cluster does not support yet.
When a GPU helps
Scenario batches are a good fit when each scenario does substantial work. In the insurance run, each path evaluated nearly 20 million nodes for 240 input values and four outputs. In the 49-node tax and benefits model, transfers dominated the GPU time. The CPU estimate was about the same, based on a smaller sample.
Setup matters too. The insurance graph and plan took about 40 s to prepare. Reusing the plan spreads that cost over later batches. For incremental work, the size of the change matters. Changing the entire market path on the CPU took 5.3 s.
See the seven benchmark models and the other workload timings, including runs that kept inputs on the GPU.
Method and limitations
In the mixed-portfolio check, Watershed’s scenario results agreed with the independent lifelib reference to a maximum relative difference of 3.1 × 10⁻¹³. The measurements page gives the checks for each workload, the timing boundaries and the commands used to reproduce the runs.
Scenario plans support a subset of Watershed formulas, including double-precision arithmetic, comparisons and conditionals. Plans must be rebuilt when the graph structure changes. The caller explicitly chooses incremental evaluation or a scenario plan, and selects the CPU or CUDA backend for the plan.