Isle of Savreen is the 3D follow-up to my 2D water simulator: a glass cube of water, a meter on a side, with a rusting fishing boat run aground among mossy boulders inside. You look down on it like a model on a table, and the arrows let you pick the tank up. Tip it and the water pours to the low side and breaks over the rocks; jerk it sideways and it slams into the far wall and climbs it. As in the 2D version, every bit of water is a brush stroke, deep blue when slow and white foam when fast, and the sound is built from the same physics.

How FLIP turns particles into water

The water is about 7,500 particles, each a 2.5 cm drop that carries its own position and velocity. Particles are good at being water: wherever there are particles there’s water, everywhere else is air, so splashes, droplets and a free surface come for free. What particles are bad at is the one rule water lives by, that it can’t be squeezed. Checking that every bit of water has as much flowing out as flowing in is easy on a grid and hard among scattered points.

FLIP (Fluid-Implicit-Particle) uses both. The particles carry the water; a coarse grid, 20 × 20 × 20 cells of 5 cm, enforces incompressibility. Brackbill and Ruppel invented it for plasma physics in 1986, and Zhu and Bridson brought it to graphics in 2005; it’s what film effects use for splashing water. Every 1/60 of a second, each step goes:

1. Move the particles. Add gravity (and the tank’s forces, below), then move each particle along its velocity. For a moment the water just flies.
2. Keep them spaced out. Anywhere two particles overlap, push them apart. Without this they clump in some places and leave gaps in others.
3. Collide. Push any particle that ended up in a wall, the ship or a rock back out to the surface, and stop it moving into it.
4. Particles to grid. Each particle spreads its velocity onto the eight nearest cell faces, weighted by how close it is. The grid stores velocities on cell faces rather than centers, so the flow through each face is exactly what’s needed to count what enters and leaves each cell.
5. Make it incompressible. For every cell with water in it, add up the flow out through its six faces. In real water that’s zero. Here it won’t be, so nudge the faces until it is: push flow out of cells that are being crammed and into ones that are draining. Each fix disturbs the neighbors, so the grid is swept 40 times. That nudging is the pressure in the Navier–Stokes equations. Cells packed with more particles than water at rest would hold get an extra push outward, so the water doesn’t slowly lose volume.
6. Grid back to particles. Each particle reads the corrected velocity from the faces around it. Here’s the trick in the name: it doesn’t take the grid’s velocity (that method, PIC, works but smears motion away and the water turns to syrup); it keeps its own velocity and adds only the change the grid made. That keeps all the small-scale motion the coarse grid can’t represent. Pure FLIP is noisy, so the blend here is 90% FLIP and 10% PIC, and that 10% stands in for water’s tiny viscosity.

If the fastest water would move more than two cells in a step, the step is split into smaller ones so nothing tunnels through a thin wall. In 3D each of those stages touches three times as many neighbors as in 2D, which is why the grid is coarser here, and why speed mattered (more below).

Moving the tank without moving the water

The obvious way to tip a tank is to tip the grid. That’s the hard way: the walls would cut diagonally through the cells. So the simulation never moves. It runs in the tank’s own frame, where the walls and rocks always line up with the grid, and moving the tank becomes forces the water feels, the ones you feel in a car or on a roundabout:

• Tilt turns gravity: seen from inside the tank, “down” points at a different wall.
• Sliding pushes the water back, the way coffee sloshes when you start walking.
• Turning adds an outward push (centrifugal), a sideways drift of anything moving (Coriolis), and a jolt as a turn starts and stops.

Each arrow press is a short, smooth move that starts and ends at rest, so these forces are known exactly. I checked them against textbook answers: a tank accelerating sideways tilts its water surface to a slope of a/g, a tank turned on its side ends with its water against the new floor, and water in a spinning tank rises up the walls into a curved bowl.

A glowing light in the sky adds one more force. It swings from far left to far right and back every 2.5 seconds, pulling the water toward it with about a third the strength of gravity. A wave takes about 1.25 seconds to cross this tank and back, so the light keeps the water sloshing without the motion building up out of control.

A shipwreck and some rocks

The wreck comes from a painting of a beached fishing boat, pale teal paint flaking to orange rust. It’s built in code: a hull lofted from cross-sections narrowing to a raked bow, a deckhouse and wheelhouse, railings with posts missing, and a leaning mast, painted with the same brush strokes as everything else. The rocks are a reusable boulder model: give it a box and a seed and it grows a unique rock in that box, with flat facets where slate has split, layered ledges, clefts, a lean, and moss on whatever faces up.

The water meets them through signed distance fields: grids storing, at every point, how far it is to the nearest surface, negative inside. Any shape that can answer “is this point inside me?” can be baked into one, and step 3 above becomes a single lookup per particle: if a drop is too close, push it out along the direction the distance grows fastest. The mesh you see and the shape the water hits come from the same function, so they always agree.

From physics to pixels

The simulation and the drawing run on separate threads. A worker thread owns the water and steps it 60 times a second on its own clock; the page only draws, and only when the worker has sent something new:

The model-railway look is a fake tilt-shift lens. A real one keeps only a thin band of the scene sharp, which is what your eye expects of something a few centimeters away. Here the scene is drawn sharp, blurred copies are made, and each pixel picks its blur from how far it is from the tank, above, below, in front or behind, with a little extra saturation and a vignette on top.

Quieting the fans

The first version made my laptop sound like it was taking off, so I measured. The surprise: two-thirds of each step went to step 2, pushing overlapping particles apart, and only a few percent to the pressure solve. Sorting the particles by grid cell, so neighbors sit next to each other in memory, and checking each nearby pair once instead of twice made every step 35 to 40 percent faster.

The rest was wasted drawing. The screen runs at 120 frames a second, but the water only changes 60 times a second, and the page kept redrawing even while paused. Now the page draws only when the worker sends a new frame, and the worker sleeps when nothing is moving. Idle, CPU use went from about three cores to a third of one in a software-rendering test, and the water runs at real speed whatever the frame rate.

Try it

Isle of Savreen is live here, and the code is on GitHub: AndrewKL/IsleOfSavreen. It’s TypeScript with three.js, a Web Worker for the simulation and the Web Audio API for sound. Turn your sound on, click to start, then tip the tank: the arrow keys slide it, Shift with the arrows tilts it, and Q and E spin it.

• Launch Isle of Savreen
• View the code on GitHub

Helios is a gravity sandbox that runs in the browser. Click anywhere a few times to drop some bodies, press Play, and watch them pull on each other. Every body is a glowing sphere that pulses and hums a low sci-fi drone. The closer it gets to another body, the bigger and brighter it gets and the faster it pulses and wubs. It never plays out the same way twice, and that is the point: it is a toy built around one of the oldest unsolved problems in physics.

The three-body problem

Given three masses, their positions and their velocities, where will they be at any later time? Newton answered this for two bodies in 1687. Two bodies trace ellipses, parabolas or hyperbolas forever, and a formula tells you exactly where they will be at any time you like. Add a third body and there is no such formula.

Why it’s hard to solve

It is not for lack of trying. The problem has too many unknowns and too few rules. Three bodies moving in space have 18 numbers that change over time: three position and three velocity components each. Conservation of energy, momentum and angular momentum pins down only 10 of them. In 1887 Heinrich Bruns proved there are no other algebraic conserved quantities to make up the difference.

Henri Poincaré went further around 1890 and showed that the motion generally can’t be reduced to a neat formula at all. Karl Sundman did find an infinite series that technically solves it, in 1912, but it converges so slowly that it is useless in practice.

So we simulate. Astronomers step the bodies forward in small time increments, computing the forces and nudging every position and velocity a little at a time, and so does Helios. That works, but every step adds a tiny error, and with three bodies tiny errors don’t stay tiny.

Why it’s unstable

Poincaré’s other discovery was chaos. A difference in starting positions far smaller than you could ever measure grows exponentially over time. Two nearly identical setups soon look nothing alike, so long-term prediction is impossible even though the laws themselves are exact. Nothing is random here; it is deterministic and still unpredictable.

The mechanism is the close encounter. When two bodies pass near each other they swap a lot of energy very quickly, and one can be flung onto a wide orbit or out of the system entirely. That is the usual ending: most three-body systems eventually throw one body out, and the other two settle into a tight pair carrying away the leftover energy.

There are rare exceptions, special starting positions that repeat forever. Euler found three bodies in a line in 1767, and Lagrange found them at the corners of an equilateral triangle in 1772; Jupiter’s Trojan asteroids ride that triangle. In 1993 Cris Moore found three equal masses chasing each other around a figure-eight. But most of these orbits are balanced on a knife-edge. Nudge them and chaos takes over.

Science, art and music

Helios is honest physics. Every body pulls on every other body with Newton’s inverse-square law, and a velocity Verlet integrator steps it forward 240 times a second, with a fixed timestep so the result doesn’t depend on your frame rate. There are two deliberate compromises, and the info panel in the app says so. Gravity is softened at very short range so bodies pass through each other instead of reaching infinite speeds. And a weak attractor sits at the center of the screen, pulling strays back instead of letting the usual ejection end the show.

The art is in what the physics is made to look like. The quantity that matters most in a three-body system is invisible: how close each body is to its nearest neighbor, because that is where the close encounters, the energy swaps and the chaos come from. Helios draws that. As two bodies approach, each grows, its core burns whiter, its pulse speeds up and deepens, and it stretches toward its neighbor like a body being pulled apart by tides. A near miss reads as a flare. The drama on screen is the same thing that is making the math hard.

The music does the same thing for your ears. Each body is its own voice: two detuned sawtooth waves and a sub-octave sine running through a resonant filter, with a slow oscillator sweeping the filter open and closed. That sweep is the “wub.” Far from everything, a body wubs lazily, about one and a half times a second. As it closes in on another body, the wub speeds up to ten times a second and the tone gets brighter and louder. Each body is panned left or right by its position on screen, so you can hear where it is.

The notes are chosen so the chaos stays listenable. Each new body takes the next note of a pentatonic scale, the five-note scale with no half steps in it, so however many bodies you add and however they move, the chord never clashes. The physics decides the rhythm and the scale keeps the harmony.

Put together, a simulation becomes a composition that no one wrote. You choose the starting positions, and after that Newton writes the score: long quiet drifts where every voice wubs slowly and apart, then a close pass where two voices race and flare together, and one body gets slung across the stereo field. Place the same bodies a few pixels differently and you get an entirely different piece. That is chaos, heard rather than read about.

Try it

Helios is live here, and the code is on GitHub: AndrewKL/Helios3BodyProblemVisualizer. It is TypeScript and three.js, with the Web Audio API for the sound and no other libraries. Turn your sound on, click a handful of times, and press Play.

• Launch Helios
• View the code on GitHub

When I was a new SDE at ZS Associates, more than 15 years ago, I spent a significant amount of time playing with very large in-memory lists. Calling ToList() on a few million records is easy to write, and it’s easy to miss what it costs: memory climbs, the garbage collector kicks in over and over, and more and more of the program’s time goes to the runtime cleaning up after it.

A few years later, in 2014, I wrote a small library to test an idea: what if the list didn’t live on the managed heap at all? Serialize every object into one big block of memory, and turn the bytes back into objects only while you are looping over them. I recently got it building again on .NET 10, moved its storage into native memory, and wrote a benchmark to see whether the idea holds up.

Why a big list hurts

.NET’s garbage collector is generational. New objects start in generation 0, which is collected often and cheaply, on the bet that most objects die young. Objects that survive a few collections get promoted to generation 2, which is only collected occasionally. To collect, the GC has to find every object that is still reachable, which means following every reference from every live object. The cost of that tracing grows with the number of live objects, not with how much memory they take up.

A big list breaks the generational bet. Every record you add survives, because the list holds on to it. So each gen 0 collection finds almost nothing to free and has to copy the survivors up a generation, and the gen 1 and gen 2 collections that follow have to trace millions of objects and references each time. Five million records with one string each is ten million objects, plus the list’s internal array, which is big enough to go on the Large Object Heap. Every one of those objects also carries a 16-byte header and is reached through an 8-byte pointer.

None of this shows up as an error. The program just gets slower, and it gets slower the most in the code that has nothing to do with the list, because a long gen 2 collection pauses every thread.

Getting the data off the heap

The library is OffHeapIEnumerable<T>. You give it any IEnumerable<T>, and it walks through it once, writing each object’s properties into a stream with a BinaryWriter: an int is 4 bytes, a string is a length prefix plus UTF-8, and so on. Enumerating it reads the stream back and builds a fresh object for each record as you reach it. To the GC, the whole data set is nothing but the stream.

What that stream sits on has changed three times. The first version used a plain MemoryStream, which keeps everything in one byte[]. That fails on big data sets, because a single array needs one contiguous run of memory and a MemoryStream can’t grow past 2 GB. The 2014 version swapped in a MemoryTributary, which is a stream backed by a list of 64 KB arrays. The chunks are small enough to stay off the Large Object Heap, and the GC only sees a few thousand of them instead of millions of objects. They are still managed memory, though.

The version I have now allocates its 1 MB blocks with NativeMemory.Alloc, outside the GC heap entirely. The GC never scans them, never moves them and doesn’t count them. The catch is that nothing frees them for you. The collection is IDisposable, and you have to dispose it, the same as a file handle.

The benchmark

The benchmark builds 5 million records, each with an int, a string, a decimal, a double and a bool, and stores them three ways. The first is a plain List<T>. The second is the off-heap collection with the original serializer. The third is the off-heap collection with a newer, faster serializer I’ll come back to. For each one it measures how long it takes to build, how long it takes to loop over once, how much memory it holds, and how long a forced full, blocking garbage collection takes while the data is still alive. That last number is the one this whole project is about.

The off-heap version is three times faster to build, and the reason is in the second row. More than half of the List<T> build time is the garbage collector copying and tracing records that were never going to die. The off-heap version makes plenty of garbage while building too, but all of it dies young, which is the case the GC is designed for.

It also takes 2.7 times less memory. The list pays a 16-byte header for every record and every string, an 8-byte pointer to each, and two bytes per character in UTF-16. The stream pays a byte or two of framing instead.

A full collection with the list alive takes 124 ms, and that grows with the list. With the data off heap it takes a tenth of a millisecond, because there’s nothing left to trace.

And then there’s the last row. Looping over the list takes 84 ms, because the objects are already there. Looping over the off-heap version is about 5 to 9 times slower, because every pass has to build 5 million new objects and 5 million new strings from bytes.

Boxing, and a faster serializer

The original serializer made that last row worse than it needed to be. It worked through reflection in the most general way: a list of properties, each with a getter and setter delegate typed as object, and a switch on the property’s type. Every int, bool, decimal and double it read got boxed into a short-lived heap object on the way to its setter. That’s four extra allocations and five delegate calls per record.

The typed serializer builds one reader and one writer per type with expression trees and compiles them, so the result is the same code you would write by hand:

reader => new Record
{
    Id = reader.ReadInt32(),
    Name = reader.ReadBoolean() ? reader.ReadString() : null,
    Price = reader.ReadDecimal(),
    Score = reader.ReadDouble(),
    Active = reader.ReadBoolean()
}

That halves the garbage made while enumerating, down to exactly one record and one string each, and it makes enumeration 1.3 to 1.9 times faster across runs. It is still slower than walking a list. Most of the remaining cost is going through BinaryReader and Stream for every 4- or 8-byte field; reading straight out of the native blocks would be the next step.

On heap vs off heap

On heap is the right default, and it isn’t close for most programs.

• Reading is as fast as it gets. The objects already exist, so a loop is just following pointers.
• You get random access, updates in place, LINQ, and any type you like, including ones with nested objects and collections.
• The GC frees it all for you, and there is no serialization format to keep in sync with your classes.
• The cost is the GC. Every live object is something to trace, so collections get slower as the data grows, and it’s the whole process that pauses, not just the code using the list.
• Small objects are memory-hungry: 16 bytes of header each, 8 bytes per reference, and UTF-16 strings.

Off heap trades read speed for everything else.

• A full GC stays at a tenth of a millisecond no matter how much data you hold, and building the data set doesn’t push the rest of the program through gen 2 collections.
• It takes much less memory, because there are no object headers or pointers, and strings are stored as UTF-8.
• Every read deserializes. If you loop over the data many times, you pay that cost every time, and it is several times the cost of reading a list.
• It only goes forward. There’s no indexing and no updating in place; you append and you enumerate.
• It only handles the types the serializer knows about. Here that’s int, bool, decimal, float, double and string properties on a class with a public parameterless constructor.
• You own the memory. Forget to dispose it and the GC won’t know it’s there, so the finalizer that would eventually free it may not run for a long time.
• The objects you get back are copies. Changing one doesn’t change what’s stored.

The trade is worth it when the data set is big, lives a long time, is read only a few times, and shares a process with code that cares about pause times, like a web server that keeps a large cache or reference data set in memory. It isn’t worth it for a list you build, loop over once and throw away. A few million short-lived objects are exactly what the GC is good at.

What I’d reach for today

There are other ways to get most of the way there. Background GC does most of the gen 2 work while your threads keep running, although the gen 0 and gen 1 collections that promote a growing list still stop them. If your records can be structs with no reference fields, an array of them is one object to the GC, and the GC doesn’t look inside it, which gets you most of the benefit with none of the serialization. And if the data is really a table, an in-process columnar engine like DuckDB, or Apache Arrow, will do this better than a hand-rolled stream.

Still, the core idea holds. The garbage collector’s cost is in the number of live objects, not their size, so when you have millions of them, the best thing you can do is give it fewer to look at.

The code, the tests and the benchmark are on GitHub at AndrewKL/CSharpOffHeapStorage. Running dotnet run -c Release --project OffHeapStorage.Benchmark -- 1000000 5000000 reproduces the numbers on your own machine.

• View the code on GitHub

The water simulator is a small river that runs in the browser. Click once and water pours from a ledge high on the left, crashes onto a row of mossy boulders, pools behind them, spills over and runs out the right side. Drag anywhere to drop in a new boulder and watch the water find its way around it. The whole scene is painted in brush strokes: still water is deep blue, and the faster the water moves, the paler it gets, until the fastest water on screen is white foam. It sounds like water too, with a rush that follows the current, splashes where it lands, and a low surf that swells as it sloshes.

The Navier–Stokes equations

Water, air, honey and blood all move by the same two rules. Claude-Louis Navier wrote them down in 1822 and George Gabriel Stokes put them on firm footing in

1. They are Newton’s force equals mass times acceleration, applied to every tiny parcel of a fluid.

The first rule is about momentum. A parcel of water speeds up or turns because pressure pushes on it, gravity pulls on it, and viscosity, the fluid’s internal friction, drags it toward the speed of its neighbors. It also carries its own velocity along as it moves, and that self-carrying term is what makes flow curl back on itself into swirls and eddies.

The second rule is that water can’t be squeezed. Whatever flows into any small region has to flow out again. Pressure is exactly the force needed to keep that true: it builds up wherever water is being crammed together and pushes it apart.

Why they’re still unsolved

Two short equations, and still nobody knows whether they always make sense. In three dimensions, no one has proved that smooth solutions always exist, or that the velocity can’t suddenly blow up to infinity at a point. It is one of the Clay Mathematics Institute’s seven Millennium Prize Problems, with a million-dollar reward for either answer.

The practical problem is turbulence. How chaotic a flow gets depends on its Reynolds number: speed times size, divided by viscosity. A stream running at a meter per second over a meter-sized rock has a Reynolds number around a million. At that point the flow breaks into eddies, the eddies break into smaller eddies, and the cascade doesn’t stop until it reaches swirls tens of microns across, where viscosity finally turns the motion into heat. Resolving all of that in 3D would take something like ten trillion grid cells. Engineers don’t do it. They model the small eddies statistically and simulate only the big ones, and even that takes supercomputers.

How a few thousand particles fake a waterfall

This simulation cheats in the way film effects cheat. It uses a method called FLIP, short for Fluid-Implicit-Particle. Brackbill and Ruppel invented it for plasma physics in 1986, and Zhu and Bridson brought it to computer graphics in

1. The water is about six thousand particles, each carrying a position and a velocity. Every step, their velocities are copied onto a coarse 120 × 60 grid. A pressure solve adjusts the grid until no cell gains or loses water, and the change is copied back to the particles. The particles give you a free surface, where water meets air, for free, and they carry momentum without smearing it, which is what keeps the water lively and splashy instead of syrupy.

It looks right because at this scale water is dominated by gravity, inertia and incompressibility, and FLIP gets those three right. Jets fall in parabolas, pools find their level, waves travel and water piles up against rocks. What it gets wrong is everything small. There is no viscosity term, the only damping comes from numerical side effects, and the grid cells are about a thousand times too coarse to see real turbulence. It is also 2D, and 2D turbulence is a different animal: small swirls merge into big, long-lived whirlpools instead of shattering into smaller ones. The big lazy eddies you see in the pools are partly that.

Science, art and sound

The physics decides where every bit of water goes. The art is in what it is made to look like, and the look comes from an impasto painting of a woodland waterfall: white water tumbling between dark slate boulders with bright moss on top.

Color is speed. The quantity your eye can’t normally see in moving water is how fast each part of it is going, so the simulation paints it. Each particle’s speed is compared with the fastest water currently in the scene. The slowest water is deep blue, faster water is paler, and the fastest is white. The scale follows the water, so a gentle trickle and a torrent both use the full range. Press Surge and the whole jet flashes white, then the colors settle back over a second or two as the scale catches up. This happens to be how rivers look. Real white water is white because it is fast and broken up, and still pools are dark.

Every particle is a brush stroke. Each one is stretched along the direction it is moving, with bristle streaks and a little variation in tone, so fast water becomes long, thin, pale streaks that trace the flow, and still water becomes short horizontal dabs, like a painter’s pool. The rocks are painted the same way: slate strokes with light catching the top, a band of moss that creeps down the sides, and a seed per rock so it looks the same every frame.

The sound is built the same way, from the physics up, with no recorded audio at all. A rush of filtered noise follows the average speed of the water, getting louder and brighter as it speeds up, and pans toward wherever the fast water is. Splashes are short bursts of noise, triggered when water hits something hard enough to change its velocity sharply. Under both is a low surf that swells and fades like waves on a shore and grows as the water sloshes up and down. Draw a boulder into a quiet pool and you hear the slap. Press Surge and the whole river roars.

Try it

The water simulator is live here, and the code is on GitHub: AndrewKL/NavierStokesSimulation. It is TypeScript and WebGL2, with the Web Audio API for the sound and no other libraries. Turn your sound on, click to let the water flow, and drag a few boulders into its path.

• Launch the water simulator
• View the code on GitHub

The problem is one every large finance organization eventually runs into. Financial transactions used to fit on a single relational database. They stopped fitting a long time ago, and the data ended up spread across a lot of teams — each of which understood its own data best, and none of which wanted to wait on a central team to publish it.

A data mesh is the usual answer: let the teams that own the data own the publishing too, and centralize the governance rather than the data. The interesting part is what that costs. Democratizing data product creation removes the central bottleneck, but it hands every producer a new job — onboarding a dataset to the catalog, then managing permissions on it, before anyone else can use it. Get that wrong and adoption quietly stalls, because the path of least resistance goes back to copying data around.

So most of the work went into making the obligations cheap. Producer accounts sync their AWS Glue Data Catalog metadata into a central catalog. Lake Formation in a central governance account holds the fine-grained permissions. Consumers browse a front-end application, request access, and the request routes to the data owner for approval, with a ticket behind it for auditing. On approval, automation provisions an AWS RAM resource share, and the consumer queries the data with Redshift Spectrum, Athena, EMR or QuickSight.

It currently hosts around 850 discoverable datasets and more than 300 curated data products.

The diagram above is the shape of it: metadata copies inward in blue, permissions flow outward in red, and no bulk data moves between the accounts at all. The post has the rest of the diagrams and the details on the catalog sync, which is the part that took the most care.

When a free radical meets a molecule, where does it attack? For benzene the answer is obvious, because every carbon is the same. For azulene, pyrene or a new ligand nobody has made before, it isn’t, and finding out in the lab takes time, material and sometimes a particle accelerator.

Radmap is a program I wrote as a chemistry student at Saint Mary’s University in Halifax, in the Atlantic Centre for Green Chemistry, to answer that question computationally. It shows the earliest moment of a reaction between a molecule and a single atom (a hydrogen atom, a proton, a hydride, a bromine atom or anything else monoatomic) as a colored surface wrapped around the molecule. Red marks the low-energy spots, where the incoming atom is drawn in and a radical is most likely to add.

We described the method in the Journal of Molecular Graphics and Modelling:

Andrew K. Long, Jason A. Fawcett, Jason A.C. Clyburne and Cory C. Pye. RADMAP: Simple probes for rapid assessment of complex reactivity: A method and case studies on the reaction of hydrogen atoms with unsaturated organic molecules. J. Mol. Graph. Model. 64, 147–152 (2016).

Why muons

The problem came from muon chemistry. A positive muon that captures an electron forms muonium, which behaves chemically like a hydrogen atom with about a ninth of the mass. Fire muons into a sample and the muonium adds to the molecules in it, just as a hydrogen atom would. In a transverse-field muon spin resonance (µSR) experiment, each distinct radical that forms gives its own pair of signals.

So before running an experiment you want to know how many places the muonium can add, and roughly how much of each adduct will form. That tells you how many signals to look for and how strong each should be. Beam time at a muon facility is scarce, and a good prediction makes it go further.

How it works

Azulene + H·: the question, a probe atom moved over the molecule, the surface it samples, and the finished map.

Radmap works in four steps:

1. Build a surface. Starting from a structure loaded from a .mol file, Radmap draws a solvent-accessible surface around the molecule. Every point on it sits a fixed distance outside the atoms (by default a 1.2 Å probe radius beyond each atom’s van der Waals radius), which is what isodistance means here. A resolution setting controls how many points the mesh has.
2. Place the probe. For every point on the mesh, Radmap writes a Gaussian input file containing the molecule plus the probe atom at that point, and a batch file to run them all. You choose the probe, the charge and multiplicity, and the method and basis set. The default is a short two-step optimization at the unrestricted B3LYP/6-31G level, which suits an open-shell radical.
3. Compute. Gaussian runs every job. This is the slow part: one quantum chemistry calculation per point on the surface, which is why the batch file can be split across machines.
4. Paint. Radmap reads the energy out of each output file, smooths the values onto the displayed surface with a Gaussian-weighted average, and colors it from blue (high energy, where the probe is pushed away) to red (low energy, where the probe is drawn in).

The result is a potential energy surface that isn’t a flat slice through space but wraps the whole molecule, so every side of it can be compared at once. The red regions predict where addition happens. The relative energies of the red spots also give a qualitative estimate of how much of each adduct forms, which says something about the molecule’s kinetic stability as well as its reactivity.

The interface is built on Jmol, the open source molecular viewer, so the finished map can be rotated and inspected like any other Jmol model. Radmap itself is open source too.

Reading the maps

Pyridine + H·. The strongest red sits just outside the nitrogen, and the center of the ring is blue.

Pyridine is a good first example because it has only one feature that isn’t a C–H. The map picks it out straight away: the hot spot is at the nitrogen, and the ring carbons are all much cooler.

Pyrene + H·. Red over the outer carbons, weaker at the 2 and 7 positions, and blue over the ring centers.

Pyrene is where the method starts to earn its keep. It has 10 C–H carbons, and the map separates them into groups: the carbons at the ends of the upper and lower double bonds and the carbons next to the ring junctions glow red, while the carbons in the middle of the left and right edges are noticeably cooler. Each distinct red site is a distinct radical, and a distinct µSR signal.

Carbon disulfide + H·. The low-energy regions are the caps over the two sulfur atoms, not the carbon.

1,3-Dimethylimidazol-2-ylidene, an N-heterocyclic carbene, + H·. The red patch is over the carbene carbon between the two nitrogens.

The carbene is a nice check. Earlier muonium experiments from the Clyburne group, with Paul Percival’s team, showed that the hydrogen atom adds to the carbene carbon, and that is exactly where Radmap puts the red.

A phosphorus pincer compound + H·. The reactive site is the phosphorus at the center.

This is also where Radmap is useful beyond muons. When you design a new ligand to bind to a metal, you want to know where its reactive sites are before you spend weeks making it. A Radmap surface, run with whichever probe fits the question, gives a quick first look.

Azulene, rendered in grayscale.

Limitations

Radmap is deliberately simple, and that has costs:

• It only sees the first contact. The probe is placed at a fixed distance and relaxed for only a couple of steps, so the map shows where an atom is drawn in, not the final product or the barrier to get there.
• It’s expensive. Every point on the mesh is a separate quantum chemistry job. A fine mesh on a large molecule means thousands of calculations.
• It’s qualitative. The colors rank sites against each other on one molecule. They aren’t rate constants, and the level of theory limits how far the numbers can be trusted.

Within those limits, it turns a question that used to need an experiment into a picture you can rotate on screen.

Credits

• Main developer: Andrew K. Long
• Advisers: Jason A.C. Clyburne and Cory C. Pye
• Co-author: Jason A. Fawcett
• Art design: Emily Rose

Thanks to Robert M. Hanson and everyone on the Jmol-Development mailing list for their help with this program, and to Jason Masuda for his input during development.

It started as a port of the Fortran IV program in Appendix B of Szabo and Ostlund’s Modern Quantum Chemistry. The step #N comments in the solver follow the SCF procedure on page 146 of that book, which is the honest way to describe it: the physics is theirs, the code is mine. I was advised by Jason A. C. Clyburne, with additional help from Cory Pye.

The shape of the calculation is the part worth knowing, because it is the same shape in every serious quantum chemistry package. Put a contracted Gaussian 1s orbital on each atom. Compute the one-electron integrals — the overlap matrix, the kinetic energy, the nuclear attraction — and add the last two to get the core Hamiltonian. Diagonalize the overlap matrix to build a transformation that orthogonalizes the basis, because the atomic orbitals you started with are not orthogonal and nothing downstream wants to deal with that.

Then the expensive part. Every two-electron integral is a four-index object, so the count grows as the fourth power of the basis size, and you need all of them before the iteration can start. JackSolver precomputes them across a thread pool sized to the number of cores, which is the one place where writing this in Java instead of Fortran paid for itself.

The self-consistent loop is the idea the whole method rests on. The Fock matrix depends on the electron density, and the density is what you are trying to find, so you guess, solve, and use the answer to build a better guess. JackSolver starts from a zero density matrix, then each pass builds the Fock matrix, transforms and diagonalizes it, and forms a new density — mixing 90% of the new one with 10% of the old, because taking the new one outright makes it oscillate instead of settle. It stops when the RMS change in the density falls below 1e-4.

Everything is in atomic units: bohr for distance, hartree for energy.

The built-in example reproduces the Szabo and Ostlund HeH⁺ calculation at a bond length of 1.4632 bohr, and should print a total energy of about −2.8607 hartree. That number is how you know the thing works — it is the same one in the back of the book.

The code is on GitHub: AndrewKL/JackSolver.