Getting nerdsniped by weird artifacts in neural cellular automata

Wesley Maa

The chameleon below is grown by a neural cellular automaton (NCA): a grid of cells that all follow the same learned ruleset. Imagine Conway's Game of Life but ML-ified. You can click to damage it, plant more seeds, and train your own in this browser in ~20 seconds.

loading…
Click or drag to damage it and watch it regrow, or switch “click to” to plant extra seeds. Right-drag (two-finger click on a trackpad) to move the picture (content wraps). Pinch or use + and − to zoom. On a phone, switch “click to” to move.

A few months ago I saw a new project on Twitter called Cells to Pixels. It builds on growing NCAs: the cells grow a hidden state, 32 numbers each, and a small decoder reads that state at any point and returns a colour, so a coarse grid of cells can be drawn at any resolution.

I played around with it a lot and kept getting annoyed by this one faint lattice artifact. If you zoom in, it looks like there's a square mesh over everything. Immediately I thought to myself that this must've been a result of some weird sampling the authors did. Which, as it turned out, was absolutely the case!

A chameleon rendered by Cells to Pixels and by my version, each with a zoomed-in crop and a frequency spectrum
Left: Cells to Pixels. Right: my version. The middle row is a zoom. The bottom row is the frequency spectrum, where the mesh shows up as the grid of faint dots on the left.

AI slop summary

Why it happens

In the original paper, the decoder that turns cell state into RGBA is conditioned on a coordinate (u, v) that is sampled from outside the cells. It corresponds to where the pixel should be relative to the current cell. Every cell sees the exact same (u, v) pattern, which creates a strong built-in bias toward repeating the same detail in every cell. That creates a meshing artifact.

It also means the renderer is not translation equivariant. The coordinate is fixed to the grid, so if the cell state shifts by a fraction of a cell, (u, v) does not shift with it, and the picture changes when it should just move.

Implementing a simple fix

Rather than computing the coordinate from the pixel grid, I reserve two dimensions of each cell's internal state for position. Each cell already has 32 values. The decoder used to get all of them plus the outside (u, v). Now it gets 30 of them, and the other two go in the slot where (u, v) used to be. That adds no parameters: the same update rule writes those two values along with the rest.

Nothing forces them to be coordinates, so the cells store whatever helps. You can see for yourself what this position code looks like by setting "show" under the demo to "position code". Red and green are the two values. For example, on the chameleon they aren't a tight coordinate grid but rather form soft patches that vary slowly across the body.

Detail inside a cell comes from blending. A pixel that sits between cells gets a mix of its neighbours' values, so what the decoder sees changes smoothly from cell to cell and never restarts at a border.

A single coordinate for the whole image, fed in from outside, would also stop the repetition. But it would be pinned to the canvas, so the picture still could not move. Values kept in the state travel with the cells, preserving the translation equivariance that lets you more or less copy and paste the learned image.

Diagram: a seed grows into a grid of cells with 32 channels each; the decoder reads the cells at a point, with position coming either from two of the channels or from the grid
Green: position kept in the cells. Purple: position computed from the grid.

The mesh goes away. Sliding also becomes exact: move the cells by any whole number of pixels and the picture moves with them, pixel for pixel. That is why a seed planted anywhere in the demo grows a clean copy.

Numbers

where position comes fromshift errormesh strengthPSNR (dB)LPIPS
the cells (mine)0.07 × 10−619.30.317
grid (u, v) (Cells to Pixels)18.91.9820.40.211
LIIF21.72.4821.50.173
LTE, 128 frequencies22.22.4520.40.173
LTE, 5 frequencies20.72.5519.30.234
hash grid46.5n/a32.30.016

Medians of three training runs. Shift error: how many grey levels (out of 255) the picture changes by when it should only have moved. Mesh strength: how much of the image's frequency spectrum sits at the cell spacing.

From the table, you can see that while the artifacting is reduced, the learned NCA produces blurrier images (PSNR up to ~2 dB lower, LPIPS worse).

The demo

It runs on your GPU with WebGPU (a recent Chrome, Edge or Safari) and nothing is uploaded. The model is 126 KB. On an M3 Max, 20 seconds is about 4,700 training steps. The pretrained examples were trained offline for 80,000.

Paper

The formal version is an extended abstract at the NeurReps workshop, NeurIPS 2026.

@misc{maa2026interface,
  title  = {Equivariance as an interface property: state-carried positional
            codes in neural cellular automata},
  author = {Maa, Wesley},
  year   = {2026},
  note   = {Extended abstract, NeurIPS 2026 Workshop on Symmetry and Geometry
            in Neural Representations (NeurReps)}
}