Getting nerdsniped by weird artifacts in neural cellular automata
Wesley Maa
The chameleon below is grown by a neural cellular automaton (NCA): a grid of cells that all follow the same learned ruleset. Imagine Conway's Game of Life but ML-ified. You can click to damage it, plant more seeds, and train your own in this browser in ~20 seconds.
A few months ago I saw a new project on Twitter called Cells to Pixels. It builds on growing NCAs: the cells grow a hidden state, 32 numbers each, and a small decoder reads that state at any point and returns a colour, so a coarse grid of cells can be drawn at any resolution.
I played around with it a lot and kept getting annoyed by this one faint lattice artifact. If you zoom in, it looks like there's a square mesh over everything. Immediately I thought to myself that this must've been a result of some weird sampling the authors did. Which, as it turned out, was absolutely the case!
AI slop summary
Why it happens
In the original paper, the decoder that turns cell state into RGBA is conditioned on a coordinate (u, v) that is sampled from outside the cells. It corresponds to where the pixel should be relative to the current cell. Every cell sees the exact same (u, v) pattern, which creates a strong built-in bias toward repeating the same detail in every cell. That creates a meshing artifact.
It also means the renderer is not translation equivariant. The coordinate is fixed to the grid, so if the cell state shifts by a fraction of a cell, (u, v) does not shift with it, and the picture changes when it should just move.
Implementing a simple fix
Rather than computing the coordinate from the pixel grid, I reserve two dimensions of each cell's internal state for position. Each cell already has 32 values. The decoder used to get all of them plus the outside (u, v). Now it gets 30 of them, and the other two go in the slot where (u, v) used to be. That adds no parameters: the same update rule writes those two values along with the rest.
Nothing forces them to be coordinates, so the cells store whatever helps. You can see for yourself what this position code looks like by setting "show" under the demo to "position code". Red and green are the two values. For example, on the chameleon they aren't a tight coordinate grid but rather form soft patches that vary slowly across the body.
Detail inside a cell comes from blending. A pixel that sits between cells gets a mix of its neighbours' values, so what the decoder sees changes smoothly from cell to cell and never restarts at a border.
A single coordinate for the whole image, fed in from outside, would also stop the repetition. But it would be pinned to the canvas, so the picture still could not move. Values kept in the state travel with the cells, preserving the translation equivariance that lets you more or less copy and paste the learned image.
The mesh goes away. Sliding also becomes exact: move the cells by any whole number of pixels and the picture moves with them, pixel for pixel. That is why a seed planted anywhere in the demo grows a clean copy.
Numbers
| where position comes from | shift error | mesh strength | PSNR (dB) | LPIPS |
|---|---|---|---|---|
| the cells (mine) | 0.0 | 7 × 10−6 | 19.3 | 0.317 |
| grid (u, v) (Cells to Pixels) | 18.9 | 1.98 | 20.4 | 0.211 |
| LIIF | 21.7 | 2.48 | 21.5 | 0.173 |
| LTE, 128 frequencies | 22.2 | 2.45 | 20.4 | 0.173 |
| LTE, 5 frequencies | 20.7 | 2.55 | 19.3 | 0.234 |
| hash grid | 46.5 | n/a | 32.3 | 0.016 |
Medians of three training runs. Shift error: how many grey levels (out of 255) the picture changes by when it should only have moved. Mesh strength: how much of the image's frequency spectrum sits at the cell spacing.
From the table, you can see that while the artifacting is reduced, the learned NCA produces blurrier images (PSNR up to ~2 dB lower, LPIPS worse).
The demo
It runs on your GPU with WebGPU (a recent Chrome, Edge or Safari) and nothing is uploaded. The model is 126 KB. On an M3 Max, 20 seconds is about 4,700 training steps. The pretrained examples were trained offline for 80,000.
Paper
The formal version is an extended abstract at the NeurReps workshop, NeurIPS 2026.
@misc{maa2026interface,
title = {Equivariance as an interface property: state-carried positional
codes in neural cellular automata},
author = {Maa, Wesley},
year = {2026},
note = {Extended abstract, NeurIPS 2026 Workshop on Symmetry and Geometry
in Neural Representations (NeurReps)}
}