Follow-up to: nilmamano.com/posts/cnn-vs-transformer-for-the-wall-game-2546007
I went back and retrained 36 generations of the ResNet for a fair comparison with the transformer.

This plot: everything equal, only different model architectures.

Now cleanly: Transformer (8M params) > ResNet (7M params)

One dimension in which the ResNet wins is inference speed, making each gen about 3x faster.