Getting started with pixels, surfaces and SIMD

What is SIMD?

Single Instruction, Multiple Data. This is intended to contrast with the 'normal' programming paradigm (SISD - Single Instruction Single Data) where when you perform an operation between on one set of data at a time e.g.:

result = 1 + 1

Your single instruction (here an addition operation) only added one pair of numbers together. In a SIMD paradigm you could imagine the operation working as in the pseudocode below:

result = [1,2,3,4] + [1,2,3,4]
// result = [2,4,6,8]

where a single add operation works on all our individual pieces of data at the same time. To understand why this is useful and how we use this approach in Pygame CE I'm, going to take a quick swerve to talk about pixels.

Pixels & Surfaces

In SDL (and most graphics libraries I'm familiar with) pixels have a format that describes how the pixel data is structured in memory. Generally the important questions answered by the pixel format is 'How much memory is this pixel going to use?', 'In what order are the colour channels?' and, to a lesser extent, 'Does this pixel have an alpha channel or not?'.

Most of our SIMD functions only operate on 32bit pixel data - this is partly because almost all pixel data in Pygame CE programs is 32bit - so we get the biggest bang for our buck here, but also because the SIMD paradigm works best on this pixel format because of how the available hardware instructions line up with the pixel colour channel data size.

In a 32bit pixel each colour channel gets 8bits of data devoted to it. If you are now struggling to recall the binary number system, or what computer memory looks like - 8 bits means you get 8 0s or 1s to work with for each channel:

Red   = 1111 1111 // 255 red
Green = 0000 0000 // 0 green
Blue  = 0100 0000 // 64 blue
Alpha = 1100 1000 // 200 alpha

With Pygame CE surfaces we cannot be certain exactly which order our pixels will be in, sadly in the dawn of computing there was no agreement on the best ordering of pixel channels; now many orderings exist and the same code can even produce different surfaces depending on the platform it runs on. So, to manage this we have to carefully structure our operations on surfaces to be 'channel ordering aware' - at least when this matters to the effect we are trying to create.

SDL (and thus Pygame's) Surface's 32bit pixels are stored in memory as a single block, with each pixel's 32bits following directly after the pixel before it. If we were to represent the pixel above as hex and imagine 8 of identical colour in a surface, they might look like the below in memory:

FF004014FF004014FF004014FF004014FF004014FF004014FF004014FF004014
//FF004014|FF004014|FF004014|FF004014|FF004014|FF004014|FF004014|FF004014

That's the basics of pixels and Surfaces under the hood.

Bitwise operators in C

Now that we know that our standard pixel's channels are stored in four chunks of 8 bits, it is also handy to have some tools to manipulate those 32 bits. This is where C's bitwise operators step in:

// The bitwise operators
& // Bitwise AND
| // Bitwise OR
^ // Bitwise XOR
~ // Bitwise complement (I like to think of it as an invert)
<< // Bitshift left
>> // Bitshift right

With these we can combine channel masks together, invert masks, shift a pixel along by one channel to add channels together and lots of other tricks. There are lots of explainers out there on the internet about how each of these operators work, but you can go far wrong if you imagine the operation described occurring with each and every bit of the data (e.g. 1010 & 1101 = 1000).

These bitwise operations also make an appearance in SIMD format later on.