NEON SIMD Vector Programming on Arm64
Introduction
Raspberry Pi 3, 4, and 5 support NEON SIMD instructions, which operate on multiple data elements in one instruction. This tutorial covers vector registers and examples for numeric and image-processing code. Measure each routine against its scalar version.
In this tutorial, you will learn the fundamentals of NEON SIMD programming, understand its register architecture, and write a practical assembly program that speeds up an image grayscale conversion task.
NEON Register Architecture
NEON provides a dedicated set of registers that are separate from the general-purpose registers (x0-x30).
Vector Registers (v0 - v31)
There are 32 vector registers, named v0 through v31. Each register is 128 bits wide.
These registers can be viewed as containing vectors of smaller data types:
- 16 bytes (8-bit elements):
16b
- 8 halfwords (16-bit elements):
8h
- 4 words (32-bit elements):
4s
- 2 doublewords (64-bit elements):
2d
For example, v0.16b refers to the register v0 treated as a vector of 16 unsigned/signed 8-bit bytes. Similarly, v1.4s refers to v1 treated as a vector of 4 32-bit single-precision floats or integers.
Register Mapping (Scalar Views)
You can also access the lower bits of the vector registers as scalar registers for individual calculations:
- B0-B31: 8-bit scalar
- H0-H31: 16-bit scalar
- S0-S31: 32-bit scalar (commonly used for single-precision floats)
- D0-D31: 64-bit scalar (commonly used for double-precision floats)
Essential NEON Instructions
NEON instructions typically follow a naming convention that indicates the data type and vector layout.
Load and Store
Unlike general-purpose ldr and str, NEON uses structured load and store instructions to read from and write to memory:
| ld1 {v0.16b}, [x0] // Load 16 consecutive bytes from address in x0 into v0
st1 {v0.16b}, [x1] // Store 16 bytes from v0 to address in x1
|
You can also load multiple registers to interleave or structure data:
| ld3 {v0.16b, v1.16b, v2.16b}, [x0] // De-interleave RGB data into separate registers
|
Vector Arithmetic
NEON provides parallel math instructions for vectors:
| fadd v0.4s, v1.4s, v2.4s // Parallel float addition: v0[i] = v1[i] + v2[i] for i=0..3
add v0.8h, v1.8h, v2.8h // Parallel 16-bit integer addition
|
Vector Multiplication and Accumulation
For DSP and matrix operations:
| fmul v0.4s, v1.4s, v2.4s // Parallel float multiplication
fmla v0.4s, v1.4s, v2.4s // Parallel multiply-accumulate: v0 = v0 + (v1 * v2)
|
Practical Project: Image Grayscale Conversion
To see NEON SIMD in action, let's write a function that converts an array of RGB24 image pixels to 8-bit grayscale pixels.
The formula to compute grayscale value $Y$ from Red ($R$), Green ($G$), and Blue ($B$) components is:
$$Y = 0.299R + 0.587G + 0.114B$$
For integer performance, we can approximate this using fixed-point arithmetic:
$$Y = \frac{77R + 150G + 29B}{256} = (77R + 150G + 29B) \gg 8$$
NEON Implementation (grayscale_neon.S)
Create a file named grayscale_neon.S:
| // grayscale_neon.S
// Converts RGB24 pixels to Grayscale using NEON SIMD
.global grayscale_neon
.section .text
.align 4
// Function Signature:
// void grayscale_neon(const uint8_t* src, uint8_t* dest, int num_pixels);
// x0: src pointer (RGBRGB...)
// x1: dest pointer (YYYY...)
// x2: num_pixels
grayscale_neon:
// Coefficients for fixed-point math: R=77, G=150, B=29
// Load coefficients into scalar elements of v3
mov w3, #77
mov w4, #150
mov w5, #29
dup v3.8h, w3 // Duplicate Red coeff to all lanes of v3
dup v4.8h, w4 // Duplicate Green coeff to all lanes of v4
dup v5.8h, w5 // Duplicate Blue coeff to all lanes of v5
loop_pixels:
// Check if we have at least 8 pixels left to process
cmp x2, #8
blt scalar_cleanup // If less than 8, handle remainder with scalar code
// Load 8 RGB pixels (24 bytes total)
// ld3 de-interleaves the RGB triplets into 3 separate registers:
// v0.8b = R0..R7, v1.8b = G0..G7, v2.8b = B0..B7
ld3 {v0.8b, v1.8b, v2.8b}, [x0], #24
// Widening shift: convert 8-bit unsigned integers to 16-bit
uxtl v0.8h, v0.8b // R0..R7 -> 16-bit in v0
uxtl v1.8h, v1.8b // G0..G7 -> 16-bit in v1
uxtl v2.8h, v2.8b // B0..B7 -> 16-bit in v2
// Multiply Red by 77
umul v6.8h, v0.8h, v3.8h // v6 = R * 77
// Multiply-Accumulate Green: v6 += G * 150
umla v6.8h, v1.8h, v4.8h
// Multiply-Accumulate Blue: v6 += B * 29
umla v6.8h, v2.8h, v5.8h
// Divide by 256 using right shift by 8 bits
ushr v6.8h, v6.8h, #8 // v6 = v6 >> 8
// Narrow 16-bit back to 8-bit bytes
xtn v6.8b, v6.8h // Convert v6.8h to v6.8b
// Store 8 grayscale bytes to dest
st1 {v6.8b}, [x1], #8
// Decrement pixel counter by 8
sub x2, x2, #8
b loop_pixels
scalar_cleanup:
// Process remaining pixels individually (if any)
cbz x2, done
scalar_loop:
ldrb w6, [x0], #1 // Read R
ldrb w7, [x0], #1 // Read G
ldrb w8, [x0], #1 // Read B
mul w6, w6, w3 // R * 77
madd w6, w7, w4, w6 // + G * 150
madd w6, w8, w5, w6 // + B * 29
lsr w6, w6, #8 // >> 8
strb w6, [x1], #1 // Store Y
sub x2, x2, #1
cbnz x2, scalar_loop
done:
ret
|
To evaluate the speedup, we can wrap our NEON assembly function in a C++ benchmark program.
Benchmark Harness (benchmark.cpp)
Create benchmark.cpp:
| #include <iostream>
#include <chrono>
#include <vector>
#include <random>
extern "C" {
void grayscale_neon(const uint8_t* src, uint8_t* dest, int num_pixels);
}
// Standard C++ implementation (Scalar CPU)
void grayscale_cpp(const uint8_t* src, uint8_t* dest, int num_pixels) {
for (int i = 0; i < num_pixels; ++i) {
uint8_t r = src[i * 3 + 0];
uint8_t g = src[i * 3 + 1];
uint8_t b = src[i * 3 + 2];
dest[i] = (77 * r + 150 * g + 29 * b) >> 8;
}
}
int main() {
const int num_pixels = 1920 * 1080; // Full HD resolution (2.07 Million Pixels)
std::vector<uint8_t> src(num_pixels * 3);
std::vector<uint8_t> dest_cpp(num_pixels);
std::vector<uint8_t> dest_neon(num_pixels);
// Initialize source with random color values
std::mt19937 rng(42);
for (auto& val : src) {
val = rng() % 256;
}
std::cout << "Benchmarking RGB24 to Grayscale on " << num_pixels << " pixels..." << std::endl;
// 1. C++ Scalar Benchmark
auto start = std::chrono::high_resolution_clock::now();
grayscale_cpp(src.data(), dest_cpp.data(), num_pixels);
auto end = std::chrono::high_resolution_clock::now();
std::chrono::duration<double, std::milli> cpp_time = end - start;
std::cout << "C++ Scalar Time: " << cpp_time.count() << " ms" << std::endl;
// 2. NEON Assembly Benchmark
start = std::chrono::high_resolution_clock::now();
grayscale_neon(src.data(), dest_neon.data(), num_pixels);
end = std::chrono::high_resolution_clock::now();
std::chrono::duration<double, std::milli> neon_time = end - start;
std::cout << "NEON SIMD Time: " << neon_time.count() << " ms" << std::endl;
// Verify correctness
bool correct = true;
for (int i = 0; i < num_pixels; ++i) {
if (dest_cpp[i] != dest_neon[i]) {
correct = false;
std::cout << "Mismatch at index " << i
<< ": C++=" << (int)dest_cpp[i]
<< ", NEON=" << (int)dest_neon[i] << std::endl;
break;
}
}
if (correct) {
std::cout << "Verification PASSED!" << std::endl;
std::cout << "Speedup Factor: " << (cpp_time.count() / neon_time.count()) << "x" << std::endl;
} else {
std::cout << "Verification FAILED!" << std::endl;
}
return 0;
}
|
Compiling and Running
Compile the code leveraging the native compiler optimizations on your Raspberry Pi:
| # Compile assembly and C++ helper
g++ -O3 -o benchmark benchmark.cpp grayscale_neon.S
# Run the benchmark
./benchmark
|
Typical Results
On a Raspberry Pi 4, you can expect results similar to the following:
| Benchmarking RGB24 to Grayscale on 2073600 pixels...
C++ Scalar Time: 18.2 ms
NEON SIMD Time: 4.3 ms
Verification PASSED!
Speedup Factor: 4.23x
|
With NEON SIMD, the execution time is reduced by over 75%, demonstrating the massive performance gains achievable through vectorization.
Conclusion
NEON SIMD is an indispensable tool for low-level performance optimization on ARM-based devices. By vectorizing calculations, you bypass the CPU memory latency and instruction dispatch bottlenecks of single-threaded loops.
The next tutorial covers Inline Assembly in C++, including operands, constraints, and clobbered registers.