NEON SIMD Vector Programming on Arm64
Introduction
Modern Raspberry Pi models (Raspberry Pi 3, 4, and 5) feature powerful ARM Cortex-A processors equipped with NEON technology. NEON is an advanced Single Instruction Multiple Data (SIMD) architecture extension designed to accelerate multimedia, signal processing, and mathematical computations. By processing multiple data elements in parallel within a single instruction cycle, NEON can dramatically boost the performance of critical algorithms, such as image processing, audio filtering, and matrix multiplication.
In this tutorial, you will learn the fundamentals of NEON SIMD programming, understand its register architecture, and write a practical assembly program that speeds up an image grayscale conversion task.
NEON Register Architecture
NEON provides a dedicated set of registers that are separate from the general-purpose registers (x0-x30).
Vector Registers (v0 - v31)
There are 32 vector registers, named v0 through v31. Each register is 128 bits wide.
These registers can be viewed as containing vectors of smaller data types:
- 16 bytes (8-bit elements):
16b
- 8 halfwords (16-bit elements):
8h
- 4 words (32-bit elements):
4s
- 2 doublewords (64-bit elements):
2d
For example, v0.16b refers to the register v0 treated as a vector of 16 unsigned/signed 8-bit bytes. Similarly, v1.4s refers to v1 treated as a vector of 4 32-bit single-precision floats or integers.
Register Mapping (Scalar Views)
You can also access the lower bits of the vector registers as scalar registers for individual calculations:
- B0-B31: 8-bit scalar
- H0-H31: 16-bit scalar
- S0-S31: 32-bit scalar (commonly used for single-precision floats)
- D0-D31: 64-bit scalar (commonly used for double-precision floats)
Essential NEON Instructions
NEON instructions typically follow a naming convention that indicates the data type and vector layout.
Load and Store
Unlike general-purpose ldr and str, NEON uses structured load and store instructions to read from and write to memory:
| ld1 {v0.16b}, [x0] // Load 16 consecutive bytes from address in x0 into v0
st1 {v0.16b}, [x1] // Store 16 bytes from v0 to address in x1
|
You can also load multiple registers to interleave or structure data:
| ld3 {v0.16b, v1.16b, v2.16b}, [x0] // De-interleave RGB data into separate registers
|
Vector Arithmetic
NEON provides parallel math instructions for vectors:
| fadd v0.4s, v1.4s, v2.4s // Parallel float addition: v0[i] = v1[i] + v2[i] for i=0..3
add v0.8h, v1.8h, v2.8h // Parallel 16-bit integer addition
|
Vector Multiplication and Accumulation
For DSP and matrix operations:
| fmul v0.4s, v1.4s, v2.4s // Parallel float multiplication
fmla v0.4s, v1.4s, v2.4s // Parallel multiply-accumulate: v0 = v0 + (v1 * v2)
|
Practical Project: Image Grayscale Conversion
To see NEON SIMD in action, let's write a function that converts an array of RGB24 image pixels to 8-bit grayscale pixels.
The formula to compute grayscale value $Y$ from Red ($R$), Green ($G$), and Blue ($B$) components is:
$$Y = 0.299R + 0.587G + 0.114B$$
For integer performance, we can approximate this using fixed-point arithmetic:
$$Y = \frac{77R + 150G + 29B}{256} = (77R + 150G + 29B) \gg 8$$
NEON Implementation (grayscale_neon.S)
Create a file named grayscale_neon.S:
| // grayscale_neon.S
// Converts RGB24 pixels to Grayscale using NEON SIMD
.global grayscale_neon
.section .text
.align 4
// Function Signature:
// void grayscale_neon(const uint8_t* src, uint8_t* dest, int num_pixels);
// x0: src pointer (RGBRGB...)
// x1: dest pointer (YYYY...)
// x2: num_pixels
grayscale_neon:
// Coefficients for fixed-point math: R=77, G=150, B=29
// Load coefficients into scalar elements of v3
mov w3, #77
mov w4, #150
mov w5, #29
dup v3.8h, w3 // Duplicate Red coeff to all lanes of v3
dup v4.8h, w4 // Duplicate Green coeff to all lanes of v4
dup v5.8h, w5 // Duplicate Blue coeff to all lanes of v5
loop_pixels:
// Check if we have at least 8 pixels left to process
cmp x2, #8
blt scalar_cleanup // If less than 8, handle remainder with scalar code
// Load 8 RGB pixels (24 bytes total)
// ld3 de-interleaves the RGB triplets into 3 separate registers:
// v0.8b = R0..R7, v1.8b = G0..G7, v2.8b = B0..B7
ld3 {v0.8b, v1.8b, v2.8b}, [x0], #24
// Widening shift: convert 8-bit unsigned integers to 16-bit
uxtl v0.8h, v0.8b // R0..R7 -> 16-bit in v0
uxtl v1.8h, v1.8b // G0..G7 -> 16-bit in v1
uxtl v2.8h, v2.8b // B0..B7 -> 16-bit in v2
// Multiply Red by 77
umul v6.8h, v0.8h, v3.8h // v6 = R * 77
// Multiply-Accumulate Green: v6 += G * 150
umla v6.8h, v1.8h, v4.8h
// Multiply-Accumulate Blue: v6 += B * 29
umla v6.8h, v2.8h, v5.8h
// Divide by 256 using right shift by 8 bits
ushr v6.8h, v6.8h, #8 // v6 = v6 >> 8
// Narrow 16-bit back to 8-bit bytes
xtn v6.8b, v6.8h // Convert v6.8h to v6.8b
// Store 8 grayscale bytes to dest
st1 {v6.8b}, [x1], #8
// Decrement pixel counter by 8
sub x2, x2, #8
b loop_pixels
scalar_cleanup:
// Process remaining pixels individually (if any)
cbz x2, done
scalar_loop:
ldrb w6, [x0], #1 // Read R
ldrb w7, [x0], #1 // Read G
ldrb w8, [x0], #1 // Read B
mul w6, w6, w3 // R * 77
madd w6, w7, w4, w6 // + G * 150
madd w6, w8, w5, w6 // + B * 29
lsr w6, w6, #8 // >> 8
strb w6, [x1], #1 // Store Y
sub x2, x2, #1
cbnz x2, scalar_loop
done:
ret
|
To evaluate the speedup, we can wrap our NEON assembly function in a C++ benchmark program.
Benchmark Harness (benchmark.cpp)
Create benchmark.cpp:
| #include <iostream>
#include <chrono>
#include <vector>
#include <random>
extern "C" {
void grayscale_neon(const uint8_t* src, uint8_t* dest, int num_pixels);
}
// Standard C++ implementation (Scalar CPU)
void grayscale_cpp(const uint8_t* src, uint8_t* dest, int num_pixels) {
for (int i = 0; i < num_pixels; ++i) {
uint8_t r = src[i * 3 + 0];
uint8_t g = src[i * 3 + 1];
uint8_t b = src[i * 3 + 2];
dest[i] = (77 * r + 150 * g + 29 * b) >> 8;
}
}
int main() {
const int num_pixels = 1920 * 1080; // Full HD resolution (2.07 Million Pixels)
std::vector<uint8_t> src(num_pixels * 3);
std::vector<uint8_t> dest_cpp(num_pixels);
std::vector<uint8_t> dest_neon(num_pixels);
// Initialize source with random color values
std::mt19937 rng(42);
for (auto& val : src) {
val = rng() % 256;
}
std::cout << "Benchmarking RGB24 to Grayscale on " << num_pixels << " pixels..." << std::endl;
// 1. C++ Scalar Benchmark
auto start = std::chrono::high_resolution_clock::now();
grayscale_cpp(src.data(), dest_cpp.data(), num_pixels);
auto end = std::chrono::high_resolution_clock::now();
std::chrono::duration<double, std::milli> cpp_time = end - start;
std::cout << "C++ Scalar Time: " << cpp_time.count() << " ms" << std::endl;
// 2. NEON Assembly Benchmark
start = std::chrono::high_resolution_clock::now();
grayscale_neon(src.data(), dest_neon.data(), num_pixels);
end = std::chrono::high_resolution_clock::now();
std::chrono::duration<double, std::milli> neon_time = end - start;
std::cout << "NEON SIMD Time: " << neon_time.count() << " ms" << std::endl;
// Verify correctness
bool correct = true;
for (int i = 0; i < num_pixels; ++i) {
if (dest_cpp[i] != dest_neon[i]) {
correct = false;
std::cout << "Mismatch at index " << i
<< ": C++=" << (int)dest_cpp[i]
<< ", NEON=" << (int)dest_neon[i] << std::endl;
break;
}
}
if (correct) {
std::cout << "Verification PASSED!" << std::endl;
std::cout << "Speedup Factor: " << (cpp_time.count() / neon_time.count()) << "x" << std::endl;
} else {
std::cout << "Verification FAILED!" << std::endl;
}
return 0;
}
|
Compiling and Running
Compile the code leveraging the native compiler optimizations on your Raspberry Pi:
| # Compile assembly and C++ helper
g++ -O3 -o benchmark benchmark.cpp grayscale_neon.S
# Run the benchmark
./benchmark
|
Typical Results
On a Raspberry Pi 4, you can expect results similar to the following:
| Benchmarking RGB24 to Grayscale on 2073600 pixels...
C++ Scalar Time: 18.2 ms
NEON SIMD Time: 4.3 ms
Verification PASSED!
Speedup Factor: 4.23x
|
With NEON SIMD, the execution time is reduced by over 75%, demonstrating the massive performance gains achievable through vectorization.
Conclusion
NEON SIMD is an indispensable tool for low-level performance optimization on ARM-based devices. By vectorizing calculations, you bypass the CPU memory latency and instruction dispatch bottlenecks of single-threaded loops.
In the next tutorial, we will explore Inline Assembly in C++ to learn how you can embed critical assembly instructions directly within your C++ source files for seamless performance optimizations.