SIMD: SSE, AVX, NEON, SVE, Intrinsics

This chapter covers SIMD (single instruction, multiple data): the instruction sets (SSE, AVX, NEON, SVE) and how to use them from C via intrinsics.

Why This Matters

SIMD processes multiple data elements per instruction, giving 2x–16x speedups on data-parallel work. It is the backbone of graphics, audio, numerical, and machine-learning kernels.

Prerequisites

• c.cpu.5 — architectures.
• c.opt.3 — auto-vectorization.

Core Concept

SIMD instructions operate on vector registers holding multiple lanes of the same type (e.g., 4 floats in an SSE register, 8 in AVX). A single instruction adds/multiplies all lanes in parallel.

Using SIMD from C

Auto-vectorization

The compiler may vectorize simple, dependency-free loops (c.opt.3). Add restrict and aligned, contiguous access to help it.

Intrinsics

Intrinsics map directly to SIMD instructions:

#include <immintrin.h>   /* x86 SIMD */

void add4(float *a, float *b, float *c)
{
    __m128 va = _mm_loadu_ps(a);
    __m128 vb = _mm_loadu_ps(b);
    __m128 vc = _mm_add_ps(va, vb);
    _mm_storeu_ps(c, vc);
}

ARM NEON uses <arm_neon.h> with float32x4_t and vaddq_f32.

Examples

SSE sum of four floats

#include <immintrin.h>

float sum4(const float *a)
{
    __m128 v = _mm_loadu_ps(a);
    v = _mm_hadd_ps(v, v);
    v = _mm_hadd_ps(v, v);
    return _mm_cvtss_f32(v);
}

Alignment

Aligned loads (_mm_load_ps) require 16-byte alignment; unaligned loads (_mm_loadu_ps) work anywhere but may be slower on older hardware.

How It Works

The CPU has wide vector registers (XMM/YMM/ZMM on x86, V/Q on ARM). A SIMD instruction performs the same operation on each lane simultaneously. Intrinsics compile to these instructions with the right register allocation.

Variations

Masked and gather/scatter (AVX-512/SVE)

Advanced SIMD supports predication (masked lanes) and gather/scatter (loading non-contiguous elements), useful for sparse/irregular data.

Portable SIMD wrappers

Libraries or _Generic can abstract over SSE/NEON, but raw intrinsics are architecture-specific.

Common Mistakes

• Mixing aligned and unaligned loads incorrectly (UB if misaligned with

_mm_load_ps).

• Assuming all target CPUs support AVX-512 (runtime dispatch needed).
• Overlooking that auto-vectorization often suffices.

Undefined Behavior

• Using an aligned load on an unaligned address. VERIFIED
• Reading past the end of an array (the intrinsics do not bounds-check).

Portability

• Intrinsics are architecture-specific. Isolate them behind feature detection

or use a portable wrapper.

Under the Hood

The compiler lowers intrinsics to the matching instruction. Vectorization requires data-parallel, dependency-free loops and often alignment/restrict.

Practical Usage

• First try auto-vectorization; write intrinsics only where it fails.
• Use restrict and SoA layout to enable vectorization.
• Check CPU features at runtime (__builtin_cpu_supports) before using

AVX/AVX-512.

Exercises

1. Write an SSE function to add two float arrays and verify results. 2. Vectorize a dot product with intrinsics. 3. Compare auto-vectorized vs. intrinsics code for the same loop.

Deep Challenge

Implement a vectorized matrix-vector multiply (float) using AVX intrinsics, handling arbitrary sizes with a scalar remainder. Explain your data layout and alignment choices.

Related Concepts

• c.opt.3 — auto-vectorization.
• c.cpu.5 — architectures.
• c.perf.2 — SoA layout.

References

• Intel Intrinsics Guide, ARM NEON/SVE docs.

Verification

• SIMD instruction sets and intrinsic behavior are hardware/compiler-specific.

HARDWARE/COMPILER-SPECIFIC

• No example was executed during generation unless noted. `Execution not

verified.`

Progress

• [ ] Read
• [ ] Understand
• [ ] Complete examples
• [ ] Complete exercises
• [ ] Complete deep challenge

Concept checkboxes

• [ ] SSE/AVX/NEON/SVE
• [ ] Intrinsics
• [ ] Aligned vs. unaligned loads
• [ ] Auto-vectorization vs. manual

Mastery levels