Review & Verify Updated 2026-09 10 min read View as Markdown

The performance traps in generated C

In C the compiler is a better optimiser than you are, so the interesting problems are the ones it cannot fix: memory layout, allocation in hot loops, and asking the wrong question.

Generated C is usually written for clarity, which is the right default and occasionally the wrong one. The performance problems that survive review are not the obvious ones — the compiler handles those — they are structural: how memory is laid out, how often it is allocated, and how many times it is copied.

Allocation#

1. malloc inside a hot loop#

The most common real performance bug in generated C, and the one that scales worst.

for (size_t i = 0; i < n; i++) {
    char *buf = malloc(256);            /* n allocations */
    format_row(buf, 256, &rows[i]);
    emit(buf);
    free(buf);
}

Correct: hoist it. One allocation, or none.

char buf[256];                          /* stack — no allocator involved at all */
for (size_t i = 0; i < n; i++) {
    format_row(buf, sizeof buf, &rows[i]);
    emit(buf);
}

Look for: any malloc, calloc, strdup or realloc inside a loop body. Ask whether the buffer can be allocated once outside, or live on the stack.

2. realloc growth one element at a time#

for (size_t i = 0; i < n; i++) {
    arr = realloc(arr, (i + 1) * sizeof *arr);   /* quadratic copying */
    arr[i] = compute(i);
}

Each realloc may copy the whole array. Growing geometrically makes it amortised constant:

if (len == cap) {
    size_t new_cap = cap ? cap * 2 : 16;
    void *p = realloc(arr, new_cap * sizeof *arr);
    if (!p) { free(arr); return -ENOMEM; }
    arr = p; cap = new_cap;
}
arr[len++] = compute(i);

Better still, when you know the count: malloc(n * sizeof *arr) once. And note the temporary — arr = realloc(arr, …) leaks the original on failure, which is failure mode 7.

3. An arena instead of a thousand small frees#

For parse trees, request handling, or anything with a natural lifetime, a bump allocator with one bulk free is dramatically faster than per-node malloc/free, and it removes a whole class of leak.

typedef struct { char *base; size_t cap, used; } arena_t;

static void *arena_alloc(arena_t *a, size_t n, size_t align) {
    size_t off = (a->used + align - 1) & ~(align - 1);
    if (off + n > a->cap) return NULL;
    a->used = off + n;
    return a->base + off;
}
/* teardown is: a->used = 0;  or one free of a->base. */

Memory layout#

4. Array of structs where struct of arrays was wanted#

typedef struct { float x, y, z; char name[64]; int flags; } particle_t;
particle_t parts[100000];

for (size_t i = 0; i < n; i++) parts[i].x += parts[i].vx * dt;

Each iteration pulls an entire ~80-byte struct into cache to touch four bytes. You are using a small fraction of every cache line you fetch.

typedef struct {                        /* struct of arrays */
    float *x, *y, *z, *vx, *vy, *vz;
    char (*name)[64];
} particles_t;

for (size_t i = 0; i < n; i++) p.x[i] += p.vx[i] * dt;   /* dense, prefetchable */

This is the layout change that produces order-of-magnitude differences in numeric code, and no compiler will make it for you. It is only worth doing where you sweep large arrays touching a few fields — for occasional access to whole records, array-of-structs is correct and clearer.

5. Struct padding nobody looked at#

struct rec { char flag; double value; char code; int id; };   /* 24 bytes */
struct rec { double value; int id; char flag, code; };        /* 16 bytes */

Same fields, a third less memory, a third fewer cache lines for an array of them. Order members largest-to-smallest and check:

pahole -C rec ./app          # shows holes and padding explicitly

6. Pointer chasing#

A linked list walk is a dependent load chain: the processor cannot prefetch the next node until the current one arrives. For anything you iterate more than you insert into the middle of, a flat array wins by a wide margin — often 5-10x — regardless of what the complexity analysis says.

Generated C reaches for linked lists because they are the textbook structure. Ask what the actual access pattern is.

Copies#

7. strcpy where a pointer would do#

Generated string handling copies defensively — often copying a substring out of a buffer purely to read it. const char * plus a length avoids the copy entirely:

typedef struct { const char *p; size_t n; } str_t;    /* a view, not a copy */

A string-view type threaded through your parser removes most allocation from it, and is also usually clearer.

8. Passing large structs by value#

void process(config_t cfg);        /* copies the whole struct, per call */
void process(const config_t *cfg); /* one pointer */

Harmless for small structs, real for anything over a couple of cache lines in a frequently-called function.

9. Line-at-a-time I/O on a large file#

fgetc in a loop, or unbuffered read of small chunks. Use a large buffered read, or mmap for a file you will scan repeatedly. Generated file handling defaults to the simplest correct thing, which is not the fastest.

Micro-optimisation that is not#

Modern compilers at -O2 already do these. Generated code sometimes includes them, and they cost readability for nothing:

What actually helps and is worth asking for: restrict on non-aliasing pointer parameters (it unlocks vectorisation the compiler cannot otherwise prove is safe), const where it is true, and static on functions with no external linkage.

gcc -O2 -fopt-info-vec-missed -c hot.c    # why did this loop not vectorise?

Threading#

10. False sharing#

Two threads writing to adjacent counters in the same cache line will fight over it, and the code looks perfectly parallel.

struct { long count; } counters[NTHREADS];              /* all in one cache line */
struct { long count; char pad[64 - sizeof(long)]; }     /* one line each */
     counters[NTHREADS];

Generated parallel code produces the first version routinely, and the symptom is parallel code that is slower than serial. perf c2c finds it.

11. A lock around something atomic#

pthread_mutex around a counter increment where atomic_fetch_add would do. Orders of magnitude cheaper under contention.

What to do, in order#

1. perf stat        -> is it cache, branches, or instruction count?
2. perf record      -> which function?
3. Is that function allocating, copying, or chasing pointers in a loop?
4. Fix the layout or the allocation. Re-measure.
5. Only then consider the kernel itself.

If you are running model inference in C, the cost model is different again — bandwidth-bound rather than compute-bound, with its own set of traps. That is covered in local inference economics.

Common questions#

Should I ask the agent to optimise the code?#

Not upfront. Ask for correct and clear, profile, then optimise the one thing that showed up. Generated "optimised" C tends to be the micro-optimisation table above — less readable for no measurable gain — while missing the layout and allocation issues that actually matter.

Is -O3 worth it over -O2?#

Sometimes, and it needs measuring rather than assuming — -O3 mostly enables more aggressive inlining and vectorisation, which can hurt through code bloat and instruction-cache pressure. Never use -Ofast unless you have deliberately decided to give up IEEE floating-point guarantees.

Do sanitizers change the performance picture?#

Yes, substantially — ASan is roughly 2x and distorts memory behaviour, so never benchmark a sanitized build. Develop with sanitizers on, profile with a separate -O2 build, and keep both in your Makefile so nobody has to remember.