Zero-cost abstractions (and where they leak)

"Zero-cost abstraction" does not mean "abstractions are free." It means C++ tries to make unused abstraction machinery cost nothing, and used abstraction machinery compile to code as good as a hand-written lower-level version. Templates, inlining, RAII, std::span, and many STL algorithms can do that. The leak is that some design choices necessarily create runtime facts: allocation, virtual dispatch, type erasure, reference counting, exception metadata, code size, and cache misses.

The reset: zero-cost is a claim about generated code for a concrete abstraction. Verify it at the assembly, allocation, and cache level.

How it really works

C++ has several abstraction mechanisms that disappear well when the optimizer has enough information:

The optimizer is not magic. It needs visibility, aliasing facts, simple control flow, and a target ABI. A small template that takes a lambda can inline into the caller and become the same loop you would have written in C. A std::function parameter usually cannot do that: it type-erases the callable behind an indirect call and may allocate for large captures. Virtual dispatch similarly preserves runtime polymorphism through a vtable call unless the compiler can devirtualize. Exceptions may have near-zero cost on the non-throwing path on some ABIs, but they still influence binary metadata, code layout, and what happens when an exception is thrown.

This is where the "C++ lets you write high-level code" story meets the machine model from earlier branches. The CPU still executes loads, stores, branches, calls, and cache misses. The linker still sees symbols. The allocator still serves heap requests. The ABI still decides how virtual calls, exception handling, and object layout work.

Executable artifact: compare C, templates, type erasure, and virtual dispatch

The runnable demo lives in examples/modern-cpp/zero-cost-abstractions-and-where-they-leak/. It builds a C baseline and a C++ version that uses std::span, a template projection, std::function, and a virtual interface.

cd examples/modern-cpp/zero-cost-abstractions-and-where-they-leak
./run.sh

The C baseline is exactly the low-level loop:

static int square_sum_c(IntSlice slice) {
    int total = 0;
    for (size_t i = 0; i < slice.count; ++i) {
        int value = slice.items[i];
        total += value * value;
    }
    return total;
}

The C++ template version expresses the operation generically:

template <typename T, typename Projection>
static T sum_projected(std::span<const T> values, Projection project) {
    T total{};
    for (T value : values) {
        total += project(value);
    }
    return total;
}

auto square = [](int value) {
    return value * value;
};

At -O2, the template/lambda version has a good chance of becoming the same basic loop as the C function because the callable type and body are visible. The same demo also includes:

static int sum_with_function(std::span<const int> values,
                             const std::function<int(int)> &project);

struct Cost {
    virtual ~Cost() = default;
    virtual int value() const = 0;
};

Those are intentionally not the same kind of abstraction. std::function is type erasure; virtual functions are runtime dispatch. Sometimes that is the correct design. It is just not free in the same way a visible template can be. The script writes c_manual.O2.s and demo.O2.s; compare them locally or paste the functions into Compiler Explorer.

Where the abstraction leaks

• Allocation leaks through. std::vector, std::string, std::make_shared, and type-erased callables can allocate. The abstraction may be clean, but the allocator and cache still matter.
• Dispatch leaks through. Virtual calls, function pointers, and std::function can become indirect calls that block inlining and prediction.
• Code size leaks through. Templates can instantiate many copies. More code can hurt instruction cache even if each specialization is fast.
• Error handling leaks through. Exceptions, RTTI, and unwind metadata affect binary layout and build flags, especially in kernels or freestanding environments.
• Layout leaks through. A neat class hierarchy may scatter objects across the heap. Data-oriented layout may beat object-oriented shape in hot loops.
• Compile time leaks through. Header-heavy templates move work from runtime to build time. That can be the right trade, but it is still a cost.

Failure modes & trade-offs

• Trusting the slogan. "Zero-cost" is not proof. Measure assembly, allocations, cache misses, branch misses, and binary size when it matters.
• Over-abstracting the hot path. A type-erased callback in a cold configuration path is fine. The same callback in a per-packet loop can dominate runtime.
• Forgetting ABI boundaries. Templates do not form stable binary interfaces. Virtual interfaces do, but they commit you to layout and dispatch choices.
• Debug builds lie differently. Inlining and optimization may be absent. Debug iterator checks and abstraction layers can look far more expensive than release code.
• Microbenchmarks can lie too. A tiny loop may inline perfectly alone and behave differently inside a real binary with cache pressure and unpredictable data.

In practice

• Write the clear abstraction first, then inspect. If the generated code is the same, keep the clearer code. If not, decide whether the cost is on the path that matters.
• Prefer compile-time polymorphism in hot generic code. Templates, concepts, and lambdas keep the concrete type visible.
• Use runtime polymorphism when the runtime choice is real. Virtual dispatch and std::function are good tools when you need substitution across separately compiled or dynamically chosen behavior.
• Keep ownership and layout visible. std::span is a view, std::vector is contiguous ownership, std::list is nodes, and std::shared_ptr is reference counting. The type should remind you of the machine cost.
• Use the same tools as C. Compiler Explorer, -S, nm, profilers, sanitizers, and cache-aware thinking still apply.

Connects to: Templates and generic programming · The STL: containers, iterators, algorithms · C vs C++: what you actually gain (and pay) · Optimization: what -O2 does to your code · Data layout and cache-friendliness · Object files and what's inside them · The dynamic loader, relocation, PLT, and GOT

Sources

• C++ Core Guidelines - zero-overhead principle framing and performance/resource-management guidance for modern C++. https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines
• Bjarne Stroustrup - A Tour of C++ - concise overview of the language and standard-library abstractions this branch relies on. https://www.stroustrup.com/tour3.html
• cppreference - Templates - instantiation model that makes compile-time abstraction concrete. https://en.cppreference.com/w/cpp/language/templates
• cppreference - Virtual functions - dynamic dispatch model and when overriding behavior is selected at runtime. https://en.cppreference.com/w/cpp/language/virtual
• cppreference - std::function - type-erased callable wrapper and its callable-storage semantics. https://en.cppreference.com/w/cpp/utility/functional/function
• cppreference - Exceptions - exception handling and stack unwinding model that can affect generated code and binary metadata. https://en.cppreference.com/w/cpp/language/exceptions
• Ulrich Drepper - What Every Programmer Should Know About Memory - cache and memory-system reality behind abstraction costs. https://akkadia.org/drepper/cpumemory.pdf
• Compiler Explorer - inspect whether a specific abstraction optimizes to the expected machine code. https://godbolt.org/