#pragma GCC system_header // Overview / Examples / API / FAQ / Resources

perf: C++23 Performance library

Performance is not a number!

Overview

Single header/module performance library that combines the power of:
c++23, linux/perf/intel_pt, llvm/mca, gnuplot/sixel, ...

Features

Benchmarking, Tuning, Profiling, Tracing, Analyzing

name	description
perf::info::*	compiler, cpu, memory, sys, proc, bin
perf::core::*	code, compiler, cpu, memory
perf::time::timer	steady_clock, cpu, thread, real, monotonic, tsc
perf::stat::counter	perf stat -e
perf::stat::sampler	perf record -e / perf mem record
perf::trace::tracer	perf record -e intel_pt/
perf::mc::disassembler	assembly, address, encoding, size, uops, latency, rthroughput, may_load, may_store, has_side_effects, branch::*, source
perf::mca::analyzer	timeline, resource_pressure, bottleneck
perf::runner	baseline, latency, throughput
perf::plot::*	hist, box, bar, line, ecdf
perf::*	log, json, report, annotate

Requirements

Minimal

• (clang-16+ | gcc-12+) / c++23+

Optimal (recommended)

• (clang-19+ | gcc-13+) / c++23+
  • llvm-19+ - apt-get install llvm-dev
• linux-6.x+
  • perf_event_open - apt-get install linux-tools-common
• intel-12th+ with PEBS, IPT support
  • libipt - apt-get install libipt-dev
• terminal with sixel support
  • gnuplot - apt-get install gnuplot

Dockerfile

Optional

• gh - apt-get install gh
• prof - https://github.com/qlibs/prof
  • linux-perf - apt get install linux-tools-common
  • intel-vtune - apt get install intel-oneapi-vtune
  • amd-uprof - https://www.amd.com/en/developer/uprof.html#downloads
  • gperftools - apt get install google-perftools
  • llvm-xray - apt-get install llvm
  • callgrind - apt-get install valgrind
• uefi - https://github.com/qlibs/uefi
• ut - https://github.com/qlibs/ut

Dockerfile

Examples

Info/Core

Profiling

Tracing

Analyzing

Plotting

Benchmarking

Miscellaneous

API

Configuration

/**
 * PERF version # https://semver.org
 */
#define PERF (MAJOR, MINOR, PATCH) // ex. (1, 0, 0)

/**
 * GNU # default: deduced based on `__GNUC__`
 * - 0 not compatible
 * - 1 compatible
 */
#define PERF_GNU 0/1

/**
 * Linux # default: deduced based on `__linux__` and `perf_event_open.h`
 * - 0 not supported
 * - 1 supported
 */
#define PERF_LINUX 0/1

/**
 * LLVM # default: deduced based on `llvm-dev` headers
 * - 0 not supported
 * - 1 supported
 */
#define PERF_LLVM 0/1

/**
 * Intel Processor Trace # default: deduced based on `<intel_pt.h>` header
 * - 0 not supported
 * - 1 supported
 */
#define PERF_INTEL 0/1

/**
 * Output support # default: 1
 * - 0 no output support compiled in
 * - 1 output supported (`log, json, report, annotate, plot`)
 */
#define PERF_OUT 0/1

/**
 * tests # default: not-defined
 * - defined:     disables all compile-time, run-time tests
 * - not-defined: compile-time tests executed,
 *                run-time tests available by `test::run()` API
 */
#define NTEST

/**
 * gnuplot terminal # see `gnuplot -> set terminal` # default: 'sixel'
 * - 'sixel'                  # console image # https://www.arewesixelyet.com
 * - 'wxt'                    # popup window
 * - 'dumb size 150,25 ansi'  # console with colors
 * - 'dumb size 80,25'        # console
 */
ENV:PERF_PLOT_TERM

/**
 * style # default: dark
 * - light
 * - dark
 */
ENV:PERF_PLOT_STYLE

Synopsis

• https://github.com/qlibs/perf/blob/main/perf.cppm

FAQ

Setup

• How to setup docker?

  Dockerfile

  docker build -t perf .

  docker run \
    -it \
    --privileged \
    --network=host \
    -e DISPLAY=${DISPLAY} \
    -v ${PWD}:${PWD} \
    -w ${PWD} \
    perf

• How to build depenencies?

  apt-get install linux-tools-common
  apt-get install llvm-dev
  apt-get install libipt-dev
  apt-get install gnuplot

• How to setup linux performance counters?

  setup.sh

  .github/scripts/setup.sh --perf # --rdpmc --max-sample-rate 10000

  linux

  sudo mount -o remount,mode=755 /sys/kernel/debug
  sudo mount -o remount,mode=755 /sys/kernel/debug/tracing
  sudo chown `whoami` /sys/kernel/debug/tracing/uprobe_events
  sudo chmod a+rw /sys/kernel/debug/tracing/uprobe_events
  echo 0 | sudo tee /proc/sys/kernel/kptr_restrict
  echo -1 | sudo tee /proc/sys/kernel/perf_event_paranoid
  echo 1000 | sudo tee /proc/sys/kernel/perf_event_max_sample_rate

  echo 2 | sudo tee /sys/devices/cpu_core/rdpmc

• How to find out which performance events are supported by the cpu?

  https://perfmon-events.intel.com

  perf list

• How to reduce execution variaility?

  tune.sh

  .github/scripts/tune.sh

  pyperf

  pip3 install pyperf

  sudo pyperf system tune
  sudo pyperf system show
  sudo pyperf system reset

  linux

  # Set Process CPU Affinity (apt install util-linux)
  taskset -c 0 ./a.out
  
  # Set Process Scheduling Priority (apt install coreutils)
  nice -n -20 taskset -c 0 ./a.out # -20..19 (most..less favorable to the process)
  
  # Disable CPU Frequency Scaling (apt install cpufrequtils)
  sudo cpupower frequency-set --governor performance
  # cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
  
  # Disable Address Space Randomization
  echo 0 > /proc/sys/kernel/randomize_va_space
  
  # Disable Processor Boosting
  echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost
  
  # Disable Turbo Mode
  echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo
  
  # Disable Hyperthreading/SMT
  echo off | sudo tee /sys/devices/system/cpu/smt/control
  
  # Restrict memory to a single socket
  numactl -m 0 -N 0 ./a.out
  
  # Enable Huge Pages
  sudo numactl --cpunodebind=1 --membind=1 hugeadm \
    --obey-mempolicy --pool-pages-min=1G:64
  sudo hugeadm --create-mounts

  boot / grub

  # Enable Kernel Mode Task-Isolation (https://lwn.net/Articles/816298)
  isolcpus=<cpu number>,...,<cpu number> # cat /sys/devices/system/cpu/isolated
  
  # Disable P-states and C-states
  idle=pool intel_pstate=disable intel_idle.max_cstate=0 processor.max_cstate=1 # cat /sys/devices/system/cpu/intel_pstate/status
  
  # Disable NMI watchdog
  nmi_watchdog=0 # cat /proc/sys/kernel/nmi_watchdog

  https://github.com/qlibs/uefi

Usage

• How to use perf with modules?

  clang++-20 -std=c++23 -O3 -I. --precompile perf.cppm
  clang++-20 -std=c++23 -O3 -fprebuilt-module-path=. perf.pcm *.cpp -lLLVM-18 -lipt

• What is prevent_elision and when it`s needed?

  Optimizing compiler may elide code completely if its considered as not needed - doesnt have side effects. prevent_elision enforces that the code wont be elided by the compiler

  verify(perf::compiler::is_elided([] { }));
  verify(perf::compiler::is_elided([] { auto value = 4 + 2; }));
  verify(perf::compiler::is_elided([] { int i{}; i++; }));
  verify(not perf::compiler::is_elided([&] { i++; }));
  verify(not perf::compiler::is_elided([] { static int i; i++; }));
  verify(not perf::compiler::is_elided([=] {
    int i{};
    perf::compiler::prevent_elision(i++);
  }));

• How to change assembly syntax?

  perf::llvm llvm{
    {.syntax = perf::arch::syntax::att} // default: intel
  };

• How to disassemble for a different platform?

  perf::llvm llvm{
    .triple = "x86_64-pc-linux-gnu" // see `llvm-llc` for details
  };

• How to write custom profiler?

  struct profiler {
    constexpr auto start();
    constexpr auto stop();
    [[nodiscard]] constexpr auto *operator() const;
  };

  static_assert(perf::profiler_like<profiler>);

• How to integrate with unit-testing framework?

  import perf;
  import ut; // https://github.com/qlibs/ut
  
  int main() {
    "benchmark"_test = [] {
      // ...
    };
  }

• Which terminal can display images?

  Any terminal with sixel support - https://www.arewesixelyet.com

• How to plot on the server without sixel?

  PERF_PLOT_TERM='dumb' ./a.out
  PERF_PLOT_TERM='dumb size 80,25' ./a.out
  PERF_PLOT_TERM='dumb size 150,25 ansi' ./a.out

• How to plot using windows-based charts?

  PERF_PLOT_TERM='wxt' ./a.out

• How to change plots style?

  PERF_PLOT_STYLE='dark' ./perf # default
  PERF_PLOT_STYLE='light' ./perf

• How to save plots?

  perf::plot::gnuplot plt{{.term = "png"}};
  plt.send("set output 'output.png'");
  perf::plot::bar(plt, ...);

• How to export results?

  export.sh

  ./a.out 2>&1 | .github/scripts/export.sh markdown > results.md
  ./a.out 2>&1 | .github/scripts/export.sh notebook > results.ipynb
  ./a.out 2>&1 | .github/scripts/export.sh html > results.html

• How to share results?

  gh - apt-get install gh

  # https://jbt.github.io/markdown-editor
  gh gist create --public --web results.md

  # https://jupyter.org
  gh gist create --public --web results.ipynb

  # https://htmlpreview.github.io
  gh gist create --public --web results.html

• How perf tests are working?

  compile-time tests are executed upon include/import (enabled by default)
  run-time/sanity check tests can be executed at run-time

  int main() {
    perf::test::run({.verbose = true});
  }

  Tests can be disabled with -DNTEST (not recommended)

  $CXX -DNTEST ... # tests will NOT be compiled in

  #ifndef NTEST
  "perf"_suite = [] {
    "run-time and compile-time"_test = [] constexpr {
      expect(3 == accumulate({1, 2, 3}, 0));
    };
  
    "run-time only"_test = [] mutable {
      expect(std::rand() >= 0);
    };
  
    "compile-time"_test = [] consteval {
      expect(sizeof(int) == sizeof(0));
    };
  };
  #endif

Performance

• Latency vs. Throughput?

  Latency is the time it takes for a single operation to complete (ns)
  Throughput is the total number of operations or tasks completed in a given amount of time (op/s)
  

• What are performance compilation flags?

  -O1                     # optimizations (O1) [0]
  -O2                     # optimizations (O1 + O2) [0]
  -O3                     # [unsafe] optimizations (O1 + O2 + O3) [0]
  -DNDEBUG                # disables asserts, etc.
  -march=native           # defines architecture [1]
  -ffast-math             # [unsafe] faster but non-conforming math [2]
  -g                      # debug symbols
  -fno-omit-frame-pointer # always keep the frame pointer in a register
  -fcf-protection=none    # [unsafe] stops emmitting `endbr64`

  [0] https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
  [1] https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html
  [2] https://gcc.gnu.org/wiki/FloatingPointMath
  

• What are performance compiler attributes?

  // target
  [[gnu::target("avx2")]]
  [[gnu::target("bmi")]]
  
  // optimize
  [[gnu::optimize("O3")]
  [[gnu::optimize("ffast-math")]

  https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html

• What is Top-Down Microarchitecture Analysis method?

  https://www.intel.com/content/www/us/en/docs/vtune-profiler/cookbook/2023-0/top-down-microarchitecture-analysis-method.html
  https://github.com/andikleen/pmu-tools/wiki/toplev-manual
  

• What is Jupyter Notebook?

  Jupter Notebook can be used for data analysis (python)

  # apt install jupyter
  jupyter notebook -ip 0.0.0.0 --no-browser notebook.ipynb

• What are different benchmarking workflows?

  Dev (cpp -> ./a.out)
    (+) easy editing and integration with exising tooling
    (+) output to the console (plots with sixel)
    (+) easy to share via gist/markdown
    (+) can assert and verify expectations
    (+) can be executed in headless mode on the server without UI
    (-) harder analysis than in python
    (-) single compiler workflow
  

  Research (cpp -> ./a.out (json) -> notebook/python)
    (+) powerful data analysis (python)
    (+) can run different compilers/options
    (+) easy to share via jupyter notebook
    (+) can be used on kaggle/google colab
    (-) cpp is not well supported by notebooks but building/running works fine
    (~) usually requires running through a browser
    (-) not suited for assembly analysis
    (-) might be slow with a lot of data
  

• How to avoid benchmarking pitfalls?

  WHERE (cpu, os effects)
  WHAT (latency vs. throughput)
  HOW (statistical, realistic, structured/top-down, visual)
  WHY (understanding, analysis, verfication, sharing)
  

Resources

Specs

• Manuals

  • Intel - https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html
  • AMD - https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/programmer-references/40332.pdf
  • ARM - https://developer.arm.com/documentation/ddi0487/latest
  • Apple - https://developer.apple.com/documentation/apple-silicon/cpu-optimization-guide

• Metrics

  Speed of light ............................ ~1 foot/ns
  L1 cache reference ......................... 0.5 ns
  Branch mispredict ............................ 5 ns
  L2 cache reference ........................... 7 ns
  Mutex lock/unlock ........................... 25 ns
  Main memory reference ...................... 100 ns
  Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
  SSD random read ........................ 150,000 ns  = 150 µs
  Read 1 MB sequentially from memory ..... 250,000 ns  = 250 µs
  Round trip within same datacenter ...... 500,000 ns  = 0.5 ms
  Read 1 MB sequentially from SSD .....  1,000,000 ns  =   1 ms
  Read 1 MB sequentially from disk .... 20,000,000 ns  =  20 ms
  Send packet CA->UK->CA ....          150,000,000 ns  = 150 ms

  # System topology (apt install hwloc)
  lstopo # lstopo-no-graphics
  
  # CPU info (apt install util-linux)
  lscpu | grep -E ^CPU|^Model|^Core|^Socket|^Thread
  
  # Cache info
  lscpu | grep cache
  getconf -a | grep CACHE_LINESIZE
  
  # Numa nodes
  lscpu | grep -E ^NUMA
  
  # Huge pages
  cat /proc/meminfo | grep -i huge

  • Latency, Throughput, and Port Usage Information - https://uops.info
  • Latency, Memory Latency and CPUID dumps - http://instlatx64.atw.hu
  • CPUID data repository - https://x86-cpuid.org
  • Instruction Reference - https://www.felixcloutier.com/x86
  • Processor Information - https://sandpile.org
  • Opcode and Instruction Reference - http://ref.x86asm.net/
  • SIMD Instruction List - https://www.officedaytime.com/simd512e
  • Instruction Discovery And Analysis - https://explore.liblisa.nl
  • Intrinsics Cheatsheet - https://db.in.tum.de/~finis/x86-intrin-cheatsheet-v2.1.pdf
  • Instruction Matrix - https://github.com/google/highway/blob/master/g3doc/instruction_matrix.pdf
  • Memory Latency Data - https://chipsandcheese.com/memory-latency-data
  • Core To Core Latency - https://github.com/nviennot/core-to-core-latency
  • Operation Costs in CPU Clock Cycles - http://ithare.com/infographics-operation-costs-in-cpu-clock-cycles
  • Microarchitecture Metrics - https://dougallj.github.io/applecpu/firestorm.html
  • Microarchitecture Cheatsheet - https://docs.google.com/spreadsheets/d/18ln8SKIGRK5_6NymgdB9oLbTJCFwx0iFI-vUs6WFyuE
  • Top-Down Metrics - https://github.com/intel/perfmon/blob/main/TMA_Metrics-full.xlsx
  • Performance Monitoring Events - https://perfmon-events.intel.com
  • Performance Monitor Counters - https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/programmer-references/58550-0.01.pdf

• Books / Publications

  • Optimizing Software in C++: An Optimization Guide for Windows, Linux and Mac platforms - https://www.agner.org/optimize/optimizing_cpp.pdf
  • Optimizing Subroutines in Assembly Language: An Optimization Guide for x86 platforms - https://www.agner.org/optimize/optimizing_assembly.pdf
  • The Microarchitecture of Intel, AMD and VIA CPUs: An Optimization Guide for Assembly programmers and compiler makers - https://www.agner.org/optimize/microarchitecture.pdf
  • Instruction Tables: Lists of instruction latencies, throughputs and micro-operation breakdowns for Intel, AMD and VIA CPUs - https://www.agner.org/optimize/instruction_tables.pdf
  • Calling Conventions for different C++ compilers and operating systems - https://www.agner.org/optimize/calling_conventions.pdf
  • What Every Programmer Should Know About Memory - https://www.akkadia.org/drepper/cpumemory.pdf
  • Performance Analysis and Tuning on Modern CPUs - https://github.com/dendibakh/perf-book/releases
  • Algorithms for Modern Hardware - https://en.algorithmica.org/hpc
  • The Art of Writing Efficient Programs - https://www.packtpub.com/product/the-art-of-writing-efficient-programs
  • Computer Architecture - https://dl.acm.org/doi/book/10.5555/1999263
  • Is Parallel Programming Hard, And, If So, What Can You Do About It? - https://www.kernel.org/pub/linux/kernel/people/paulmck/perfbook/perfbook.html
  • SIMD for C++ Developers - http://const.me/articles/simd/simd.pdf
  • Hackers Delight - https://doc.lagout.org/security/Hackers%20Delight.pdf
  • Data-Oriented Design - https://www.dataorienteddesign.com/dodbook
  • The Art of Assembly Language - https://www.plantation-productions.com/Webster/www.artofasm.com/Linux/HTML/AoATOC.html
  • Memory Models - https://research.swtch.com/mm
  • High-Precision Branch Target Injection Attacks Exploiting the Indirect Branch Predictor - https://indirector.cpusec.org
  • The Linux Scheduler: a Decade of Wasted Cores - https://people.ece.ubc.ca/sasha/papers/eurosys16-final29.pdf
  • The Tail At Scale - https://www.barroso.org/publications/TheTailAtScale.pdf
  • Producing wrong data without doing anything obviously wrong! - https://dl.acm.org/doi/10.1145/1508284.1508275
  • nanoBench: A Low-Overhead Tool for Running Microbenchmarks on x86 Systems - https://arxiv.org/abs/1911.03282
  • Robust benchmarking in noisy environments - https://arxiv.org/abs/1608.04295
  • Can Seqlocks Get Along With Programming Language Memory Models - https://www.hpl.hp.com/techreports/2012/HPL-2012-68.pdf
  • Cache-Oblivious Algorithms and Data Structures - https://erikdemaine.org/papers/BRICS2002
  • Top-Down Microarchitecture Analysis Method - https://www.intel.com/content/www/us/en/docs/vtune-profiler/cookbook/2023-0/top-down-microarchitecture-analysis-method.html

• Guides

  • RHEL Performance Guide - https://myllynen.github.io/rhel-performance-guide/
  • Performance Analysis Methodology - https://www.brendangregg.com/methodology.html
  • Active Benchmarking - https://www.brendangregg.com/activebenchmarking.html
  • Intel Intrinsics Guide - https://www.intel.com/content/www/us/en/docs/intrinsics-guide/index.html
  • Low Latency Tuning Guide - https://rigtorp.se/low-latency-guide
  • Measuring workloads with Top-down Microarchitecture Analysis - https://github.com/andikleen/pmu-tools/wiki/toplev-manual
  • A CPU research kernel with minimal noise for cycle-by-cycle micro-architectural introspection - https://gamozolabs.github.io/metrology/2019/08/19/sushi_roll.html
  • Envisioning a Simplified Intel Architecture - https://www.intel.com/content/www/us/en/developer/articles/technical/envisioning-future-simplified-architecture.html
  • Apple Silicon Guide - https://github.com/mikeroyal/Apple-Silicon-Guide
  • Monitoring and Managing System Status and Performance - https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/monitoring_and_managing_system_status_and_performance
  • X3 Low Latency Quickstart - https://docs.amd.com/r/en-US/ug1586-onload-user/X3-Low-Latency-Quickstart
  • Modern Microprocessors A 90-Minute Guide! - https://www.lighterra.com/papers/modernmicroprocessors

• Tutorials

  • Performance Ninja Class - https://github.com/dendibakh/perf-ninja
  • Hardware Effects - https://github.com/Kobzol/hardware-effects
  • Performance Tuning - https://github.com/NAThompson/performance_tuning_tutorial
  • Mastering C++ with Google Benchmark - https://ashvardanian.com/posts/google-benchmark
  • Learning to Write Less Slow C, C++, and Assembly Code - https://github.com/ashvardanian/less_slow.cpp
  • A Top-Down method for performance analysis and counters architecture - https://www.researchgate.net/publication/269302126_A_Top-Down_method_for_performance_analysis_and_counters_architecture
  • Measuring Workloads With TopLev - https://github.com/andikleen/pmu-tools/wiki/toplev-manual
  • Bits Of Architecture - https://github.com/CoffeeBeforeArch/bits_of_architecture
  • Bit Twiddling Hacks - https://graphics.stanford.edu/~seander/bithacks.html
  • The Linux Kernel - https://www.kernel.org/doc/html/latest/index.html
  • gcc Optimization - https://wiki.gentoo.org/wiki/GCC_optimization
  • gcc Assembler Syntax - https://www.felixcloutier.com/documents/gcc-asm
  • llvm Scheduling Models - https://github.com/llvm/llvm-project/tree/main/llvm/lib/Target
  • llvm Optimization Passes - https://llvm.org/docs/Passes.html
  • llvm Vectorizers - https://llvm.org/docs/Vectorizers.html
  • linux Referencer - https://elixir.bootlin.com/linux

• Benchmarks

  • Phoronix Test Suite - https://www.phoronix-test-suite.com
  • CPU benchmark - https://www.cpubenchmark.net
  • Geekbench benchmark - https://browser.geekbench.com
  • CPU Benchmarks - https://curiouscoding.nl/posts/cpu-benchmarks
  • 7-Zip LZMA Benchmark - https://www.7-cpu.com

Feeds

• News

  • Linux News - https://lwn.net
  • Chips and Cheese - https://chipsandcheese.com
  • WikiChip - https://wikichip.org
  • CPUID - https://www.cpuid.com/news.html
  • Real World Tech - https://www.realworldtech.com
  • Tom`s Hardware - https://www.tomshardware.com
  • Phoronix - https://www.phoronix.com
  • comp.lang.asm.x86 - https://groups.google.com/g/comp.lang.asm.x86

• Blogs

  • Agner Fog - https://www.agner.org
  • Denis Bakhvalov - https://easyperf.net/blog
  • Daniel Lemire - https://lemire.me/blog
  • Wojciech Mula - http://0x80.pl/articles/index.html
  • Erik Rigtorp - https://rigtorp.se
  • Brendan Gregg - https://brendangregg.com/blog
  • Geoff Langdale - https://branchfree.org
  • Ragnar Groot Koerkamp - https://curiouscoding.nl/posts/cpu-benchmarks
  • Travis Downs - https://travisdowns.github.io
  • Stefanos Baziotis - https://sbaziotis.com/#blog
  • Dmitry Vyukov - https://www.1024cores.net
  • John Farrier - https://johnfarrier.com
  • Tanel Poder - https://tanelpoder.com
  • Arseny Kapoulkine - https://zeux.io
  • Johnny Software Blog - https://johnnysswlab.com
  • JabPerf - https://jabperf.com/blog
  • Number World - http://www.numberworld.org
  • Gamozo Labs - https://gamozolabs.github.io
  • Mechanical Sympathy - https://mechanical-sympathy.blogspot.com
  • Performance Engineering - https://pramodkumbhar.com
  • Performance Matters - https://thume.ca/archive.html
  • Performance Tricks - https://www.performetriks.com/blog
  • Coding Confessions - https://blog.codingconfessions.com
  • The Netflix Tech - https://netflixtechblog.com
  • Cloudflare - https://blog.cloudflare.com

• Lists

  • C++ Links - https://github.com/MattPD/cpplinks
  • Awesome Performance C++ - https://github.com/fenbf/AwesomePerfCpp
  • Awesome Lock Free - https://github.com/rigtorp/awesome-lockfree
  • Awesome SIMD - https://github.com/awesome-simd/awesome-simd
  • Computer, Enhance! - https://www.computerenhance.com
  • Low Latency Trading Insights - https://lucisqr.substack.com

• Miscellaneous

  • Conferences - https://www.p99conf.io, https://supercomputing.org, https://hotchips.org, https://microarch.org
  • Podcasts - https://signals-threads.simplecast.com, https://microarch.club, https://tlbh.it, https://twoscomplement.org
  • C++ Low Latency Group (SG14) - https://github.com/WG21-SG14/SG14

Videos

• Channels

  • Computer Architecture - Onur Mutlu - https://www.youtube.com/@OnurMutluLectures
  • Computer, Enhance - Casey Muratori - https://www.youtube.com/@MollyRocket
  • Assembly - Creel - https://www.youtube.com/c/WhatsACreel
  • EasyPerf - Denis Bakhvalov - https://www.youtube.com/@easyperf3992
  • SIMD algorithms - Denis Yaroshevskiy - https://www.youtube.com/playlist?list=PLYCMvilhmuPEM8DUvY6Wg_jaSFHpmlSBD

• Benchmarking

  • Tuning C++: Benchmarks, and CPUs, and Compilers! Oh My! - Chandler Carruth - https://www.youtube.com/watch?v=nXaxk27zwlk
  • Counting Nanoseconds Microbenchmarking C++ Code - David Gross - https://www.youtube.com/watch?v=Czr5dBfs72U
  • Benchmarking C++ Code - Bryce Adelstein-Lelbach - https://www.youtube.com/watch?v=zWxSZcpeS8Q
  • Benchmarking C++, From video games to algorithmic trading - Alexander Radchenko - https://www.youtube.com/watch?v=7YVMC5v4qCA
  • Tuning C++: Benchmarks, and CPUs, and Compilers! Oh My! - Chandler Carruth - https://www.youtube.com/watch?v=nXaxk27zwlk
  • Going Nowhere Faster - Chandler Carruth - https://www.youtube.com/watch?v=2EWejmkKlxs
  • Measurement and Timing - Performance suiteering of Software Systems - https://www.youtube.com/watch?v=LvX3g45ynu8
  • How NOT to Measure Latency - Gil Tene - https://www.youtube.com/watch?v=lJ8ydIuPFeU

• Profiling

  • From Top-down Microarchitecture Analysis to Structured Performance Optimizationsa - https://cassyni.com/events/YKbqoE4axHCgvQ9vuQq7Cy
  • Coz: finding code that counts with causal profiling - ACM - https://www.youtube.com/watch?v=jE0V-p1odPg
  • Take Advantage for Intel Instrumentation and Tracing Technology for Performance Analysis - https://www.youtube.com/watch?v=1zdVFLajewM&list=PLg-UKERBljNw3_6Q598CS3DE7KqDXjP-d
  • LIKWID Performance Tools - https://www.youtube.com/playlist?list=PLxVedhmuwLq2CqJpAABDMbZG8Whi7pKsk
  • Introduction to the Tracy Profiler - Bartosz Taudul - https://youtu.be/fB5B46lbapc
  • Performance Matters - Emery Berger - https://www.youtube.com/watch?v=r-TLSBdHe1A

• Analyzing

  • Understanding the Performance of code using LLVM-MCA - A. Biagio & M. Davis - https://www.youtube.com/watch?v=Ku2D8bjEGXk
  • LLVM Optimization Remarks - Ofek Shilon - https://www.youtube.com/watch?v=qmEsx4MbKoc

• Optimizing

  • Understanding Compiler Optimization - Chandler Carruth - https://www.youtube.com/watch?v=haQ2cijhvhE

  • Efficiency with Algorithms, Performance with Data Structures - Chandler Carruth - https://www.youtube.com/watch?v=fHNmRkzxHWs

  • Design for Performance - Fedor Pikus - https://www.youtube.com/watch?v=m25p3EtBua4

  • Unlocking Modern CPU Power - Next-Gen C++ Optimization Techniques - https://www.youtube.com/watch?v=wGSSUSeaLgA

  • Branchless Programming in C++ - Fedor Pikus - https://www.youtube.com/watch?v=g-WPhYREFjk

  • CPU design effects - Jakub Beranek - youtube.com/watch?v=ICKIMHCw--Y

  • Fastware - Andrei Alexandrescu - https://www.youtube.com/watch?v=o4-CwDo2zpg

  • Performance Tuning - Matt Godbolt - https://www.youtube.com/watch?v=fV6qYho-XVs

  • Memory & Caches - Matt Godbolt - https://www.youtube.com/watch?v=4_smHyqgDTU

  • What Every Programmer Should Know about How CPUs Work - Matt Godbolt - https://www.youtube.com/watch?v=-HNpim5x-IE

  • There Are No Zero-cost Abstractions - Chandler Carruth - https://www.youtube.com/watch?v=rHIkrotSwcc&

  • Understanding Optimizers: Helping the Compiler Help You - Nir Friedman - https://www.youtube.com/watch?v=8nyq8SNUTSc

  • C++ Algorithmic Complexity, Data Locality, Parallelism, Compiler Optimizations, & Some Concurrency - Avi Lachmish - https://www.youtube.com/watch?v=0iXRRCnurvo

  • Software Optimizations Become Simple with Top-Down Analysis on Intel Skylake - Ahmad Yasin - https://www.youtube.com/watch?v=kjufVhyuV_A

  • Being Friendly to Your Computer Hardware in Software Development - Ignas Bagdonas - https://www.youtube.com/watch?v=eceFgsiPPmk

  • Want fast C++? Know your hardware - Timur Doumler - https://www.youtube.com/watch?v=BP6NxVxDQIs

  • What is Low Latency C++ - Timur Doumler - https://www.youtube.com/watch?v=EzmNeAhWqVs, https://www.youtube.com/watch?v=5uIsadq-nyk

  • Where Have All the Cycles Gone? - Sean Parent - https://www.youtube.com/watch?v=B-aDBB34o6Y

  • Understanding CPU Microarchitecture to Increase Performance - https://www.youtube.com/watch?v=rglmJ6Xyj1c

  • Performance Analysis & Tuning on Modern CPU - Denis Bakhvalov - https://www.youtube.com/watch?v=Ho3bCIJcMcc

  • Comparison of C++ Performance Optimization Techniques for C++ Programmers - Eduardo Madrid - https://www.youtube.com/watch?v=4DQqcRwFXOI

  • Simple Code, High Performance - Molly Rocket - https://www.youtube.com/watch?v=Ge3aKEmZcqY

  • Assembly, System Calls, and Hardware in C++ - David Sankel - https://www.youtube.com/watch?v=7xwjjolDnwg

  • Optimizing Binary Search - Sergey Slotin - https://www.youtube.com/watch?v=1RIPMQQRBWk

  • A Deep Dive Into Dispatching Techniques in C++ - Jonathan Muller - https://www.youtube.com/watch?v=vUwsfmVkKtY

  • Dive into the general purpose GPU programming - Ashot Vardanian - https://www.youtube.com/watch?v=AA4RI6o0h1U

  • C++ Memory Model: from C++11 to C++23 - Alex Dathskovsky - https://www.youtube.com/watch?v=SVEYNEWZLo4

  • Abusing Your Memory Model for Fun and Profit - Samy Al Bahra, Paul Khuong - https://www.youtube.com/watch?v=N07tM7xWF1U&t=1s

  • The speed of concurrency (is lock-free faster?) - Fedor Pikus - https://www.youtube.com/watch?v=9hJkWwHDDxs

  • Read, Copy, Update, then what? RCU for non-kernel programmers - Fedor Pikus - https://www.youtube.com/watch?v=rxQ5K9lo034

  • Single Producer Single Consumer Lock-free FIFO From the Ground Up - Charles Frasch - https://www.youtube.com/watch?v=K3P_Lmq6pw0

  • Introduction to Hardware Efficiency in Cpp - Ivica Bogosavljevic - https://www.youtube.com/watch?v=Fs_T070H9C8

  • The Performance Price of Dynamic Memory in C++ - Ivica Bogosavljevic - https://www.youtube.com/watch?v=LC4jOs6z-ZI

  • The Hidden Performance Price of C++ Virtual Functions - Ivica Bogosavljevic - https://www.youtube.com/watch?v=n6PvvE_tEPk

  • Why do Programs Get Slower with Time? - Ivica Bogosavljevic - https://www.youtube.com/watch?v=nS5vjnPKX0I

  • CPU Cache Effects - Sergey Slotin - https://www.youtube.com/watch?v=mQWuX_KgH00

  • Cpu Caches and Why You Care - Scott Meyers - https://www.youtube.com/watch?v=WDIkqP4JbkE

  • CPU vs FPGA - https://www.youtube.com/watch?v=BML1YHZpx2o

  • Designing for Efficient Cache Usage - Scott McMillan - https://www.youtube.com/watch?v=3-ityWN-FdE

  • Cache consistency and the C++ memory model - Yossi Moale - https://www.youtube.com/watch?v=Sa08x_NMZIg

  • std::simd: How to Express Inherent Parallelism Efficiently Via Data-parallel Types - Matthias Kretz - https://www.youtube.com/watch?v=LAJ_hywLtMA

  • The Art of SIMD Programming - Sergey Slotin - https://www.youtube.com/watch?v=vIRjSdTCIEU

  • Advanced SIMD Algorithms in Pictures - Denis Yaroshevskiy - https://www.youtube.com/watch?v=vGcH40rkLdA

  • Performance Optimization, SIMD and Cache - Sergiy Migdalskiy - https://www.youtube.com/watch?v=Nsf2_Au6KxU

  • Data-Oriented Design and C++ - Mike Acton - https://www.youtube.com/watch?v=rX0ItVEVjHc

  • Practical Data Oriented Design (DoD) - Andrew Kelley - https://www.youtube.com/watch?v=IroPQ150F6c

  • Data Orientation For The Win - Eduardo Madrid - https://www.youtube.com/watch?v=QbffGSgsCcQ

  • You Can Do Better than std::unordered_map - Malte Skarupke - https://www.youtube.com/watch?v=M2fKMP47slQ

  • Faster than Rust and C++: the PERFECT hash table - https://www.youtube.com/watch?v=DMQ_HcNSOAI

  • Designing a Fast, Efficient, Cache-friendly Hash Table, Step by Step - Matt Kulukundis - https://www.youtube.com/watch?v=ncHmEUmJZf4

  • C++ Run-Time Optimizations for Compile-Time Reflection - Kris Jusiak - https://www.youtube.com/watch?v=ncHmEUmJZf4 - https://www.youtube.com/watch?v=kCATOctR0BA

  • When Nanoseconds Matter: Ultrafast Trading Systems in C++ - David Gross - https://www.youtube.com/watch?v=sX2nF1fW7kI

  • When a Microsecond Is an Eternity: High Performance Trading Systems in C++ - Carl Cook - https://www.youtube.com/watch?v=NH1Tta7purM

  • The Speed Game: Automated Trading Systems in C++ - Carl Cook - https://www.youtube.com/watch?v=ulOLGX3HNCI

  • Low-Latency Trading Systems in C++ - Jason McGuiness - https://www.youtube.com/watch?v=FnMfhWiSweo

  • High Frequency Trading and Ultra Low Latency development techniques - Nimrod Sapir - https://www.youtube.com/watch?v=_0aU8S-hFQI

  • Trading at light speed: designing low latency systems in C++ - David Gross - https://www.youtube.com/watch?v=8uAW5FQtcvE&list=PLSkBiuVO9yj1MvDkYJ5WOnPeKsoRi3eiW&index=2

  • Optimizing Trading Strategies for FPGAs in C/C++ - https://www.youtube.com/watch?v=4Wklh0XS5i0

  • C++ Electronic Trading for Cpp Programmers - Mathias Gaunard - https://www.youtube.com/watch?v=ltT2fDqBCEo

  • Achieving performance in financial data processing through compile time introspection - Eduardo Madrid - https://www.youtube.com/watch?v=z6fo90R8q5U

  • How to Simulate a Low Latency Exchange in C++ - Benjamin Catterall - https://www.youtube.com/watch?v=QQrTE4YLkSE

  • Building Low Latency Trading Systems - https://www.youtube.com/watch?v=yBNpSqOOoRk

  • Cache Warming: Warm Up The Code - Jonathan Keinan - https://www.youtube.com/watch?v=XzRxikGgaHI

  • How Linux Took Over the World of Finance - Christoph H Lameter - https://www.youtube.com/watch?v=UUOM4KdaHkY

Tools

• Benchmarking

  • google-benchmark - https://github.com/google/benchmark / https://quick-bench.com
  • nanobench - https://github.com/martinus/nanobench
  • celero - https://github.com/DigitalInBlue/Celero
  • nanobench - https://github.com/andreas-abel/nanoBench
  • uarch-bench - https://github.com/travisdowns/uarch-bench
  • llvm-exegesis - https://llvm.org/docs/CommandGuide/llvm-exegesis.html

• Profiling

  • linux-perf - https://perf.wiki.kernel.org
  • intel-vtune - https://www.intel.com/content/www/us/en/docs/vtune-profiler
  • amd-uprof - https://www.amd.com/en/developer/uprof.html
  • pmu-tools - https://github.com/andikleen/pmu-tools
  • perf-tools - https://github.com/brendangregg/perf-tools
  • magictrace - https://github.com/janestreet/magic-trace
  • tracy - https://github.com/wolfpld/tracy
  • likwid - https://github.com/RRZE-HPC/likwid
  • coz - https://github.com/plasma-umass/coz
  • ebpf - https://ebpf.io
  • callgrind - https://valgrind.org/docs/manual/cl-manual.html
  • yperf - https://github.com/aayasin/perf-tools
  • dtrace - https://www.oracle.com/linux/downloads/linux-dtrace.html
  • ftrace - https://www.kernel.org/doc/html/latest/trace/ftrace.html
  • utrace - https://github.com/Gui774ume/utrace
  • strace - https://strace.io
  • omnitrace - https://github.com/ROCm/omnitrace
  • optick - https://github.com/bombomby/optick
  • easy_profiler - https://github.com/yse/easy_profiler
  • gprof - https://ftp.gnu.org/old-gnu/Manuals/gprof-2.9.1/html_mono/gprof.html
  • gperftools - https://github.com/gperftools/gperftools
  • oprofile - https://oprofile.sourceforge.io
  • llvm-opt-report - https://llvm.org/docs/CommandGuide/llvm-opt-report.html
  • optview2 - https://github.com/OfekShilon/optview2
  • llvm-xray - https://llvm.org/docs/XRay.html
  • lttng - https://lttng.org
  • bcc - https://github.com/iovisor/bcc
  • sysprof - https://www.sysprof.com

• Analyzing

  • Compiler Explorer - https://compiler-explorer.com
  • llvm-mca - https://llvm.org/docs/CommandGuide/llvm-mca.html
  • osaca - https://github.com/RRZE-HPC/OSACA
  • uica - https://uica.uops.info

• Optimizing / Profile-Guided Optimization

  • clang-pgo - https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization
  • gcc-pgo - https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
  • llvm-bolt - https://github.com/llvm/llvm-project/blob/main/bolt/README.md
  • llvm-propelleer - https://github.com/google/llvm-propeller
  • autofdo - https://github.com/google/autofdo
  • e-graphs - https://egraphs-good.github.io

• Utilities

  • pyperf - https://github.com/psf/pyperf
  • hyperfine - https://github.com/sharkdp/hyperfine
  • numatop - https://github.com/intel/numatop
  • bpftop - https://github.com/Netflix/bpftop
  • pahole - https://github.com/acmel/dwarves
  • perfetto - https://perfetto.dev
  • speedscope - https://github.com/jlfwong/speedscope
  • kcachegrind - https://kcachegrind.sourceforge.net/html/Home.html
  • hotspot - https://github.com/KDAB/hotspot
  • jupyter - https://jupyter.org

• Libraries

  • perf_event_open - https://man7.org/linux/man-pages/man2/perf_event_open.2.html
  • llvm-dev - https://llvm.org
  • perfmon2 - https://perfmon2.sourceforge.net
  • papi - https://github.com/icl-utk-edu/papi
  • libpfc - https://github.com/obilaniu/libpfc
  • intel_pt - https://github.com/intel/libipt

License

MIT / Apache2:LLVM*