New Benchmark Is Neat - Rpi5 numbers

[jkrauska]

Here are my numbers from a RPi5 Dev setup.

# birdnet-go benchmark
🐦 BirdNET-Go build date: 2024-12-28T18:20:03Z, using config file: /root/.config/birdnet-go/config.yaml
🚀 Testing with XNNPACK delegate:
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 330, Average time: 90ms

🐌 Testing standard CPU inference:
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 170, Average time: 168ms
Results:
Method         Inference Time   Throughput
─────────────  ───────────────  ──────────────────────
Standard        168.0 ms           5.97 inferences/sec
XNNPACK          90.0 ms          11.13 inferences/sec

Curious what other platforms see.

[tphakala]

MacBook M4 Pro, I limited number of threads to 5 which gave best performance on this system

# ./bin/birdnet-go benchmark
🐦 BirdNET-Go build date: 2024-12-30T08:09:37Z, using config file: /Users/x/.config/birdnet-go/config.yaml
🚀 Testing with XNNPACK delegate:
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 5 threads of available 12 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 2280, Average time: 13ms

🐌 Testing standard CPU inference:
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 5 threads of available 12 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 1590, Average time: 18ms
Results:
Method         Inference Time   Throughput
─────────────  ───────────────  ──────────────────────
Standard         18.0 ms          53.20 inferences/sec
XNNPACK          13.0 ms          76.00 inferences/sec
─────────────  ───────────────  ──────────────────────

🚀 Speed improvement with XNNPACK: 27.8%

[tphakala]

Intel Core i7-13700K with 8 threads, Windows 11

 .\birdnet-go.exe benchmark
🐦 BirdNET-Go build date: 2024-12-28T18:09:55Z, using config file: C:\Users\Tomi\birdnet\config.yaml
🚀 Testing with XNNPACK delegate:
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
BirdNET-Go_v1.4.5.tflite model initialized, using 8 threads of available 24 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 2020, Average time: 14ms

🐌 Testing standard CPU inference:
BirdNET-Go_v1.4.5.tflite model initialized, using 8 threads of available 24 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 950, Average time: 31ms
Results:
Method         Inference Time   Throughput
─────────────  ───────────────  ──────────────────────
Standard         31.0 ms          31.70 inferences/sec
XNNPACK          14.0 ms          67.37 inferences/sec
─────────────  ───────────────  ──────────────────────

🚀 Speed improvement with XNNPACK: 54.8%

[tphakala]

Intel Core i5-7260U with all 4 cores, Debian 12

 ./bin/birdnet-go benchmark
🐦 BirdNET-Go build date: 2024-12-29T09:36:16Z, using config file: /home/thakala/.config/birdnet-go/config.yaml
🚀 Testing with XNNPACK delegate:
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 790, Average time: 37ms

🐌 Testing standard CPU inference:
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 480, Average time: 62ms
Results:
Method         Inference Time   Throughput
─────────────  ───────────────  ──────────────────────
Standard         62.0 ms          16.13 inferences/sec
XNNPACK          37.0 ms          26.63 inferences/sec
─────────────  ───────────────  ──────────────────────

🚀 Speed improvement with XNNPACK: 40.3%

[tphakala]

Raspberry Pi 3 Model B, Debian 12 (rpi os bookworm)

./birdnet-go benchmark
🐦 BirdNET-Go build date: 2024-12-28T18:12:41Z, using config file: /home/thakala/.config/birdnet-go/config.yaml
🚀 Testing with XNNPACK delegate:
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 50, Average time: 542ms

🐌 Testing standard CPU inference:
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 30, Average time: 848ms
Results:
Method         Inference Time   Throughput
─────────────  ───────────────  ──────────────────────
Standard        847.0 ms           1.20 inferences/sec
XNNPACK         541.0 ms           1.87 inferences/sec
─────────────  ───────────────  ──────────────────────

🚀 Speed improvement with XNNPACK: 36.1%

[tphakala]

Raspberry Pi 4 Model B, Debian 12 (rpi os bookworm)

🐦 BirdNET-Go build date: 2024-12-30T08:32:32Z, using config file: /home/thakala/.config/birdnet-go/config.yaml
🚀 Testing with XNNPACK delegate:
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 180, Average time: 166ms

🐌 Testing standard CPU inference:
BirdNET GLOBAL 6K V2.4 FP32 model initialized, using 4 threads of available 4 CPUs
⏳ Running benchmark for 30 seconds...
🔄 Inferences: 120, Average time: 240ms
Results:
Method         Inference Time   Throughput
─────────────  ───────────────  ──────────────────────
Standard        240.0 ms           4.17 inferences/sec
XNNPACK         166.0 ms           6.00 inferences/sec
─────────────  ───────────────  ──────────────────────

🚀 Speed improvement with XNNPACK: 30.8%