Skip to content

Symbol collision when using other native gems that depend on libzstd #102

Open
@dmariassy

Description

@dmariassy

Problem Description

zstd-ruby currently exports all ZSTD symbols with default visibility, which causes symbol collisions when used alongside other gems that dynamically link to system libzstd (e.g., rdkafka-ruby).

Reproduction Steps

  1. Install both zstd-ruby and rdkafka-ruby in the same application
  2. Run code that triggers ZSTD compression in rdkafka
  3. Observe intermittent segmentation faults

Root Cause

zstd-ruby statically links libzstd but exports all ZSTD functions globally due to:

// In ext/zstdruby/libzstd/zstd.h
#define ZSTDLIB_VISIBLE __attribute__ ((visibility ("default")))
#define ZSTDLIB_API ZSTDLIB_VISIBLE

This creates a symbol collision where:

  1. rdkafka-ruby expects to use system libzstd (/lib/x86_64-linux-gnu/libzstd.so.1)
  2. Dynamic linker resolves ZSTD function calls to zstd-ruby's symbols instead
  3. Memory allocated by system libzstd gets freed by zstd-ruby's libzstd implementation
  4. Results in segfaults like:
    #0  edata_szind_set (szind=36, edata=0x0) at include/jemalloc/internal/edata.h:472
    ...
    #8  0x00007a1243a49528 in ZSTD_freeCCtx () from zstd-ruby/zstdruby.so
    #9  0x00007a125eace71d in rd_kafka_zstd_compress (...) at rdkafka_zstd.c:220
    

Proposed solution

  • Modify the build configuration to hide ZSTD symbols
  • Add RUBY_FUNC_EXPORTED macro to Init_zstdruby to explicitly mark it as publicly visible

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions