Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crash in XS_DBI_dispatch by cloning $dbh #170

Open
daxim opened this issue Mar 13, 2025 · 4 comments
Open

crash in XS_DBI_dispatch by cloning $dbh #170

daxim opened this issue Mar 13, 2025 · 4 comments

Comments

@daxim
Copy link

daxim commented Mar 13, 2025

DBI 1.647

❯ gdb --args perl -Mblib -mDBI -MClone=clone -E'clone(DBI->connect("dbi:Pg:host=localhost"))'
GNU gdb (GDB; openSUSE Tumbleweed) 15.2
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.opensuse.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from perl...
(gdb) r
Starting program: /home/larsd/.local/share/perlbrew/perls/perl-5.40.1/bin/perl -Mblib -mDBI -MClone=clone -Eclone\(DBI-\>connect\(\"dbi:Pg:host=localhost\"\)\)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
XS_DBI_dispatch (cv=0x5555558dce28) at /home/larsd/.cache/cpanm/work/1741862763.21292/DBI-1.647/DBI.xs:3362
3362        if ((i = DBIc_DEBUGIV(imp_xxh))) { /* merge handle into global */
Missing separate debuginfos, use: zypper install libcrypt1-debuginfo-4.4.38-1.1.x86_64
Missing separate debuginfos, use: zypper install libpq5-debuginfo-17.4-1.1.x86_64
Missing separate debuginfos, use: zypper install libopenssl3-x86-64-v3-debuginfo-3.2.4-1.1.x86_64
Missing separate debuginfos, use: zypper install krb5-debuginfo-1.21.3-2.1.x86_64
Missing separate debuginfos, use: zypper install libldap2-debuginfo-2.6.8-2.2.x86_64
Missing separate debuginfos, use: zypper install libz-ng-compat1-debuginfo-2.2.3-1.1.x86_64
Missing separate debuginfos, use: zypper install libjitterentropy3-debuginfo-3.4.1-4.3.x86_64
Missing separate debuginfos, use: zypper install libcom_err2-debuginfo-1.47.0-4.4.x86_64
Missing separate debuginfos, use: zypper install libsasl2-3-debuginfo-2.1.28-9.2.x86_64
Missing separate debuginfos, use: zypper install libkeyutils1-debuginfo-1.6.3-7.3.x86_64
Missing separate debuginfos, use: zypper install libselinux1-debuginfo-3.8-2.1.x86_64
Missing separate debuginfos, use: zypper install libpcre2-8-0-debuginfo-10.44-1.2.x86_64
(gdb) bt full
#0  XS_DBI_dispatch (cv=0x5555558dce28) at /home/larsd/.cache/cpanm/work/1741862763.21292/DBI-1.647/DBI.xs:3362
        sp = 0x555555617558
        ax = 1
        mark = 0x555555617550
        items = 1
        origmark = 0
        h = 0x5555559ac1f8
        st1 = 0x5555558dce28
        st2 = 0x5555559ac8e8
        orig_h = 0x5555559ac1f8
        err_sv = <optimized out>
        tmp_svp = <optimized out>
        hook_svp = 0x0
        mg = <optimized out>
        gimme = 1
        trace_flags = 0
        trace_level = 0
        is_DESTROY = 1
        meth_type = methtype_DESTROY
        is_unrelated_to_Statement = 0
        keep_error = 1
        ErrCount = 18446744073709551615
        i = <optimized out>
        outitems = <optimized out>
        call_depth = <optimized out>
        is_nested_call = <optimized out>
        profile_t1 = 0
        is_orig_method_name = 1
        meth_name = 0x555555581340 "DESTROY"
        ima = 0x5555558df7a0
        ima_flags = 65540
        imp_xxh = 0x555555a03110
        imp_msv = 0x0
--Type <RET> for more, q to quit, c to continue without paging--c
        qsv = 0x0
        buf = <optimized out>
        avi = <optimized out>
        p = <optimized out>
        s = <optimized out>
        agg = <optimized out>
        p_ = <optimized out>
#1  0x00007ffff7d10cfd in Perl_rpp_invoke_xs (cv=0x5555558dce28)
    at /home/larsd/.local/share/perlbrew/build/perl-5.40.1/perl-5.40.1/inline.h:1193
No locals.
#2  Perl_pp_entersub () at pp_hot.c:6495
        markix = 0
        is_scalar = false
        gv = 0x5555559bc0d8
        cv = 0x5555558dce28
        cx = <optimized out>
        old_savestack_ix = <optimized out>
        sv = 0x5555558dce28
        do_ref = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
#3  0x00007ffff7d945f3 in Perl_runops_standard () at run.c:41
        op = <optimized out>
#4  0x00007ffff7c72c6b in Perl_call_sv (sv=<optimized out>, arg_flags=arg_flags@entry=45) at perl.c:3216
        old_cxix = -1
        p_ = <optimized out>
        myop = {op_next = 0x0, op_sibparent = 0x0, op_ppaddr = 0x7ffff7d10aa0 <Perl_pp_entersub>, op_targ = 0, op_type = 187, 
          op_opt = 0, op_slabbed = 0, op_savefree = 0, op_static = 0, op_folded = 0, op_moresib = 0, op_spare = 0, 
          op_flags = 65 'A', op_private = 0 '\000', op_first = 0x0, op_other = 0x7fffffffc900}
        method_op = {op_next = 0x5555559bc648, op_sibparent = 0x5555559bc150, op_ppaddr = 0x0, op_targ = 140737350638266, 
          op_type = 304, op_opt = 1, op_slabbed = 0, op_savefree = 0, op_static = 0, op_folded = 0, op_moresib = 1, op_spare = 1, 
          op_flags = 155 '\233', op_private = 85 'U', op_u = {op_first = 0x5555556b9fd0, op_meth_sv = 0x5555556b9fd0}, 
          op_rclass_sv = 0x555500000020}
        oldmark = 0
        retval = 0
        oldcatch = true
        ret = <optimized out>
        oldop = 0x0
        flags = 45
        cur_env = {je_prev = 0x7ffff7fb3500 <PL_start_env>, je_buf = {{__jmpbuf = {140737353824392, -6856996284238243355, 1, 0, 
                93824996786680, 140737353522696, -6856996284179523099, -6856978628723713563}, __mask_was_saved = 0, __saved_mask = {
                __val = {13816973012072644575, 140737353522696, 93824995938000, 93824995948912, 0, 93824995948912, 93824994656816, 
                  140737350608258, 0, 1821615850, 0, 93824994576240, 42949672960, 18446744069414584320, 0, 93824994656888}}}}, 
          je_ret = 0, je_mustcatch = false, je_old_delaymagic = 0, je_old_stack_hwm = 7}
#5  0x00007ffff7d9a1f6 in S_curse (sv=sv@entry=0x5555559ac4b0, check_refcnt=check_refcnt@entry=true) at sv.c:7213
        tmpref = 0x5555559ac1f8
        p_ = <optimized out>
        p_ = <optimized out>
        destructor = <optimized out>
        meta = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        sp = <optimized out>
        stash = <optimized out>
#6  0x00007ffff7d9a96d in Perl_sv_clear (orig_sv=orig_sv@entry=0x5555559ac4b0) at sv.c:6753
        stash = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        type = 12
        iter_sv = 0x0
        next_sv = 0x0
        sv = 0x5555559ac4b0
        hash_index = 0
#7  0x00007ffff7d9afdf in Perl_sv_free2 (sv=0x5555559ac4b0, rc=<optimized out>) at sv.c:7314
No locals.
#8  0x00007ffff7d9b775 in Perl_SvREFCNT_dec_NN (sv=<optimized out>)
    at /home/larsd/.local/share/perlbrew/build/perl-5.40.1/perl-5.40.1/sv_inline.h:730
        rc = <optimized out>
        rc = <optimized out>
#9  do_clean_objs (ref=0x5555559ac4c8) at sv.c:451
        target = <optimized out>
        p_ = <optimized out>
#10 S_visit (f=<optimized out>, flags=2048, mask=2048) at sv.c:394
        svend = 0x5555559acfc0
        sv = 0x5555559ac4c8
        sva = 0x5555559abfd0
        visited = <optimized out>
        sva = <optimized out>
        visited = <optimized out>
        svend = <optimized out>
        sv = <optimized out>
        p_ = <optimized out>
#11 Perl_sv_clean_objs () at sv.c:545
        olddef = <optimized out>
        olderr = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
#12 0x00007ffff7c760e0 in perl_destruct (my_perl=<optimized out>) at perl.c:927
        destruct_level = 0 '\000'
        hv = <optimized out>
        i = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
        p_ = <optimized out>
#13 0x000055555555520c in main (argc=<optimized out>, argv=<optimized out>, env=<optimized out>) at perlmain.c:137
        exitstatus = <optimized out>
        i = <optimized out>

❯ perl -V
Summary of my perl5 (revision 5 version 40 subversion 1) configuration:
   
  Platform:
    osname=linux
    osvers=6.12.9-1-default
    archname=x86_64-linux-ld
    uname='linux denkheft 6.12.9-1-default #1 smp preempt_dynamic fri jan 10 08:30:10 utc 2025 (0ae2136) x86_64 x86_64 x86_64 gnulinux '
    config_args='-de -Dprefix=/home/larsd/.local/share/perlbrew/perls/perl-5.40.1 -Dusedevel -Duseshrplib -Dman1dir=none -Dman3dir=none -Dcf_email=root@localhost -Dperladmin=root@localhost -Dusemorebits -Dusesitecustomize -Accflags=-DNO_TAINT_SUPPORT -Accflags=-march=native -Accflags=-fPIC -Accflags=-DPERL_DISABLE_PMC -Accflags=-ggdb -Aeval:scriptdir=/home/larsd/.local/share/perlbrew/perls/perl-5.40.1/bin'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=define
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-DNO_TAINT_SUPPORT -march=native -fPIC -DPERL_DISABLE_PMC -ggdb -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2'
    cppflags='-DNO_TAINT_SUPPORT -march=native -fPIC -DPERL_DISABLE_PMC -ggdb -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='14.2.1 20241007 [revision 4af44f2cf7d281f3e4f3957efce10e8b2ccb2ad3]'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='long double'
    nvsize=16
    Off_t='off_t'
    lseeksize=8
    alignbytes=16
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /lib/../lib64 /usr/lib/../lib64 /lib /usr/lib /lib64 /usr/lib64 /usr/local/lib64
    libs=-lpthread -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
    libc=/lib/../lib64/libc.so.6
    so=so
    useshrplib=true
    libperl=libperl.so
    gnulibc_version='2.40'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E -Wl,-rpath,/home/larsd/.local/share/perlbrew/perls/perl-5.40.1/lib/5.40.1/x86_64-linux-ld/CORE'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_LONG_DOUBLE
    HAS_STRTOLD
    HAS_TIMES
    NO_TAINT_SUPPORT
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DISABLE_PMC
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_LONG_DOUBLE
    USE_PERLIO
    USE_PERL_ATOF
    USE_SITECUSTOMIZE
  Built under linux
  Compiled at Feb 20 2025 14:06:26
  %ENV:
    PERLBREW_HOME="/home/larsd/.local/share/perlbrew"
    PERLBREW_MANPATH="/home/larsd/.local/share/perlbrew/perls/perl-5.40.1/man"
    PERLBREW_PATH="/home/larsd/.local/share/perlbrew/bin:/home/larsd/.local/share/perlbrew/perls/perl-5.40.1/bin"
    PERLBREW_PERL="perl-5.40.1"
    PERLBREW_ROOT="/home/larsd/.local/share/perlbrew"
    PERLBREW_SHELLRC_VERSION="0.98"
    PERLBREW_VERSION="0.98"
    PERLCRITIC="/home/larsd/.config/perlcriticrc"
    PERLREPL_HISTFILE="/home/larsd/.local/state/perlreplhist"
    PERLTIDY="/home/larsd/.config/perltidyrc"
    PERL_CPANM_HOME="/home/larsd/.cache/cpanm"
    PERL_CPANM_OPT="-M https://cpan.metacpan.org/ --verify --with-recommends --with-suggests --with-develop --with-all-features"
    PERL_EXTUTILS_AUTOINSTALL="--default-deps"
    PERL_MM_NONINTERACTIVE="1"
    PERL_MM_USE_DEFAULT="1"
  @INC:
    /home/larsd/.local/share/perlbrew/perls/perl-5.40.1/lib/site_perl/5.40.1/x86_64-linux-ld
    /home/larsd/.local/share/perlbrew/perls/perl-5.40.1/lib/site_perl/5.40.1
    /home/larsd/.local/share/perlbrew/perls/perl-5.40.1/lib/5.40.1/x86_64-linux-ld
    /home/larsd/.local/share/perlbrew/perls/perl-5.40.1/lib/5.40.1
@Tux
Copy link
Member

Tux commented Mar 14, 2025

I just finished rebuilding my complete perl tree on openSUSE TumbleWeed, and I cannot reproduce your core dump. Might well be releated to Leap-15.2

Summary of my perl5 (revision 5 version 40 subversion 1) configuration:

  Platform:
    osname=linux
    osvers=6.13.6-1-default
    archname=x86_64-linux-thread-multi-quadmath
    uname='linux lx09 6.13.6-1-default #1 smp preempt_dynamic mon mar 10 08:49:24 utc 2025 (495d82a) x86_64 x86_64 x86_64 gnulinux '
    config_args='-Duse64bitall -Dusequadmath -Dusethreads -Duseithreads -Duseshrplib -Uversiononly -Dinc_version_list=none -Dextras='V\ Test::Warnings' -des'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-D_REENTRANT -D_GNU_SOURCE -pie -fPIE -fPIC -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/pro/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2'
    cppflags='-D_REENTRANT -D_GNU_SOURCE -pie -fPIE -fPIC -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/pro/local/include'
    ccversion=''
    gccversion='14.2.1 20250220 [revision 9ffecde121af883b60bbe60d00425036bc873048]'
:
:
  Linker and Libraries:
    ld='cc'
    ldflags ='-L/pro/local/lib -fstack-protector-strong'
    libpth=/usr/local/lib /usr/x86_64-suse-linux/lib /usr/lib /pro/local/lib /usr/lib64 /usr/local/lib64
    libs=-lpthread -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat -lquadmath
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc -lquadmath
    libc=/lib/../lib64/libc.so.6
    so=so
    useshrplib=true
    libperl=libperl.so
    gnulibc_version='2.41'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E -Wl,-rpath,/pro/lib/perl5/5.40.1/x86_64-linux-thread-multi-quadmath/CORE'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -L/pro/local/lib -fstack-protector-strong'
DBI 🐧 perl -MConfig::Perl::V=summary -MData::Peek -wE'DDumper (summary)'
{   HAS_LONG_DOUBLE  => 1,
    HAS_STRTOLD      => 1,
    HAS_TIMES        => 1,
    MULTIPLICITY     => 1,
    PERLIO_LAYERS    => 1,
    PERL_COPY_ON_WRITE => 1,
    PERL_DONT_CREATE_GVSV => 1,
    PERL_HASH_FUNC_SIPHASH13 => 1,
    PERL_HASH_USE_SBOX32 => 1,
    PERL_MALLOC_WRAP => 1,
    PERL_OP_PARENT   => 1,
    PERL_PRESERVE_IVUV => 1,
    PERL_USE_SAFE_PUTENV => 1,
    USE_64_BIT_ALL   => 1,
    USE_64_BIT_INT   => 1,
    USE_ITHREADS     => 1,
    USE_LARGE_FILES  => 1,
    USE_LOCALE       => 1,
    USE_LOCALE_COLLATE => 1,
    USE_LOCALE_CTYPE => 1,
    USE_LOCALE_NUMERIC => 1,
    USE_LOCALE_TIME  => 1,
    USE_PERLIO       => 1,
    USE_PERL_ATOF    => 1,
    USE_QUADMATH     => 1,
    USE_REENTRANT_API => 1,
    USE_THREAD_SAFE_LOCALE => 1,
    archname         => 'x86_64-linux-thread-multi-quadmath',
    cc               => 'cc',
    ccversion        => '',
    config_args      => '-Duse64bitall -Dusequadmath -Dusethreads -Duseithreads -Duseshrplib -Uversiononly -Dinc_version_list=none -Dextras=\'V\\ Test::Warnings\' -des',
    d_longdbl        => 'define',
    d_longlong       => 'define',
    default_inc_excludes_dot => 'define',
    doublesize       => '8',
    gccversion       => '14.2.1 20250220 [revision 9ffecde121af883b60bbe60d00425036bc873048]',
    inc_version_list => ' ',
    intsize          => '4',
    ivsize           => '8',
    longdblsize      => '16',
    longlongsize     => '8',
    lseeksize        => '8',
    nvsize           => '16',
    osname           => 'linux',
    osvers           => '6.13.6-1-default',
    patchlevel       => '40',
    revision         => '5',
    subversion       => '1',
    use64bitall      => 'define',
    use64bitint      => 'define',
    useithreads      => 'define',
    uselongdouble    => 'undef',
    usemultiplicity  => 'define',
    usemymalloc      => 'n',
    useperlio        => 'define',
    useshrplib       => 'true',
    version          => '5.40.1'
    }
DBI 🐧 perl -Mblib -mDBI -MClone=clone -E'clone(DBI->connect("dbi:Pg:host=localhost"))'
DBI 🐧
DBI 🐧 perl -d:TraceUse=hidecore -Mblib -mDBI -MClone=clone -E'clone(DBI->connect("dbi:Pg:host=localhost"))'
Modules used from -e:
  11.  DBI 1.648, -e line 0 [main]
  24.    DBD::Pg 3.18.0, DBI.pm line 795 (eval 5) [DBI::_firesafe]
  26.      version::vxs 0.9933, version.pm line 18 (eval 6)
  20.  Clone 0.47, -e line 0 [main]
Possible proxies:
   5 -e line 0, sub main::BEGIN

@daxim
Copy link
Author

daxim commented Mar 21, 2025

I have prepared a repro environment for you. Obtain Containerfile from https://gist.github.com/daxim/c4671438127e6ee3cd570e5b01cee7bd

build image (result < 2 GB)

yourhost:> podman build -t dbi-170 -f Containerfile

run image as container and enter a shell

yourhost:> podman run -it dbi-170 bash
container:# /etc/init.d/postgresql restart
container:# gdb --args perl5.40.1 -mDBI -MClone=clone -E'clone(DBI->connect("dbi:Pg:host=localhost", "postgres"))'

@Tux
Copy link
Member

Tux commented Mar 21, 2025

🐧 podman build -t dbi-170 -f Containerfile
STEP 1/6: FROM debian:12
Resolved "debian" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/debian:12...
Getting image source signatures
Copying blob 7cd785773db4 done   |
Error: creating build container: copying system image from manifest list: writing blob: adding layer with blob "sha256:7cd785773db44407e20a679ce5439222e505475eed5b99f1910eb2cda51729ab": processing tar file(potentially insufficient UIDs or GIDs available in user namespace (requested 0:42 for /etc/gshadow): Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate": lchown /etc/gshadow: invalid argument): exit status 1

(Never used podman before, forgive my ignorance)
I'll be unavailable for the next week, sorry again

@daxim
Copy link
Author

daxim commented Mar 21, 2025

potentially insufficient UIDs or GIDs available in user namespace […] Check /etc/subuid and /etc/subgid if configured locally and run "podman system migrate"

https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md#etcsubuid-and-etcsubgid-configuration

I'll be unavailable

No worries, enjoy your time off.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants