should provide an --exclude parameter to avoid shipping test-data and other binary blobs

[dirkmueller]

in the case of "vcpkg", we seem to include files that should not be needed for building, and could raise concerns. we should be supporting an exclude parameter to strip those files from the vendor.tar.gz tarball:

tar tvf vendor.tar.zst|grep -i -E '(lzm|xz)'
-rw-r--r-- 0/0             981 1970-01-01 01:00 vendor/vcpkg/test-data/no-status/installed/vcpkg/info/liblzma_5.2.3-2_x64-windows.list
-rw-r--r-- 0/0               0 1970-01-01 01:00 vendor/vcpkg/test-data/no-status/installed/x64-windows/bin/lzma.dll
-rw-r--r-- 0/0               0 1970-01-01 01:00 vendor/vcpkg/test-data/no-status/installed/x64-windows/lib/lzma.lib
-rw-r--r-- 0/0            1012 1970-01-01 01:00 vendor/vcpkg/test-data/normalized/installed/vcpkg/info/liblzma_5.2.3-2_x64-windows-static.list
-rw-r--r-- 0/0             981 1970-01-01 01:00 vendor/vcpkg/test-data/normalized/installed/vcpkg/info/liblzma_5.2.3-2_x64-windows.list
-rw-r--r-- 0/0               0 1970-01-01 01:00 vendor/vcpkg/test-data/normalized/installed/x64-windows/bin/lzma.dll
-rw-r--r-- 0/0               0 1970-01-01 01:00 vendor/vcpkg/test-data/normalized/installed/x64-windows/debug/bin/lzma.dll
-rw-r--r-- 0/0               0 1970-01-01 01:00 vendor/vcpkg/test-data/normalized/installed/x64-windows/debug/lib/lzma.lib
-rw-r--r-- 0/0               0 1970-01-01 01:00 vendor/vcpkg/test-data/normalized/installed/x64-windows/lib/lzma.lib
-rw-r--r-- 0/0               0 1970-01-01 01:00 vendor/vcpkg/test-data/normalized/installed/x64-windows-static/debug/lib/lzma.lib
-rw-r--r-- 0/0               0 1970-01-01 01:00 vendor/vcpkg/test-data/normalized/installed/x64-windows-static/lib/lzma.lib````

[jfkw]

Agreed. I'd like to investigate their layout and the mechanism. I'm not familliar with vcpkg other than the Microsoft C++ package manager. Do you have a link in OBS or upstream Github?

Normally test-only Go packages are not imported by go mod vendor. The heuristic is to vendor only packages which are transitively imported by the Go module. This sometimes trips up packages which have a subdirectory with e.g. C code and no Go code reachable from main(). In those cases the subdirectory is omitted from the generated vendor/.

[jfkw]

As we plan an --exclude parameter, here is a sample of unique file extensions present in the vendor/ directory of Hugo, a Go application with a large number of Go dependencies, including linking to C libraries:

$ find ./hugo/vendor -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort -u
abnf
badjson
badxml
bash
bicep
c
cfg
conf
contributebot
cpp
der
dockerignore
editorconfig
gitattributes
gitignore
gitmodules
go
h
hpp
ini
jpg
json
json5
libyaml
lock
mailmap
markdown
md
pem
png
proto
ps1
py
replay
s
sh
template
tif
tmpl
toml
txt
vim
xml
yaml
yml
zsh

Given the number of file extensions one might enounter, we would want any --exclude parameter be capable of multi-invocations and/or wildcards. I'd like to align with conventions used by other OBS Source Services.

[hennevogel]

JFYI: For ruby we ship an RPM macro/script to do this in the spec file...

https://build.opensuse.org/projects/devel:languages:ruby/packages/ruby-common/files/gem_build_cleanup?expand=1