back2source: Run d2d analysis on many projects

[pombredanne]

For each of the ecosystems below, we should define a set of interesting projects then analyze, and then review results for accuracy

The high level steps are:

• Collect a list of PURLs and download URL pairs
• Run the d2d scans
• Collect and review issues and accuracy
• Fix bugs and run again
• Eventually find issues build a report of these and reach out to projects for key issues

The ecosystems to consider are:

• back2source: Run pipeline for 100 Windows C++ projects #1470
• back2source: Run pipeline for 100 macOS projects #1471
• back2source: Run pipeline for 100 .Net projects #1472
• back2source: Run pipeline for Java and native code and Android apk and dex classes #1473
• back2source: Run pipeline for Python and C/C++ source and the python bytecode and native binaries #1474
• back2source: Run pipeline for Rust source and binaries #1475
• back2source: Run pipeline for Ruby source and gem packages #1476
• back2source: Run pipeline for JavaScript #1477

[tdruez]

Progress update [on Jan 25 2024]

Reports available for:

• Windows C++
• macOS
• .Net
• Java
• Ruby
• Python
• JavaScript
• Rust

XLSX files @ https://github.com/aboutcode-org/back2source-data/tree/main/reports/2025-01-17

Next: "Collect and review issues and accuracy"

@pombredanne How do you want to proceed for the "review" part?

[pombredanne]

Next: "Collect and review issues and accuracy"

@pombredanne How do you want to proceed for the "review" part?

@tdruez @AyanSinhaMahapatra the review consist in:

1. looking at all discrepancies with a TODO/REVIEW status
2. Fix most problems, or enter issues for larger more complex problems
3. Collect a clean set of issues and determine if these are things we could report upstream as problems
   3.1. ideally, we would want to determine unknown bundled packages and check for unknown vulnerabilities or license issues from these inclusions