Bug: ResourceWarnings

[LeamHall]

Pre-submission checks

• I am not filing an auditing error (false positive or negative). These must be reported to pypa/advisory-database instead.
• I agree to follow the PSF Code of Conduct.
• I have looked through the open issues for a duplicate report.

Expected behavior

I'm getting Resource warnings when running pip-audit.

Actual behavior

$ python -m pip_audit -r tmp/requirements.txt
/home/leam/.local/lib/python3.12/site-packages/pip_api/_vendor/pyparsing.py:108: DeprecationWarning: module 'sre_constants' is deprecated
  import sre_constants
/home/leam/.local/lib/python3.12/site-packages/pip_audit/_virtual_env.py:134: ResourceWarning: unclosed file <_io.FileIO name=4 mode='rb' closefd=True>
  run(pip_upgrade_cmd, state=self._state)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/leam/.local/lib/python3.12/site-packages/pip_audit/_virtual_env.py:134: ResourceWarning: unclosed file <_io.FileIO name=6 mode='rb' closefd=True>
  run(pip_upgrade_cmd, state=self._state)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/leam/.local/lib/python3.12/site-packages/pip_audit/_virtual_env.py:166: ResourceWarning: unclosed file <_io.FileIO name=5 mode='rb' closefd=True>
  run(package_install_cmd, log_stdout=True, state=self._state)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/home/leam/.local/lib/python3.12/site-packages/pip_audit/_virtual_env.py:166: ResourceWarning: unclosed file <_io.FileIO name=7 mode='rb' closefd=True>
  run(package_install_cmd, log_stdout=True, state=self._state)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Found 3 known vulnerabilities in 1 package
Name  Version ID             Fix Versions
----- ------- -------------- ------------
flask 0.5     PYSEC-2019-179 1.0
flask 0.5     PYSEC-2018-66  0.12.3
flask 0.5     PYSEC-2023-62  2.2.5,2.3.2
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='tmp/requirements.txt' mode='r' encoding='UTF-8'>
ResourceWarning: Enable tracemalloc to get the object allocation traceback`

Reproduction steps

Run pip-audit.

Logs

Additional context

No response

OS name, version, and architecture

Fedora Linux on x86_64

pip-audit version

2.7.3

pip version

25.0

Python version

3.12.1+

[woodruffw]

Hi @LeamHall, thanks for the report. Those ResourceWarnings should be pretty easy to remove; I'll look into them in a bit.

(The DeprecationWarning comes from pip-api, which vendors a copy of pyparsing. So that'll need a separate upstream fix.)

[LeamHall]

Hi @LeamHall, thanks for the report. Those ResourceWarnings should be pretty easy to remove; I'll look into them in a bit.

(The DeprecationWarning comes from pip-api, which vendors a copy of pyparsing. So that'll need a separate upstream fix.)

Thanks! The DeprecationWarning is already noted as an issue on pip-api for Python 3.11, I added a note that it occurs on 3.12 as well.

[woodruffw]

Reproduced this with the following:

$ python -Wall -m pip_audit -r <(echo 'flask==0.5') --no-deps

yielding:

/Users/william/devel/pip-audit/env/lib/python3.11/site-packages/pip_api/_vendor/pyparsing.py:108: DeprecationWarning: module 'sre_constants' is deprecated
  import sre_constants
WARNING:pip_audit._cli:--no-deps is supported, but users are encouraged to fully hash their pinned dependencies
WARNING:pip_audit._cli:Consider using a tool like `pip-compile`: https://pip-tools.readthedocs.io/en/latest/#using-hashes
/Users/william/devel/pip-audit/pip_audit/_virtual_env.py:134
: ResourceWarning: unclosed file <_io.FileIO name=4 
mode='rb' closefd=True>
  run(pip_upgrade_cmd, state=self._state)
ResourceWarning: Enable tracemalloc to get the object 
allocation traceback
/Users/william/devel/pip-audit/pip_audit/_virtual_env.py:134
: ResourceWarning: unclosed file <_io.FileIO name=6 
mode='rb' closefd=True>
  run(pip_upgrade_cmd, state=self._state)
ResourceWarning: Enable tracemalloc to get the object 
allocation traceback
/Users/william/devel/pip-audit/pip_audit/_virtual_env.py:166
: ResourceWarning: unclosed file <_io.FileIO name=5 
mode='rb' closefd=True>
  run(package_install_cmd, log_stdout=True, 
state=self._state)
ResourceWarning: Enable tracemalloc to get the object 
allocation traceback
/Users/william/devel/pip-audit/pip_audit/_virtual_env.py:166
: ResourceWarning: unclosed file <_io.FileIO name=7 
mode='rb' closefd=True>
  run(package_install_cmd, log_stdout=True, 
state=self._state)
ResourceWarning: Enable tracemalloc to get the object 
allocation traceback
Found 3 known vulnerabilities in 1 package
Name  Version ID             Fix Versions
----- ------- -------------- ------------
flask 0.5     PYSEC-2019-179 1.0
flask 0.5     PYSEC-2018-66  0.12.3
flask 0.5     PYSEC-2023-62  2.2.5,2.3.2
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/fd/63' mode='r' encoding='UTF-8'>
ResourceWarning: Enable tracemalloc to get the object allocation traceback

I'm a bit confused as to why our run(...) wrapper leaks a file, but I'll try and figure that out 🙂

Edit: Figured it out: I thought Popen(...) would close pipe fds by default, but apparently not. Using it as a context manager instead does the trick.

[LeamHall]

If you have something I can drop in and test, let me know.

[woodruffw]

If you have something I can drop in and test, let me know.

Thanks! I'll have a PR up later today that'll be testable.

[woodruffw]

#878 fixes these leakages for me locally. Please give it a test if you can!

[LeamHall]

Here's what I have:

[leam@shaphan pip-audit]$ pwd
/usr/local/src/forks/pip-audit

[leam@shaphan pip-audit]$ git branch
* main

[leam@shaphan pip-audit]$ make run -r ~/lang/python/toys/tmp/requirements.txt

Which did it's own venv, and then ran, giving only the pip_api DeprecationWarning. No ResourceWarnings, yay! I'm not great at packages yet, sorry. What else can I do to test?

[LeamHall]

In the normal venv

• with pip-audit 2.7.3 unmodified
• from branch ww/resource-warnings
• I copied in your new _subprocess.py and ran:

python -m pip_audit -r tmp/requirements.txt

And it also ran without ResourceWarnings. Looks good to me!

[woodruffw]

Thanks a ton @LeamHall!