pyvinecopulib > 0.6.5

[Martin-Molinero]

Latest versions seem to be causing runtime explosions, it's not deterministic though, explodes sometimes

Fatal Python error: take_gil: PyMUTEX_LOCK(gil->mutex) failed
Python runtime state: initialized
Current thread 0x00007f082c8d2640 (most recent call first):
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1233 in create_module
  File "<frozen importlib._bootstrap>", line 573 in module_from_spec
  File "<frozen importlib._bootstrap>", line 676 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "/opt/miniconda3/lib/python3.11/site-packages/pyvinecopulib/__init__.py", line 1 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "none", line 2 in <module>
Thread 0x00007f09e0d86640 (most recent call first):
  <no Python frame>

The active test run was aborted. Reason: Test host process crashed : 20241210 20:09:36.942 ERROR:: LocalDiskMapFileProvider.GetMapFileResolver(sgx): The specified directory does not exist: ../../../Data/equity/sgx/map_files
2024-12-10 20:09:37.793993: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2024-12-10 20:09:37.803252: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1733861377.813161   28197 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1733861377.816198   28197 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-10 20:09:37.826603: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
/opt/miniconda3/lib/python3.11/site-packages/keras/src/layers/core/dense.py:87: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.
  super().__init__(activity_regularizer=activity_regularizer, **kwargs)
I0000 00:00:1733861379.559346   28197 gpu_device.cc:2022] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 13512 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 4090 Laptop GPU, pci bus id: 0000:01:00.0, compute capability: 8.9

How to reproduce:

• docker image quantconnect/lean:foundation
• git clone https://github.com/QuantConnect/Lean.git && cd Lean && dotnet build /p:Configuration=Release /v:quiet /p:WarningLevel=1 QuantConnect.Lean.sln
• pip install pyvinecopulib==0.6.8
• dotnet test ./Tests/bin/Release/QuantConnect.Tests.dll --filter "FullyQualifiedName=QuantConnect.Tests.Python.PythonPackagesTests.ShapTest|KerasTest|PyvinecopulibTest" -> might need to run a couple of times for it to fail

Related QuantConnect/Lean#8455

[tvatter]

Hi,

Sorry for missing this issue.

We're preparing a pretty big update of pyvinecopulib, switching the backend from pyvind11 to nanobind, including some huge perf improvements from vinecopulib (around 8x speedup in internal benchmarks). Things are currently in the nanobind branch which pass unit tests on all platforms.

Given the nature of the error, I suspect this might either solve it (yay) or worsen it. I'm seeing in your raised issue something that is GIL-related, and nanobind has changed how they handle this from pybind11. Importantly, they've introduced a FREE_THREADED option:

   - Compile an Python extension that opts into free-threaded (i.e.,
      GIL-less) Python behavior, which requires a special free-threaded
      build of Python 3.13 or newer. The flag is ignored on unsupported
      Python versions.

Can you try replicating using this new version of the package?

[tvatter]

So, the nanobind update is merged into the dev branch. The update will be made available on PyPI in the next few days as version 0.7.0

In the meantime, from the root of the docker image:

apt-get update
apt-get install -y libeigen3-dev libboost-all-dev
export Boost_INCLUDE_DIR=/usr/include
export EIGEN3_INCLUDE_DIR=/usr/include/eigen3
git clone --recursive https://github.com/vinecopulib/pyvinecopulib.git
cd pyvinecopulib
git checkout dev
pip install ./pyvinecopulib

Then dotnet test ./Tests/bin/Release/QuantConnect.Tests.dll --filter "FullyQualifiedName=QuantConnect.Tests.Python.PythonPackagesTests.ShapTest|KerasTest|PyvinecopulibTest" as suggested. Could not make it fail despite trying 10 times.

@Martin-Molinero can you make the use of pyvinecopulib >= 0.7.0 mandatory?

[Martin-Molinero]

Nice!

can you make the use of pyvinecopulib >= 0.7.0 mandatory

Next time we update our base image, every couple of months, we will look into this 💯

[tvatter]

Great, then it will definitely be available by then, I'm closing the issue, but feel free to reopen if needed!