Ensure shared library using MSYS2 for a commercial code #223
Comments
|
As I recall, *.dll libraries create a separate copy for of the variables that are static in a function, *.so libraries only have one copy for each of these variables. I suggest you try suppressing the change on MSYS2 and see if the CppAD make checks pass or fail. |
|
I've made the following changes: I'm a new user to ninja and that is how the code is built. 'make check' is therefore not a command, should I be running 'ninja test'? |
I'm not sure of the point you're making here. Could you explain? Thanks |
|
|
Ok thanks for that. I've remade your latest tagged version using the suggestion above using the code diff I've already posted. I've attached the full output of make check. There were 10 failures that I can see, all seem to be that the libraries cannot find system libraries, Some of the failures include the ipopt tests against your ipopt wrappers - these are the ones I'm making use of so that's the focus. What would you suggestion to fix these build errors? |
|
Is this a Debug or Release build ? If it is a release, try with debugging for more error messaging. |
|
release.
will debugging give more info on the missing library? |
More info. You can run |
|
using
|
|
Try adding the directory that contains the missing library to your PATH (windows looks for dlls in its path). |
|
Ok, so that is to fix the most recent debug build failing that you asked for? Or that is to fix Also, do you know where that location of these can typically be found, it's not a dll I've actively installed? Inside MSYS2 or inside windows? |
|
How did you get Ipopt ? The check for nan error means the directory with the lib_cppad.dll may not be in your path. |
|
I just downloaded and compiled it from source under the same process/os/compilers etc |
|
There is an option to have cppad get its optional packages and install them locally; see |
|
getting a bit lost with what I should try next. do you have any advice on where I could typically find I will go back to a release build and ensure ipopt is in the path |
|
Thanks. I'll take a look. I'm on windows 11 and already have MSVC installed. It's not clear that it shouldn't be there. But I'll look and get back to this thread. |
|
You can get a conda-forege install of cppad on windows; see But this uses static libraries. If it turns out form your testing that a dll will work, perhaps that can be changed in the conda-forge version. |
|
I have a fix that may avoid some of the linking problems you are having. Start with the original 20250000.2 source and in the top level directory create the file CMakeLists.patch with the contents below. Then execute the command |
|
I'm working my way through this I've a full stack to compile before I can be sure. Get back to you. Thanks |
|
Ok, so I've managed to build my code entirely via MSYS2 (ucrt64), in release, run it on windows, that is via a powershell and nothing to do with an MSYS2 shell, and get the same results from my code as run on linux. What I can say is that make check still fails - however, something odd is going on. When I run the exe file from a powershell command, it doesn't print anything to the screen, but does run, but the make check command issued from MSYS2 does fail: So within MSYS2 terminal using ucrt64: However, going to a powershell and running the exe: I'm assuming nothing printed to the screen but otherwise running is a pass here? I needed the changes you committed and I needed the diff I've noted above, but in this instance I'm not sure running make check is really telling the whole story. For instance if make check just, made the checks, and I was able to 'run check' in the powershell we may see this as a pass. I would say finally, that a debug build on MSYS2 (ucrt) does not link properly do to missing symbols, so I cannot run the make check process under debug due to that. I noted that build failure above and repeat here for convenience: Would you be content to absorb my changes along side your own? I would add that the shared lib under linux is useful too. Also, I should probably I have protected the shared build more consistently: Should probably respect the What do you think? |
|
I think that this is coming from the following: in |
|
This got to 100% with only a few failures (used -k), it's simply because it's running the exe through MSYS2 which doesn't have the Is that what you expected? |
|
Try installing this in some local location and see if you can use the cppad_lib.dll with a simple program. |
|
I am currently looking into this using conda on windows (conda lets you build with the Visual Studio compiler). I got it all to compile and run the tests using NMake mafiles, but my first attempt at getting it to link did not work. I think I do not have the correct flags either on compile or link. |
That's what I was saying, it does work. I've build my main program, built a cpack installer, installed it under windows, and ran it, somewhere totally different from a powershell. It works fine. Just the 'make checks' that do not run from the build dir when they are executed under MSYS2. |
|
I got it to work in Conda (all the tests passed). The real trick to getting a test program (outside the cppad build directory) to work (for me) was using vcvarsall.bat and using CMake to compile and link the separate test of the dll: Note that CMake can use unix like paths (even though we are on windows). |
|
Yep agreed. As I said in my previous post that's how I compile my code, cmake. I really think the only problem here is MSYS2 when running the tests. |
|
I am going to fully test the change on MSYS2 and if it works, change CMakeLists.txt so that it builds dlls on windows. |
|
Ok. Let me know if I can be any help. I used the UCRT64 variant |
|
I think you mean to path cppad_ipopt/src/CMakeLists.txt ? |
|
Yep. Or rather, just all targets that are made across the code base should respect the user flag for dynamic or static. |
|
cppad_lib and cppad_ipopt are the only two that get installed (and cppad_ipopt has been deprecated for over ten years and perhaps should be removed). |
|
No worries. Just kept them consistent. |
|
OK. I think I have it correct now; see Thanks. |
|
Looks good. Thanks for that. I'll look to write up MSYS2 notes over the next couple of days. |
|
Sorry this took so long. Download MSYS2, and Only use the UCRT64 framework for these notes. Open from the start menu the UCRT64 variant of MSYS2. Type Repeat Then install:
Then after cloning the master of cppad (a build of Ipopt may be required depending on your commands below):
Is this any help? |
|
@a-jp |
|
I think we said above that we expect this: we need to separate the compilation of the test-target, from running it. If you run |
|
I keep forgetting that this is like cross compiling. Replacing the And then opening a powershell window, changing to the cppad.git directory, and executing I see that the executable is there. But if I try to execute it. I do not get any output. Here is what I expect when executing it: |
|
Ok, that one I cannot explain, that is very odd |
|
Would you please try the same test and report what you get. |
|
yep doing that now |
|
same result, runs with no output |
|
Adding a std::cout << "here" << std::endl; at the top of int main inside ipopt_solve.cpp prints nothing - are you sure something is not set to redirect std cout cerr? |
|
Same thing happens for me with the get_started example (which is much simpler) At least I get an error message when I execute the program in dos. |
Nothing prints out on the MSYS2 terminal |
|
Perhaps we should try building a Debug version and see what happens ? |
|
Can you try the new version of powershell from Microsoft pwsh.exe and prove that using that it works fine? I got it for free from the Microsoft store and using it the program runs fine. |
|
OK, please carefully read this, I've resolved the issue and it works just fine in normal Windows Powershell (powershell.exe) PS C:\msys64\home\user\codebase\toolchain\cppad-build> $PSVersionTable.PSVersion
Major Minor Build Revision
----- ----- ----- --------
5 1 26100 2161
PS C:\msys64\home\user\codebase\toolchain\cppad-build> .\example\ipopt_solve\example_ipopt_solve.exe
PS C:\msys64\home\user\codebase\toolchain\cppad-build> $LASTEXITCODE
-1073741515
PS C:\msys64\home\user\codebase\toolchain\cppad-build> $env:PATH +=";C:\msys64\home\user\codebase\toolchain\ipopt-install\bin\"
PS C:\msys64\home\user\codebase\toolchain\cppad-build> $env:PATH +=";C:\msys64\home\user\codebase\toolchain\mumps-install\bin\"
PS C:\msys64\home\user\codebase\toolchain\cppad-build> $env:PATH +=";C:\msys64\ucrt64\bin"
PS C:\msys64\home\user\codebase\toolchain\cppad-build> .\example\ipopt_solve\example_ipopt_solve.exe
here
Begin test group example/ipopt_solve
get_started OK
ode_inverse OK
retape OK
memory_leak OK
All 4 tests passed.
End test group example/ipopt_solve
PS C:\msys64\home\user\codebase\toolchain\cppad-build> $LASTEXITCODE
0The issue is that powershell just fails silently, but if you inspect the exit code, it fails. There is nothing wrong with the build, this is just down to accurately setting the paths. Ignore previous comments about needing to use Powershell (pwsh.exe), version 7.5.0. Although, this also works just fine in that too. Finally, this also works totally fine under MSYS2 terminal once I realised it was the paths that were wrong, moreover, $PATH, not LD_LIBRARY_PATH. user@comp UCRT64 ~/codebase/toolchain/cppad-build
$ ./example/ipopt_solve/example_ipopt_solve.exe
user@comp UCRT64 ~/codebase/toolchain/cppad-build
$ export PATH=$PATH:/home/user/codebase/toolchain/ipopt-install/bin:/home/user/codebase/toolchain/mumps-install/bin
user@comp UCRT64 ~/codebase/toolchain/cppad-build
$ ./example/ipopt_solve/example_ipopt_solve.exe
here
Begin test group example/ipopt_solve
get_started OK
ode_inverse OK
retape OK
memory_leak OK
All 4 tests passed.
End test group example/ipopt_solveThere are no issues, tests work fine, in all terminals, but you'll need to update your test scripts (make check) to ensure PATH! is correctly set. |
|
There is no directory called codebase orr toolchain below my c:\msys64 directory. |
|
These are just commands I typed, you need to point them to your mumps and ipopt bin dirs |
|
The |
|
when I run it under MSYS2, I get: I don't need a script...I've already provided powershell instructions, and MSYS2 instructions. I don't understand your last post can you be more detailed. Also, did you try what I posted and did it work? The point of my last post was to show it working in two terminals just by making sure path is set properly, I don't understand the point of the script? |
|
If I execute the following script below in the powershell, I get a message that a certain entry point is missing (see image below):
|
|
send output |
|
|
Sure you've not mixed debug and release builds? |
|
That was a good idea, but it turned out to not be the problem. I change CMAKE_BUILD_TYPE back to Release, deleted the old P.S. |
Hi,
I'd like to use (unmodified) CppAD within a commercial closed source code that I distribute. To comply with EPL2.0 I must (for practical purposes) link to a shared library, as static libraries require me to provide a mechanism for the person at the other end to link to a different version of the cppad libraries if they wish, which is practically very challenging while keeping the source closed.
There is a section in the cmakelists:
If I'm on MSYS2 and request cppad_static_lib to be false, it will be ignored. Is this something that can be reworked, to allow all cppad libs (including the bindings to ipopt also which I use) to be dynamic rather than static?
On a related note, I see on linux that a shared object will be built, but that libcppad_ipopt seems to always be a static library. Again, is this something that could be changed to always be dynamic?
Many thanks,
The text was updated successfully, but these errors were encountered: