Skip to content

Fix Vulkan build issues, enable Vulkan build in CI #6709

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

vmedea
Copy link
Contributor

@vmedea vmedea commented May 4, 2025

In 133b033, SCP_vector was made into a subclass, so direct assignment from the vulkan-hpp result isn't possible anymore. Use an intermediate vector.

@Goober5000 Goober5000 added the vulkan Issues and Features related to the vulkan render backend label May 4, 2025
@Goober5000
Copy link
Contributor

@vmedea In order to fix the CI checks, could you change .github/workflows/test-pull_request.yaml line 14 from

cmake_options: [""]

to

cmake_options: -DFSO_BUILD_WITH_VULKAN=ON

@vmedea
Copy link
Contributor Author

vmedea commented May 5, 2025

@vmedea In order to fix the CI checks, could you change .github/workflows/test-pull_request.yaml line 14 from

Done!

@Goober5000
Copy link
Contributor

Goober5000 commented May 5, 2025

D'oh...

Invalid workflow file: .github/workflows/test-pull_request.yaml#L14
The workflow is not valid. .github/workflows/test-pull_request.yaml (Line: 14, Col: 24): Unexpected value '-DFSO_BUILD_WITH_VULKAN=ON'

@vmedea Can you try it with brackets and quotes? So:

cmake_options: ["-DFSO_BUILD_WITH_VULKAN=ON"]

@vmedea vmedea force-pushed the fix-vulkan-build branch from 1646807 to 4606a4f Compare May 5, 2025 17:04
@vmedea
Copy link
Contributor Author

vmedea commented May 5, 2025

Did that now! But it looks like the Linux build image doesn't have Vulkan installed, or the path needs to be passed in:

CMake Error at /usr/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Vulkan (missing: Vulkan_INCLUDE_DIR) (found version "")
Call Stack (most recent call first):
  /usr/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.28/Modules/FindVulkan.cmake:600 (find_package_handle_standard_args)
  lib/imgui/CMakeLists.txt:42 (find_package)

@Goober5000
Copy link
Contributor

Well I'm not exactly sure what to do here, but try taking the following code (copied from the Windows and Mac sections) and putting it in the Linux section:

      - name: Prepare Vulkan SDK
        uses: humbletim/[email protected]
        with:
          vulkan-query-version: 1.4.304.1
          vulkan-components: Vulkan-Headers, Vulkan-Loader
          vulkan-use-cache: true

@vmedea vmedea force-pushed the fix-vulkan-build branch 3 times, most recently from 732ba0d to 475548b Compare May 7, 2025 12:21
@vmedea
Copy link
Contributor Author

vmedea commented May 7, 2025

It looks like the problem it now runs into comes from: https://github.com/scp-fs2open/docker-images/blob/main/build/linux/Dockerfile#L14

The path https://packages.lunarg.com/vulkan/1.3.224 doesn't exist anymore (LunarG must have deleted it). The newest version available there that still has Ubuntu Focal is https://packages.lunarg.com/vulkan/1.3.283/.

Edit: But that script already installs libvulkan-dev through apt-get so theoretically it shouldn't need to be installed through an explicit step. Let's see if it's just a matter of finding it.

Aargh. Installing libvulkan-dev from lunarg's repository for focal does not seem to actually give the development header files, only cmake metadata:

/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/cmake
/usr/lib/x86_64-linux-gnu/cmake/VulkanLoader
/usr/lib/x86_64-linux-gnu/cmake/VulkanLoader/VulkanLoaderConfig-none.cmake
/usr/lib/x86_64-linux-gnu/cmake/VulkanLoader/VulkanLoaderConfig.cmake
/usr/lib/x86_64-linux-gnu/cmake/VulkanLoader/VulkanLoaderConfigVersion.cmake
/usr/lib/x86_64-linux-gnu/pkgconfig
/usr/lib/x86_64-linux-gnu/pkgconfig/vulkan.pc
/usr/share
/usr/share/doc
/usr/share/doc/libvulkan-dev
/usr/share/doc/libvulkan-dev/changelog.Debian.gz
/usr/share/doc/libvulkan-dev/copyright
/usr/lib/x86_64-linux-gnu/libvulkan.so

Not sure if this is a mistake in the package or something else. Ubuntu 24.04's package has them, but that's the distro's default package not LunarG's.

It's intentional: vulkan-headers contains the headers. This is not currently installed.

@Goober5000
Copy link
Contributor

Huh, good detective work. 👍 I've approved and merged the other ticket.

@Goober5000
Copy link
Contributor

Well merging that ticket produced an error in the build-images repository:

#16 [linux/amd64  7/16] RUN apt-get -yq update && apt-get -yq install cmake=3.28.1-0kitware1ubuntu20.04.1 	ninja-build libopenal-dev libreadline6-dev libpng-dev libjpeg62-dev 	liblua5.1-0-dev libjansson-dev libsdl2-dev libfreetype6-dev valgrind g++-9 g++-13 	libvulkan-dev vulkan-headers vulkan-validationlayers libvulkan1 python3.8 python3.8-dev libxcb-glx0-dev
#16 0.384 Hit:1 https://packages.lunarg.com/vulkan/1.3.283 focal InRelease
#16 0.407 Hit:2 http://security.ubuntu.com/ubuntu focal-security InRelease
#16 0.489 Hit:3 http://archive.ubuntu.com/ubuntu focal InRelease
#16 0.586 Hit:4 https://apt.kitware.com/ubuntu focal InRelease
#16 0.630 Hit:5 http://archive.ubuntu.com/ubuntu focal-updates InRelease
#16 0.646 Hit:6 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu focal InRelease
#16 0.772 Hit:7 http://archive.ubuntu.com/ubuntu focal-backports InRelease
#16 0.789 Hit:8 http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu focal InRelease
#16 1.125 Reading package lists...
#16 2.001 Reading package lists...
#16 2.891 Building dependency tree...
#16 3.045 Reading state information...
#16 3.104 python3.8 is already the newest version (3.8.10-0ubuntu1~20.04.18).
#16 3.104 python3.8 set to manually installed.
#16 3.104 Some packages could not be installed. This may mean that you have
#16 3.104 requested an impossible situation or if you are using the unstable
#16 3.104 distribution that some required packages have not yet been created
#16 3.104 or been moved out of Incoming.
#16 3.104 The following information may help to resolve the situation:
#16 3.104 
#16 3.104 The following packages have unmet dependencies:
#16 3.164  cmake : Depends: cmake-data (= 3.28.1-0kitware1ubuntu20.04.1) but 4.0.2-0kitware1ubuntu20.04.1 is to be installed
#16 3.164          Recommends: gcc but it is not going to be installed
#16 3.164          Recommends: make
#16 3.175 E: Unable to correct problems, you have held broken packages.
#16 ERROR: process "/bin/sh -c apt-get -yq update && apt-get -yq install cmake=3.28.1-0kitware1ubuntu20.04.1 \tninja-build libopenal-dev libreadline6-dev libpng-dev libjpeg62-dev \tliblua5.1-0-dev libjansson-dev libsdl2-dev libfreetype6-dev valgrind g++-9 g++-13 \tlibvulkan-dev vulkan-headers vulkan-validationlayers libvulkan1 python3.8 python3.8-dev libxcb-glx0-dev" did not complete successfully: exit code: 100

@vmedea
Copy link
Contributor Author

vmedea commented May 8, 2025

Well merging that ticket produced an error in the build-images repository:

My guess is that something in kitware's cmake repository changed since last image build. None of the vulkan packages depends on any version of cmake so that's not the reason.

This fixes the conflict locally: scp-fs2open/docker-images#5

@Goober5000
Copy link
Contributor

Goober5000 commented May 9, 2025

Great catch! The PR has been merged so I restarted the workflow checks.

EDIT: Bleh, it's back to failing on the Vulkan_INCLUDE_DIR

@vmedea vmedea force-pushed the fix-vulkan-build branch from 73c8578 to 8eda5ca Compare May 11, 2025 12:05
@vmedea
Copy link
Contributor Author

vmedea commented May 11, 2025

Bumped the image, it gets past the cmake stuff now ! 🎉

It's still failing, but at least in a useful way, will look into handling the return codes here:

/__w/fs2open.github.com/fs2open.github.com/code/graphics/vulkan/RenderFrame.cpp:23:31: error: ignoring return value of 'vk::Result vk::Device::waitForFences(const vk::ArrayProxy<const vk::Fence>&, vk::Bool32, uint64_t, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic; vk::Bool32 = unsigned int; uint64_t = long unsigned int]', declared with attribute 'nodiscard' [-Werror=unused-result]
/__w/fs2open.github.com/fs2open.github.com/code/graphics/vulkan/RenderFrame.cpp:44:37: error: ignoring return value of 'vk::Result vk::Device::acquireNextImageKHR(vk::SwapchainKHR, uint64_t, vk::Semaphore, vk::Fence, uint32_t*, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic; uint64_t = long unsigned int; uint32_t = unsigned int]', declared with attribute 'nodiscard' [-Werror=unused-result]
/__w/fs2open.github.com/fs2open.github.com/code/graphics/vulkan/RenderFrame.cpp:88:34: error: ignoring return value of 'vk::Result vk::Queue::presentKHR(const vk::PresentInfoKHR&, const Dispatch&) const [with Dispatch = vk::DispatchLoaderDynamic]', declared with attribute 'nodiscard' [-Werror=unused-result]

@vmedea
Copy link
Contributor Author

vmedea commented May 12, 2025

No failing checks anymore, but it looks like 6 of the Linux builds are hanging (have been since yesterday). There appears to be no way to view the logs in progress.

@Goober5000
Copy link
Contributor

Strange, it's attempting to check more builds than it has run. Those checks don't even have logs; they are waiting for something that doesn't even exist.

I have to assume this is related to the change in build configuration, but I'm not sure how.

@Goober5000
Copy link
Contributor

Ok let me double check a few things. Thanks for your perseverance!

@vmedea vmedea force-pushed the fix-vulkan-build branch from c2dfb0e to 7e269ca Compare May 14, 2025 13:59
@Goober5000
Copy link
Contributor

I'd like to ask if you could remove your workaround in configure_cmake.sh so that we can see the checks that are in limbo again. I want to see if this can be fixed in the configuration without requiring the hack. We can try a few things, and if it doesn't work we can always put the hack back in.

Copy link
Contributor

@Goober5000 Goober5000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's try something first - see previous reply

@vmedea vmedea force-pushed the fix-vulkan-build branch from 7e269ca to 02e0f6c Compare May 18, 2025 12:39
@vmedea
Copy link
Contributor Author

vmedea commented May 18, 2025

I'd like to ask if you could remove your workaround in configure_cmake.sh so that we can see the checks that are in limbo again.

Sure, restored the yaml change instead.

@Goober5000
Copy link
Contributor

Thanks. I think I'm getting closer to solving this. Could you revert the change from this comment above, so I can see what that does?
#6709 (comment)

@vmedea vmedea force-pushed the fix-vulkan-build branch from 02e0f6c to 044550d Compare May 19, 2025 06:56
@Goober5000
Copy link
Contributor

In the current PR, at least, it looks like cmake_options still has -DFSO_BUILD_WITH_VULKAN=ON. Could you put that back to [""]?

@vmedea vmedea force-pushed the fix-vulkan-build branch from 044550d to f5938f2 Compare May 21, 2025 13:22
@vmedea
Copy link
Contributor Author

vmedea commented May 21, 2025

In the current PR, at least, it looks like cmake_options still has -DFSO_BUILD_WITH_VULKAN=ON. Could you put that back to [""]?

Sorry, thought reverting to before making it a yaml list. It's gone completely now.

Doesn't look like it's building with Vulkan now:

-- With Vulkan: OFF

There's some clang tidy error though, will look into it:

/__w/fs2open.github.com/fs2open.github.com/code/graphics/vulkan/VulkanRenderer.cpp:343:32: error: no member named 'init' in 'vk::DispatchLoaderStatic' [clang-diagnostic-error]
        VULKAN_HPP_DEFAULT_DISPATCHER.init(vkGetInstanceProcAddr);
                                      ^
/__w/fs2open.github.com/fs2open.github.com/code/graphics/vulkan/VulkanRenderer.cpp:430:32: error: no member named 'init' in 'vk::DispatchLoaderStatic' [clang-diagnostic-error]
        VULKAN_HPP_DEFAULT_DISPATCHER.init(instance.get());
                                      ^

Edit: fixed this by passing the right define to clang-tidy to use the dynamic dispatcher. It doesn't have to do with cmake settings. The compile flags used by clang-tidy are entirely separate from those derived by cmake.

@vmedea vmedea force-pushed the fix-vulkan-build branch 4 times, most recently from 8b53f2d to 028ab50 Compare May 21, 2025 19:57
@Goober5000
Copy link
Contributor

Goober5000 commented May 22, 2025

I believe you've squashed the last issue. Well done! All the checks have passed, there are no more fiddly cmake configuration changes, and that extra clang-tidy arg looks like it should work with or without Vulkan. I want to double-check with someone before I approve it, but I think this is good to go. Thanks again for your perseverance!

@vmedea
Copy link
Contributor Author

vmedea commented May 22, 2025

Yess!

The only thing is, as far as i can see vulkan is no longer enabled for the CI. The vulkan code is checked by clang-tidy, which is good, but not compiled in any of the runs:

-- Found Vulkan: /usr/lib/x86_64-linux-gnu/libvulkan.so (found version "1.3.283") found components: glslc glslangValidator 
-- Enabling Vulkan support
...
-- With Vulkan: OFF

"Enabling Vulkan support" is misleading here. It comes from OpenXR, not from FSO's own cmake file.

@Goober5000
Copy link
Contributor

Good point. I'll see about adding it to the CI runs.

@Goober5000
Copy link
Contributor

Goober5000 commented May 27, 2025

@vmedea So here is a conundrum. Take a look at this line:
https://github.com/scp-fs2open/fs2open.github.com/blob/master/code/CMakeLists.txt#L121C108-L121C108

If Vulkan is present, that flag should be automatically included. So it shouldn't need to be explicitly specified to clang-tidy.

...Actually, that probably answers the question. The build is being performed without Vulkan support, but clang is forced to run on that file because the file was edited as part of the PR.

So let me ask you to try the following:

  1. Remove the changes to clang_tidy.sh
  2. Add the changes from the new PR Add Vulkan Build Checks #6734 to your PR (I think that PR's CI checks are failing because they need the fixes from this PR)
  3. ???
  4. Profit!

(I believe, if all of that works, we'll have to override, suppress, or force-approve the clang-tidy failures since clang-tidy is created a build configuration that shouldn't normally be possible.)

@Goober5000
Copy link
Contributor

@vmedea bump

@vmedea
Copy link
Contributor Author

vmedea commented Jun 1, 2025

If Vulkan is present, that flag should be automatically included. So it shouldn't need to be explicitly specified to clang-tidy.

When i checked it seemed to me like there is no mechanism that passes the cmake-provided flags to clang-tidy. But it'd make sense.

E.g. there's nothing here that passes any compiler flags at all: https://github.com/scp-fs2open/fs2open.github.com/blob/master/ci/linux/clang_tidy.sh#L22 . It's not launched through cmake, nor make/ninja where DEFINES could be passed in. But directly from the CI runner. But maybe i'm missing something.

So let me ask you to try the following:

Will try!

vmedea and others added 4 commits June 1, 2025 13:30
In 133b033, SCP_vector was made into a
subclass, so direct assignment from the vulkan-hpp result isn't possible
anymore. Use an intermediate vector.
Because these functions can return statuses that aren't strictly an
error (and thus don't raise an exception), but might have to be handled,
they return a `vk::Result` that's `[[nodiscard]]`. Ideally some of the
result statuses would be handled here, but to get it to compile, cast
the result to void.
Switch to the new Linux docker image with Vulkan headers from
scp-fs2open/docker-images#7, for all github builds.
@vmedea vmedea force-pushed the fix-vulkan-build branch from 028ab50 to 6c19386 Compare June 1, 2025 13:31
@Goober5000
Copy link
Contributor

If Vulkan is present, that flag should be automatically included. So it shouldn't need to be explicitly specified to clang-tidy.

When i checked it seemed to me like there is no mechanism that passes the cmake-provided flags to clang-tidy. But it'd make sense.

E.g. there's nothing here that passes any compiler flags at all: https://github.com/scp-fs2open/fs2open.github.com/blob/master/ci/linux/clang_tidy.sh#L22 . It's not launched through cmake, nor make/ninja where DEFINES could be passed in. But directly from the CI runner. But maybe i'm missing something.

Ah, I see what you mean. I believe what's happening is that the cmake configuration causes that define to be associated with the code project that is built as part of Visual Studio or the makefiles or what have you. But as you pointed out, none of that is passed through to clang-tidy when that's called by the shell script.

So please put that definition back into the script and let's see what happens. In fact, to be consistent with the cmake file, put all the definitions in: WITH_VULKAN, VULKAN_HPP_DISPATCH_LOADER_DYNAMIC=1, and VK_NO_PROTOTYPES

`clang-tidy` doesn't pass in the compilation flags normally provided by cmake,
so add the vulkan flags manually to make sure that everything is checked
as it should be, and thus prevent it from choosing the static dispatcher
and complaining about `.init()` being used.
@vmedea
Copy link
Contributor Author

vmedea commented Jun 2, 2025

I believe what's happening is that the cmake configuration causes that define to be associated with the code project that is built as part of Visual Studio or the makefiles or what have you. But as you pointed out, none of that is passed through to clang-tidy when that's called by the shell script.

Exactly. In some projects, a header file is generated with options and defines, and included. But not here, it's all passed on the compiler command line by the build system. Which is fine, but it means external tooling can't access them.

So please put that definition back into the script and let's see what happens. In fact, to be consistent with the cmake file, put all the definitions in: WITH_VULKAN, VULKAN_HPP_DISPATCH_LOADER_DYNAMIC=1, and VK_NO_PROTOTYPES

Yes good idea, added!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
vulkan Issues and Features related to the vulkan render backend
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants