openGL 2.0 Alpha blending

[xDarkDelux]

hello, help me deal with the problem, because I do not have enough knowledge or experience with rendering.
I have a game that uses directx9 rendering (of course, there is no access to the sources :)), I want to use your library in it to write my GUI. What I want to do - for the best performance, I came up with the idea to use the opengl2 renderer in a separate independent thread with my context, that is, let the game render its main scene with its direct9, and the UI will spin in its thread on opengl2, I managed to do this but there is one problem when the gui element is rendered on top of the main scene - the translucency of the alpha channel does not work, this is how it looks like https://prnt.sc/EOzrM0m_FiSR if you look closely - you will see that the alpha channel works correctly when the element is on top of another element but on the background of the main scene is solid black instead of a light shadow..
this is how i initialize the window to render the gui

static HWND InitializeWindow(HINSTANCE instance_handle, const std::wstring& name, int& inout_width, int& inout_height, bool allow_resize)
{
RECT main_rect;
GetWindowRect(GTopWnd, &main_rect);

WNDCLASSW windraw;
//windraw.cbSize = sizeof(WNDCLASSEX);
windraw.style = CS_HREDRAW | CS_VREDRAW;
windraw.lpfnWndProc = &WindowProcedureHandler;
windraw.cbClsExtra = 0;
windraw.cbWndExtra = 0;
windraw.hInstance = instance_handle;
windraw.hIcon = NULL;
windraw.hCursor = LoadCursor(nullptr, IDC_ARROW);
windraw.hbrBackground = nullptr;
windraw.lpszMenuName = NULL;
windraw.lpszClassName = name.data();

RegisterClassW(&windraw);

HWND window_handle = CreateWindowExW(WS_EX_TOPMOST | WS_EX_LAYERED | WS_EX_NOACTIVATE, name.data(), name.data(),
WS_POPUP, 0, 0, inout_width, inout_height, GTopWnd, NULL, instance_handle, NULL);

SetParent(window_handle, GTopWnd);

//Set the window to transparent
SetLayeredWindowAttributes(window_handle, RGB(0, 0, 0), 0, LWA_COLORKEY);


SetWindowPos(window_handle, NULL, 0, 0, inout_width, inout_height, SWP_SHOWWINDOW);

return window_handle;
}

please, if you can help me suggest what needs to be changed or added to correctly display the alpha channel on the background of the main scene - I will be very grateful

[mikke89]

It sounds pretty wild to use both DirectX and OpenGL in the same application. To me it seems a lot simpler to write a DirectX backend instead.

I guess in your setup there could be many reasons why translucency isn't working. You need to ensure the RmlUi output is actually alpha blended correctly with the DX target when compositing the two, I can't really help you with that.

[xDarkDelux]

Yes, the idea is a bit wild, but the result is more than satisfying. I have no problem synchronizing the interaction with the interface, with all the gui rendered and updated in a separate thread and does not affect the performance of the main game scene, which is already "barely breathing" on the old unreal engine. I have a renderer for your lib on directx 9, the basis are taken from libRocket, but it is very poor performance, for example during the playback of such animation render on Directx9 and OpenGL 2.0 takes 5 ms per tick, which is very bad for me, it is very much.
UBOOL DrawerWindow::ShowWithAnim(const FLOAT& duration)
{
if (Animate(direction, Rml::Property(showing_width, Rml::Property::DP), duration, Rml::Tween{ Rml::Tween::Bounce, Rml::Tween::Out }, 1, false))
{
Show();
return TRUE;
}
return FALSE;
}

That's why I decided to render the ui in a different thread.
What's most interesting, I connected GLFW backend yesterday, and implemented it all in the game, in another thread and in another window on top of the main one. There rendering this animation on OpenGL 3.0 works steadily with no lag in time and fps. And what's funny - the alpha channel out of the box is rendered correctly on the background of the main scene. But there is a problem that some textures are rendered with the wrong transparency indicator. https://prnt.sc/87bdRud_rA3N here you can see how the inner texture is more transparent than the one below it, while it should be so that this texture on top of its background together should give less transparency on the main scene even than the window texture itself. Once again I'm stumped and don't know how to fix it, because the rendering on OpenGL 3 .0 is even more complicated than on 2.0 and with shaders :)
I searched a little and realized that my problem is something like this, but how to fix it, I have no idea :(
https://ru.stackoverflow.com/questions/1304499/%D0%A0%D0%B5%D0%B6%D0%B8%D0%BC-%D1%81%D0%BC%D0%B5%D1%88%D0%B8%D0%B2%D0%B0%D0%BD%D0%B8%D1%8F-%D0%BF%D0%BE%D0%BB%D1%83%D0%BF%D1%80%D0%BE%D0%B7%D1%80%D0%B0%D1%87%D0%BD%D0%BE%D0%B9-%D1%82%D0%B5%D0%BA%D1%81%D1%82%D1%83%D1%80%D1%8B-%D0%B8-%D0%BF%D0%BE%D0%BB%D1%83%D0%BF%D1%80%D0%BE%D0%B7%D1%80%D0%B0%D1%87%D0%BD%D0%BE%D0%B3%D0%BE-%D0%B1%D1%83%D1%84%D0%B5%D1%80%D0%B0

I would be glad to use render gui on directx9 if it were more productive than now or render on OpenGL 2.0 in another thread if it rendered alpha channel on the main scene correctly... I'm even ready to pay someone who can help me with any of my problems, but I don't even know who to ask :(
The only option that I have - is to render to texture in OpenGL 3.0 and create textures for the direct in the same separate thread and with minimal synchronization render that texture in the main thread on Directx9, I think this in any case will be more productive than the render on directx9 which I now have, but not sure that again there will not be some problems that I can not solve :)

[xDarkDelux]

I could give you more information if you are interested, if I had a way to contact you directly via some messenger for example.

[mikke89]

Hm, I don't know what's going on here. This issue is just too specific for me to be of any help... I would rather choose to go the DX backend route, and instead figure out why it is slow. Out of curiosity, what sort of game(s) are you working on top of?

Just an idea, perhaps it would be simpler to blit the DX background over to GL, and then do all the UI rendering on top. Then blit that result back to DX? At least that way you don't have to deal with transparency or blending between graphics API, since all the blending would be done on the GL side.

[xDarkDelux]

In general, I solved the problem with alpha blending, now the alpha channel is working correctly and the GUI is rendered in a separate thread on the GLFW / OpenGL 3, now it remains to figure out how to make this window a child window of the main scene (of course, the window of the main scene is on winapi (win32), and it seems that there is no problem with the code, you can call SetParent - but then the background appears and the window with the gui either blinks (if the fps is low), or simply does not render (if there are a lot of fps (it feels like it is being rendered under the parent)) and again tried many options and can't come up with a single correct one :)

[xDarkDelux]

https://prnt.sc/Qv-Vp4Vusng6
Here's another point that I don't understand at all. Pay attention to the shadow of the elements above and below. Above - this is a window created on the GLFW platform with the OpenGL 3.0 renderer with transparent background settings. Below - the same platform and renderer and the same code, only in the window initialization I added the following code:

` HWND hwNative = glfwGetWin32Window(window);

long ex_style = GetWindowLong(hwNative, GWL_EXSTYLE);
SetWindowLong(hwNative, GWL_EXSTYLE, ex_style | WS_EX_LAYERED | WS_EX_NOACTIVATE);

SetLayeredWindowAttributes(hwNative, RGB(0, 0, 0), 0, LWA_COLORKEY);
SetWindowPos(hwNative, HWND_TOPMOST, 0, 0, width, height, SWP_SHOWWINDOW);

ShowWindow(hwNative, SW_SHOW);`

Why is this happening and how to solve it? What is the problem with win32 windows? :)