WebGPURenderer: Add Storage3DTexture and StorageArrayTexture

[Spiri0]

#31167

This is initially a draft to be able to use texture_starage_2d_array in threejs.webgpu.
I see in the code that preparations for the extension have already been made in some places and therefore I would like to share my ideas here before I unnecessarily go in the wrong direction.

[github-actions]

📦 Bundle size

Full ESM build, minified and gzipped.

	Before	After	Diff
WebGL	337.57 78.74	337.57 78.74	+0 B +0 B
WebGPU	556.8 154.16	557.44 154.26	+646 B +107 B
WebGPU Nodes	556.15 154	556.36 154.05	+217 B +51 B

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

	Before	After	Diff
WebGL	468.75 113.42	468.75 113.42	+0 B +0 B
WebGPU	632.54 171.18	632.75 171.23	+217 B +48 B
WebGPU Nodes	587.59 160.53	587.81 160.58	+217 B +49 B

[Spiri0]

I've started by creating a StorageTexture node for array textures. I see in the code that some things like texture_storage_2d_array and 2d-array for the viewDimension have already been created.

This code at the end of StorageTextureNode.js makes me think. uvNode and storeNode are currently undefined

/**
 * TODO: Explain difference to `storageTexture()`.
 *
 * @tsl
 * @function
 * @param {StorageTexture} value - The storage texture.
 * @param {Node<vec2|vec3>} uvNode - The uv node.
 * @param {?Node} [storeNode=null] - The value node that should be stored in the texture.
 * @returns {StorageTextureNode}
 */
export const textureStore = ( value, uvNode, storeNode ) => {

	const node = storageTexture( value, uvNode, storeNode );

	if ( storeNode !== null ) node.toStack();

	return node;

};

Here, however, the possibility of 3 dimensionality is already hinted at @param {Node<vec2|vec3>} uvNode - The uv node.
My idea was to use this.isStorageTextureArray = true; as an indicator for a StorageTextureArray and then to incorporate the appropriate case distinctions wherever necessary in the threejs code. But that only makes sense if my thoughts are in line with any ideas that may already be in the planning.
@sunag Have you already considered arrays and 3D textures with the uvNode, or does the uvNode have a different background? Currently, it's nonfunctional.
If that doesn't mean anything, I'll continue with my approach with the StorageTextureArray and this.isStorageTextureArray = true; if you think that's a good way.

[Spiri0]

I can now use the texture with the textureStorage node in the compute shader with texture_storage_2d_array.

However, the extension for correctly reading it into the fragment shader with the texture node is still missing. I'm working on that. Currently, it's still recognized as texture_2d.

[sunag]

@Spiri0 What do you think about creating just a new parameter (depth) in StorageTexture and removing the StorageTextureArray.

[Spiri0]

@Spiri0 What do you think about creating just a new parameter (depth) in StorageTexture and removing the StorageTextureArray.

I originally had it that way. Then I just imagined that you might have preferred to separate it. I had depth = 0 in the constructor so that it would be treated as a 2D texture by default. With depth greater than 0, it would be treated as a texture array. I can continue working on it this evening (CET). I'll then look for the reason why the fragment shader recognizes it as texture_2d. I already have an idea where to look.

Here, I'm using texture arrays and copyTextureToTexture to stream large numbers of tiles (4 tiles per layer). Please excuse the poor video quality. I'm new to making videos.

https://youtu.be/UXe4SPWI_8U

All textures are upscaled to 32k. Unfortunately, I can't find any models with very high-quality, detailed textures. I would like to have a city as a model.
With storage texture arrays, shared storage buffers and compute shaders this can be done without copyTextureToTexture.

[Spiri0]

Now it's working. But I'm not satisfied with it yet. I admit I still don't understand why there are two indicators for texture arrays.

isArrayTexture
isTextureArray

I used isTextureArray because I saw it in the StorageTexture when reading it from the console. However, isArrayTexture is from Texture. I would prefer to use isArrayTexture if that's okay. If so, I'll rework that

[Mugen87]

Why do we have isArrayTexture and isTextureArray now? Can't you use the existing isArrayTexture?

[Spiri0]

I generally only use what is already available as a parameter, so I used isTextureArray.
screenshot from a StorageTexture node with r176 without my extension

If isArrayTexture were available, I would have used it. I can, of course, implement it, and that's why I'm asking. But then there would be two indicators for the array state in the StorageTextures, which I don't think is right, because then both would have to be set.
I assume there were reasons why isTextureArray was introduced, but I don't know them, and that's why I'm asking. One indicator is certainly better than two.

I could trace isTextureArray back in the code to where it plays a role. Perhaps it was simply created but not used anywhere yet, since storage textures aren't array-capable yet. Then we could remove isTextureArray from the code and replace it with isArrayTexture.

[mrdoob]

@Spiri0

All textures are upscaled to 32k. Unfortunately, I can't find any models with very high-quality, detailed textures. I would like to have a city as a model.

If you want a challenge...

https://disneyanimation.com/resources/moana-island-scene/

😁

[Spiri0]

If you want a challenge...

https://disneyanimation.com/resources/moana-island-scene/

😁

Wow, I'm downloading this right now. I'm curious to see what it looks like.
With such high-resolution models, I'll surely need my virtual texture system and my virtual geometry system in conjunction to manage the models efficiently.
I hope it's possible to open the models in Blender to convert them to gltf and open the textures in GIMP.
The download will take a while, but I'm really curious to see the data.

Thank you very much mrdoob

[Spiri0]

@Mugen87 I think I've found the cause. On the left is the current release, r176, and on the right is the current dev. I updated before I started the PR 4 days ago. The best thing to do will be to wait for r177 and then I test it with all the changes made then, and with isArrayTexture to ensure it's ready for r178.

[Spiri0]

@sunag Now it's ready for review. Thanks for changing isTextureArray to isArrayTexture. I assume you did that. I didn't want to change a global state parameter without asking. Now the PR looks much more elegant

[Spiri0]

@sunag Can it be merged as it is now or do you have any changes you would like?

[sunag]

I think the default should be 1? and array more than 1...

[Spiri0]

The reason I chose 0 was because someone might come up with the idea of ​​wanting to define an array with a length of 1, for example, when providing an app with a dynamic image array length, because that's possible, and then complain that it doesn't work.

If you're working with a TextureArray on the Threejs side and the WGSLNodeBuilder suddenly wants to use a normal texture because of the length 1, it crashes because of the wrong shader texture format.

A value of 0 clearly defines a normal texture, and a depth > 0 clearly defines a TextureArray.

But if you'd like, I can set that to 1.
In this case, it can be argued that the user has to take this into account on the threejs side, but this significantly increases the effort.

[sunag]

Following this spec, 3DTexture does not support array, so the fact that it has count does not seem to make sense. *ArrayTexture and *3DTexture are distinct and should exist individually.

I think that #30959 needs to be reviewed or the other textures need to be, as there seems to be an incompatibility.

Textures

Property	Description	Type
*Texture	width, height	2D
*ArrayTexture	height, height, depth	Array
*3DTexture	height, height, depth	3D

RenderTarget

Property	Description	Type
*RenderTarget	width, height, { count }	2D
*RenderTargetArray	height, height, depth, { count }	Array
*RenderTarget3D	height, height, depth	3D

RenderTarget (Currently)

Property	Description	Type
*RenderTarget	width, height, { count, depth }	2D/Array
*RenderTarget3D	height, height, depth	3D

Property	Type
width	2D
height	2D
depth	Array/3D
count	RenderTargets count

I have a feeling we should go back to RenderTargetArray and its logic but now preserve the functionality of the count property so we can have Multi Render Target Arrays.

So I would go back in the same way so that we have a StorageArrayTexture and soon Storage3DTexture too.

Changing texture properties I imagine could be a lot of work for us and the users, and this seems like it can be avoided.

@Mugen87 @RenaudRohlinger any thoughts on this?

[Spiri0]

So, should we go back to a separate class like I originally had?

I would prefer that because it allows the user to clearly define the texture format by choosing the respective class:

StorageTexture,
StorageArrayTexture
Storage3DTexture

[Spiri0]

As far as I can see, the places for the case distinction in the WGSLNodeBuilder and WebGPUBindingUtils are the same, which makes things easier.

[sunag]

So, should we go back to a separate class like I originally had?

Yes, it seems like the best way to me.

[Spiri0]

I'll take care of it in the new week. I'll also validate the Storage3DTexture in my app.
Then we have elegantly covered all three variants, because I think the separate classes make things very clear for the user.

[Spiri0]

Okay, both are working as they should. The 3DTexture was a good idea, @sunag. It's important for fog and clouds, like in my first repo, which I made in threejs with WebGL: https://github.com/Spiri0/volumetric-clouds?tab=readme-ov-file
These were my first steps in threejs. With the Storage3DTexture on GPU side, I can create higher resolution cloud structures much faster than with the Data3DTextures on CPU side.

I changed the order in "getUniforms(shaderStage)" in the WGSLNodeBuilder. The reason was that if normal 3DTextures or arrayTextures were checked first, the storage condition would not be met, and this required complicated extra conditions and restrictions. The changed order makes it simple.

isSampledTexture3D replaced by is3DTexture, since it does not matter for the viewDimension whether sampled or not. This avoids unnecessary extra conditions

[sunag]

These TSL functions were declared in some point?

[Spiri0]

Ah, a valid point. For WebGPU, only the export from src/Three.WebGPU.js is necessary.
I can remove the export from src/Three.TSL.js or, what seems more appealing to me, add it for Three.TSL.js, because it should be available there too. But I have to look where.