Ability to paint a lot of different textures

[Zylann]

Currently the plugin is limited to 4, or maybe up to 8 different textures. Adding more would drop performance significantly.

In order to support more, we could use texture arrays in a new shader, but Godot doesn't support them yet.

Writing an atlas-based shader could help emulating texture arrays, however it has annoying limitations such as pixel bleeding with mipmaps and filter.

Eventually there should be a more flexible API so shaders can be switched and the editor can adapt to which features it supports.

Depends on godotengine/godot#9008

[pgruenbacher]

looks like the sampler3D got implemented. godotengine/godot#20597

[Zylann]

Yep, didn't get the time to work on it yet

[Cyrus-Harding]

@Zylann Any possibilty to impliment this? Is it an easy task, or does it require a large rewrite?

[Zylann]

@kimamb it needs to introduce a new type of splatmap and an entirely new shader. It will also look a bit different in quality, from the few tests I've done in the past. But so far I didnt get the time to get it implemented.

[Cyrus-Harding]

@Zylann I see. If it's anything to go by; I would really apreciate if you could look at it, though I realize that you are a busy person.

I believe that the two most important things to make this usable in a game is this and #55.

I wish I knew enough to contribute to the addon,but this is way above my level of knowledge.

[omomthings]

+1 for this

[Calinou]

@omomthings Please don't bump issues without contributing significant new information; use the 👍 reaction button on the first post instead.

[TokisanGames]

I modified the shader to provide separate UV scale and opacity per channel. This allows us to scale a stone path separately from the grass or mountain. Also it allows us to use a "snow" texture (white w/ a noise texture for normal/roughness) and then paint it any color we want for dirt or other textures. We can also paint textures to avoid repeating patterns, then use the opacity as a colorist would.

Here the color map opacity for the grass is turned off and for the mountain turned down. The left side is what it would look like in the regular shader. It was impossible to get a good looking blend between grass and colormap dirt without this.

The shader cost is negligible, and the "extra" materials it provides can be a good workaround for the time being. I now have snow, stone path, grass, and rock. Then snow can be painted to unlimited other colors.

This is brown snow depthmap blended with the grass. It took only a few seconds to paint the white snow texture, blend in the grass with the mottled brush on low opacity, then dump brown over the area. The grass has the color map turned off. And of course I could have added more colors to the dirt and blended them all. Since the color map is turned off for the grass we're free to blend the dirt without making a mess in the grass.

Here's the shader so I don't have to fork and submit a PR. But it would be great if added to master.
hterrain_shader.txt

[Zylann]

@tinmanjuggernaut it could be added to the main shader, probably using vec4 for those factors instead of individual floats. That's a clever way to reduce tiling.

[Zylann]

I started experimenting with a new kind of shader, and as you can see the amount of different textures is greatly increased:

And in theory it can go up to 256.
It's done with a texture array and blending two indexed layers at a time with a new kind of splatmap, where R=texture1, G=texture2, B=blend.
The only downside of this approach is that a ground pixel can only blend two textures, so choosing how the brush paints this splatmap is quite a challenge.

At the moment I made it so only textures with an odd index can blend with textures with an even index, while the others don't... it's a bit technical. If you try to paint them, it results in this:

But it can probably be overcomed with careful painting, or maybe baking the map from a color-coded image with carefully chosen indexes using I-dont-know-which-algorithm. Smoke and mirrors :D

[RonanZe]

Really nice!
The even/odd system is need to keep good performances when painting?

[Zylann]

Well, yes and no, it's actually the first way I found how to paint it at all xD I had another idea but sounded too complicated and could still mess up.
I'm not really sure what kind of algorithm should be used. Something that somehow makes sure there is no illegal pixels next to each other, I don't know really. I have a vague idea of what it would be like but it seems it would be painfully slow to do, yeah.
Eventually, importing a color-coded map with a (heavy) algorithm figuring out all the indexes and weights, that could do the trick, but that won't be realtime and again I didnt get any thought into how that would work.

So for now I'm still experimenting, that workflow would need a whole section in the doc because there is a bit of setup to do. I should probably make a branch for people to try out, you could have some ideas.

[Cyrus-Harding]

Looks very promesing.

Yeah, it would be nice if you pushed it to a branch; so that we could try it.

[Zylann]

Pushed my WIP to the texture_array branch

[RonanZe]

I just downloaded the texture array branch but I'm not sure how to use it.
by default I only see the ground0, ground1, ground2, cliff.

I've tried to switch to "Shader Type, Custom",
then load "Custom Shader, array.shader",
then "U Ground Albedo Bump Array" -->New TextureArray
but it's doing nothing

[Zylann]

@RonanZe it needs a whole doc section. There are a bunch of things to do before you can use it, although you did most of it.

1. Create a big square image, with each of your ground textures inside it, arranged in a grid like an atlas, and put it in your project.
2. In the import tab, change the import type to Texture Array. Turn on Repeat, modify the tile counts below to match the grid counts, and click Re-import. Godot will ask you to restart the editor, do that.
3. Now create or find an existing terrain, change its shader to CUSTOM, and assign the shader at res://addons/zylann.hterrain/shaders/array.shader.
4. Under Shader Params, assign the U Ground Albedo Bump Array to the texture array you created earlier
5. At this point, if you don't see the bottom panel update with plenty of texture slots, select another node, then select the terrain again. If that still doesn't work, restart the editor maybe.
6. Now you should be able to paint.

Note that this is a WIP, there can be bugs and things that I have yet to figure out.

[RonanZe]

Thanx! seem to works. Not in the right way --> it paint only flat colors.
But I have the textures slots visible in the editor.
Maybe it's a wrong option in the Import?

I didn't realize that texture array was the same has color atlas.
Isn't to restrictive in terme of resolution? And having large textures (like 4 or 8k) do not sound like an optimal way of using textures.

[Calinou]

And having large textures (like 4 or 8k) do not sound like an optimal way of using textures.

I don't think it's any different from having more (but smaller) textures. Recent desktop/laptop GPUs all support 16384×16384 textures. Even 10-year-old NVIDIA Fermi GPUs do 🙂

Older GPUs like the GeForce 8/9 series are limited to 8192×8192, but these will most likely not work well with the GLES3 renderer anyway (which is a requirement for this plugin).

[Zylann]

@RonanZe

Not in the right way --> it paint only flat colors.

Enable repeat flag in your texture import settings.

Isn't to restrictive in terme of resolution? And having large textures (like 4 or 8k) do not sound like an optimal way of using textures.

I hope that's actually not a limitation for the final game, because it does get imported as a texture array, not as is. So you can go higher if you like, in reality the imported texture will be 3D. It may consume as much memory though, unless compression settings work on these.

I tried another way to kind of automatically assign sensible indexes and weight, but it's barely any better than odd-even. It's still really finnicky to paint. The potential is there, authoring just needs a lot more technique than a classic 4-texture splatmap. nVidia gamework's example also conveniently eludes that :p Other tools like MegaSplat in Unity use texture arrays, I just have no idea how they use them in such a way painting isn't a hassle.

[RonanZe]

I also had to erase the .import dir. Without that, the texture stay locked (by the plug?) and no way to select the flag repeat.

But I also feel the potential! 😋

[Zylann]

I reported it godotengine/godot#38870

[RonanZe]

Not the more efficient way but can be a temp workaround before discovering a better system:
Why not double the texture in the ArrayTexture:
put "Texture 1" in odd (slot 1) and even (slot 2), the same for "Texture 2" (slot 3 and 4) and so on...

I take twice the memory but then you can do additionnals checks after im.get_pixel(x,y) in the "for y ... for x" loops of the hterrain_brush.gd,
and select if you need to paint the odd or even version of "Texture 1".

if c.b > 0.5:
paint "Texture 1" odd --> textureArray slot 1
else:
paint "Texture 1" even --> textureArray slot 2

But maybe it's to hacky or I don't have to whole picture 😄

[Zylann]

@RonanZe I tried the if thing

I tried another way to kind of automatically assign sensible indexes and weight, but it's barely any better than odd-even. It's still really finnicky to paint.

More indexes only dilute the problem but surely allow more wiggle room. It needs to be experimented.

[Zylann]

Ok, this is now merged to master. Painting is still not perfect, but it's relatively usable when used with caution :D

Take a look at the docs for more info: https://github.com/Zylann/godot_heightmap_plugin/blob/master/addons/zylann.hterrain/doc/main.md#array-workflow

[RonanZe]

Great!
Thanx Zylann

[Zylann]

Closing this now that it's possible to paint many textures. That said, there is still room for improvement, so changes can still happen with this workflow.

[TokisanGames]

Master e51b7bd

1. I created a 4x2 texture array for albedo, and another for normals. The plugin detects all 8 channels of albedo. However when I paint, it ignores the albedo and only paints colors of the channels.
   

2. When I run the game:
   

3. Why doesn't this shader doesn't include bump or roughness? Are they not needed for depth blending or realism?

4. Here are my albedo and normal maps, which are in two separate files. Why not have them all in one file? e.g. Each row of the array: albdeo, bump, normal, roughness

[Zylann]

I created a 4x2 texture array for albedo, and another for normals. The plugin detects all 8 channels of albedo. However when I paint, it ignores the albedo and only paints colors of the channels.

Channels have no "color". What you see looks like an overstretch, bad UVs, global map not blending properly, or wrong mip. What are your shader settings, and the import settings of your texture array?

When I run the game

Looks different. Import settings got overwritten I guess. I had to reverse-engineer Godot's .import files for that to not happen, but looks like something went missing maybe? The index map looks interpolated, it should not be.

Why doesn't this shader doesn't include bump or roughness? Are they not needed for depth blending or realism?

It does have the same features as the other shaders, and uses the same packing convention.

Here are my albedo and normal maps, which are in two separate files. Why not have them all in one file? e.g. Each row of the array: albdeo, bump, normal, roughness

It's actually more than that. As I said, it uses the same packing convention. If you want to use all features of the shader, you must make two texture arrays, so two RGBA atlases:

• One where albedo is packed in RGB, and bump in A
• One where normals are packed in RGB, and roughness in A

[TokisanGames]

Same for both normal and albedo arrays:

Ok, on the alpha packing.

[Zylann]

So assuming Godot did import your texture array correctly, I don't see anything wrong here. Maybe the problem is in the shader settings then. UV scale of 0 (like your screenshot shows) or globalmap fading too close are possible scenarios to cause this. Looks like there is an internal problem with the format of the index map as well.

[TokisanGames]

UV Scale of 0 was indeed the cause of the albedo issue.

When I close and reopen Godot now in the editor it looks the same as in the game, with the rainbow borders.

Once I start painting again, the textures shift and go back to the first image, with the borders looking somewhat smooth.

Scale, blend distance or depth blending have no positive effect on the borders.

[Zylann]

Depth blending only works if you alpha-pack bump next to albedo.

[TokisanGames]

Right, and I haven't done that yet. But even with it turned off I still get multicolor borders.

And not just that. The map isn't even accurate. Here's w/ depthmap turned off, and adjusting the painting. Baking the globalmap didn't help. Closing and reopening Godot didn't update it.

[RonanZe]

@Zylann
Maybe you already think about that but just to be sure;

Why not hiding the packed RGB+A optmization think behind the plugin?
The more user friendly solution would be to put the code of you Channel Packer directly in the plugin. Then ask the user to load the Albedo, Bump, Normal and Roughness one by one directly in Godot.

Right now, makes the changes on textures can be slow because you have to always repack the channels. And if you want to test several textures, it became a bit paintful.

It would be good to have that also for Arrays. If arrays where not directly accessible in the Plug but Array where rebuild when you change one texture directly in Godot, it whould make the things easier.

Is it because GDScript is to slow?

[Zylann]

@RonanZe it's not because GDScript is slow (well it does that slowly but it does the job), and in fact I have a branch in which I tried to implement a tool for that. The problem is how to integrate that nicely...
While you may expect to specify every map individually in Godot, these are designer-side images. They should not be imported as is or be present in the final game. That's what will happen if you place them in the project. At least, you should put them in a folder with a .gdignore, and make sure they don't get included in the export. Besides, conversion must happen in editor, not in game, because it's very slow, and after that it need to go through the TextureArray importer which can apply compression and other settings.

For the classic shader, packing was relatively easy with the channel packer, but atlasing is one more step to go through.
So in my test branch, I created a resource type which I called "texture config", which contains the path to a bunch of textures, inside the project or outside of it. It's better as a file because there are lots of paths to specify, and better not re-type them all the time. I spent a while figuring out a custom inspector UI (because defaults were horrible), with a button to generate the final textures from all the inputs. Then you'd run the TextureArray importer on the results. While this tool works, it still feels not well integrated, and not really what you'd expect from a resource file. It just serves as an editor-only tool and does nothing else in the game.
I thought about a custom importer, however such importer needs to import multiple files, and generate multiple files, while chaining with another one. I don't know how to make this work properly, and I defaulted to a custom resource because the inspector of the import dock doesn't seem to be customizable (it only supports key/value properties without any form of nesting).

I can consider merging this "tool" though, but it still needs some work to support either regular textures or atlased ones.