[vulkan] Different behaviour for max_color_attachment_bytes_per_sample in 23.0

[Azkellas]

I have a program that sets the device required limits of max_color_attachment_bytes_per_sample to 128 and later creates a pipeline requiring 48 bytes of color attachments.

In 22.0 the adapter limits returns 32 bytes but I can request how much I want in the device creation and the pipeline runs fine.
In 23.0 the adapter limits also returns 32 bytes but I can't request 128 in the device required limits. If I request 32 then the pipeline creation fails because the pipeline is asking for 48 bytes while the device only allows for up to 32.

Panics in adapter.request_device: Unable to find a suitable GPU adapter!: RequestDeviceError { inner: Core(LimitsExceeded(FailedLimit { name: "max_color_attachment_bytes_per_sample", requested: 128, allowed: 32 })) }

or in device.create_render_pipeline: [2025-01-03T21:11:49Z ERROR wgpu_core::device::global] Device::create_render_pipeline error: The total number of bytes per sample in color attachments 48 exceeds the limit 32

My adapter is the same in both cases: Adapter Vulkan AdapterInfo { name: "NVIDIA GeForce RTX 3080 Ti", vendor: 4318, device: 8712, device_type: DiscreteGpu, driver: "NVIDIA", driver_info: "560.94", backend: Vulkan } on Windows 11.

Requesting a dx12 backend gives me 64 bytes available in both versions of wgpu, but, again, let me ask for more in the device if I want. I haven't tried a pipeline requiring more than 64 though.

Which behaviour is correct? Is the adapter limit wrong and device correct as in 22.0, or is it impossible to have a pipeline of more that 32 bytes per sample for my adapter (in which case, what happened before in 22.0?)

On a side note the pipeline that needs 48 bytes has the following layout:

[
    Rgba8UnormSrgb = 4,
    Rgba32Float = 16,
    R8Unorm = 1,
    R8Unorm = 1,
    R8Unorm = 1,
    Rgba32Float = 16, 
    R32Float = 4
]

How does it count for 48? I would expect 43, 44 or 52 depending on internal optimizations and alignment, but not 48.

Thanks!

Minimal reproducible code

This project runs fine in 22.0 but crashes in adapter.request_device in 23.0 due to the required limits.
Dependencies are

[dependencies]
env_logger = "0.11.6"
futures = "0.3.31"
log = "0.4.22"
logging = "0.1.0"
wgpu = "22.0.0"

or wgpu = "23.0.0" to switch version.

Log files
v22.0.0.txt
v23.0.0.txt

async fn start_wgpu() {
    env_logger::Builder::new().filter_level(log::LevelFilter::Trace).init();

    let backends: wgpu::Backends = wgpu::Backends::VULKAN;
    let dx12_shader_compiler = wgpu::Dx12Compiler::default();
    let gles_minor_version = wgpu::Gles3MinorVersion::default();

    let instance = wgpu::Instance::new(wgpu::InstanceDescriptor {
        backends,
        flags: wgpu::InstanceFlags::default(),
        dx12_shader_compiler,
        gles_minor_version,
    });

    // create high performance adapter
    let adapter = instance
        .request_adapter(&wgpu::RequestAdapterOptions {
            power_preference: wgpu::PowerPreference::HighPerformance,
            compatible_surface: None,
            force_fallback_adapter: false,
        })
        .await
        .expect("Unable to find a suitable GPU adapter!");

    println!("Adapter: {:?}", adapter.get_info());
    dbg!(adapter.limits());

    let required_features = wgpu::Features::default();
    let adapter_features = adapter.features();

    let needed_limits = wgpu::Limits {
        max_bind_groups: 8,
        max_color_attachments: 8,
        // ----> This works in 22.0 but not in 23.0
        max_color_attachment_bytes_per_sample: 128,
        ..Default::default()
    }.using_resolution(adapter.limits());

    let trace_dir = std::env::var("WGPU_TRACE");
    let (device, _) = adapter
        .request_device(
            &wgpu::DeviceDescriptor {
                label: Some("Device Descriptor"),
                required_features: adapter_features | required_features,
                required_limits: needed_limits,
                memory_hints: wgpu::MemoryHints::Performance,
            },
            trace_dir.ok().as_ref().map(std::path::Path::new),
        )
        .await
        .expect("Unable to find a suitable GPU adapter!");

    dbg!(device.limits());
}


fn main() {
    futures::executor::block_on(start_wgpu());
}

[teoxoy]

The limit on Vulkan is currently static since there isn't a limit that Vulkan exposes for this but as the comment says we could probably increase it for non-tiled GPUs.

wgpu/wgpu-hal/src/vulkan/adapter.rs

Lines 1053 to 1056 in 826db5e

	// TODO: programmatically determine this, if possible. It's unclear whether we can
	// as of https://github.com/gpuweb/gpuweb/issues/2965#issuecomment-1361315447.
	// We could increase the limit when we aren't on a tiled GPU.
	let max_color_attachment_bytes_per_sample = 32;

You weren't running into this in v22 because the limit check was missing, it got added in 9619a43.

[cwfitzgerald]

Talking with the Dawn devs, they use MAX_ATTACHMENT_COUNT * LARGEST_ATTACHMENT_SIZE for dx12, and do normal binning for vulkan. As discussed in triage today, we should use a similar limit on vulkan. This is a very easy PR, so I'll tackle this shortly.

[cwfitzgerald]

As for your example, there's padding and one bizzare definition making this confusing.

Format	Size	Alignment	Range
Rgba8UnormSrgb	8 (!!!)	1	0..8
Rgba32Float	16	4	8..24
R8Unorm	1	1	24..25
R8Unorm	1	1	25..26
R8Unorm	1	1	26..27
(Padding)			27..28
Rgba32Float	16	4	28..44
R32Float	4	4	44..48
Total			48

Rgba8UnormSrgb being 8 bytes is the surprising number and stems from metal being metal. You can see the whole table here https://gpuweb.github.io/gpuweb/#texture-format-caps under "Render Target Pixel Byte Cost`

[Azkellas]

Awesome, thanks for answer and the resource!