Incorrect adjoint gradients with overlapping design variables

[mochen4]

When the material grids of design variables are overlapped, e.g. the eigenmode tutorial example, the adjoint gradients don't agree with the finite difference gradients.

In the aforementioned example, the gradients are correct if only one block of the design region is kept in the geometry but not both:

geometry=[
...
mp.Block(center=design_region.center, size=design_region.size, material=design_variables), # design region
mp.Block(center=design_region.center, size=design_region.size, material=design_variables,
             e1=mp.Vector3(x=-1).rotate(mp.Vector3(z=1), np.pi/2), e2=mp.Vector3(y=1).rotate(mp.Vector3(z=1), np.pi/2))
]

[smartalecH]

IIRC, the error is a constant error, right? meaning we are just off by a scale factor (i.e. it's not as if the entire gradient is just wrong)?

If that's the case, it should be easy to debug. Perhaps we are double-counting the number of grids somewhere...

[mochen4]

IIRC, the error is a constant error, right? meaning we are just off by a scale factor (i.e. it's not as if the entire gradient is just wrong)?

Yes, I think the gradients are off by a scale factor, but that scale varies a bit with different random structures.

[smartalecH]

I'm pretty sure the issue is due to the following snippet introduced in #1780.

meep/src/meepgeom.cpp

Lines 2758 to 2771 in 11545a1

	if (ukind == material_data::U_MIN && u != uval) return; // TODO look into this
	if (ukind == material_data::U_PROD) scaleby *= uval / u;
	for (int xi=0;xi<lx;xi++){
	for (int yi=0;yi<ly;yi++){
	for (int zi=0;zi<lz;zi++){
	int x=x_list[xi], y=y_list[yi], z=z_list[zi];
	int u_idx = IDX(x,y,z);
	std::complex<double> prod = adj*get_material_gradient(
	r,adjoint_c,forward_c,fwd,freq,geps,gv,du,udata,u_idx);
	D(x, y, z) += prod.real() * scaleby;
	}
	}
	}