inconsistent gradients with fd simulations

[MarkMa1990]

Hi all,

I have a question about the scaling factor for setting adjoint sources. I found it was set to meep.GaussianSource(fcen, fwidth=0.1*fcen) for the testing example (see code link below). then, I did a robust test by tuning this bandwidth factor. The scaling factor turns out to be very sensitive to the bandwidth. It seems that the adjoint field E_adj are really different by a phase factor. and the phase factor is not well considered for adjoint sources at several different bandwidth.

Structure, monitor, and source setting

Test

df is a factor for the adjoint source profile meep.GaussianSource(fcen, fwidth=df*fcen).

Code in python library

Code: https://nbviewer.org/github/NanoComp/meep/blob/master/python/examples/adjoint_optimization/01-Introduction.ipynb

Meep version: 1.18.0, 1.19.0, 1.20.0, 1.21.0, 1.22.0, 1.23.0

[smartalecH]

Hmm as @mochen4 pointed out, I thought we fixed this in #1515...

I know our tests currently check at arbitrary bandwidths...

[mochen4]

Hmm as @mochen4 pointed out, I thought we fixed this in #1515...

I know our tests currently check at arbitrary bandwidths...

For the tests, #1515 changed df to 0.23 but #1855 changed it back to 0.2.

meep/python/tests/test_adjoint_solver.py

Line 50 in 91d36ba

df = 0.2*fcen

[smartalecH]

Sure but test_adjoint_jax.py still tests different values for the bandwidth:

meep/python/tests/test_adjoint_jax.py

Lines 168 to 179 in 91d36ba

	('1500_1550bw_01relative_gaussian_port1',
	onp.linspace(1 / 1.50, 1 / 1.55, 3).tolist(), 0.1, 0.5, 0),
	('1550_1600bw_02relative_gaussian_port1',
	onp.linspace(1 / 1.55, 1 / 1.60, 3).tolist(), 0.2, 0.5, 0),
	('1500_1600bw_03relative_gaussian_port1',
	onp.linspace(1 / 1.50, 1 / 1.60, 4).tolist(), 0.3, 0.5, 0),
	('1500_1550bw_01relative_gaussian_port2',
	onp.linspace(1 / 1.50, 1 / 1.55, 3).tolist(), 0.1, 0.5, 1),
	('1550_1600bw_02relative_gaussian_port2',
	onp.linspace(1 / 1.55, 1 / 1.60, 3).tolist(), 0.2, 0.5, 1),
	('1500_1600bw_03relative_gaussian_port2',
	onp.linspace(1 / 1.50, 1 / 1.60, 4).tolist(), 0.3, 0.5, 1),

[ianwilliamson]

Also see #1541.

Despite the tests checking different values for the forward source bandwidth, this still seemed to be an issue for me and I had trouble pinning it down.

[mochen4]

Actually, I realized that the issue is about the bandwidth of the adjoint source, i.e. the hardcoded 0.1 here:

meep/python/adjoint/objective.py

Line 102 in 619c879

def _create_time_profile(self, fwidth_frac=0.1):

In this case, #1515 and the tests are probably irrelevant, because they are about the bandwidth of the forward source.

I did some more tests, and found something similar to what was originally suggested in #1507 (comment). The additional phase for adjoint source is np.exp(2j * np.pi * cutoff / fwidth_frac), where cutoff is the another parameter for the Gaussian source and is default to 5. Also notice that this phase is the conjugate of the phase in #1507 (comment), which was about the bandwidth of the forward source.