Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support for single-precision floating point for fields array functions #1833

Merged
merged 11 commits into from
Dec 1, 2021

Conversation

oskooi
Copy link
Collaborator

@oskooi oskooi commented Nov 20, 2021

#1544 and #1675 added support for the single-precision floating point type for the time-domain and DFT fields. However, the fields array functions such as fields::get_array_slice and fields::get_dft_array only use double precision which unfortunately exacerbates the memory bottleneck described in #1797. This PR modifies all of the array functions to support single precision.

@codecov-commenter
Copy link

codecov-commenter commented Nov 23, 2021

Codecov Report

Merging #1833 (a852f73) into master (e4b845c) will decrease coverage by 0.13%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master    #1833      +/-   ##
==========================================
- Coverage   73.53%   73.40%   -0.14%     
==========================================
  Files          17       17              
  Lines        4879     4896      +17     
==========================================
+ Hits         3588     3594       +6     
- Misses       1291     1302      +11     
Impacted Files Coverage Δ
python/adjoint/optimization_problem.py 56.77% <ø> (ø)
python/adjoint/wrapper.py 98.50% <ø> (ø)
python/simulation.py 76.85% <75.00%> (+0.14%) ⬆️
python/adjoint/utils.py 95.28% <100.00%> (ø)
python/visualization.py 45.65% <0.00%> (-1.12%) ⬇️

@oskooi
Copy link
Collaborator Author

oskooi commented Nov 23, 2021

This is finally ready to be merged. There were lots of (little) changes necessary to make everything work. The documentation for get_array and get_dft_array has been updated to mention that the type of the array elements returned by these two functions is the same as the fields themselves.

For now, the gradient computed by the adjoint solver is still in double precision regardless of the type of the forward and adjoint DFT fields. This is not a performance limitation since the gradient is much smaller in memory consumption than the DFT fields .To support computing the gradient in single precision would require changes to geom_epsilon::eff_chi1inv_row and other material-related functions which currently return the ε tensor in double precision. This can be addressed in a separate PR, if necessary.

@oskooi oskooi marked this pull request as ready for review November 23, 2021 23:40
src/energy_and_flux.cpp Outdated Show resolved Hide resolved
src/meep.hpp Outdated Show resolved Hide resolved
@stevengj
Copy link
Collaborator

The basic principle is to try to keep scalar calculations, like accumulating the integral, in double precision. Use realnum only for arrays like the fields. So, when you take the fields array and integrate it, you convert the value to double precision first (since C++ doesn't support mixed-precision calculations on complex numbers).

It might be helpful to write a little utility function, e.g.

std::complex<double> cdouble(std::complex<float> z) { return std::complex<double>(real(z), imag(z)); }
std::complex<double> cdouble(std::complex<double> z) { return z; }

@oskooi
Copy link
Collaborator Author

oskooi commented Nov 24, 2021

The scalar calculations have been reverted to double precision and the realnum type is now used only for the fields arrays.

There are two slight adjustments to the tolerances of the checks in test_adjoint_solver.py which are actually unrelated to this PR. These were necessary because they were causing test failures on my local i7-7700k machine when using single precision.

@stevengj stevengj merged commit f9bd757 into NanoComp:master Dec 1, 2021
@oskooi oskooi deleted the dft_realnum branch December 1, 2021 02:20
mochen4 pushed a commit to mochen4/meep that referenced this pull request Feb 16, 2022
NanoComp#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup
mochen4 pushed a commit to mochen4/meep that referenced this pull request Feb 16, 2022
NanoComp#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup
mochen4 pushed a commit to mochen4/meep that referenced this pull request Feb 16, 2022
NanoComp#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup
mochen4 pushed a commit to mochen4/meep that referenced this pull request Feb 22, 2022
NanoComp#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup
mochen4 pushed a commit to mochen4/meep that referenced this pull request Feb 22, 2022
NanoComp#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup
mochen4 pushed a commit to mochen4/meep that referenced this pull request Feb 22, 2022
NanoComp#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup
mochen4 pushed a commit to mochen4/meep that referenced this pull request Feb 22, 2022
NanoComp#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup
stevengj added a commit that referenced this pull request Mar 10, 2022
* first draft of loop_in_chunks using grid_vol calculations

* try to only use owned points in loop_in_chunks

* don't unpad if !use_symmetry

* stop vscode from complaining about comments

* rm hack to loop over centered grid

* fix and example

* almost fix

* before fix

* one more loop over num_chunk

* include set

* different approach

* minor

* include climits

* include climits

* fix retval

* clean up

* using flipped

* add missing changes

* fix bug

* fix pml

* Fix memory leaks in SWIG wrappers. (#1826)

* Fix memory leaks in SWIG wrappers.

* move delete into material_free

Co-authored-by: Steven G. Johnson <stevenj@alum.mit.edu>

* Fix adjoint gradient with conductivities (#1830)

* damp_fix

* increase run time

Co-authored-by: Mo Chen <mochen@Mos-MacBook-Pro.local>

* plot geometry for dispersive materials without initializing structure object (#1827)

* plot geometry without initializing structure class

* update docstrings

* rotate epsilon grid by 90 degrees to properly orient axes

* add support for dispersive ε

* update markdown pages from docstrings

* make frequency and resolution parameters of plot2D dictionary keys of eps_parameters

* reinstate frequency parameter of plot2D and add warning that it is deprecated

* fix order of frequency check

* unit test for `get_epsilon_grid` (#1835)

* unit test for get_epsilon_grid

* fix

* limit test points to homogeneous regions (i.e. no material interfaces) and away from potential chunk boundaries

* support for single-precision floating point for fields array functions (#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup

* Fix memory leaks (#1839)

* Fix memory leaks

* Add kkg to authors list

* Fix for Issue #1834  (#1840)

* Fix memory leaks

* Add kkg to authors list

* Expose set_default_material and use it in libpympb/pympb.cpp

* use unique_ptr (C++11) instead of make_unique (C++14) (#1844)

* Use None instead of empty list in constructors (#1846)

* use None

* minor fix

Co-authored-by: Mo Chen <mochen@Mos-MacBook-Pro.local>

* Define what happens when `β=∞` and `u=η` (#1842)

* define what happens when beta=inf and u=0.5

* use eta not 0.5

* Update src/meepgeom.cpp

Co-authored-by: Steven G. Johnson <stevenj@mit.edu>

* fix for arrays (#1845)

* minor improvements to docs (#1848)

* update homebrew instructions for hdf5 and openblas

(fixes #1850)

* recommend python3 on macos

* silence compiler warnings

* whoops, missing commit

* tests need scipy and autograd

* missing sudo

* parameterized package is also used for tests

* h5py and jax on mac

* note on autogen.sh for git clone

* xcode installation shortcut

* bug fix for get_epsilon_point and cell boundary in parallel simulation (#1849)

* bug fix for get_epsilon_point and cell boundary in parallel simulation

* check for six digits in test_material_grid.py because of single precision

* unit test for conductivity (#1857)

* unit test for conductivity

* describe in the docs how to model the attenutation coefficient using conductivity

* Update python/tests/test_conductivity.py

Co-authored-by: Steven G. Johnson <stevenj@mit.edu>

* Fix the failure message for absorber 1D test (#1859)

* add missing fixed field phase to MPB unit test (#1860)

* fix memory leak in array-slice-ll.cpp (#1865)

* fix memory leak in array-slice-ll.cpp

* reinstate line break

* fix memory leak in cyl-ellipsoid-ll.cpp (#1866)

* level parameter for contour plot calls epsilon data (#1869)

* fix heap buffer overflow error for update E from D in cylindrical coordinates (#1871)

* Add cylindrical coordinates support for `plot2d` (#1873)

* add visualization support for plot2d

* bug fix with cartesian plotting

* fix near-field monitor position in cylindrical coordinate tutorial (#1874)

* fix memory leaks in structure and fields load during checkpointing (#1872)

* fix memory leaks in structure and fields load during checkpointing

* delete the chi1inv and fields array if it exists and reallocate

* in unit test, set gaussian source cutoff to 0 due to off-by-1 timestep counter bug

* remove cutoff=0 from unit tests

* lazily allocate H only if B is not NULL

* allocate fields array for H in PML region

* fix two memory leaks in geom_epsilon class (#1877)

* fix two memory leaks in geom_epsilon class

* delete global variable default_material at the end of unit tests

* add unset_default_material function to class meep_geom

* include ring-ll.cpp in C++ unit tests (#1878)

* include ring-ll.cpp in C++ unit tests

* only validate Harminv modes with error below some threshold

* fix and example

* first draft of loop_in_chunks using grid_vol calculations

* try to only use owned points in loop_in_chunks

* don't unpad if !use_symmetry

* stop vscode from complaining about comments

* rm hack to loop over centered grid

* almost fix

* before fix

* one more loop over num_chunk

* include set

* different approach

* minor

* include climits

* include climits

* fix retval

* clean up

* using flipped

* add missing changes

* fix bug

* fix pml

* Fix memory leaks in SWIG wrappers. (#1826)

* Fix memory leaks in SWIG wrappers.

* move delete into material_free

Co-authored-by: Steven G. Johnson <stevenj@alum.mit.edu>

* Fix adjoint gradient with conductivities (#1830)

* damp_fix

* increase run time

Co-authored-by: Mo Chen <mochen@Mos-MacBook-Pro.local>

* plot geometry for dispersive materials without initializing structure object (#1827)

* plot geometry without initializing structure class

* update docstrings

* rotate epsilon grid by 90 degrees to properly orient axes

* add support for dispersive ε

* update markdown pages from docstrings

* make frequency and resolution parameters of plot2D dictionary keys of eps_parameters

* reinstate frequency parameter of plot2D and add warning that it is deprecated

* fix order of frequency check

* unit test for `get_epsilon_grid` (#1835)

* unit test for get_epsilon_grid

* fix

* limit test points to homogeneous regions (i.e. no material interfaces) and away from potential chunk boundaries

* support for single-precision floating point for fields array functions (#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup

* Fix memory leaks (#1839)

* Fix memory leaks

* Add kkg to authors list

* Fix for Issue #1834  (#1840)

* Fix memory leaks

* Add kkg to authors list

* Expose set_default_material and use it in libpympb/pympb.cpp

* use unique_ptr (C++11) instead of make_unique (C++14) (#1844)

* Use None instead of empty list in constructors (#1846)

* use None

* minor fix

Co-authored-by: Mo Chen <mochen@Mos-MacBook-Pro.local>

* Define what happens when `β=∞` and `u=η` (#1842)

* define what happens when beta=inf and u=0.5

* use eta not 0.5

* Update src/meepgeom.cpp

Co-authored-by: Steven G. Johnson <stevenj@mit.edu>

* fix for arrays (#1845)

* minor improvements to docs (#1848)

* update homebrew instructions for hdf5 and openblas

(fixes #1850)

* recommend python3 on macos

* silence compiler warnings

* whoops, missing commit

* tests need scipy and autograd

* missing sudo

* parameterized package is also used for tests

* h5py and jax on mac

* note on autogen.sh for git clone

* xcode installation shortcut

* bug fix for get_epsilon_point and cell boundary in parallel simulation (#1849)

* bug fix for get_epsilon_point and cell boundary in parallel simulation

* check for six digits in test_material_grid.py because of single precision

* unit test for conductivity (#1857)

* unit test for conductivity

* describe in the docs how to model the attenutation coefficient using conductivity

* Update python/tests/test_conductivity.py

Co-authored-by: Steven G. Johnson <stevenj@mit.edu>

* Fix the failure message for absorber 1D test (#1859)

* add missing fixed field phase to MPB unit test (#1860)

* fix memory leak in array-slice-ll.cpp (#1865)

* fix memory leak in array-slice-ll.cpp

* reinstate line break

* fix memory leak in cyl-ellipsoid-ll.cpp (#1866)

* level parameter for contour plot calls epsilon data (#1869)

* fix heap buffer overflow error for update E from D in cylindrical coordinates (#1871)

* Add cylindrical coordinates support for `plot2d` (#1873)

* add visualization support for plot2d

* bug fix with cartesian plotting

* fix near-field monitor position in cylindrical coordinate tutorial (#1874)

* fix memory leaks in structure and fields load during checkpointing (#1872)

* fix memory leaks in structure and fields load during checkpointing

* delete the chi1inv and fields array if it exists and reallocate

* in unit test, set gaussian source cutoff to 0 due to off-by-1 timestep counter bug

* remove cutoff=0 from unit tests

* lazily allocate H only if B is not NULL

* allocate fields array for H in PML region

* fix two memory leaks in geom_epsilon class (#1877)

* fix two memory leaks in geom_epsilon class

* delete global variable default_material at the end of unit tests

* add unset_default_material function to class meep_geom

* include ring-ll.cpp in C++ unit tests (#1878)

* include ring-ll.cpp in C++ unit tests

* only validate Harminv modes with error below some threshold

* fix and example

* first draft of loop_in_chunks using grid_vol calculations

* try to only use owned points in loop_in_chunks

* don't unpad if !use_symmetry

* stop vscode from complaining about comments

* rm hack to loop over centered grid

* almost fix

* before fix

* one more loop over num_chunk

* include set

* different approach

* minor

* include climits

* include climits

* fix retval

* clean up

* using flipped

* add missing changes

* fix bug

* fix pml

* Fix adjoint gradient with conductivities (#1830)

* damp_fix

* increase run time

Co-authored-by: Mo Chen <mochen@Mos-MacBook-Pro.local>

* plot geometry for dispersive materials without initializing structure object (#1827)

* plot geometry without initializing structure class

* update docstrings

* rotate epsilon grid by 90 degrees to properly orient axes

* add support for dispersive ε

* update markdown pages from docstrings

* make frequency and resolution parameters of plot2D dictionary keys of eps_parameters

* reinstate frequency parameter of plot2D and add warning that it is deprecated

* fix order of frequency check

* support for single-precision floating point for fields array functions (#1833)

* switch dft-related functions to using realnum from double

* more fixes

* more type conversions from double to realnum

* adjust check tolerance of tests/integrate.cpp based on floating-point precision

* more fixes

* rebase from master from fix merge conflicts

* slight adjustment to tolerances in unit tests and update docs

* remove type check in test_adjoint_solver.py

* revert return types of integration functions to double

* revert return type of process_dft_component to double

* cleanup

* Fix memory leaks (#1839)

* Fix memory leaks

* Add kkg to authors list

* Fix for Issue #1834  (#1840)

* Fix memory leaks

* Add kkg to authors list

* Expose set_default_material and use it in libpympb/pympb.cpp

* use unique_ptr (C++11) instead of make_unique (C++14) (#1844)

* update homebrew instructions for hdf5 and openblas

(fixes #1850)

* rm hack to loop over centered grid

* almost fix

* one more loop over num_chunk

* different approach

* include climits

* include climits

* clean up

* using flipped

* Revert "fix pml"

This reverts commit 9ed741e.

* fix rebase

* fix rebase

* fix rebase

* fix rebase

* fix rebase

* fix error

* increase tol

* cleanup

* cleanup

Co-authored-by: Steven G. Johnson <stevenj@alum.mit.edu>
Co-authored-by: Mo Chen <mochen@Mos-MacBook-Pro.local>
Co-authored-by: Mo Chen <mochen@dhcp-10-29-91-247.dyn.MIT.EDU>
Co-authored-by: Krishna Gadepalli <34969407+kkg4theweb@users.noreply.github.com>
Co-authored-by: Ardavan Oskooi <ardavano@google.com>
Co-authored-by: Steven G. Johnson <stevenj@mit.edu>
Co-authored-by: Alec Hammond <alec.m.hammond@gmail.com>
Co-authored-by: Andreas Hoenselaar <ahoens@google.com>
Co-authored-by: simbilod <46427609+simbilod@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants