Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refact: the DPA1 descriptor #3696

Merged
merged 22 commits into from
Apr 24, 2024
Merged

refact: the DPA1 descriptor #3696

merged 22 commits into from
Apr 24, 2024

Conversation

iProzd
Copy link
Collaborator

@iProzd iProzd commented Apr 21, 2024

This PR reformat DPA1 descriptor consistent for TF/PT/DP

  • Clean up the implementation of DPA1 descriptor in TF/PT
  • Add numpy implementation for DPA1 descriptor
  • Add serialize and deserialize for DPA1 descriptor in TF/PT/DP
  • Add consistence test for TF/PT/DP

Now DPA1 descriptor in TF and PT&DP are consistent when:
smooth_type_embdding == False
env_protection == 0.0
stripped_type_embedding == False

DPA1 descriptor in PT and DP are always consistent.

Need to fix

  • consistence in TF: when excluded_types != [] and attn_layer > 0.

Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

Copy link

codecov bot commented Apr 21, 2024

Codecov Report

Attention: Patch coverage is 89.03509% with 100 lines in your changes are missing coverage. Please review.

Project coverage is 82.12%. Comparing base (1985631) to head (2a67cea).

Files Patch % Lines
deepmd/dpmodel/descriptor/dpa1.py 88.81% 35 Missing ⚠️
deepmd/tf/descriptor/se_atten.py 81.72% 34 Missing ⚠️
deepmd/dpmodel/utils/network.py 85.71% 10 Missing ⚠️
deepmd/pt/model/descriptor/se_atten.py 95.43% 10 Missing ⚠️
deepmd/pt/model/descriptor/dpa1.py 84.61% 6 Missing ⚠️
deepmd/pt/model/network/layernorm.py 93.61% 3 Missing ⚠️
deepmd/tf/utils/network.py 91.66% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##            devel    #3696      +/-   ##
==========================================
+ Coverage   81.89%   82.12%   +0.23%     
==========================================
  Files         509      511       +2     
  Lines       46591    47364     +773     
  Branches     2952     2952              
==========================================
+ Hits        38156    38899     +743     
- Misses       7542     7572      +30     
  Partials      893      893              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@njzjz njzjz linked an issue Apr 21, 2024 that may be closed by this pull request
deepmd/tf/env.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@wanghan-iapcm wanghan-iapcm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stripped type embedding should be added in a future PR.

deepmd/dpmodel/descriptor/dpa1.py Outdated Show resolved Hide resolved
deepmd/dpmodel/descriptor/dpa1.py Outdated Show resolved Hide resolved
deepmd/dpmodel/descriptor/dpa1.py Outdated Show resolved Hide resolved
deepmd/dpmodel/descriptor/dpa1.py Show resolved Hide resolved
deepmd/dpmodel/descriptor/dpa1.py Show resolved Hide resolved
deepmd/dpmodel/utils/network.py Outdated Show resolved Hide resolved
deepmd/pt/model/network/layernorm.py Outdated Show resolved Hide resolved
source/tests/pt/model/test_dpa1.py Show resolved Hide resolved
deepmd/dpmodel/descriptor/dpa1.py Show resolved Hide resolved
source/tests/pt/model/test_dpa1.py Outdated Show resolved Hide resolved
deepmd/dpmodel/descriptor/dpa1.py Outdated Show resolved Hide resolved
deepmd/dpmodel/descriptor/dpa1.py Outdated Show resolved Hide resolved
deepmd/pt/model/descriptor/se_atten.py Outdated Show resolved Hide resolved
source/tests/tf/test_data_large_batch.py Outdated Show resolved Hide resolved
source/tests/pt/model/test_dpa1.py Outdated Show resolved Hide resolved
source/tests/pt/model/test_dpa1.py Outdated Show resolved Hide resolved
source/tests/pt/model/test_dpa1.py Outdated Show resolved Hide resolved
deepmd/dpmodel/descriptor/dpa1.py Show resolved Hide resolved
@wanghan-iapcm wanghan-iapcm changed the title Reformat DPA1 refact: the DPA1 descriptor Apr 23, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Out of diff range and nitpick comments (1)
deepmd/dpmodel/descriptor/dpa1.py (1)

279-285: Initialization of TypeEmbedNet with hardcoded activation function 'Linear'. Consider making this configurable if flexibility is required.

coderabbitai[bot]
coderabbitai bot previously requested changes Apr 23, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8

Out of diff range and nitpick comments (3)
deepmd/dpmodel/descriptor/dpa1.py (2)

206-206: Ensure clarity on parameter usage in the constructor.

It's important to clearly document which parameters are in use and which are kept for compatibility. This helps maintain clarity and prevent confusion for future developers or users of the API.


614-664: Ensure proper handling of attention layers.

When implementing complex layers like NeighborGatedAttention, it's crucial to ensure that each layer is handled correctly, especially in serialization and deserialization processes to maintain state consistency across sessions.

deepmd/tf/descriptor/se_atten.py (1)

3-3: Consider removing unused imports to keep the code clean and maintainable.

deepmd/tf/descriptor/se_atten.py Show resolved Hide resolved
deepmd/pt/model/descriptor/dpa1.py Show resolved Hide resolved
Copy link
Member

@njzjz njzjz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@deepmodeling deepmodeling deleted a comment from coderabbitai bot Apr 24, 2024
@deepmodeling deepmodeling deleted a comment from coderabbitai bot Apr 24, 2024
@deepmodeling deepmodeling deleted a comment from coderabbitai bot Apr 24, 2024
@iProzd iProzd dismissed coderabbitai’s stale review April 24, 2024 06:23

Some are unnecessary changes requested.

@iProzd iProzd enabled auto-merge April 24, 2024 06:23
@iProzd iProzd added this pull request to the merge queue Apr 24, 2024
Merged via the queue into deepmodeling:devel with commit d0cd9f3 Apr 24, 2024
48 checks passed
@iProzd iProzd deleted the rf_dpa1_0415 branch April 24, 2024 07:36
github-merge-queue bot pushed a commit that referenced this pull request Apr 25, 2024
After #3696, we don't need keras anymore.

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
mtaillefumier pushed a commit to mtaillefumier/deepmd-kit that referenced this pull request Sep 18, 2024
This PR reformat DPA1 descriptor consistent for TF/PT/DP

- Clean up the implementation of DPA1 descriptor in TF/PT
- Add numpy implementation for DPA1 descriptor
- Add `serialize` and `deserialize` for DPA1 descriptor in TF/PT/DP
- Add consistence test for TF/PT/DP

Now DPA1 descriptor in TF and PT&DP are consistent when:
`smooth_type_embdding` == False
`env_protection` == 0.0
`stripped_type_embedding` == False

 DPA1 descriptor in PT and DP are always consistent.

Need to fix 
- consistence in TF: when `excluded_types` != [] and `attn_layer` > 0.

---------

Signed-off-by: Duo <50307526+iProzd@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
mtaillefumier pushed a commit to mtaillefumier/deepmd-kit that referenced this pull request Sep 18, 2024
After deepmodeling#3696, we don't need keras anymore.

Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] pt: refactor DPA-1 in the PyTorch backend
3 participants