Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add approximate support for GeLU #36

Merged
merged 6 commits into from
Dec 17, 2024

Conversation

aturker-synnada
Copy link
Collaborator

Description

GeLU now takes approximate flag as init argument. Closes #35.

What is Changed

  • GeLU approximate support added.

Checklist:

  • Tests that cover the code added.
  • Corresponding changes documented.
  • All tests passed.
  • The code linted and styled (pre-commit run --all-files has passed).

@mertbozkir
Copy link

Hello @aturker-synnada, if we add a return, could the CI build pass, right? Looks like fails because of return, anything that I should be aware of? 😅

The other ‘on_failure’ CI pipeline seems to require credentials. How should we handle that?

@aturker-synnada
Copy link
Collaborator Author

Hello @aturker-synnada, if we add a return, could the CI build pass, right? Looks like fails because of return, anything that I should be aware of? 😅

The other ‘on_failure’ CI pipeline seems to require credentials. How should we handle that?

Yes, you are right 😅. I guess it is not about credentials; it should do something when CI failed @mehmetozsoy-synnada

@aturker-synnada
Copy link
Collaborator Author

Fixed mypy error and updated GeLU init arguments.

Copy link

codecov bot commented Dec 17, 2024

Codecov Report

Attention: Patch coverage is 95.65217% with 3 lines in your changes missing coverage. Please review.

Project coverage is 86.51%. Comparing base (4ab5d44) to head (51f8f1c).
Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
mithril/backends/with_autograd/mlx_backend/ops.py 25.00% 3 Missing ⚠️
@@            Coverage Diff             @@
##             main      #36      +/-   ##
==========================================
+ Coverage   86.48%   86.51%   +0.03%     
==========================================
  Files          61       61              
  Lines       15544    15600      +56     
==========================================
+ Hits        13443    13497      +54     
- Misses       2101     2103       +2     
Files with missing lines Coverage Δ
mithril/backends/with_autograd/jax_backend/ops.py 84.81% <100.00%> (ø)
...ithril/backends/with_autograd/torch_backend/ops.py 79.26% <100.00%> (+0.05%) ⬆️
...hril/backends/with_manualgrad/numpy_backend/ops.py 85.13% <100.00%> (+0.12%) ⬆️
...backends/with_manualgrad/numpy_backend/ops_grad.py 86.73% <100.00%> (+0.06%) ⬆️
mithril/framework/constraints.py 94.80% <100.00%> (+0.12%) ⬆️
mithril/framework/logical/essential_primitives.py 99.57% <100.00%> (+<0.01%) ⬆️
mithril/models/primitives.py 97.42% <100.00%> (+<0.01%) ⬆️
mithril/backends/with_autograd/mlx_backend/ops.py 31.65% <25.00%> (-0.23%) ⬇️

@kberat-synnada kberat-synnada merged commit 18000e7 into synnada-ai:main Dec 17, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEATURE] GeLU tanh approximation
4 participants