deps: limit dependency ranges #54
Conversation
- automatic minor upgrades, must check for major Signed-off-by: Anh-Uong <anh.uong@ibm.com>
- splitup flash-attn and core requirements Signed-off-by: Anh-Uong <anh.uong@ibm.com>
|
relates/depends on #68 |
requirements.txt
Outdated
| sentencepiece>=0.1.99,<2.0 | ||
| tokenizers>=0.13.3,<1.0 | ||
| tqdm>=4.66.2,<5.0 | ||
| trl>=0.7.10,<1.0 |
There was a problem hiding this comment.
for libraries that are not yet 1.0, I think it is not guaranteed that there won't be any API breaking change between subversions, so it is probably better to be more conservative about their versions in range.
There was a problem hiding this comment.
applies less for accelerate though, since that library just never went 1.0 but is very popular, so unlikely to make such changes now without big splash.
Signed-off-by: Anh-Uong <anh.uong@ibm.com>
Signed-off-by: Anh-Uong <anh.uong@ibm.com>
Signed-off-by: Anh-Uong <anh.uong@ibm.com>
Signed-off-by: Anh-Uong <anh.uong@ibm.com>
|
@gkumbhat do you mind giving this another review? I agree with your point that
but I'm not sure what more conservative version to set as the upper limit, any suggestions? |
|
@tedhtchang if you can also give this another review pass, many thanks! |
There was a problem hiding this comment.
@anhuong Could you rebase.
For the important packages, I think we could set more conservative limit for those lower than 1.0.
In new PR we can figure out how we freeze packages versions in case we need to create older commit.
pyproject.toml
Outdated
| "fire", | ||
| "simpleeval", | ||
| "numpy>=1.26.4,<2.0", | ||
| "accelerate>=0.20.3,<1.0", |
There was a problem hiding this comment.
perhaps set more conservative 0.4 (current 0.31) since it's lower than 1.0
pyproject.toml
Outdated
| "accelerate>=0.20.3,<1.0", | ||
| "transformers>=4.34.1,<5.0,!=4.38.2", | ||
| "torch>=2.2.0,<3.0", | ||
| "sentencepiece>=0.1.99,<1.0", |
There was a problem hiding this comment.
perhaps set more conservative limit say 0.3 (current 0.2). This package updates very slow.
pyproject.toml
Outdated
| "sentencepiece>=0.1.99,<1.0", | ||
| "tokenizers>=0.13.3,<1.0", | ||
| "tqdm>=4.66.2,<5.0", | ||
| "trl>=0.7.10,<1.0", |
There was a problem hiding this comment.
perhaps 0.10.0 current is 0.9.4
pyproject.toml
Outdated
| "tokenizers>=0.13.3,<1.0", | ||
| "tqdm>=4.66.2,<5.0", | ||
| "trl>=0.7.10,<1.0", | ||
| "peft>=0.8.0,<1.0", |
There was a problem hiding this comment.
perhaps set limit to 0.13 current 0.11.1
Signed-off-by: Anh-Uong <anh.uong@ibm.com>
Signed-off-by: Anh-Uong <anh.uong@ibm.com>
Signed-off-by: Anh-Uong <anh.uong@ibm.com>
Limit dependency ranges to automatic minor upgrades, must manually check for major upgrades to see what breaking changes are included. Matches current versions that are being used