Bump trl from 0.7.4 to 0.7.10 (#309)

Bumps [trl](https://github.com/huggingface/trl) from 0.7.4 to 0.7.10.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/huggingface/trl/releases">trl's
releases</a>.</em></p>
<blockquote>
<h2>v0.7.10: Automatic templating, <code>setup_chat_format</code> API,
stronger tests</h2>
<h1>v0.7.10: Minor fixes, Automatic templating,
<code>setup_chat_format</code> API, stronger tests</h1>
<p>This Patch release adds a new feature in TRL for dealing with chat
datasets - you can load a directly formatted dataset without the need of
formatting it beforehand.</p>
<p>Read more about it here: <a
href="https://huggingface.co/docs/trl/sft_trainer#dataset-format-support">https://huggingface.co/docs/trl/sft_trainer#dataset-format-support</a></p>
<p>The release also introduces a new API <code>setup_chat_format</code>
to correctly resize the model embeddings with the target size when
adding new tokens to comply with the chat format. Currently we only
support <code>chatml</code> format and we can add more formats in the
future</p>
<p>Read more about it here: <a
href="https://huggingface.co/docs/trl/sft_trainer#add-special-tokens-for-chat-format">https://huggingface.co/docs/trl/sft_trainer#add-special-tokens-for-chat-format</a></p>
<p>We also extensively test SFTTrainer and DPOTrainer and the example
scripts, <code>dpo.py</code> and <code>sft.py</code> should be well
-battletested. If you see any issue with the script, please let us know
on GitHub.</p>
<h2>What's Changed</h2>
<ul>
<li>set dev version by <a
href="https://github.com/younesbelkada"><code>@​younesbelkada</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1207">huggingface/trl#1207</a></li>
<li>Check tokenize params on DPOTrainer by <a
href="https://github.com/pablovicente"><code>@​pablovicente</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1197">huggingface/trl#1197</a></li>
<li>Fix shape descriptions in calculate_loss method by <a
href="https://github.com/yuta0x89"><code>@​yuta0x89</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1204">huggingface/trl#1204</a></li>
<li>Fix FSDP error by <a
href="https://github.com/mgerstgrasser"><code>@​mgerstgrasser</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1196">huggingface/trl#1196</a></li>
<li>Update Unsloth SFT, DPO docs by <a
href="https://github.com/danielhanchen"><code>@​danielhanchen</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1213">huggingface/trl#1213</a></li>
<li>Fix args type by <a
href="https://github.com/zspo"><code>@​zspo</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1214">huggingface/trl#1214</a></li>
<li>[<code>core</code> / <code>Docker</code>] Add workflow to build TRL
docker images by <a
href="https://github.com/younesbelkada"><code>@​younesbelkada</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1215">huggingface/trl#1215</a></li>
<li>Refactor RewardConfig to own module by <a
href="https://github.com/lewtun"><code>@​lewtun</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1221">huggingface/trl#1221</a></li>
<li>Add support for ChatML dataset format in by <a
href="https://github.com/philschmid"><code>@​philschmid</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1208">huggingface/trl#1208</a></li>
<li>Add slow test workflow file by <a
href="https://github.com/younesbelkada"><code>@​younesbelkada</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1223">huggingface/trl#1223</a></li>
<li>Remove a repeating line in how_to_train.md by <a
href="https://github.com/kykim0"><code>@​kykim0</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1226">huggingface/trl#1226</a></li>
<li>Logs metrics on all distributed processes when using DPO &amp; FSDP
by <a href="https://github.com/AjayP13"><code>@​AjayP13</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1160">huggingface/trl#1160</a></li>
<li>fix: improve error message when <code>pad_token_id</code> is not
configured by <a
href="https://github.com/yumemio"><code>@​yumemio</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1152">huggingface/trl#1152</a></li>
<li>[<code>core</code> / tests ] v1 slow tests by <a
href="https://github.com/younesbelkada"><code>@​younesbelkada</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1218">huggingface/trl#1218</a></li>
<li>[<code>core</code> / SFTTrainer] Fix breaking change by <a
href="https://github.com/younesbelkada"><code>@​younesbelkada</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1229">huggingface/trl#1229</a></li>
<li>Fixes slow tests by <a
href="https://github.com/younesbelkada"><code>@​younesbelkada</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1241">huggingface/trl#1241</a></li>
<li>Fix weird doc bug by <a
href="https://github.com/younesbelkada"><code>@​younesbelkada</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1244">huggingface/trl#1244</a></li>
<li>Add <code>setup_chat_format</code> for adding new special tokens to
model for training chat models by <a
href="https://github.com/philschmid"><code>@​philschmid</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1242">huggingface/trl#1242</a></li>
<li>Fix chatml template by <a
href="https://github.com/philschmid"><code>@​philschmid</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1248">huggingface/trl#1248</a></li>
<li>fix: fix loss_type and some args desc by <a
href="https://github.com/zspo"><code>@​zspo</code></a> in <a
href="https://redirect.github.com/huggingface/trl/pull/1247">huggingface/trl#1247</a></li>
<li>Release: v0.7.10 by <a
href="https://github.com/younesbelkada"><code>@​younesbelkada</code></a>
in <a
href="https://redirect.github.com/huggingface/trl/pull/1253">huggingface/trl#1253</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/yuta0x89"><code>@​yuta0x89</code></a>
made their first contribution in <a
href="https://redirect.github.com/huggingface/trl/pull/1204">huggingface/trl#1204</a></li>
<li><a
href="https://github.com/danielhanchen"><code>@​danielhanchen</code></a>
made their first contribution in <a
href="https://redirect.github.com/huggingface/trl/pull/1213">huggingface/trl#1213</a></li>
<li><a href="https://github.com/zspo"><code>@​zspo</code></a> made their
first contribution in <a
href="https://redirect.github.com/huggingface/trl/pull/1214">huggingface/trl#1214</a></li>
<li><a
href="https://github.com/philschmid"><code>@​philschmid</code></a> made
their first contribution in <a
href="https://redirect.github.com/huggingface/trl/pull/1208">huggingface/trl#1208</a></li>
<li><a href="https://github.com/kykim0"><code>@​kykim0</code></a> made
their first contribution in <a
href="https://redirect.github.com/huggingface/trl/pull/1226">huggingface/trl#1226</a></li>
<li><a href="https://github.com/AjayP13"><code>@​AjayP13</code></a> made
their first contribution in <a
href="https://redirect.github.com/huggingface/trl/pull/1160">huggingface/trl#1160</a></li>
<li><a href="https://github.com/yumemio"><code>@​yumemio</code></a> made
their first contribution in <a
href="https://redirect.github.com/huggingface/trl/pull/1152">huggingface/trl#1152</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/huggingface/trl/compare/v0.7.9...v0.7.10">https://github.com/huggingface/trl/compare/v0.7.9...v0.7.10</a></p>
<h1>v0.7.9: Patch release for DPO &amp; SFTTrainer</h1>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/huggingface/trl/commit/09ca7607d50714a80c9f0feb206b29f1c5a49bab"><code>09ca760</code></a>
Release: v0.7.10 (<a
href="https://redirect.github.com/huggingface/trl/issues/1253">#1253</a>)</li>
<li><a
href="https://github.com/huggingface/trl/commit/1e68753216105a4aca733b32f7c4b50ff1fedfca"><code>1e68753</code></a>
fix: fix loss_type and some args desc (<a
href="https://redirect.github.com/huggingface/trl/issues/1247">#1247</a>)</li>
<li><a
href="https://github.com/huggingface/trl/commit/1f59eeb9bb4298aa6857754284d49cecc483ec36"><code>1f59eeb</code></a>
Fix chatml template (<a
href="https://redirect.github.com/huggingface/trl/issues/1248">#1248</a>)</li>
<li><a
href="https://github.com/huggingface/trl/commit/928d14445e31b3586ce8b73ca70ecb02dc603369"><code>928d144</code></a>
Add <code>setup_chat_format</code> for adding new special tokens to
model for training c...</li>
<li><a
href="https://github.com/huggingface/trl/commit/3319993bd1f8c1e43a8490ece2b4e9d36a7279ef"><code>3319993</code></a>
Fix weird doc bug (<a
href="https://redirect.github.com/huggingface/trl/issues/1244">#1244</a>)</li>
<li><a
href="https://github.com/huggingface/trl/commit/4fb3d0c86047661279e024c2e5d085ff432ea648"><code>4fb3d0c</code></a>
Update sft_trainer.py (<a
href="https://redirect.github.com/huggingface/trl/issues/1241">#1241</a>)</li>
<li><a
href="https://github.com/huggingface/trl/commit/bcccdeb6f939cf5b6993e722f015f3bb2b6e65c4"><code>bcccdeb</code></a>
[<code>core</code> / SFTTrainer] Fix breaking change (<a
href="https://redirect.github.com/huggingface/trl/issues/1229">#1229</a>)</li>
<li><a
href="https://github.com/huggingface/trl/commit/ef209e311f25a017518cedd95a7964eea09c87b3"><code>ef209e3</code></a>
[<code>core</code> / tests ] v1 slow tests (<a
href="https://redirect.github.com/huggingface/trl/issues/1218">#1218</a>)</li>
<li><a
href="https://github.com/huggingface/trl/commit/341f6a6787016a0cc8ce1a68bb8ea574be72830c"><code>341f6a6</code></a>
fix: improve error message when <code>pad_token_id</code> is not
configured (<a
href="https://redirect.github.com/huggingface/trl/issues/1152">#1152</a>)</li>
<li><a
href="https://github.com/huggingface/trl/commit/97b9fa212af8d5723c891ebc55a73c763f69a978"><code>97b9fa2</code></a>
Update dpo_trainer.py (<a
href="https://redirect.github.com/huggingface/trl/issues/1160">#1160</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/huggingface/trl/compare/v0.7.4...v0.7.10">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=trl&package-manager=pip&previous-version=0.7.4&new-version=0.7.10)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

pyproject.toml

@@ -39,7 +39,7 @@ pytest = "^7.4.4"
 
 
 [tool.poetry.group.remoteworker.dependencies]
-trl = "^0.7.2"
+trl = "^0.7.10"
 transformers = "^4.36.2"
 httpx = "^0.26.0"
 peft = "^0.7.1"