[BUG] Fix 404 NOT FOUND issue caused by endpoint tail slash

[Mingqi2]

The issue was triggered when user added a tail slash '/' in HF_ENDPOINT, like $set HF_ENDPOINT=https://hf-mirror.com/.

Then it's interesting (stupid?) to visit a link like - https://hf-mirror.com//api/models/sentence-transformers/all-MiniLM-L6-v2/revision/main by huggingface.

Okay then you got 404 and don't know why. Bad experience and you should take care of them in the code with urljoin and carefully strip.

[Mingqi2]

@Wauplin could you please review this PR? thanks.

[hanouticelina]

Hi @Mingqi2, one of the maintainers of huggingface_hub here 👋
thanks a lot for your contribution! indeed, some servers might block double slashes in URLs as they can potentially lead to security vulnerabilities (see this stackoverflow comment for more context). This is not the case for https://huggingface.co but since we also allow custom endpoints, we can handle this case properly with a minimal endpoint normalization!

[Mingqi2]

@hanouticelina thanks for your warm reply! I have commited most of your changes. Please take a look again and help to merge it thanks.

[hanouticelina]

just noticed a typo in the imports, once it's fixed, we will be good to merge! 🎉

[Mingqi2]

just noticed a typo in the imports, once it's fixed, we will be good to merge! 🎉

Good catch. It's committed thanks! Let's merge 🎉

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[hanouticelina]

All good! thanks @Mingqi2 for your contribution! 🤗
failing tests are unrelated and will be fixed when #2728 will be merged.