Merge branch 'main' into automatically-handle-outdated-inference-params

.github/workflows/build_repocard_examples.yaml

@@ -17,7 +17,7 @@ jobs:
       - name: Set up Python
         uses: actions/setup-python@v2
         with:
-          python-version: 3.12
+          python-version: 3.13
 
       # Install dependencies
       - name: Configure and install dependencies

.github/workflows/python-tests.yml

@@ -21,15 +21,15 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.8", "3.12"]
+        python-version: ["3.8", "3.13"]
         test_name:
           [
             "Repository only",
             "Everything else",
-            "torch_latest",
+
           ]
         include:
-          - python-version: "3.12" # LFS not ran on 3.8
+          - python-version: "3.13" # LFS not ran on 3.8
             test_name: "lfs"
           - python-version: "3.8"
             test_name: "fastai"
@@ -41,6 +41,8 @@ jobs:
             test_name: "tensorflow"
           - python-version: "3.8" # test torch~=1.11 on python 3.8 only.
             test_name: "Python 3.8, torch_1.11"
+          - python-version: "3.12" # test torch latest on python 3.12 only.
+            test_name: "torch_latest"
     steps:
       - uses: actions/checkout@v2
       - name: Set up Python ${{ matrix.python-version }}

docs/source/cn/installation.md

@@ -97,7 +97,7 @@ cd huggingface_hub
 pip install -e .
 ```
 
-这些命令将你克隆存储库的文件夹与你的 Python 库路径链接起来。Python 现在将除了正常的库路径之外，还会在你克隆到的文件夹中查找。例如，如果你的 Python 包通常安装在`./.venv/lib/python3.12/site-packages/`中，Python 还会搜索你克隆的文件夹`./huggingface_hub/`
+这些命令将你克隆存储库的文件夹与你的 Python 库路径链接起来。Python 现在将除了正常的库路径之外，还会在你克隆到的文件夹中查找。例如，如果你的 Python 包通常安装在`./.venv/lib/python3.13/site-packages/`中，Python 还会搜索你克隆的文件夹`./huggingface_hub/`
 
 ## 通过 conda 安装
 

docs/source/de/installation.md

@@ -90,7 +90,7 @@ cd huggingface_hub
 pip install -e .
 ```
 
-Diese Befehle verknüpfen den Ordner, in den Sie das Repository geklont haben, mit Ihren Python-Bibliothekspfaden. Python wird nun zusätzlich zu den normalen Bibliothekspfaden im geklonten Ordner suchen. Wenn Ihre Python-Pakete normalerweise in `./.venv/lib/python3.12/site-packages/` installiert sind, wird Python auch den geklonten Ordner `./huggingface_hub/` durchsuchen.
+Diese Befehle verknüpfen den Ordner, in den Sie das Repository geklont haben, mit Ihren Python-Bibliothekspfaden. Python wird nun zusätzlich zu den normalen Bibliothekspfaden im geklonten Ordner suchen. Wenn Ihre Python-Pakete normalerweise in `./.venv/lib/python3.13/site-packages/` installiert sind, wird Python auch den geklonten Ordner `./huggingface_hub/` durchsuchen.
 
 ## Installieren mit conda
 

docs/source/en/guides/hf_file_system.md

@@ -6,6 +6,14 @@ rendered properly in your Markdown viewer.
 
 In addition to the [`HfApi`], the `huggingface_hub` library provides [`HfFileSystem`], a pythonic [fsspec-compatible](https://filesystem-spec.readthedocs.io/en/latest/) file interface to the Hugging Face Hub. The [`HfFileSystem`] builds on top of the [`HfApi`] and offers typical filesystem style operations like `cp`, `mv`, `ls`, `du`, `glob`, `get_file`, and `put_file`.
 
+<Tip warning={true}>
+
+  [`HfFileSystem`] provides fsspec compatibility, which is useful for libraries that require it (e.g., reading
+  Hugging Face datasets directly with `pandas`). However, it introduces additional overhead due to this compatibility
+  layer. For better performance and reliability, it's recommended to use [`HfApi`] methods when possible.
+
+</Tip>
+
 ## Usage
 
 ```python

docs/source/en/installation.md

@@ -104,7 +104,7 @@ pip install -e .
 
 These commands will link the folder you cloned the repository to and your Python library paths.
 Python will now look inside the folder you cloned to in addition to the normal library paths.
-For example, if your Python packages are typically installed in `./.venv/lib/python3.12/site-packages/`,
+For example, if your Python packages are typically installed in `./.venv/lib/python3.13/site-packages/`,
 Python will also search the folder you cloned `./huggingface_hub/`.
 
 ## Install with conda

docs/source/fr/installation.md

@@ -104,7 +104,7 @@ pip install -e .
 
 Python regardera maintenant à l'intérieur du dossier dans lequel vous avez cloné le dépôt en
 plus des chemins de librairie classiques. Par exemple, si vos packages Python sont installés dans
-`./.venv/lib/python3.12/site-packages/`, Python regardera aussi dans le dossier que vous avez
+`./.venv/lib/python3.13/site-packages/`, Python regardera aussi dans le dossier que vous avez
 cloné `./huggingface_hub/`.
 
 ## Installation avec conda

docs/source/hi/installation.md

@@ -103,7 +103,7 @@ pip install -e .
 
 ये कमांड उस फ़ोल्डर को लिंक करेंगे जिसे आपने रिपॉजिटरी में क्लोन किया है और आपके पायथन लाइब्रेरी पथ।
 पाइथॉन अब सामान्य लाइब्रेरी पथों के अलावा आपके द्वारा क्लोन किए गए फ़ोल्डर के अंदर भी देखेगा।
-उदाहरण के लिए, यदि आपके पायथन पैकेज आमतौर पर `./.venv/lib/python3.12/site-packages/` में स्थापित हैं,
+उदाहरण के लिए, यदि आपके पायथन पैकेज आमतौर पर `./.venv/lib/python3.13/site-packages/` में स्थापित हैं,
 पायथन आपके द्वारा क्लोन किए गए फ़ोल्डर `./huggingface_hub/` को भी खोजेगा।
 
 ## कोंडा के साथ स्थापित करें

docs/source/ko/installation.md

@@ -94,7 +94,7 @@ pip install -e .
 
 이렇게 클론한 레포지토리 폴더와 Python 경로를 연결합니다.
 이제 Python은 일반적인 라이브러리 경로 외에도 복제된 폴더 내부를 찾습니다.
-예를 들어 파이썬 패키지가 일반적으로 `./.venv/lib/python3.12/site-packages/`에 설치되어 있다면, Python은 복제된 폴더 `./huggingface_hub/`도 검색하게 됩니다.
+예를 들어 파이썬 패키지가 일반적으로 `./.venv/lib/python3.13/site-packages/`에 설치되어 있다면, Python은 복제된 폴더 `./huggingface_hub/`도 검색하게 됩니다.
 
 ## conda로 설치하기 [[install-with-conda]]
 

setup.py

@@ -134,6 +134,7 @@ def get_version() -> str:
         "Programming Language :: Python :: 3.10",
         "Programming Language :: Python :: 3.11",
         "Programming Language :: Python :: 3.12",
+        "Programming Language :: Python :: 3.13",
         "Topic :: Scientific/Engineering :: Artificial Intelligence",
     ],
     include_package_data=True,

src/huggingface_hub/file_download.py

@@ -390,9 +390,8 @@ def http_get(
 
     consistency_error_message = (
         f"Consistency check failed: file should be of size {expected_size} but has size"
-        f" {{actual_size}} ({displayed_filename}).\nWe are sorry for the inconvenience. Please retry"
-        " with `force_download=True`.\nIf the issue persists, please let us know by opening an issue "
-        "on https://github.com/huggingface/huggingface_hub."
+        f" {{actual_size}} ({displayed_filename}).\nThis is usually due to network issues while downloading the file."
+        " Please retry with `force_download=True`."
     )
 
     # Stream file to buffer

src/huggingface_hub/hf_api.py

@@ -760,6 +760,8 @@ class ModelInfo:
             List of spaces using the model.
         safetensors (`SafeTensorsInfo`, *optional*):
             Model's safetensors information.
+        security_repo_status (`Dict`, *optional*):
+            Model's security scan status.
     """
 
     id: str
@@ -788,6 +790,7 @@ class ModelInfo:
     siblings: Optional[List[RepoSibling]]
     spaces: Optional[List[str]]
     safetensors: Optional[SafeTensorsInfo]
+    security_repo_status: Optional[Dict]
 
     def __init__(self, **kwargs):
         self.id = kwargs.pop("id")
@@ -853,7 +856,7 @@ def __init__(self, **kwargs):
             if safetensors
             else None
         )
-
+        self.security_repo_status = kwargs.pop("securityRepoStatus", None)
         # backwards compatibility
         self.lastModified = self.last_modified
         self.cardData = self.card_data
@@ -1546,6 +1549,36 @@ def _inner(self, *args, **kwargs):
 
 
 class HfApi:
+    """
+    Client to interact with the Hugging Face Hub via HTTP.
+
+    The client is initialized with some high-level settings used in all requests
+    made to the Hub (HF endpoint, authentication, user agents...). Using the `HfApi`
+    client is preferred but not mandatory as all of its public methods are exposed
+    directly at the root of `huggingface_hub`.
+
+    Args:
+        endpoint (`str`, *optional*):
+            Endpoint of the Hub. Defaults to <https://huggingface.co>.
+        token (Union[bool, str, None], optional):
+            A valid user access token (string). Defaults to the locally saved
+            token, which is the recommended method for authentication (see
+            https://huggingface.co/docs/huggingface_hub/quick-start#authentication).
+            To disable authentication, pass `False`.
+        library_name (`str`, *optional*):
+            The name of the library that is making the HTTP request. Will be added to
+            the user-agent header. Example: `"transformers"`.
+        library_version (`str`, *optional*):
+            The version of the library that is making the HTTP request. Will be added
+            to the user-agent header. Example: `"4.24.0"`.
+        user_agent (`str`, `dict`, *optional*):
+            The user agent info in the form of a dictionary or a single string. It will
+            be completed with information about the installed packages.
+        headers (`dict`, *optional*):
+            Additional headers to be sent with each request. Example: `{"X-My-Header": "value"}`.
+            Headers passed here are taking precedence over the default headers.
+    """
+
     def __init__(
         self,
         endpoint: Optional[str] = None,
@@ -1555,32 +1588,6 @@ def __init__(
         user_agent: Union[Dict, str, None] = None,
         headers: Optional[Dict[str, str]] = None,
     ) -> None:
-        """Create a HF client to interact with the Hub via HTTP.
-
-        The client is initialized with some high-level settings used in all requests
-        made to the Hub (HF endpoint, authentication, user agents...). Using the `HfApi`
-        client is preferred but not mandatory as all of its public methods are exposed
-        directly at the root of `huggingface_hub`.
-
-        Args:
-            token (Union[bool, str, None], optional):
-                A valid user access token (string). Defaults to the locally saved
-                token, which is the recommended method for authentication (see
-                https://huggingface.co/docs/huggingface_hub/quick-start#authentication).
-                To disable authentication, pass `False`.
-            library_name (`str`, *optional*):
-                The name of the library that is making the HTTP request. Will be added to
-                the user-agent header. Example: `"transformers"`.
-            library_version (`str`, *optional*):
-                The version of the library that is making the HTTP request. Will be added
-                to the user-agent header. Example: `"4.24.0"`.
-            user_agent (`str`, `dict`, *optional*):
-                The user agent info in the form of a dictionary or a single string. It will
-                be completed with information about the installed packages.
-            headers (`dict`, *optional*):
-                Additional headers to be sent with each request. Example: `{"X-My-Header": "value"}`.
-                Headers passed here are taking precedence over the default headers.
-        """
         self.endpoint = endpoint if endpoint is not None else constants.ENDPOINT
         self.token = token
         self.library_name = library_name
@@ -1791,8 +1798,8 @@ def list_models(
                 A tuple of two ints or floats representing a minimum and maximum
                 carbon footprint to filter the resulting models with in grams.
             sort (`Literal["last_modified"]` or `str`, *optional*):
-                The key with which to sort the resulting models. Possible values
-                are the properties of the [`huggingface_hub.hf_api.ModelInfo`] class.
+                The key with which to sort the resulting models. Possible values are "last_modified", "trending_score",
+                "created_at", "downloads" and "likes".
             direction (`Literal[-1]` or `int`, *optional*):
                 Direction in which to sort. The value `-1` sorts by descending
                 order while all other values sort by ascending order.
@@ -1904,7 +1911,15 @@ def list_models(
         if len(search_list) > 0:
             params["search"] = search_list
         if sort is not None:
-            params["sort"] = "lastModified" if sort == "last_modified" else sort
+            params["sort"] = (
+                "lastModified"
+                if sort == "last_modified"
+                else "trendingScore"
+                if sort == "trending_score"
+                else "createdAt"
+                if sort == "created_at"
+                else sort
+            )
         if direction is not None:
             params["direction"] = direction
         if limit is not None:
@@ -2003,8 +2018,8 @@ def list_datasets(
             search (`str`, *optional*):
                 A string that will be contained in the returned datasets.
             sort (`Literal["last_modified"]` or `str`, *optional*):
-                The key with which to sort the resulting datasets. Possible
-                values are the properties of the [`huggingface_hub.hf_api.DatasetInfo`] class.
+                The key with which to sort the resulting models. Possible values are "last_modified", "trending_score",
+                "created_at", "downloads" and "likes".
             direction (`Literal[-1]` or `int`, *optional*):
                 Direction in which to sort. The value `-1` sorts by descending
                 order while all other values sort by ascending order.
@@ -2114,7 +2129,15 @@ def list_datasets(
         if len(search_list) > 0:
             params["search"] = search_list
         if sort is not None:
-            params["sort"] = "lastModified" if sort == "last_modified" else sort
+            params["sort"] = (
+                "lastModified"
+                if sort == "last_modified"
+                else "trendingScore"
+                if sort == "trending_score"
+                else "createdAt"
+                if sort == "created_at"
+                else sort
+            )
         if direction is not None:
             params["direction"] = direction
         if limit is not None:
@@ -2186,8 +2209,8 @@ def list_spaces(
             linked (`bool`, *optional*):
                 Whether to return Spaces that make use of either a model or a dataset.
             sort (`Literal["last_modified"]` or `str`, *optional*):
-                The key with which to sort the resulting Spaces. Possible
-                values are the properties of the [`huggingface_hub.hf_api.SpaceInfo`]` class.
+                The key with which to sort the resulting models. Possible values are "last_modified", "trending_score",
+                "created_at" and "likes".
             direction (`Literal[-1]` or `int`, *optional*):
                 Direction in which to sort. The value `-1` sorts by descending
                 order while all other values sort by ascending order.
@@ -2223,7 +2246,15 @@ def list_spaces(
         if search is not None:
             params["search"] = search
         if sort is not None:
-            params["sort"] = "lastModified" if sort == "last_modified" else sort
+            params["sort"] = (
+                "lastModified"
+                if sort == "last_modified"
+                else "trendingScore"
+                if sort == "trending_score"
+                else "createdAt"
+                if sort == "created_at"
+                else sort
+            )
         if direction is not None:
             params["direction"] = direction
         if limit is not None:
@@ -2493,7 +2524,7 @@ def model_info(
                 Whether to set a timeout for the request to the Hub.
             securityStatus (`bool`, *optional*):
                 Whether to retrieve the security status from the model
-                repository as well.
+                repository as well. The security status will be returned in the `security_repo_status` field.
             files_metadata (`bool`, *optional*):
                 Whether or not to retrieve metadata for files in the repository
                 (size, LFS metadata, etc). Defaults to `False`.
@@ -9186,7 +9217,8 @@ def _prepare_upload_folder_additions(
                 token=token,
             )
         if len(filtered_repo_objects) > 30:
-            logger.info(
+            log = logger.warning if len(filtered_repo_objects) > 200 else logger.info
+            log(
                 "It seems you are trying to upload a large folder at once. This might take some time and then fail if "
                 "the folder is too large. For such cases, it is recommended to upload in smaller batches or to use "
                 "`HfApi().upload_large_folder(...)`/`huggingface-cli upload-large-folder` instead. For more details, "