Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Explicitly support URL params in string form #1623

Merged
merged 8 commits into from
Apr 25, 2023
25 changes: 21 additions & 4 deletions singer_sdk/streams/rest.py
Original file line number Diff line number Diff line change
Expand Up @@ -279,18 +279,35 @@ def get_url_params(
self,
context: dict | None, # noqa: ARG002
next_page_token: _TToken | None, # noqa: ARG002
) -> dict[str, Any]:
"""Return a dictionary of values to be used in URL parameterization.
) -> dict[str, Any] | str:
"""Return a dictionary or string of URL query parameters.

If paging is supported, developers may override with specific paging logic.

If your source needs special handling and, for example, parentheses should not
be encoded, you can return a string constructed with
`urllib.parse.urlencode`_:

.. code-block:: python

from urllib.parse import urlencode

class MyStream(RESTStream):
def get_url_params(self, context, next_page_token):
params = {"key": "(a,b,c)"}
return urlencode(params, safe="()")

Args:
context: Stream partition or context dictionary.
next_page_token: Token, page number or any request argument to request the
next page of data.

Returns:
Dictionary of URL query parameters to use in the request.
Dictionary or encoded string with URL query parameters to use in the
request.

.. _urllib.parse.urlencode:
https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlencode
"""
return {}

Expand Down Expand Up @@ -341,7 +358,7 @@ def prepare_request(
"""
http_method = self.rest_method
url: str = self.get_url(context)
params: dict = self.get_url_params(context, next_page_token)
params: dict | str = self.get_url_params(context, next_page_token)
request_data = self.prepare_request_payload(context, next_page_token)
headers = self.http_headers

Expand Down