Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

URL params are not handled cleanly when defined #45

Closed
rickheil opened this issue Jan 31, 2022 · 2 comments
Closed

URL params are not handled cleanly when defined #45

rickheil opened this issue Jan 31, 2022 · 2 comments
Assignees
Labels
bug Something isn't working resolved in dev Resolved in the dev branch, but not committed to main yet.

Comments

@rickheil
Copy link
Collaborator

In the current SimpleMDM.py, a starting_after=0 is force-applied to each URL in _get_data, and any passed parameters tacked on the end. When overriding the starting value (e.g. for Logs, found while testing #25) this ends up with two start values in the URL, eg: https://a.simplemdm.com/api/v1/logs?limit=100&starting_after=0&starting_after=aldskjrgawop4fjasdlkfjasfw93f4war

Obviously the SimpleMDM service is honoring the second one of these because it works, but we should add some more intelligent parsing of the parameters and use the existing start_id value if specific params are passed from the various functions.

@rickheil rickheil self-assigned this Jan 31, 2022
@carlashley
Copy link

carlashley commented May 23, 2022

Probably the easiest way to ensure params are appropriately dealt with is to utilise the ability of requests to parse the params=dict_obj out into an encoded URL in the same manner that urllib.parse.urlencode does.

For example:

    def paginate(self, url: str, limit: int = 100, starting_after: str = "0", **kwargs) -> Optional[List[Any]]:
        """Paginate over results from API query.
        :param url: url string to perform pagination on
        :param limit: maximum number of objects returned in response when paginating
        :param starting_after: initial ID value to use for starting the pagination from,
                               the default is '0' so as to start from the first page in the
                               pagination process"""
        result = list()

        if not kwargs:
            kwargs = dict()

        kwargs["limit"] = limit
        kwargs["starting_after"] = starting_after
        has_more = True

        while has_more:
            req = self.session.get(self._urljoin(url), params=kwargs, timeout=self.timeout)

            if has_more and req.json() and req.json().get("data"):
                result.extend(req.json()["data"])
                kwargs["starting_after"] = str(req.json()["data"][-1].get("id"))  # Update next ID value
                has_more = req.json().get("has_more", False)
            else:
                if req.json().get("data"):
                    result.extend(req.json()["data"])
                    has_more = req.json().get("has_more", False)
                else:
                    break

        return result

In the case of querying devices using list_all from the API, the URL would end up as such using the default arg values provided to the paginate method:
https://a.simplemdm.com/api/v1/devices?limit=100&starting_after=0

Any methods that then call on the paginate method could then subsequently also have an optional starting_after arg passed through.

bryanheinz added a commit to bryanheinz/simpleMDMpy that referenced this issue May 30, 2022
- params input is now converted to req_params while preserving any input params
bryanheinz added a commit that referenced this issue May 30, 2022
- Cleaned up my bad merge on Devices.get_device() and adds some help docs
- Closes the session on deinit that the Connection class now opens
- Resolves issue #45 by preserving input parameters instead of overwriting them
- Added setup.cfg and pyproject.toml files for packaging new releases
- Added a few basic tests
- Updates the changelog and gitignore files
@MagerValp
Copy link
Collaborator

I think this should finally be fixed now with the merge of PR #59

@MagerValp MagerValp added bug Something isn't working resolved in dev Resolved in the dev branch, but not committed to main yet. labels Nov 24, 2022
@MagerValp MagerValp mentioned this issue Dec 1, 2022
rickheil pushed a commit that referenced this issue Dec 2, 2022
* resolves the bug in issue #25 by removing id_override, replacing data with params, and adding specific input parameters.

* required changes to resolve the bug in issue #25. migrated from updating the url in _get_data to a local _params variable that is updated with the input params var.

* added Unreleased section and updated with issue #25 changes.

* Fix calls that return a single item.

* Return single items without wrapping in list.

* added params= to be explicit, and marked a potential bug.

* updated CHANGELOG

* added docstring

* updated CHANGELOG

* Resolves Issue 38 (#1)

* Resolves issue #38
* Resolves issue #24

- Updated update_device input to accept both name and device_name input (breaking change)
- Data is now updated with the inputs
- Added validation that data has input
- Updated the README with update_device's new inputs

* Adding download option to profile

* Update CHANGELOG.md

* v3.0.7

* adding _get_xml connection

* update CHANGELOG

* update README

* Adding include_awaiting_enrollment option #43 (#44)

* Merging dev branch (#46)

* Use request params instead of url string in SimpleMDM._get_data()

* Fix Devices.delete_device()

* Add methods for enabling/disabling remote desktop

* Add /devices request rate limiting

* Add profile and user listing

* Add retry on 5xx errors to GET requests

* Updates gitignore and changelog (#47)

- Added ignoring egg files
- Updated changelog

* A little clean up, some fixen, and a few tests. (#48)

- Cleaned up my bad merge on Devices.get_device() and adds some help docs
- Closes the session on deinit that the Connection class now opens
- Resolves issue #45 by preserving input parameters instead of overwriting them
- Added setup.cfg and pyproject.toml files for packaging new releases
- Added a few basic tests
- Updates the changelog and gitignore files

* Add script support

* Add error handling for update_script

* Fix handling of req_params for pagination

* Update CHANGELOG.md

* Fix handling of req_params for pagination

* Update CHANGELOG.md

* Add Sample Projects

Adding some samples projects for issue #28

* Use monotonic time for rate limit and fix sleep time calc

* Update CHANGELOG.md

Co-authored-by: Steve <steve.kueng@gmail.com>
Co-authored-by: Bryan Heinz <git@bryanheinz.com>
Co-authored-by: Jon Crain <joncrain@users.noreply.github.com>
@rickheil rickheil closed this as completed Dec 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working resolved in dev Resolved in the dev branch, but not committed to main yet.
Projects
None yet
Development

No branches or pull requests

3 participants