Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blank query on search api is not giving results #12

Closed
gpremkumar opened this issue Oct 9, 2023 · 4 comments
Closed

Blank query on search api is not giving results #12

gpremkumar opened this issue Oct 9, 2023 · 4 comments

Comments

@gpremkumar
Copy link

gpremkumar commented Oct 9, 2023

Hello @dvishal485 ,

Thanks for your api on flipkart. I have been using it on a project to practice react and it is not giving response for the blank query now.

API URL: https://flipkart-scraper-api.dvishal485.workers.dev/search/

Is there any change needed to get the products on homepage ?

@dvishal485
Copy link
Owner

Thank you for bringing this to my notice, flipkart recently started blocking requests without appropriate headers, which is causing the API to get blocked and hence is not working from past few days. You can fix this error by passing default headers along with the fetch request.

I'll try my best to release a better version of the API very soon #13 , possibly patching a lot of errors which were previously encountered, including this one.

@gpremkumar
Copy link
Author

@dvishal485 thanks for the reply

sorry, I couldn't get default options to be passed for headers.

can you share the options to be passed in header to get the response

@dvishal485
Copy link
Owner

@dvishal485 thanks for the reply

sorry, I couldn't get default options to be passed for headers.

can you share the options to be passed in header to get the response

{
    "credentials": "include",
    "headers": {
        "User-Agent": "Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/118.0",
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
        "Accept-Language": "en-US,en;q=0.5",
        "Upgrade-Insecure-Requests": "1",
        "Sec-Fetch-Dest": "document",
        "Sec-Fetch-Mode": "navigate",
        "Sec-Fetch-Site": "none",
        "Sec-Fetch-User": "?1",
        "Sec-GPC": "1"
    },
    "method": "GET",
    "mode": "cors"
}

These headers work in my case (gathered from Flipkart search's network requests).

@dvishal485
Copy link
Owner

I looked more into the issue, Flipkart has intentionally limited access through remote server (may be to avoid being scraped), as a result I won't be shipping any self-hosted deployments.

Appropriate hosting methods will be shared using which user can self-host on their system, and the same will be released with #13 resolution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants