FIY another easy "fix" to empty list issue

[GordonRamsay-689]

First of all, make sure you have the correct version of bs4 installed.

You can run the following commands at your own risk if you want to completely uninstall old versions and install bs4 for python3:

$ pip uninstall beautifulsoup4
$ pip3 uninstall beautifulsoup4   
$ pip uninstall BeautifulSoup
$ pip3 uninstall BeautifulSoup
$ pip3 install beautifulsoup4

Now, this did NOT fix the empty list issue for me, but this is important first step in case you have accidentally installed the wrong version. The reason was because the description would always be None, and thus the check in modules/standard_search.py would always be true, and thus not append to results.

in 'modules/standard_search.py' on line 96:

if void is True:
    if res.description is None:
        continue

This check can prevent empty results being added, but as soon as the description parsing function is no longer current this will mess everything up, leaving you with an empty list.

I replaced it with a simpler check (which you should really expand upon if you want to avoid unpopulated responses):

if res is None:
    continue
if res.description is None: # Optional
    res.description = "No description" 

Obviously this does not fix the function to get the description, but it does make sure you can get links, title etc.

[ZeusYohaan]

Thanks mate!
This worked for me.

[GordonRamsay-689]

@ZeusYohaan I have since actually modified it so that I can get descriptions most of the time as well. Not always, but most of the time.

I didn't fork it with a fix because this only fixes this specific tiny piece, the rest is still not working.. I included the full file in another project on my profile that uses this specific function tho.

The full file is here:
https://github.com/GordonRamsay-689/Multi-Search/tree/main/for_googleapi

You can just replace the original standard_search.py with this one.

The only things I changed in this file are:

from re import findall

Line 55:
def search(query, pages=1, lang='en', area='com', ncr=True, void=True, time_period=False, sort_by_date=False, first_page=0):

The _get_description() function:

def _get_description(li):
    """Return the description of a google search.

    TODO: There are some text encoding problems to resolve."""

    sdiv = li.find("div", attrs={"class": "kb0PBd A9Y9g"})
    if sdiv:
        spans = sdiv.find_all("span")

        if spans:
            return spans[-1].text.strip()
        else:
            return None
    else:
        return None

Don't ask me why I did it this way, as I did it a while ago and don't remember how it worked. I put a weekend to looking inside the bs4 output and not knowing any CSS I just noticed this seemed to be the class holding the description and changed to that and some other minor changes. It still works for me, I use it to search from cli almost everyday.

btw, if you want to format it, look inside api_session.py (in the same project) and copy the GoogleClient.format_response(), replacing self.api_response with the output from the search function. I have since made the formatting prettier and more consistent but I haven't pushed those changes. If you want them I can send them to you tho!