imapfilter do not work with gmail.

[gollumone]

Hi, I'm trying to use imapfilter with gmail using oauth2 as verification.

I used mutt_oauth2.py with pgp to create and store my keys.

Currently I managed to establish the connection with gmail, but subsequently, any command I can insert in the configuration file to test, imapfilter remains blocked and does not execute any requests.

This is my current config file:

local function execute_command(command)
    local tmpfile = os.tmpname ()
    local exit = os.execute(command .. ' > ' .. tmpfile .. ' 2> ' .. tmpfile .. '.err')

    local stdout_file = io.open(tmpfile)
    local stdout = stdout_file:read("*all")

    local stderr_file = io.open(tmpfile .. '.err')
    local stderr = stderr_file:read("*all")

    stdout_file:close()
    stderr_file:close()

    return exit, stdout, stderr
end


local function read_oauth2_token(file_token)
    local cmd_exit, cmd_out, cmd_err = execute_command("mutt_oauth2.py " .. file_token)

    if cmd_exit then
        return cmd_out
    else
        print(cmd_err)
        return nil
    end
end

local function print_messages(mailbox)
    print("Mailbox scanning.")
    for _, message in ipairs(mailbox:select_all()) do
        print(message:info())
    end
end

--------------------------------------------------------------------------

options.timeout = 120

local oauth2_file = "<path>/oath2_token_file"

print("Setting gmail account.")
account = IMAP {
    server = 'imap.gmail.com',
    username = '<user>@gmail.com',
    oauth2 = read_oauth2_token(oauth2_file),
    ssl = 'ssl3'
}

print("Define inbox.")
inbox = account.INBOX

print("Check inbox")
inbox:check_status()

print_messages(inbox)

and this is output of debug file:

getting response (4):

* OK Gimap ready for requests from 212.162.98.74 <key_1?>

sending command (4):

1000 NOOP

getting response (4):

1000 OK Nothing Accomplished. <key_1?>

sending command (4):

1001 CAPABILITY

getting response (4):

* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH2 AUTH=PLAIN AUTH=PLAIN-CLIENTTOKEN AUTH=OAUTHBEARER
1001 OK Thats all she wrote! <key_1?>

sending command (4):

1002 AUTHENTICATE XOAUTH2 <my_oauth2_key>


getting response (4):

+ <response key>

as you can see the log stops at the server response key, the script remains suspended when check_status is executed.

if I remove the check_status command, the script remains suspended when the mailbox:select_all() command is executed.

In practice, every command that is sent to the server waits for a response that never arrives.

What am I doing wrong?

Thank you

[gollumone]

Could it be that the commands need to be sent in base64? shouldn't imapfilter handle this itself?

[gollumone]

If I wait long enough for the inbox = account.INBOX function to time out, the terminal gives me this traceback:

imapfilter: timeout period expired while waiting to read data
imapfilter: login request to my_user@gmail.com@imap.gmail.com failed
stack traceback:
[C]: in ?
[C]: in function 'error'
/usr/share/imapfilter/account.lua:75: in function '_check_result'
/usr/share/imapfilter/account.lua:93: in function '_login_user'
/usr/share/imapfilter/mailbox.lua:32: in function '_check_connection'
/usr/share/imapfilter/mailbox.lua:501: in function 'check_status'
/home/mauro/.imapfilter/config.lua:94: in main chunk

the log file don't chenge.

[bennyslbs]

Just an idea, instead of using oath2 it works for me with an app-password for an email on a google site acoount (free workspace acoount)
account = IMAP {
server = 'imap.gmail.com',
username = 'user@domain',
password = 'app-password',
ssl = 'ssl3',
}

[gollumone]

Just an idea, instead of using oath2 it works for me with an app-password for an email on a google site acoount (free workspace acoount)

 account = IMAP {
     server = 'imap.gmail.com', 
     username = 'user@domain', 
    password = 'app-password', 
    ssl = 'ssl3', 
}

Right what you say, but if it works with mutt, it MUST work with imapfilter too. Also because it seems that Google wants to remove support for this password app, sooner or later. And then it also serves me as an experience for other non-Google mailboxes, where the app-password is not really available, only aouth2.

[lefcha]

Imapfilter just supplies the XOAUTH2 string as stored in the account's oauth2 string, so in theory the string should be in the final format as it should be sent to the server. Basically imapfilter just provides a way to do this authentication instead of the standard plain login.

Here's the code:

imapfilter/src/request.c

Lines 184 to 188 in 2665c77

	if (ssn->capabilities & CAPABILITY_XOAUTH2 && oauth2) {
	TRY(t = send_request(ssn, "AUTHENTICATE XOAUTH2 %s",
	oauth2));
	TRY(rl = response_generic(ssn, t));
	}

Now what seems to happen in this case is that apparently the server needs more data... Basically the fact that it did not respond with OK XOAUTH2 authentication successful or similar but instead sent back a continuation response as denoted by the + means that it expects something more from the client.

This explains why the timeout then happens. So probably something changed, as this used to work without a second exchange of data between imapfilter and the server.

I tried to use contrib/mutt_oauth2.py script with the instructions at contrib/mutt_oauth2.py.README, but I couldn't even create a Google registration at console.developers.google.com, because the instructions seem to be out of date or something...

[lefcha]

Do you know what the 2nd exchange is about?

Because if the oauth2 script can take the reply of the server, and based on it provide another string to send to the server, we could modify imapfilter so it handles all that somehow.

[lefcha]

This is what I also see in Google's own python/oauth2.py, that seems to have been migrated to Python 3:

https://github.com/google/gmail-oauth2-tools/blob/81ced795c11ed0adb2fdbbd66a54a4f6af2586aa/python/oauth2.py#L62-L65

Maybe the oauth2 string sent is not somehow correct, and so the XOAUTH2 process isn't completed, and the server wants more data?

[lefcha]

@gollumone Can you give Google's python/oauth2.py a try, now that it's been migrated to Python 3? Maybe it produces a string that can authenticate you?

Imapfilter had an example of using this script in the past (it was removed because the script had stopped working):

imapfilter/samples/extend.lua

Lines 132 to 169 in 30bca66

	-- An alternative way to authenticate to a server is by using a OAuth2 string,
	-- if the server supports the XOAUTH2 authentication mechanism.
	--
	-- In order to generate an OAuth2 string the oauth2.py script and library can
	-- be used, and instructions on how to use it and where to download it are
	-- available at:
	--
	-- https://github.com/google/gmail-oauth2-tools/wiki/OAuth2DotPyRunThrough
	--
	-- The generated OAuth2 string is then supplied to imapfilter in order to
	-- authenticate to the IMAP server using it instead of a login
	-- username/password pair.
	--
	-- Here we assume that imapfilter has the user, the cliend id, the client
	-- secret and the refresh token, and uses them to generate a new access token
	-- (access tokens expire after one hour), and then from the new access token to
	-- generate the OAuth2 string that is used with the IMAP server:
	user = 'xoauth@gmail.com'
	clientid = '364545978226.apps.googleusercontent.com'
	clientsecret = 'zNrNsBzOOnQy8_O-8LkofeTR'
	refreshtoken = '1/q4SaB2JMQB9I-an6F1rxJE9OkOMtfjaz1bPm1tfDpQM'
	status, output = pipe_from('oauth2.py --client_id=' .. clientid ..
	' --client_secret=' .. clientsecret ..
	' --refresh_token=' .. refreshtoken)
	_, _, accesstoken = string.find(output, 'Access Token: ([%w%p]+)\n')
	status, output = pipe_from('oauth2.py --generate_oauth2_string' ..
	' --access_token=' .. accesstoken ..
	' --user=' .. user)
	_, _, oauth2string = string.find(output, 'OAuth2 argument:\n([%w%p]+)\n')
	account3 = IMAP {
	server = 'imap.gmail.com',
	ssl = 'tls1.2',
	username = user,
	oauth2 = oauth2string
	}

A few years a go I could personally authenticate to Gmail using the above example.

[dngray]

I'd stop using oauth2.py and use something more current like pizauth or oama. These are both maintained and work well with OAuth2 accounts.