[AWS] Cognito user pool admin

[hao-fang]

Number of Volunteers: 2

We will need to have a volunteer to manage the AWS Cognito user pool, specifically,

• write a Python script to import registered users to the pool (import users to AWS Cognito #57)
• implement a solution to handle registration during the conference (Figure out a way to allow registration during the conference #55)
• propose a solution to manage different groups
• write a Python script to update all current users based on the proposal
• create / delete adhoc accounts for ACL2020 website
  • we already have a bash script to create account

The person needs to have some basic knowledge on AWS CLI.

Timeline

• June 22: create accounts for all internal committee for the dry run
• June 27: create accounts for all stress test users
• July 1: import all registered users to the pool & disable/delete un-registered users
  • we will temporarily make virtual.acl2020.org hidden
• July 1: enable the AWS SES + Lambda trigger to automatically create/enable accounts

[stupid-2020]

@hao-fang I could help on the script for importing all registered users. And will check if I could help on the other tasks listed.

[hao-fang]

@hao-fang I could help on the script for importing all registered users. And will check if I could help on the other tasks listed.

Thanks. I'll get send sample file to you via email later today.

[hao-fang]

@stupid-2020 Can you write a Python script to consume a xlsx/csv file with following columns?

• Last Name
• First Name
• Middle Name
• Preferred Name
• Affiliation
• Timezone
• Country
• Email

I think you can use boto3. The equivalent AWS CLI command for each row would be

aws cognito-idp admin-create-user \
  --user-pool-id "${AWS_USER_POOL_ID}" \
  --username "${email}" \
  --user-attributes "Name=email,Value=${email}" \
  --user-attributes "Name=custom:name,Value=${name}"

where ${email} should be the Email column, and ${name} should be ${FirstName} ${LastName} columns.

Also, can you make sure to handle users that exceptions, e.g., in case there are duplicated entries in the file.
I'll send you the AWS Access Key info separately.

[stupid-2020]

@hao-fang Will do the following:

1. Read xlsx/csv file and validate the entries
2. Check duplicates (by comparing ${FIrstName}, ${LastName} and ${MiddleName})
3. Implement the function of creating user using boto3
4. Create test scripts to create and delete user

Reference:
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cognito-idp.html

[stupid-2020]

@hao-fang I got the following message:

$ aws cognito-idp admin-get-user \
  --region "us-XXXX-X" \
  --user-pool-id "us-XXXX-X_XXXXXXXXX" \
  --username "example@gmail.com"

An error occurred (UnrecognizedClientException) when calling the AdminGetUser operation: 
The security token included in the request is invalid.

Any suggestion?

[stupid-2020]

We will need to have a volunteer to manage the AWS Cognito user pool, specifically,

• implement a solution to handle registration during the conference (Figure out a way to allow registration during the conference #55)
• write a Python script to update all current users based on the proposal

Would it be possible to have a privileged page to pass the user information and call AWS Cognito API (boto3) on request?

[hao-fang]

We will need to have a volunteer to manage the AWS Cognito user pool, specifically,

• implement a solution to handle registration during the conference (Figure out a way to allow registration during the conference #55)
• write a Python script to update all current users based on the proposal

Would it be possible to have a privileged page to pass the user information and call AWS Cognito API (boto3) on request?

@stupid-2020 I have updated #55 with a Zapier + AWS Lambda solution. It can re-use some of the python code you've been developing.
Can you check that to see if this makes sense to you?

I'm not sure I understand what the privileged page look like..

[stupid-2020]

@stupid-2020 I have updated #55 with a Zapier + AWS Lambda solution. It can re-use some of the python code you've been developing.
Can you check that to see if this makes sense to you?

I'm not sure I understand what the privileged page look like..
It make sense to me. Could add function create_user to the lambda_handler. It should be fully automated then.

[stupid-2020]

@hao-fang I think I am able to complete the lambda_handler, but I haven't written any AWS lambda function before. (I think it should be implemented at AWS Lambda console, right?)

[hao-fang]

@stupid-2020 Once the python script is ready, I can help the AWS Lambda part or find another volunteer. Basically it would look like something below

import json
import boto3

def lambda_handler(event, context):
    # TODO implement
    email = event["email"]
    name = event["name"]
    
    client = boto3.client(
        "cognito-idp",
        aws_access_key_id="foo",
        aws_secret_access_key="bar",
        region_name="us-east-1",
    )
    print(client)
    
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Then event is a dictionary sent from Zapier.

[stupid-2020]

If you disable a user and re-enable the user again, will the user receive any email? Will the user be able to use the same password?

Tried. No email sent when the user is "re-enabled"

[hao-fang]

@stupid-2020 Can you start checking in code to https://github.com/acl-org/acl-2020-virtual-conference-tools . You can put things under acl2020_tools/awscognito.
I'm going to remove scripts from this main repo after the transition.

[stupid-2020]

• propose a solution to manage different groups

It is possible to write a script to disable/enable user according to the GroupName (but use with caution). For example:

python cognito_groups.py --disable attendees aws_profile.yml

And we can modify cognito_users.py to assign users to specified group while creating. For example:

python cognito_users.py --assign-group attendees sample.csv aws_profile.yml

[hao-fang]

@stupid-2020 Have you checked with this in https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cognito-idp.html#CognitoIdentityProvider.Client.list_users

PaginationToken (string) --

An identifier that was returned from the previous call to this operation, which can be used to return the next set of items in the list.

It seems that your script can only list 60 users at one time because you didn't set Limit

Limit
Maximum number of users to be returned.

Type: Integer

Valid Range: Minimum value of 0. Maximum value of 60.

Required: No

See https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_ListUsers.html#API_ListUsers_RequestSyntax

Can you double check about this? Thanks!

[stupid-2020]

@stupid-2020 Have you checked with this in https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/cognito-idp.html#CognitoIdentityProvider.Client.list_users

PaginationToken (string) --

An identifier that was returned from the previous call to this operation, which can be used to return the next set of items in the list.

It seems that your script can only list 60 users at one time because you didn't set Limit

Limit
Maximum number of users to be returned.

Type: Integer

Valid Range: Minimum value of 0. Maximum value of 60.

Required: No

See https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_ListUsers.html#API_ListUsers_RequestSyntax

Can you double check about this? Thanks!

Sorry for making such mistake. Will fix it soon.

[stupid-2020]

To find user(s) with multiple group:

python cognito_list.py -d aws_profile.yml

The sample output of duplicate.csv generated:

email,name,committee
example1@gmail.com,USER_IN_TWO_GROUPS,group-one|group-two
example2@gmail.com,USER_IN_TWO_GROUPS,group-one|group-two

To remove the user(s) from the group, it is recommended to backup the duplicate.csv and keep ONLY the user(s) should be removed from the group:

python dry_run_users.py -r group-two duplicate.csv aws_profile.yml

[hao-fang]

Just a note for future references

Remember to choose "Yes - Use Amazon SES" option. (ref https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-email.html)