[CI] Mutation testing: Automated github action to generate issue

[stackman27]

Closes: #2114

What is the purpose of the change

1. Make github action that runs once per week
2. It runs make test-mutation
3. Takes mutation_test_result.txt, and creates a github issue. (with some formatting. Perhaps only parse out failures, and split them up with a number and separate code block)

Brief Changelog

n/a

Testing and Verifying

Untested because need secrets.ADD_TO_PROJECT_PAT access

Documentation and Release Note

• Does this pull request introduce a new feature or user-facing behavior changes? (yes / no)
• Is a relevant changelog entry added to the Unreleased section in CHANGELOG.md? (yes / no)
• How is the feature or change documented? (not applicable / specification (x/<module>/spec/) / Osmosis docs repo / not documented)

[stackman27]

The current implementation outputs mutation test FAILS in a single python output. I'm not sure how i'll break each of the fails and split them up with a number and separate code block without using regex

cc @alexanderbez @ValarDragon @nikever @xBalbinus

[niccoloraspa]

You can add code blocks with sed.

Let's say the file contains this:

--- ./x/tokenfactory/keeper/createdenom.go	2022-07-21 11:57:23.000000000 +0200
+++ /var/folders/p8/6wtxs79s3x75t51d07gnv9c00000gn/T/go-mutesting-3068611658/./x/tokenfactory/keeper/createdenom.go.0	2022-07-23 10:16:35.000000000 +0200
@@ -77,7 +77,7 @@
 	if err != nil {
 		return err
 	}
-	if len(creationFee) > 0 {
+	if len(creationFee) >= 0 {
 		if err := k.distrKeeper.FundCommunityPool(ctx, creationFee, accAddr); err != nil {
 			return err
 		}

FAIL "/var/folders/p8/6wtxs79s3x75t51d07gnv9c00000gn/T/go-mutesting-3068611658/./x/tokenfactory/keeper/createdenom.go.0" with checksum 8530d085e7d2c8848191f6d9471baeeb

With

cat mutation_test_result.txt  | sed "s# @@# @@\n\`\`\`go\n#g " | sed "s#FAIL#\`\`\`\nFAIL\n\n\n#g " > mutation_test_result.txt 

It will encapsulate the go code in go block and add some new lines at the end:

--- ./x/tokenfactory/keeper/createdenom.go	2022-07-21 11:57:23.000000000 +0200
+++ /var/folders/p8/6wtxs79s3x75t51d07gnv9c00000gn/T/go-mutesting-3068611658/./x/tokenfactory/keeper/createdenom.go.0	2022-07-23 10:16:35.000000000 +0200
@@ -77,7 +77,7 @@
```go

 	if err != nil {
 		return err
 	}
-	if len(creationFee) > 0 {
+	if len(creationFee) >= 0 {
 		if err := k.distrKeeper.FundCommunityPool(ctx, creationFee, accAddr); err != nil {
 			return err
 		}

\``` # <--- I have added \ to escape the closing block
FAIL "/var/folders/p8/6wtxs79s3x75t51d07gnv9c00000gn/T/go-mutesting-3068611658/./x/tokenfactory/keeper/createdenom.go.0" with checksum 8530d085e7d2c8848191f6d9471baeeb

[alexanderbez]

@stackman27 do you have an example of what a github issue would look like when created?

[alexanderbez]

Why not off of main?

[stackman27]

just so that it runs every time i push to my branch.

All of it is just for testing

[alexanderbez]

Suggested change

# for testing

[stackman27]

@stackman27 do you have an example of what a github issue would look like when created?

Currently, this line is failing token: ${{ secrets.ADD_TO_PROJECT_PAT }} as i'm not sure i have access to creating github issues via actions

Once resolved the issues should look like this

[p0mvn]

@stackman27 do you have an example of what a github issue would look like when created?

Currently, this line is failing token: ${{ secrets.ADD_TO_PROJECT_PAT }} as i'm not sure i have access to creating github issues via actions

Once resolved the issues should look like this

It should be possible to create this token on your personal fork, merge this into the main of the fork, and test if this functions correctly by attempting to create an issue there

[ValarDragon]

Code LGTM, would like to see a sample issue before hitting merge tho

[ValarDragon]

I also think we're going to eventually need to do a more sophisticated python script to parse this and create the issue text body, but no need to block on this, getting the initial regular issue out is already awesome =)

[stackman27]

@stackman27 do you have an example of what a github issue would look like when created?

Currently, this line is failing token: ${{ secrets.ADD_TO_PROJECT_PAT }} as i'm not sure i have access to creating github issues via actions
Once resolved the issues should look like this

It should be possible to create this token on your personal fork, merge this into the main of the fork, and test if this functions correctly by attempting to create an issue there

hmm looks like i can't create issues in my osmosis-fork. I dont even have a tab for issues 🤔

[p0mvn]

@stackman27 do you have an example of what a github issue would look like when created?

Currently, this line is failing token: ${{ secrets.ADD_TO_PROJECT_PAT }} as i'm not sure i have access to creating github issues via actions
Once resolved the issues should look like this

It should be possible to create this token on your personal fork, merge this into the main of the fork, and test if this functions correctly by attempting to create an issue there

hmm looks like i can't create issues in my osmosis-fork. I dont even have a tab for issues thinking

That's weird haha. It looks like forks have to manually enable issues in settings:
https://softwareengineering.stackexchange.com/questions/179468/forking-a-repo-on-github-but-allowing-new-issues-on-the-fork

[stackman27]

@ValarDragon @p0mvn @alexanderbez Mutation testing issue sample issue: stackman27#13

[czarcas7ic]

This LGTM, are we switching this from testing to prod values before merging?

[stackman27]

This LGTM, are we switching this from testing to prod values before merging?

Yup just pushed the production version! Good to go now 🚀 285b904

[p0mvn]

Nice work

[p0mvn]

Merging since test issue was shown here: stackman27#13

Thanks, @stackman27