Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix getting stuck on refreshing postgresql grant #351

Conversation

giner
Copy link
Contributor

@giner giner commented Sep 14, 2023

This change fixes terraform refresh getting stuck forever on postgresql_grant schema when provider connection database is the same as the one we are trying to read schema information from. Until now the issue could be reproduced only on RDS (PostgreSQL 14.7). Fixed by explicitly finishing one transaction before starting a new one.

Fixes #335

Copy link

@pagalba-com pagalba-com left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can confirm that this fixed issue with stuck terraform refresh on aws rds PostgreSQL v14.7.

Compiled binary from MR branch ant it works as expected. It should go main stream.

@pagalba-com
Copy link

@cyrilgdn please approve this MR! Its impossible to work on aws prod without this.

@giner giner force-pushed the fix_getting_stuck_on_refreshing_postgresql_grant branch from 72d17d4 to 45faa28 Compare October 13, 2023 08:50
@giner giner requested a review from pagalba-com October 13, 2023 08:51
@giner
Copy link
Contributor Author

giner commented Oct 13, 2023

@kylejohnson would you take a look at this fix please?

@giner
Copy link
Contributor Author

giner commented Oct 13, 2023

@cyrilgdn

@giner giner force-pushed the fix_getting_stuck_on_refreshing_postgresql_grant branch from 45faa28 to a9554d8 Compare November 16, 2023 00:34
Copy link
Owner

@cyrilgdn cyrilgdn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @giner ,

Thanks for opening this PR and sorry for the delay

I've added one comment, let me know what do you think about it.

@@ -730,6 +730,8 @@ func checkRoleDBSchemaExists(client *Client, d *schema.ResourceData) (bool, erro
return false, nil
}

deferredRollback(txn)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should better avoid creating a new transaction and use the existing one if it's the same database.

So line 738:

var dbTxn *sql.Tx
if database == client.databaseName {
  dbTxn = txn
} else {
	var err error;
	dbTxn, err := startTransaction(client, database)
	if err != nil {
		return false, err
	}
	defer deferredRollback(dbTxn)
}

(and actually if it's for another database a transaction is not really needed as it executes only 1 request but I don't think we already have an helper to get a simple DB connection for alternative DB)

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. I'll unify this function to be more in line with the rest of the code.

giner added 3 commits March 1, 2024 17:53
This change fixes terraform refresh getting stuck forever on
postgresql_grant schema when provider connection database is the same as
the one we are trying to read schema information from. Until now the
issue could be reproduced only on RDS (PostgreSQL 14.7). Fixed by
explicitly finishing one transaction before starting a new one.

Fixes cyrilgdn#335
- Pass DBConnection instead of Client (the same as in other functions)
- Check database existence with a simple query outside of a transaction
- Use a single transaction for queries within the function
@giner giner force-pushed the fix_getting_stuck_on_refreshing_postgresql_grant branch from a159d0b to 19a04cd Compare March 1, 2024 08:54
@giner giner requested a review from cyrilgdn March 1, 2024 09:02
Copy link
Owner

@cyrilgdn cyrilgdn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 🎉

@cyrilgdn cyrilgdn merged commit 1338df7 into cyrilgdn:master Mar 2, 2024
6 checks passed
@giner
Copy link
Contributor Author

giner commented Mar 4, 2024

@cyrilgdn, Does anything need to be done for v1.22.0 to hit the registry? It's not there yet.

lemeurherve referenced this pull request in jenkins-infra/azure Mar 11, 2024
<Actions>
<action
id="703b1c5c9886f66325fdfa2d8c42994644728056afe84d8427884bee985d0729">
        <h3>Bump Terraform `postgresql` provider version</h3>
<details
id="1d9343c012f5434ac9fe8a98135bae3667b399259be16d9b14302ea3bd424a24">
            <summary>Update Terraform lock file</summary>
<p>changes detected:&#xA;&#x9;&#34;cyrilgdn/postgresql&#34; updated from
&#34;1.21.0&#34; to &#34;1.22.0&#34; in file
&#34;.terraform.lock.hcl&#34;</p>
            <details>
                <summary>1.22.0</summary>
<pre>Changelog retrieved
from:&#xA;&#x9;https://github.com/cyrilgdn/terraform-provider-postgresql/releases/tag/v1.22.0&#xA;##
What&#39;s Changed&#xD;&#xA;&#xD;&#xA;#### Bug fixes&#xD;&#xA;* Fix
getting stuck on refreshing postgresql grant by @giner in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/351&#xD;&#xA;*
fix: quote table name in postgresql_publication by @jerome-quere in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/402&#xD;&#xA;&#xD;&#xA;####
Doc&#xD;&#xA;* Add `sslinline` argument description to
index.html.markdown by @Hakkenouw in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/409&#xD;&#xA;*
Fix indentation in index.html.markdown by @hfm in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/380&#xD;&#xA;*
fix(doc): Typo in `postgresql_grant_role` doc by @cyrilgdn in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/412&#xD;&#xA;&#xD;&#xA;####
Libs&#xD;&#xA;* Bump golang.org/x/crypto from 0.11.0 to 0.17.0 by
@dependabot in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/386&#xD;&#xA;&#xD;&#xA;##
New Contributors&#xD;&#xA;* @Hakkenouw made their first contribution in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/409&#xD;&#xA;*
@jerome-quere made their first contribution in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/402&#xD;&#xA;*
@hfm made their first contribution in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/380&#xD;&#xA;*
@dependabot made their first contribution in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/386&#xD;&#xA;*
@giner made their first contribution in
https://github.com/cyrilgdn/terraform-provider-postgresql/pull/351&#xD;&#xA;&#xD;&#xA;**Full
Changelog**:
https://github.com/cyrilgdn/terraform-provider-postgresql/compare/v1.21.0...v1.22.0</pre>
            </details>
        </details>
<a
href="https://infra.ci.jenkins.io/job/updatecli/job/azure/job/main/33/">Jenkins
pipeline link</a>
    </action>
</Actions>

---

<table>
  <tr>
    <td width="77">
<img src="https://www.updatecli.io/images/updatecli.png" alt="Updatecli
logo" width="50" height="50">
    </td>
    <td>
      <p>
Created automatically by <a
href="https://www.updatecli.io/">Updatecli</a>
      </p>
      <details><summary>Options:</summary>
        <br />
<p>Most of Updatecli configuration is done via <a
href="https://www.updatecli.io/docs/prologue/quick-start/">its
manifest(s)</a>.</p>
        <ul>
<li>If you close this pull request, Updatecli will automatically reopen
it, the next time it runs.</li>
<li>If you close this pull request and delete the base branch, Updatecli
will automatically recreate it, erasing all previous commits made.</li>
        </ul>
        <p>
Feel free to report any issues at <a
href="https://github.com/updatecli/updatecli/issues">github.com/updatecli/updatecli</a>.<br
/>
If you find this tool useful, do not hesitate to star <a
href="https://github.com/updatecli/updatecli/stargazers">our GitHub
repository</a> as a sign of appreciation, and/or to tell us directly on
our <a
href="https://matrix.to/#/#Updatecli_community:gitter.im">chat</a>!
        </p>
      </details>
    </td>
  </tr>
</table>

Co-authored-by: Jenkins Infra Bot (updatecli) <60776566+jenkins-infra-bot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] terraform refresh gets stuck forever on postgresql_grant schema under certain conditions
3 participants