Code generation error: Import cycle not allowed

[mbbush]

What happened?

I've noticed that since the conversion webhooks were added, when making certain otherwise-valid changes to the resource configuration in provider-aws, the code generation pipeline returns errors. And even when it succeeds, it emits warnings about import cycles.

Warnings on successful codegen:

17:15:47 [ .. ] verify go modules dependencies have expected content
all modules verified
17:15:50 [ OK ] go modules dependencies verified
17:15:50 [ .. ] generating provider schema for hashicorp/aws 5.31.0
17:15:55 [ OK ] generating provider schema for hashicorp/aws 5.31.0

...

17:15:58 [ .. ] go generate linux_amd64

Generated 933 resources!
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta1","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/autoscaling/v1beta1 github.com/upbound/provider-aws/apis/autoscaling/v1beta2 github.com/upbound/provider-aws/apis/autoscaling/v1beta1]\n/home/matt/git/upbound-provider-aws/apis/autoscaling/v1beta1/zz_generated.resolvers.go:13:10: could not import github.com/upbound/provider-aws/apis/autoscaling/v1beta2 (import cycle: [github.com/upbound/provider-aws/apis/autoscaling/v1beta2 github.com/upbound/provider-aws/apis/autoscaling/v1beta1])\n"}
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta1","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/connect/v1beta1 github.com/upbound/provider-aws/apis/connect/v1beta2 github.com/upbound/provider-aws/apis/connect/v1beta1]\n/home/matt/git/upbound-provider-aws/apis/connect/v1beta1/zz_generated.resolvers.go:13:10: could not import github.com/upbound/provider-aws/apis/connect/v1beta2 (import cycle: [github.com/upbound/provider-aws/apis/connect/v1beta2 github.com/upbound/provider-aws/apis/connect/v1beta1])\n"}
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta1","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/elasticache/v1beta1 github.com/upbound/provider-aws/apis/elasticache/v1beta2 github.com/upbound/provider-aws/apis/elasticache/v1beta1]\n/home/matt/git/upbound-provider-aws/apis/elasticache/v1beta1/zz_generated.resolvers.go:14:10: could not import github.com/upbound/provider-aws/apis/elasticache/v1beta2 (import cycle: [github.com/upbound/provider-aws/apis/elasticache/v1beta2 github.com/upbound/provider-aws/apis/elasticache/v1beta1])\n"}
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta1","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/kafka/v1beta1 github.com/upbound/provider-aws/apis/kafka/v1beta2 github.com/upbound/provider-aws/apis/kafka/v1beta1]\n/home/matt/git/upbound-provider-aws/apis/kafka/v1beta1/zz_generated.resolvers.go:16:10: could not import github.com/upbound/provider-aws/apis/kafka/v1beta2 (import cycle: [github.com/upbound/provider-aws/apis/kafka/v1beta2 github.com/upbound/provider-aws/apis/kafka/v1beta1])\n"}
{"level":"info","ts":"2024-03-08T17:22:12-08:00","logger":"transformer-resolver","msg":"Encounter the following issues when loading a package","package":"v1beta2","issues":"-: import cycle not allowed: import stack: [github.com/upbound/provider-aws/apis github.com/upbound/provider-aws/apis/opsworks/v1beta1 github.com/upbound/provider-aws/apis/rds/v1beta2 github.com/upbound/provider-aws/apis/rds/v1beta1 github.com/upbound/provider-aws/apis/rds/v1beta2]\n/home/matt/git/upbound-provider-aws/apis/rds/v1beta2/zz_generated.resolvers.go:16:10: could not import github.com/upbound/provider-aws/apis/rds/v1beta1 (import cycle: [github.com/upbound/provider-aws/apis/rds/v1beta1 github.com/upbound/provider-aws/apis/rds/v1beta2])\n"}
17:22:13 [ OK ] go generate linux_amd64
17:22:13 [ .. ] go mod tidy
17:22:13 [ OK ] go mod tidy

Errors during unsuccessful codegen:

18:17:58 [ .. ] verify go modules dependencies have expected content
all modules verified
18:18:00 [ OK ] go modules dependencies verified
18:18:00 [ .. ] generating provider schema for hashicorp/aws 5.31.0
18:18:02 [ OK ] generating provider schema for hashicorp/aws 5.31.0

...

18:18:05 [ .. ] go generate linux_amd64
package command-line-arguments
	imports github.com/upbound/provider-aws/config
	imports github.com/upbound/provider-aws/config/autoscaling
	imports github.com/upbound/provider-aws/apis/autoscaling/v1beta1
	imports github.com/upbound/provider-aws/apis/autoscaling/v1beta2
	imports github.com/upbound/provider-aws/apis/autoscaling/v1beta1: import cycle not allowed
package command-line-arguments
	imports github.com/upbound/provider-aws/config
	imports github.com/upbound/provider-aws/config/connect
	imports github.com/upbound/provider-aws/apis/connect/v1beta1
	imports github.com/upbound/provider-aws/apis/connect/v1beta2
	imports github.com/upbound/provider-aws/apis/connect/v1beta1: import cycle not allowed
package command-line-arguments
	imports github.com/upbound/provider-aws/config
	imports github.com/upbound/provider-aws/config/elasticache
	imports github.com/upbound/provider-aws/apis/elasticache/v1beta1
	imports github.com/upbound/provider-aws/apis/elasticache/v1beta2
	imports github.com/upbound/provider-aws/apis/elasticache/v1beta1: import cycle not allowed
apis/generate.go:28: running "go": exit status 1
18:18:06 [FAIL]
make[1]: *** [build/makelib/golang.mk:240: go.generate] Error 1
make: *** [build/makelib/common.mk:434: generate] Error 2

How can we reproduce it?

I encountered this when adding new resources to provider-aws. Starting from main, do the following:

1. Add "aws_msk_cluster_policy": config.IdentifierFromProvider, to the terraform SDK external name config.
2. Run make generate. It passes.
3. Change the external name config to "aws_msk_cluster_policy": config.TemplatedStringAsIdentifier("", "{{ .parameters.cluster_arn }}"),
4. Run make generate. It fails partway through with this error:

17:31:50 [ .. ] verify go modules dependencies have expected content
all modules verified
17:31:52 [ OK ] go modules dependencies verified
17:31:52 [ .. ] generating provider schema for hashicorp/aws 5.31.0
17:31:55 [ OK ] generating provider schema for hashicorp/aws 5.31.0

...

17:31:57 [ .. ] go generate linux_amd64

Generated 933 resources!
angryjet: error: error loading packages using pattern ./...: /home/matt/git/upbound-provider-aws/apis/kafka/v1beta1/zz_generated.resolvers.go:409:62: mg.Spec.InitProvider.ClusterArn undefined (type ScramSecretAssociationInitParameters has no field or method ClusterArn)
exit status 1
apis/generate.go:34: running "go": exit status 1
17:35:45 [FAIL]
make[1]: *** [build/makelib/golang.mk:240: go.generate] Error 1
make: *** [build/makelib/common.mk:434: generate] Error 2

This makes some sense, as the new external name configuration sets ClusterArn as an identifier field, which should not be in Spec.InitProvider, but I don't understand why angryjet is still looking for it there.
5. Run make generate again. It fails with the import cycle error above.

I can work around the issue by dropping the changes made by the first run of make generate, but that's not easy to do for existing resources, so one impact of this bug is it makes it very difficult to add an identifier field to an existing resource's external name configuration. Given that (at least in provider-aws) there are still many external name configs which are un-normalized, inconvenient to use, and undocumented, this is an area that I feel is important to be able to improve upon.

[mbbush]

I determined that the error comes from the resolvers. Running find . -name zz_generated.resolvers.go -delete before make generate prevents the angryjet error.