Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not add Kotlin and Groovy sources from build/ to LST #322

Merged
merged 2 commits into from
Aug 18, 2024

Conversation

timtebeek
Copy link
Contributor

What's changed?

Reuse the mechanism we added for protos/build/generated for build/generated-sources.

What's your motivation?

Files like build-src/build/generated-sources are parsed into the LST, and modified by recipes. That last part is undesirable, as the diffs would then target files not part of the git repository, and fail to apply from a dryRun fix.patch file.

n sourcePath sourceFileType buildTool buildToolVersion checksum lineCount
1 build-src/build/generated-sources/kotlin-dsl-external-plugin-spec-builders/kotlin/gradle/kotlin/dsl/plugins/_12895b5fa930feaf40eec89923f04a18/PluginSpecBuilders.kt K.CompilationUnit modgradle 3.17.0 55d3a6f819a2d911e8dc50fcdba328c6 1186
2 build-src/build/generated-sources/kotlin-dsl-accessors/kotlin/gradle/kotlin/dsl/accessors/_f68e08f112e4a379392eaca54506dc98/Accessorsct2ylp5ktt4rxe14s34wgi9de.kt K.CompilationUnit modgradle 3.17.0 f4d3580d463adfe088c5cc21c590086f 76
3 build-src/build/generated-sources/kotlin-dsl-accessors/kotlin/gradle/kotlin/dsl/accessors/_f68e08f112e4a379392eaca54506dc98/Accessorsd6fwhfk2ip9kv56d7w2vygrh6.kt K.CompilationUnit modgradle 3.17.0 1a203f1b9aaeb9223e2565c7cd25fa37 76

Anything in particular you'd like reviewers to focus on?

There's likely a better way, but I'm having a bit of trouble working that out. Figured open this PR to start the discussion.

Have you considered any alternatives or workarounds?

What I'd done previously in an attempt to exclude these files is add them to Git ignore:
openrewrite/rewrite-recipe-bom@9f5840c
But somehow they are still picked up; I had thought we already excluded files ignored in git.

Any additional context

I know in the Maven plugin we explicitly exclude generated sources, but I'm not immediately seeing an equivalent here.
https://github.com/ammachado/rewrite-maven-plugin/blob/f900a9a6365a4d02c0083dd29e14927d6ffa6576/src/main/java/org/openrewrite/maven/MavenMojoProjectParser.java#L425-L430

@timtebeek timtebeek added the bug Something isn't working label Aug 15, 2024
@timtebeek timtebeek requested a review from shanman190 August 15, 2024 09:31
@timtebeek timtebeek self-assigned this Aug 15, 2024
@shanman190
Copy link
Collaborator

shanman190 commented Aug 15, 2024

I suspect that we're actually including all types of generated sources that are produced by various plugins as they will appear within the source files from sourceSet.getAllSources(). I think the current proto implementation was a bandaid around the problem, but never actually addressed it.

What should probably be being done is a filter operation when collecting all of the unparsed sources and excluding them when they are contained within subproject.getLayout().getBuildDirectory() (already confirmed that this is also available on the baseline Gradle 4.10, so no special logic is required for old versions). This would be directly akin to what was done within the Maven plugin.

@timtebeek timtebeek marked this pull request as ready for review August 17, 2024 21:48
@timtebeek
Copy link
Contributor Author

Thanks for the hint! Noticed we only had that pattern for Java; now also for Kotlin & Groovy + Kotlin multi platform. Should help I think, although it's a bit cumbersome to test.

@@ -737,7 +739,7 @@ public Stream<SourceFile> parse(Project subproject, Set<Path> alreadyParsed, Exe
.flatMap(jp -> jp.parse(javaPaths, baseDir, ctx))
.map(cu -> {
if (isExcluded(exclusions, cu.getSourcePath()) ||
cu.getSourcePath().startsWith(baseDir.relativize(subproject.getLayout().getBuildDirectory().get().getAsFile().toPath()))) {
cu.getSourcePath().startsWith(buildDirPath)) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to even be parsing these files at all with OpenRewrite?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we did so for the type attribution; that's at least the comment over on the Maven plugin. Hadn't thought to change that here just yet. Just excluding them after they are parsed to prevent them ending up in the LST themselves.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we have so far, yeah, but we also trigger the compile phases in Gradle as well prior to the rewrite tasks, so technically we could put the source set's own output classes onto the parser's classpath to still get the necessary type attribution and bypass parsing the generated types.

Just a thought though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could indeed be a nice improvement to stop ourselves from parsing source files whose LSTs we'll discard anyway, or perhaps even having two sources of the same type information; compiled vs source.

@timtebeek timtebeek changed the title Exclude build-generated-sources too Do not add Kotlin and Groovy sources from build/ to LST Aug 18, 2024
@timtebeek timtebeek merged commit d9c519d into main Aug 18, 2024
1 check passed
@timtebeek timtebeek deleted the exclude-build-generated-sources branch August 18, 2024 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants