Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable To Copy Files Outside Of Target Source Folder #6982

Closed
brianmichel opened this issue Oct 9, 2023 · 6 comments · Fixed by #7232
Closed

Unable To Copy Files Outside Of Target Source Folder #6982

brianmichel opened this issue Oct 9, 2023 · 6 comments · Fixed by #7232
Labels

Comments

@brianmichel
Copy link

Description

When constructing a package which may have targets which rely on vendored files (say a .lib or .h created by an external build system) it would be reasonable to create some kind of vendor folder at the root of the package and then use the .copy command to specify which resources need to be pulled into which targets, something similar to the following:
CleanShot 2023-10-09 at 17 32 29@2x

In this setup Swift PM silently fails to copy files within the package root but outside of the target's source folder.

Expected behavior

I would expect the files to be copied into a bundle under the .build folder which does happen successfully if you .copy a file within the target's source folder. This would allow people to construct arbitrary bundles that help meet the needs of the software they are building.

Actual behavior

Swift PM silently fails to copy any files referenced that are outside of the target's source folder.

Steps to reproduce

  1. Run swift build on the sample project attached
  2. Open up the .build/debug folder
  3. Look at the contents of TestCopy_App.bundle

Notice there is only 1 file there, while 3 valid file paths have been specified in the Package file.

TestCopy.zip

Swift Package Manager version/commit hash

No response

Swift & OS version (output of swift --version ; uname -a)

Target: arm64-apple-macosx14.0
Darwin zerocool.localdomain 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64

This also happens on the latest Swift on Windows toolchain as well.

@brianmichel brianmichel added the bug label Oct 9, 2023
@neonichu
Copy link
Contributor

neonichu commented Oct 9, 2023

This easily reproduces and looking at TargetSourcesBuilder also easily explains why: the process is based on looking at the files inside a target and applying the rules to them. Rules which don't match files in the target, but are technically valid (point to an existing file inside the package), are simply never considered.

Looking at SE-0271, it's not 100% clear to me whether this was intended, it feels as if there should at least be diagnostics about the unapplied rules. The proposal talks about files in targets, so I think the implication is that it is correct that this doesn't work, but it doesn't seem right that it happens silently.

@brianmichel
Copy link
Author

Yeah I think it should at least break, however I think the other behavior is desirable.

I can see maybe why you wouldn't want to package files from anywhere, but I feel like any file in the package should be fair game imo.

@neonichu
Copy link
Contributor

neonichu commented Oct 10, 2023

I can see maybe why you wouldn't want to package files from anywhere, but I feel like any file in the package should be fair game imo.

Yah, I tend to agree, was just trying to extrapolate what behavior most matches the original design.

@brianmichel
Copy link
Author

I can see maybe why you wouldn't want to package files from anywhere, but I feel like any file in the package should be fair game imo.

Yah, I tend to agree, was just trying to extrapolate what behavior most matches the original design.

Totally! I was re-reading the originally SE and saw this line which potentially supports the idea of anything being copy-able within the package, but the original authors would probably have to clarify:

Letting package authors put resources where they like in their source directories (to allow functional organization of large code bases).

@neonichu
Copy link
Contributor

neonichu commented Dec 6, 2023

see also rdar://119040426

neonichu added a commit that referenced this issue Jan 5, 2024
It is possible to declare references to resources outside of ones target and it doesn't seem like that should be problematic in any way. Currently, these declarations essentially get ignored because the algorithm looks at files inside the target and applies rules to them, so any unmatched rules simply get ignored. In the grand scheme of things it may make sense to change that approach in a more fundamental way, but to fix this particular problem we can simply record all rules that were used and apply the unmatched rules after that initial processing has finished. Since this is a significant behavior change, we only do it for tools-version 5.11 or later.

rdar://119040426
resolves #6982
neonichu added a commit that referenced this issue Jan 6, 2024
It is possible to declare references to resources outside of ones target and it doesn't seem like that should be problematic in any way. Currently, these declarations essentially get ignored because the algorithm looks at files inside the target and applies rules to them, so any unmatched rules simply get ignored. In the grand scheme of things it may make sense to change that approach in a more fundamental way, but to fix this particular problem we can simply record all rules that were used and apply the unmatched rules after that initial processing has finished. Since this is a significant behavior change, we only do it for tools-version 5.11 or later.

rdar://119040426
resolves #6982
neonichu added a commit that referenced this issue Jan 10, 2024
It is possible to declare references to resources outside of ones target
and it doesn't seem like that should be problematic in any way.
Currently, these declarations essentially get ignored because the
algorithm looks at files inside the target and applies rules to them, so
any unmatched rules simply get ignored. In the grand scheme of things it
may make sense to change that approach in a more fundamental way, but to
fix this particular problem we can simply record all rules that were
used and apply the unmatched rules after that initial processing has
finished. Since this is a significant behavior change, we only do it for
tools-version 5.11 or later.

rdar://119040426
resolves #6982

---------

Co-authored-by: Max Desiatov <m_desiatov@apple.com>
@brianmichel
Copy link
Author

Niceee, thanks @neonichu!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants