Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Duplicate symbols (filename case sensitivity issue with libsdl ) #893

Open
brentpicasso opened this issue Feb 7, 2024 · 25 comments
Open

Duplicate symbols (filename case sensitivity issue with libsdl ) #893

brentpicasso opened this issue Feb 7, 2024 · 25 comments
Assignees
@brentpicasso
Labels
bug

Comments

@brentpicasso
Copy link
Contributor

brentpicasso commented Feb 7, 2024 •
edited
Loading

Versions

  • Kivy: 2.3.0
  • Python : 3.11
  • MacOS version :
  • XCode Version :
  • Cython version :

I'm seeing duplicate symbols for libsdl libraries. This seems to be due to differences in case for libsdl files.

  • See attached output of xcode build.
  • Example from build output:
    /Users/macmini/Library/Developer/Xcode/DerivedData/racecaptureapp-gvoxmqndeqrfaxcpilbfwwppcdqu/Build/Products/Debug-iphoneos/libsdl2_mixer.a(music_drmp3.o)
    /Users/macmini/Library/Developer/Xcode/DerivedData/racecaptureapp-gvoxmqndeqrfaxcpilbfwwppcdqu/Build/Products/Debug-iphoneos/libSDL2_mixer.a(music_drmp3.o)

My initial questions

  • Which variant of these is correct?
  • Suggestions on how to fix? I'm happy to submit a PR. 👍

libsdl_duplicate_symbols.txt

Steps I did to observe this:

I did a complete rebuild of the toolchain (deleted build and dist) and re-installed with this command:

python3 toolchain.py build python3 kivy==2.3.0 pillow ffmpeg ffpyplayer

  • Additional context: I'm migrating a project from kivy 2.2.1 to kivy 2.3.0
  • I'm on the latest of kivy-ios
  • Output of toolchain status:
Brents-Mac-mini:kivy-ios macmini$ python3 toolchain.py status
audiostream  - Not built
click        - Not built
curly        - Not built
cymunk       - Not built
ffmpeg       - Build OK (built at 2024-02-07 16:18:21.205165)
ffpyplayer   - Build OK (built at 2024-02-07 16:19:05.286216)
flask        - Not built
freetype     - Build OK (built at 2024-02-07 16:13:27.340200)
hostopenssl  - Build OK (built at 2024-02-07 15:57:33.440418)
hostpython3  - Build OK (built at 2024-02-07 16:03:27.748946)
ios          - Build OK (built at 2024-02-07 16:09:54.205482)
itsdangerous - Not built
jinja2       - Not built
kivent_core  - Not built
kivy         - Build OK (built at 2024-02-07 16:12:21.744860)
kiwisolver   - Not built
libcurl      - Not built
libffi       - Build OK (built at 2024-02-07 15:59:42.695370)
libjpeg      - Build OK (built at 2024-02-07 16:13:45.282976)
libpng       - Build OK (built at 2024-02-07 16:08:31.277707)
libzbar      - Not built
markupsafe   - Not built
matplotlib   - Not built
netifaces    - Not built
numpy        - Not built
openssl      - Build OK (built at 2024-02-07 16:00:49.600563)
photolibrary - Not built
pillow       - Build OK (built at 2024-02-07 16:14:23.302231)
plyer        - Not built
py3dns       - Not built
pycrypto     - Not built
pykka        - Not built
pyobjus      - Build OK (built at 2024-02-07 16:10:04.599033)
python3      - Build OK (built at 2024-02-07 16:07:31.367719)
pyyaml       - Not built
sdl2         - Build OK (built at 2024-02-07 16:08:45.134009)
sdl2_image   - Build OK (built at 2024-02-07 16:08:54.650620)
sdl2_mixer   - Build OK (built at 2024-02-07 16:09:04.534253)
sdl2_ttf     - Build OK (built at 2024-02-07 16:09:44.320951)
werkzeug     - Not built
zbarlight    - Not built
Brents-Mac-mini:kivy-ios macmini$ python3 toolchain.py status
audiostream  - Not built
click        - Not built
curly        - Not built
cymunk       - Not built
ffmpeg       - Build OK (built at 2024-02-07 16:18:21.205165)
ffpyplayer   - Build OK (built at 2024-02-07 16:19:05.286216)
flask        - Not built
freetype     - Build OK (built at 2024-02-07 16:13:27.340200)
hostopenssl  - Build OK (built at 2024-02-07 15:57:33.440418)
hostpython3  - Build OK (built at 2024-02-07 16:03:27.748946)
ios          - Build OK (built at 2024-02-07 16:09:54.205482)
itsdangerous - Not built
jinja2       - Not built
kivent_core  - Not built
kivy         - Build OK (built at 2024-02-07 16:12:21.744860)
kiwisolver   - Not built
libcurl      - Not built
libffi       - Build OK (built at 2024-02-07 15:59:42.695370)
libjpeg      - Build OK (built at 2024-02-07 16:13:45.282976)
libpng       - Build OK (built at 2024-02-07 16:08:31.277707)
libzbar      - Not built
markupsafe   - Not built
matplotlib   - Not built
netifaces    - Not built
numpy        - Not built
openssl      - Build OK (built at 2024-02-07 16:00:49.600563)
photolibrary - Not built
pillow       - Build OK (built at 2024-02-07 16:14:23.302231)
plyer        - Not built
py3dns       - Not built
pycrypto     - Not built
pykka        - Not built
pyobjus      - Build OK (built at 2024-02-07 16:10:04.599033)
python3      - Build OK (built at 2024-02-07 16:07:31.367719)
pyyaml       - Not built
sdl2         - Build OK (built at 2024-02-07 16:08:45.134009)
sdl2_image   - Build OK (built at 2024-02-07 16:08:54.650620)
sdl2_mixer   - Build OK (built at 2024-02-07 16:09:04.534253)
sdl2_ttf     - Build OK (built at 2024-02-07 16:09:44.320951)
werkzeug     - Not built
zbarlight    - Not built
@Julian-O Julian-O added the bug label Feb 7, 2024
@brentpicasso
Copy link
Contributor Author

I checked my android buildozer build and it appears the uppercase variant is used there. Does that mean the uppercase variant is the "correct" version?

libSDL2*.so

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024

It looks like this issue has come up before:
kivy/buildozer#566

If case insensitivity still the default for macOS, it might be easiest to move to a case insensitive disk.

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024

In the recipe, it is mixed case:
https://github.com/kivy/kivy-ios/blob/master/kivy_ios/recipes/sdl2/__init__.py#L8

so there is probably a string conversion going on somewhere in either kivy-ios or the rest of the build system (xcode)

@brentpicasso
Copy link
Contributor Author

I checked my mac, and it is current set to case insensitive

Brents-Mac-mini:kivy-ios macmini$ nano blah.txt
Brents-Mac-mini:kivy-ios macmini$ cat blah.txt 
asdf
Brents-Mac-mini:kivy-ios macmini$ cat BLAH.txt
asdf

@tcaduser thanks. I see in that link the SDL2 is upper case, which I think is the correct form. So we need to find out what thing is causing the lower case form?

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024

If your filesystem is case insensitive, it means that the same exact file is being referenced twice.

Looking on your linker line, I see the same libraries being listed multiple times:

-lsdl2
-lSDL2
-lsdl2_image
-lSDL2_image
-lsdl2_mixer
-lSDL2_mixer
-lsdl2_ttf
-lSDL2_ttf

ordinarily a linker shouldn't care if the same filename is listed multiple times, so it may mean that the case difference is confusing clang:

    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang

so you would need to find the recipe referencing the libraries with different case sensitivity.

@brentpicasso
Copy link
Contributor Author

So you mean grepping this repository's files for a lower case form of lsdl2? We can do that.

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024

what I am seeing is that the recipes are referencing themselves with the mixed case name:

jsanchez@jslaptop:~/git/kivy-ios/kivy_ios$ grep -s -i   SDL2 */*/*    | grep libr
recipes/sdl2_image/__init__.py:    library = "Xcode/build/Release-{plat.sdk}/libSDL2_image.a"
recipes/sdl2/__init__.py:    library = "Xcode/SDL/build/Release-{plat.sdk}/libSDL2.a"
recipes/sdl2_mixer/__init__.py:    library = "Xcode/build/Release-{plat.sdk}/libSDL2_mixer.a"
recipes/sdl2_ttf/__init__.py:    library = "Xcode/build/Release-{plat.sdk}/libSDL2_ttf.a"

and the dependent packages are using the case insensitive name:

recipes/audiostream/__init__.py:    depends = ["python", "sdl2", "sdl2_mixer"]
recipes/curly/__init__.py:    depends = ["python", "libcurl", "sdl2", "sdl2_image"]
recipes/ffpyplayer/__init__.py:    depends = ["python", "sdl2", "ffmpeg"]
recipes/kivy/__init__.py:    depends = ["sdl2", "sdl2_image", "sdl2_mixer", "sdl2_ttf", "ios",
recipes/sdl2_image/__init__.py:    depends = ["sdl2"]
recipes/sdl2_mixer/__init__.py:    depends = ["sdl2"]
recipes/sdl2_ttf/__init__.py:    depends = ["libpng", "sdl2"]

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024

Since I've never seen this problem before, but I haven't build a package in a while, I am guessing that either

Your version of XCode is too new

or

Your version of XCode is too old

@brentpicasso
Copy link
Contributor Author

Thanks, I'll check the xcode version.

When you say "mixed cased name" - do you mean the uppercase form (SDL2) and the case insensitive name is the lower case form (sdl2) ?

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024

also note that the linker is being passed:
-Xlinker -no_deduplicate

perhaps from XCode itself.

     -no_deduplicate
                 Don't run deduplication pass in linker

my hypothesis, based on:
https://forums.developer.apple.com/forums/thread/44905

is that XCode may have a bug in setting this flag.

Are you doing the final compile? It looks like you are doing a Debug build. Can you choose a Release build in XCode, and see if that flag magically goes away.

Please note that a flag like this is to prevent symbol clashes and resulting in bad behavior.

@brentpicasso
Copy link
Contributor Author

Checked two things:

  • I'm running Xcode 14.3. Latest is 15 (released Sept 2023)
  • I tried running the archive command (which I presumably does a release build) - and got the same error.

I can try upgrading to 15.

@brentpicasso
Copy link
Contributor Author

In order to go to Xcode 15 I first have to upgrade to MacOS 13.5.

Perspective: my builds were previously running correctly on kivy 2.2.1 - the problems only started when I upgraded to kivy 2.3.0 and this latest kivy-ios,

XCode was the same (14.3) under the previous setup.

Given this, could it be possible something in kivy-ios was regressed?

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024

I don't know. The owners of this project are doing a lot of changes. I would look into your XCode build configuration and see if you can remove the no_deduplicate flag manually. Release and Debug flags are things that you can control in gui

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024

or you can see if the owners of this project would be willing to filter out duplicated library names when generating the xcode project

@brentpicasso
Copy link
Contributor Author

Thanks for all of the help! I will check those and get back. Hopefully the project maintainers chime in.

@misl6
Copy link
Member

misl6 commented Feb 8, 2024

@brentpicasso can you please share a screenshot of "Frameworks, Libraries, and Embedded content" from XCode?

Since we moved from libname.a to libname.xcframework something may gone wrong during your Xcode project migration.

@brentpicasso
Copy link
Contributor Author

@misl6 Here you go:

image

image

image

@misl6
Copy link
Member

misl6 commented Feb 8, 2024 •
edited
Loading

That's why.

The libname.a files should not be there (even if still available under dist/lib/**/*)

Can you try to remove them?

That section should look like that after the changes:

Image 08-02-24 at 19 21

@brentpicasso
Copy link
Contributor Author

@misl6 thank you for the help>

I'm not following - in my screenshots I don't see libname.a listed there. Where are you seeing it?

@brentpicasso
Copy link
Contributor Author

I searched my project.pbxproj and there are no references to the string libname in there...

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024 via email

@brentpicasso
Copy link
Contributor Author

Oh, so not specifically "libname.a" - but remove anything with a ".a" extension from the gui.

I can try that. Thank you.

@tcaduser
Copy link
Contributor

tcaduser commented Feb 8, 2024 via email

@brentpicasso
Copy link
Contributor Author

I think I got it, thanks for the help!

@brentpicasso
Copy link
Contributor Author

@misl6 is there a migration guide this info could be added to?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brentpicasso brentpicasso

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Julian-O @tcaduser @brentpicasso @misl6