fs/glob doesn't match all filenames with valid UTF-8 names (emoji) on macOS

[jakedn]

bb --version
babashka v1.12.197

MacOS 15.4 (apple silicon)

I use bb to help me manipulate my notes. I use emoji in filenames and file contents, after moving to MacOS from linux I realized the glob wasn't getting all files on my mac.

You can reproduce with the following (on MacOS)

1. open terminal
2. mkdir -p ~/Downloads/bb-glob-issue
3. touch ~/Downloads/bb-glob-issue/'📷 photography.md'
4. touch ~/Downloads/bb-glob-issue/'🗞️ Article.md'
5. touch ~/Downloads/bb-glob-issue/'🗣️ talk.md'
6. touch ~/Downloads/bb-glob-issue/'🤔 interesting things.md'
7. start repl and import babashka.fs
8. (fs/glob (fs/expand-home "~/Downaloads/bb-glob-issue") "*.md")

This only returns '🤔 interesting things.md' and '📷 photography.md'

Is this a graalvm problem ?
It is confusing to me how it works fine on linux but not on MacOS; the shell on mac is able to glob as I would expect it is only going thought babashka.fs/glob that gives me this issue.

[borkdude]

It's a weird issue but on my Macbook Pro i9 Intel I see these results consistently between the JVM and babashka:

$ bb -e '(fs/glob (fs/expand-home "~/Downloads/bb-glob-issue") "*.md")'
[#object[sun.nio.fs.UnixPath 0x9b0dedf "/Users/borkdude/Downloads/bb-glob-issue/🤔 interesting things.md"] #object[sun.nio.fs.UnixPath 0x2387b5cf "/Users/borkdude/Downloads/bb-glob-issue/📷 photography.md"]]

$ clj -M -e "(require '[babashka.fs :as fs])" -e '(fs/glob (fs/expand-home "~/Downloads/bb-glob-issue") "*.md")'
[#object[sun.nio.fs.UnixPath 0x4c4f4365 "/Users/borkdude/Downloads/bb-glob-issue/🤔 interesting things.md"] #object[sun.nio.fs.UnixPath 0xacf859d "/Users/borkdude/Downloads/bb-glob-issue/📷 photography.md"]]

The JVM I'm using is Oracle GraalVM 24+36.1. So it could be a Oracle GraalVM issue. Let me try another Java version...

[borkdude]

With Java 17 Temurin I'm getting the same result.

[borkdude]

I managed to reproduce it in pure Java interop:

user=> (.matches (.getPathMatcher (java.nio.file.FileSystems/getDefault) "glob:*.*") (.toPath (clojure.java.io/file "🗞️ Article.md")))
false
user=> (.matches (.getPathMatcher (java.nio.file.FileSystems/getDefault) "glob:*.*") (.toPath (clojure.java.io/file "Article.md")))
true

Do you see the same?

[borkdude]

I posted a question about this on Stackoverflow: https://stackoverflow.com/questions/79568521/glob-not-working-on-paths-with-unicode-in-java

[borkdude]

I noticed the **.* pattern does work but this also matches subdirectories.

[jakedn]

Do you see the same?

I don't have java installed on my machine and won't be on a Mac over the weekend.
I'll be sure to check it next week and report back.

Based on your tests it does seem to be more of a java/jvm issue on MacOS. I only noticed this because I moved from linux to Mac and all the sudden bb was skipping files when manipulating them.
For now I have a workaround with list-dir and filtering names in the script.

To be honest, I wasn't expected such a fast response; I have a lot of appreciation for your work, it makes me feel I chose the right tool in babashka!

[borkdude]

Thanks!

The problem only seems to be there on macOS for me as well. I tried linux and Windows but those return true for both expressions.

[borkdude]

I posted a bug report to https://bugreport.java.com/