Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix watcher failure due to incorrect file name encoding #78

Merged
merged 1 commit into from
Oct 15, 2018

Conversation

stkao05
Copy link
Contributor

@stkao05 stkao05 commented Oct 13, 2018

When the watched file name isn't US-ASCII encoded (i.e. UTF-8), the watcher will raise error when file modification is made:

processing events: U+6211 from UTF-8 to US-ASCII Backtrace:

[2018-10-13T15:17:43.736524 #83757] ERROR -- : exception while processing events: U+6211 from UTF-8 to US-ASCII Backtrace:
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:71:in `encode!'
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:71:in `block in normalize_encoding'
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:71:in `map'
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:71:in `normalize_encoding'
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:59:in `block in listen_handler'

The issue is fixed by avoiding the need for encoding conversion when watcher outputting log to the console.

Steps to reproduce

  • Create a post with file name "2018-01-01-夏天.markdown"
  • Run jekyll (bundle exec jekyll serve)
  • Make any modification to the post file to trigger file change event, and error will be raised.

@ashmaroli
Copy link
Member

This still breaks watching on Windows and source at D:/testé-çasé

Configuration file: D:/testé-çasé/_config.yml
            Source: D:/testé-çasé
       Destination: public
 Incremental build: disabled. Enable with --incremental
      Generating...
                    done in 0.749 seconds.
 Auto-regeneration: enabled for 'D:/testé-çasé'
    Server address: http://127.0.0.1:4000
  Server running... press ctrl-c to stop.
      Regenerating: 1 file(s) changed at 2018-10-13 05:27:29
E, [2018-10-13T05:27:29.792599 #11068] ERROR -- : exception while processing events: incompatible character encodings: UTF-8 and Windows-1252

@DirtyF
Copy link
Member

DirtyF commented Oct 13, 2018

@ashmaroli this PR focuses on files inside the source dir, fine by me, but we need to add a test.

@ashmaroli
Copy link
Member

@DirtyF I agree this fix is legit for watching 2018-01-01-夏天.markdown inside a "latin-named" source_dir, and therefore should definitely be merged..
Just pointing out that this "breaks" a "non-latin-named" source_dir and a "latin-named regular" dest_dir on Windows..

DirtyF
DirtyF previously approved these changes Oct 13, 2018
Copy link
Member

@DirtyF DirtyF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to have a test checkin' this, especially if we want Jekyll 4 to properly handle i18n.

@stkao05 stkao05 force-pushed the fix-fname-encoding branch from 17ad66c to 262af9d Compare October 14, 2018 12:13
When the watched file name isn't US-ASCII encoded (i.e. UTF-8)
the watcher will fail due to error when file modification is made:

```
processing events: U+6211 from UTF-8 to US-ASCII Backtrace:

[2018-10-13T15:17:43.736524 #83757] ERROR -- : exception while processing events: U+6211 from UTF-8 to US-ASCII Backtrace:
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:71:in `encode!'
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:71:in `block in normalize_encoding'
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:71:in `map'
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:71:in `normalize_encoding'
 -- .rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/jekyll-watch-2.1.1/lib/jekyll/watcher.rb:59:in `block in listen_handler'
```
@stkao05 stkao05 force-pushed the fix-fname-encoding branch from 262af9d to 5778ee3 Compare October 14, 2018 12:20
@stkao05
Copy link
Contributor Author

stkao05 commented Oct 14, 2018

Having thought about it a bit, I think the fix I made previously was wrong.

The issue codes was outputting file path info to the console, and therefore, it is the encoding of the file name that we are working with, not the encoding of file content. The previous fix incorrectly encodes the file name encoding with the content encoding specified in the Jekyll config.

I change my commit to fix this issue with a different approach which avoids the need for encoding conversion altogether. I have also added a new post file with Chinese file name for test purpose.

@ashmaroli Could you check if the fix works on your machine? Thanks!

@ashmaroli
Copy link
Member

@stkao05 You have effectively replaced the fix introduced in #69 with a better alternative. The issue I noted in #78 (comment) is still present but that shouldn't be a concern..

Great Job!! 👍 💯

Copy link
Member

@ashmaroli ashmaroli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 For a better alternative

@ashmaroli
Copy link
Member

@DirtyF I'm dismissing your "Approval" and requesting a re-review of the "current implementation"..

@ashmaroli ashmaroli dismissed DirtyF’s stale review October 14, 2018 16:10

Branch content changed significantly since approval.

@ashmaroli ashmaroli requested a review from a team October 14, 2018 16:10
Copy link
Member

@DirtyF DirtyF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works on my end.

@DirtyF
Copy link
Member

DirtyF commented Oct 15, 2018

@jekyllbot: merge +fix

@jekyllbot jekyllbot merged commit 1fc7582 into jekyll:master Oct 15, 2018
jekyllbot added a commit that referenced this pull request Oct 15, 2018
@jekyll jekyll locked and limited conversation to collaborators Oct 15, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants