Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

util: add fast path for Latin1 decoding #55275

Merged
merged 15 commits into from
Dec 3, 2024
Merged

util: add fast path for Latin1 decoding #55275

merged 15 commits into from
Dec 3, 2024

Conversation

mertcanaltin
Copy link
Member

@mertcanaltin mertcanaltin commented Oct 5, 2024 •
edited
Loading

I added a fast path for Latin1 (windows-1252) decoding to improve performance. This change avoids using the slower ICU-based decoding for Latin1 and instead utilizes a direct approach, similar to the fast path implemented for UTF-8.
nodejs/performance#178

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/security-wg
  • @nodejs/v8-update

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run. v8 engine Issues and PRs related to the V8 dependency. labels Oct 5, 2024
@mertcanaltin mertcanaltin requested review from anonrig and joyeecheung and removed request for anonrig October 5, 2024 08:45
@mertcanaltin mertcanaltin force-pushed the dev-178 branch 2 times, most recently from 2dd0714 to b871573 Compare October 5, 2024 08:50
@codecov Codecov
Copy link

codecov bot commented Oct 5, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 21.05263% with 30 lines in your changes missing coverage. Please review.

Project coverage is 88.39%. Comparing base (bbdfeeb) to head (77bc5ee).
Report is 456 commits behind head on main.

Files with missing lines Patch % Lines
src/encoding_binding.cc 10.34% 26 Missing ⚠️
lib/internal/encoding.js 55.55% 3 Missing and 1 partial ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main   #55275      +/-   ##
==========================================
- Coverage   88.41%   88.39%   -0.03%     
==========================================
  Files         652      652              
  Lines      186612   186799     +187     
  Branches    36062    36120      +58     
==========================================
+ Hits       165001   165117     +116     
- Misses      14885    14944      +59     
- Partials     6726     6738      +12
Files with missing lines Coverage Δ
src/encoding_binding.h 100.00% <ø> (ø)
lib/internal/encoding.js 98.87% <55.55%> (-0.64%) ⬇️
src/encoding_binding.cc 71.42% <10.34%> (-12.82%) ⬇️

... and 48 files with indirect coverage changes

@anonrig
Copy link
Member

anonrig commented Oct 5, 2024

Can you update benchmarks as well?

anonrig
anonrig reviewed Oct 5, 2024
View reviewed changes
lib/internal/encoding.js Outdated Show resolved Hide resolved
@anonrig anonrig requested review from addaleax and lemire October 5, 2024 13:28
anonrig
anonrig requested changes Oct 5, 2024
View reviewed changes
lib/internal/encoding.js Show resolved Hide resolved
lib/internal/encoding.js Outdated Show resolved Hide resolved
anonrig
anonrig requested changes Oct 5, 2024
View reviewed changes
lib/internal/encoding.js Outdated Show resolved Hide resolved
lib/internal/encoding.js Show resolved Hide resolved
lib/internal/encoding.js Outdated Show resolved Hide resolved
@mertcanaltin
Copy link
Member Author

mertcanaltin commented Oct 5, 2024 •
edited
Loading

Can you update benchmarks as well?

I wonder if this is the right place.
benchmark/util/text-decoder.js

@mertcanaltin mertcanaltin added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Nov 25, 2024
@aduh95 aduh95 added commit-queue Add this label to land a pull request using GitHub Actions. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. labels Dec 3, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Dec 3, 2024
@nodejs-github-bot nodejs-github-bot merged commit 20bcaa0 into nodejs:main Dec 3, 2024
76 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 20bcaa0

aduh95 pushed a commit to aduh95/node that referenced this pull request Dec 4, 2024
@mertcanaltin @nodejs-github-bot
util: add fast path for Latin1 decoding
bbd4d9a 
PR-URL: nodejs#55275
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
aduh95 pushed a commit to aduh95/node that referenced this pull request Dec 4, 2024
@mertcanaltin @nodejs-github-bot
util: add fast path for Latin1 decoding
fd10a0a 
PR-URL: nodejs#55275
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
targos pushed a commit that referenced this pull request Dec 6, 2024
@mertcanaltin @targos
util: add fast path for Latin1 decoding
9eccd7d 
PR-URL: #55275
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
@targos targos mentioned this pull request Dec 6, 2024
Merged
@lutzroeder lutzroeder mentioned this pull request Dec 11, 2024
Closed
@aduh95 aduh95 added dont-land-on-v18.x PRs that should not land on the v18.x-staging branch and should not be released in v18.x. dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. dont-land-on-v22.x PRs that should not land on the v22.x-staging branch and should not be released in v22.x. and removed dont-land-on-v18.x PRs that should not land on the v18.x-staging branch and should not be released in v18.x. dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. dont-land-on-v22.x PRs that should not land on the v22.x-staging branch and should not be released in v22.x. labels Dec 11, 2024
ruyadorno pushed a commit that referenced this pull request Dec 20, 2024
@mertcanaltin @ruyadorno
util: add fast path for Latin1 decoding
99eddec 
PR-URL: #55275
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
@ruyadorno ruyadorno mentioned this pull request Dec 20, 2024
Merged
ruyadorno pushed a commit that referenced this pull request Jan 5, 2025
@mertcanaltin @ruyadorno
util: add fast path for Latin1 decoding
374eb41 
PR-URL: #55275
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
marco-ippolito pushed a commit that referenced this pull request Jan 22, 2025
@mertcanaltin @marco-ippolito
util: add fast path for Latin1 decoding
2c8e1f2 
PR-URL: #55275
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
marco-ippolito pushed a commit that referenced this pull request Jan 22, 2025
@mertcanaltin @marco-ippolito
util: add fast path for Latin1 decoding
bf0e5b1 
PR-URL: #55275
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
marco-ippolito pushed a commit that referenced this pull request Jan 22, 2025
@mertcanaltin @marco-ippolito
util: add fast path for Latin1 decoding
2b6bdee 
PR-URL: #55275
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
@marco-ippolito marco-ippolito mentioned this pull request Jan 22, 2025
Merged
marco-ippolito pushed a commit that referenced this pull request Jan 24, 2025
@mertcanaltin @marco-ippolito
util: add fast path for Latin1 decoding
cca7c51 
PR-URL: #55275
Reviewed-By: Rafael Gonzaga <rafael.nunu@hotmail.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
@legendecas legendecas mentioned this pull request Jan 27, 2025
Closed
nodejs-github-bot pushed a commit that referenced this pull request Jan 29, 2025
@legendecas @nodejs-github-bot
test: convert test_encoding_binding.cc to a JS test
3c105b6 
The cctest file `test_encoding_binding.cc` is never tested and it is
not a valid test. Binding functions should never be tested with V8 API
circumvented. A binding function should only be tested with JS calls.

PR-URL: #56791
Refs: #55275
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
nodejs-github-bot pushed a commit that referenced this pull request Jan 29, 2025
@legendecas @nodejs-github-bot
test: search cctest files
82ac335 
To prevent a new cctest missing from the `node.gyp`, search cctest
files with tool `search_files.py` at configure time.

PR-URL: #56791
Refs: #55275
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
targos pushed a commit that referenced this pull request Feb 2, 2025
@legendecas @targos
test: convert test_encoding_binding.cc to a JS test
86c199b 
The cctest file `test_encoding_binding.cc` is never tested and it is
not a valid test. Binding functions should never be tested with V8 API
circumvented. A binding function should only be tested with JS calls.

PR-URL: #56791
Refs: #55275
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
targos pushed a commit that referenced this pull request Feb 2, 2025
@legendecas @targos
test: search cctest files
1146f48 
To prevent a new cctest missing from the `node.gyp`, search cctest
files with tool `search_files.py` at configure time.

PR-URL: #56791
Refs: #55275
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Daniel Lemire <daniel@lemire.me>
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@targos targos targos left review comments

@lemire lemire lemire approved these changes

@jasnell jasnell jasnell approved these changes

@anonrig anonrig anonrig approved these changes

@RafaelGSS RafaelGSS RafaelGSS approved these changes

@joyeecheung joyeecheung Awaiting requested review from joyeecheung

@addaleax addaleax Awaiting requested review from addaleax

Assignees
No one assigned
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. needs-ci PRs that need a full CI run. v8 engine Issues and PRs related to the V8 dependency.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@mertcanaltin @nodejs-github-bot @anonrig @lemire @jasnell @targos @RafaelGSS @aduh95