src: some refactoring of node_url.cc #17470

addaleax · 2017-12-05T13:56:40Z

src: minor cleanups to node_url.cc
- Remove pointless pointers
- Make WriteHost take a const argument so that it’s functionality
  is clear from the signature
- Make FindLongestZeroSequence templated to accommodate the constness
  in WriteHost and because using uint16_t is an articifial,
  unnecessary restriction
- Remove string copying when no copies are needed
- Make PercentDecode just return its return value
- Make ParseHost (string-only version) take its constant argument
  as a constant reference
src: move url internals into anonymous namespace

This helps because static doesn’t work for C++ classes,
but refactoring url_host into a proper C++ class seems the
most reasonable soluation for the memory leak fixed by the next commit.
src: make url host a proper C++ class
- Gives URLHost a proper destructor that clears memory
  depending on the type of the host (This fixes a memory leak)
- Hide the host type enums and class layout as implementation details
- Make the Parse methods members of URLHost
- Turn WriteHost into a ToString() method on the URLHost class
- Verify that at the beginning of a parse attempt, the type is set
  to “failed”
- Remove a lot of gotos from the source code 🐢🚀
Fixes: Why use new URL() may lead to memory leak ? #17448
src: use correct OOB check for IPv6 parsing

last_piece pointed to the end of the 8×16 bit array,
so piece_pointer == last_piece already means that the pointer
is not writable any longer.

Previously, this still worked most of the time but could
result in an out-of-bounds-write.

Also, rename last_piece to buffer_end to avoid this pitfall.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
commit message follows commit guidelines

Affected core subsystem(s)

src/node_url.cc

Not sure how to write tests but it makes valgrind happy about @TimothyGu’s example from #17448

- Remove pointless pointers - Make `WriteHost` take a const argument so that it’s functionality is clear from the signature - Make `FindLongestZeroSequence` templated to accommodate the constness in `WriteHost` and because using `uint16_t` is an articifial, unnecessary restriction - Remove string copying when no copies are needed - Make `PercentDecode` just return its return value - Make `ParseHost` (string-only version) take its constant argument as a constant reference

This helps because `static` doesn’t work for C++ classes, but refactoring `url_host` into a proper C++ class seems the most reasonable soluation for the memory leak fixed by the next commit.

addaleax · 2017-12-05T13:58:59Z

CI: https://ci.nodejs.org/job/node-test-commit/14573/

addaleax · 2017-12-05T14:35:00Z

Some of the test failures are definitely related & a bit curious … going to look into those later

- Gives `URLHost` a proper destructor that clears memory depending on the type of the host (This fixes a memory leak) - Hide the host type enums and class layout as implementation details - Make the `Parse` methods members of `URLHost` - Turn `WriteHost` into a `ToString()` method on the `URLHost` class - Verify that at the beginning of a parse attempt, the type is set to “failed” - Remove a lot of `goto`s from the source code 🐢🚀 Fixes: nodejs#17448

`last_piece` pointed to the end of the 8×16 bit array, so `piece_pointer == last_piece` already means that the pointer is not writable any longer. Previously, this still worked most of the time but could result in an out-of-bounds-write. Also, rename `last_piece` to `buffer_end` to avoid this pitfall.

addaleax · 2017-12-05T19:14:36Z

New CI: https://ci.nodejs.org/job/node-test-commit/14583/

TimothyGu

Splendid work! Good to see this whole thing properly C++-ized, and the memory leak fixed. LGTM.

TimothyGu · 2017-12-05T19:24:05Z

src/node_url.cc

    pointer += 2;
    ch = pointer < end ? pointer[0] : kEOL;
    piece_pointer++;
    compress_pointer = piece_pointer;
  }
  while (ch != kEOL) {
-    if (piece_pointer > last_piece)
-      goto end;
+    if (piece_pointer >= buffer_end)


Wow, nice catch!

@TimothyGu well, it was breaking CI for me, soooo... ;)

Wouldn't have seen it without the tests breaking.

- Remove pointless pointers - Make `WriteHost` take a const argument so that it’s functionality is clear from the signature - Make `FindLongestZeroSequence` templated to accommodate the constness in `WriteHost` and because using `uint16_t` is an articifial, unnecessary restriction - Remove string copying when no copies are needed - Make `PercentDecode` just return its return value - Make `ParseHost` (string-only version) take its constant argument as a constant reference PR-URL: nodejs#17470 Reviewed-By: Timothy Gu <timothygu99@gmail.com>

This helps because `static` doesn’t work for C++ classes, but refactoring `url_host` into a proper C++ class seems the most reasonable soluation for the memory leak fixed by the next commit. PR-URL: nodejs#17470 Reviewed-By: Timothy Gu <timothygu99@gmail.com>

- Gives `URLHost` a proper destructor that clears memory depending on the type of the host (This fixes a memory leak) - Hide the host type enums and class layout as implementation details - Make the `Parse` methods members of `URLHost` - Turn `WriteHost` into a `ToString()` method on the `URLHost` class - Verify that at the beginning of a parse attempt, the type is set to “failed” - Remove a lot of `goto`s from the source code 🐢🚀 PR-URL: nodejs#17470 Fixes: nodejs#17448 Reviewed-By: Timothy Gu <timothygu99@gmail.com>

`last_piece` pointed to the end of the 8×16 bit array, so `piece_pointer == last_piece` already means that the pointer is not writable any longer. Previously, this still worked most of the time but could result in an out-of-bounds-write. Also, rename `last_piece` to `buffer_end` to avoid this pitfall. PR-URL: nodejs#17470 Reviewed-By: Timothy Gu <timothygu99@gmail.com>

BridgeAR · 2017-12-12T15:53:12Z

Landed in e8a5f7b, 9236dfe, 89b3746, 2050bab

MylesBorins · 2017-12-20T18:00:13Z

@TimothyGu does this need to be included in the backport?

gibfahn · 2017-12-20T20:14:22Z

@trygve-lie , in reply to #17774 (comment)

Normally we don't backport to LTS until things have been in a current release (9.x) for at least two weeks. This hasn't gone into a 9.x release yet, so we'd be talking 6 weeks before this could land in v8.x.

If this fixes a serious bug we should consider including it in the v8.9.4 release candidate build going out today, so that it goes out with v8.9.4 in two weeks.

What would be helpful is an idea of:

How much pain this bug causes
How many people are going to hit this
How minor the fix is (how much risk is there in rushing it)

Thoughts @nodejs/lts ?

- Remove pointless pointers - Make `WriteHost` take a const argument so that it’s functionality is clear from the signature - Make `FindLongestZeroSequence` templated to accommodate the constness in `WriteHost` and because using `uint16_t` is an articifial, unnecessary restriction - Remove string copying when no copies are needed - Make `PercentDecode` just return its return value - Make `ParseHost` (string-only version) take its constant argument as a constant reference Backport-PR-URL: #18324 PR-URL: #17470 Reviewed-By: Timothy Gu <timothygu99@gmail.com>

This helps because `static` doesn’t work for C++ classes, but refactoring `url_host` into a proper C++ class seems the most reasonable soluation for the memory leak fixed by the next commit. Backport-PR-URL: #18324 PR-URL: #17470 Reviewed-By: Timothy Gu <timothygu99@gmail.com>

- Gives `URLHost` a proper destructor that clears memory depending on the type of the host (This fixes a memory leak) - Hide the host type enums and class layout as implementation details - Make the `Parse` methods members of `URLHost` - Turn `WriteHost` into a `ToString()` method on the `URLHost` class - Verify that at the beginning of a parse attempt, the type is set to “failed” - Remove a lot of `goto`s from the source code 🐢🚀 Backport-PR-URL: #18324 PR-URL: #17470 Fixes: #17448 Reviewed-By: Timothy Gu <timothygu99@gmail.com>

`last_piece` pointed to the end of the 8×16 bit array, so `piece_pointer == last_piece` already means that the pointer is not writable any longer. Previously, this still worked most of the time but could result in an out-of-bounds-write. Also, rename `last_piece` to `buffer_end` to avoid this pitfall. Backport-PR-URL: #18324 PR-URL: #17470 Reviewed-By: Timothy Gu <timothygu99@gmail.com>