canvas: Move to shared memory for images and canvas backing stores. #6705

pcwalton · 2015-07-23T06:01:11Z

The idea here is to land this before making images and canvas IPC-safe,
because this will shake out bugs relating to the shared memory. There
are currently test timeouts that are preventing multiprocess images and
canvas from landing, and I believe those are due to the inefficiency of
sending large amounts of data in the unoptimized builds we test with. By
moving to shared memory, this should drastically reduce the number of
copies and serde serialization.

Under the hood, this uses Mach OOL messages on Mac and temporary
memory-mapped files on Linux.

r? @jdm

hoppipolla-critic-bot · 2015-07-23T06:01:17Z

Critic review: https://critic.hoppipolla.co.uk/r/5630

This is an external review system which you may optionally use for the code review of your pull request.

In order to help critic track your changes, please do not make in-place history rewrites (e.g. via git rebase -i or git commit --amend) when updating this pull request.

pcwalton · 2015-07-23T06:03:59Z

BTW, the alternatives here were:

Disable the tests in question.
Hack around the lack of optimization by writing in unsafe code or C.
Get Cargo to be able to optimize a subset of crates, and block on a Cargo upgrade.
Switch tests to run in optimized builds of Servo.
Wait for serde to get bincode support and hope that's enough.
Wait for the specialization RFC to be accepted and implemented, then block on a Rust upgrade.

None of these options seemed particularly compelling to me, so I elected to take this approach. There is a risk that some of this will be rolled back when images are no longer sent to layout at all, but that requires some more heavy lifting (so that the paint task can find the image/canvas contents without layout directly supplying them to it) and we can easily roll back the change for images if we want to.

jdm · 2015-07-23T06:05:23Z

@bors-servo: r+
Makes sense.

bors-servo · 2015-07-23T06:05:24Z

📌 Commit 2a4e023 has been approved by jdm

bors-servo · 2015-07-23T06:05:25Z

⌛ Testing commit 2a4e023 with merge 798db8e...

@jdm

canvas: Move to shared memory for images and canvas backing stores. The idea here is to land this before making images and canvas IPC-safe, because this will shake out bugs relating to the shared memory. There are currently test timeouts that are preventing multiprocess images and canvas from landing, and I believe those are due to the inefficiency of sending large amounts of data in the unoptimized builds we test with. By moving to shared memory, this should drastically reduce the number of copies and `serde` serialization. Under the hood, this uses Mach OOL messages on Mac and temporary memory-mapped files on Linux. r? @jdm  [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/6705)

bors-servo · 2015-07-23T06:12:30Z

💔 Test failed - android

pcwalton · 2015-07-23T06:33:00Z

@bors-servo: r=jdm

bors-servo · 2015-07-23T06:33:01Z

📌 Commit d01bca6 has been approved by jdm

bors-servo · 2015-07-23T06:33:03Z

⌛ Testing commit d01bca6 with merge 0efe98b...

@jdm

canvas: Move to shared memory for images and canvas backing stores. The idea here is to land this before making images and canvas IPC-safe, because this will shake out bugs relating to the shared memory. There are currently test timeouts that are preventing multiprocess images and canvas from landing, and I believe those are due to the inefficiency of sending large amounts of data in the unoptimized builds we test with. By moving to shared memory, this should drastically reduce the number of copies and `serde` serialization. Under the hood, this uses Mach OOL messages on Mac and temporary memory-mapped files on Linux. r? @jdm  [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/6705)

bors-servo · 2015-07-23T06:40:29Z

💔 Test failed - android

jdm · 2015-07-23T06:46:20Z

/home/servo/.cargo/git/checkouts/ipc-channel-d95a23d1f1577bfc/master/platform/linux/mod.rs:801:18: 801:28 error: mismatched types:
 expected `u16`,
    found `u32`
(expected u16,
    found u32) [E0308]
/home/servo/.cargo/git/checkouts/ipc-channel-d95a23d1f1577bfc/master/platform/linux/mod.rs:801         S_ISSOCK(st.st_mode)
                                                                                                                ^~~~~~~~~~

pcwalton · 2015-07-23T06:48:49Z

@bors-servo: r=jdm

bors-servo · 2015-07-23T06:48:50Z

📌 Commit 0d1a5e1 has been approved by jdm

bors-servo · 2015-07-23T06:48:51Z

⌛ Testing commit 0d1a5e1 with merge 1257e23...

@jdm

canvas: Move to shared memory for images and canvas backing stores. The idea here is to land this before making images and canvas IPC-safe, because this will shake out bugs relating to the shared memory. There are currently test timeouts that are preventing multiprocess images and canvas from landing, and I believe those are due to the inefficiency of sending large amounts of data in the unoptimized builds we test with. By moving to shared memory, this should drastically reduce the number of copies and `serde` serialization. Under the hood, this uses Mach OOL messages on Mac and temporary memory-mapped files on Linux. r? @jdm  [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/6705)

bors-servo · 2015-07-24T00:54:36Z

📌 Commit 0461ae9 has been approved by jdm

pcwalton · 2015-07-24T00:54:49Z

Tested locally now that we're running WPT in release.

bors-servo · 2015-07-24T01:09:42Z

⌛ Testing commit 0461ae9 with merge ace452d...

@jdm

canvas: Move to shared memory for images and canvas backing stores. The idea here is to land this before making images and canvas IPC-safe, because this will shake out bugs relating to the shared memory. There are currently test timeouts that are preventing multiprocess images and canvas from landing, and I believe those are due to the inefficiency of sending large amounts of data in the unoptimized builds we test with. By moving to shared memory, this should drastically reduce the number of copies and `serde` serialization. Under the hood, this uses Mach OOL messages on Mac and temporary memory-mapped files on Linux. r? @jdm  [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/6705)

pcwalton · 2015-07-24T02:06:03Z

Along with the rest of e10s, this is currently blocked either on generating the metadata needed to move our WPT usage to release builds, rust-lang/cargo#1826 (which won't land due to design issues), or removing IPC on Linux.

bors-servo · 2015-07-24T02:37:50Z

💔 Test failed - linux2

jdm · 2015-07-24T12:36:19Z

/dom/ranges/Range-insertNode.html
---------------------------------
TIMEOUT [Parent]
/dom/ranges/Range-mutations.html
--------------------------------
TIMEOUT [Parent]
/dom/ranges/Range-surroundContents.html
---------------------------------------
TIMEOUT [Parent]

pcwalton · 2015-07-24T16:30:13Z

@bors-servo: r=jdm

bors-servo · 2015-07-24T16:30:14Z

📌 Commit 0461ae9 has been approved by jdm

pcwalton · 2015-07-24T18:00:57Z

@bors-servo: retry

bors-servo · 2015-07-24T18:02:20Z

⌛ Testing commit 0461ae9 with merge 43c5117...

@jdm

canvas: Move to shared memory for images and canvas backing stores. The idea here is to land this before making images and canvas IPC-safe, because this will shake out bugs relating to the shared memory. There are currently test timeouts that are preventing multiprocess images and canvas from landing, and I believe those are due to the inefficiency of sending large amounts of data in the unoptimized builds we test with. By moving to shared memory, this should drastically reduce the number of copies and `serde` serialization. Under the hood, this uses Mach OOL messages on Mac and temporary memory-mapped files on Linux. r? @jdm  [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/6705)

bors-servo · 2015-07-24T18:42:13Z

💔 Test failed - linux2

The idea here is to land this before making images and canvas IPC-safe, because this will shake out bugs relating to the shared memory. There are currently test timeouts that are preventing multiprocess images and canvas from landing, and I believe those are due to the inefficiency of sending large amounts of data in the unoptimized builds we test with. By moving to shared memory, this should drastically reduce the number of copies and `serde` serialization. Under the hood, this uses Mach OOL messages on Mac and temporary memory-mapped files on Linux.

pcwalton · 2015-07-24T23:14:57Z

@bors-servo: r=jdm

bors-servo · 2015-07-24T23:14:59Z

📌 Commit 6269749 has been approved by jdm

bors-servo · 2015-07-24T23:15:00Z

⌛ Testing commit 6269749 with merge ed1b6a3...

@jdm

canvas: Move to shared memory for images and canvas backing stores. The idea here is to land this before making images and canvas IPC-safe, because this will shake out bugs relating to the shared memory. There are currently test timeouts that are preventing multiprocess images and canvas from landing, and I believe those are due to the inefficiency of sending large amounts of data in the unoptimized builds we test with. By moving to shared memory, this should drastically reduce the number of copies and `serde` serialization. Under the hood, this uses Mach OOL messages on Mac and temporary memory-mapped files on Linux. r? @jdm  [<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/6705)

bors-servo · 2015-07-24T23:59:43Z

☀️ Test successful - android, gonk, linux1, linux2, linux3, mac1, mac2, mac3

highfive added the S-awaiting-review There is new code that needs to be reviewed. label Jul 23, 2015

jdm added S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. and removed S-awaiting-review There is new code that needs to be reviewed. labels Jul 23, 2015

pcwalton force-pushed the image-cache-shmem branch from 2a4e023 to d01bca6 Compare July 23, 2015 06:32

highfive added S-awaiting-review There is new code that needs to be reviewed. and removed S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. labels Jul 23, 2015

jdm added S-needs-code-changes Changes have not yet been made that were requested by a reviewer. and removed S-awaiting-review There is new code that needs to be reviewed. labels Jul 23, 2015

pcwalton force-pushed the image-cache-shmem branch from d01bca6 to 0d1a5e1 Compare July 23, 2015 06:48

highfive added S-awaiting-review There is new code that needs to be reviewed. and removed S-needs-code-changes Changes have not yet been made that were requested by a reviewer. labels Jul 23, 2015

This was referenced Jul 23, 2015

script: Make the ImageCacheTask use IPC. #6597

Merged

script: Make most of 2D canvas and WebGL run over IPC. #6616

Merged

jdm added S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. and removed S-awaiting-review There is new code that needs to be reviewed. labels Jul 24, 2015

pcwalton force-pushed the image-cache-shmem branch from 0461ae9 to 6269749 Compare July 24, 2015 23:14

highfive added S-awaiting-review There is new code that needs to be reviewed. and removed S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. labels Jul 24, 2015

bors-servo merged commit 6269749 into servo:master Jul 24, 2015

This was referenced Jul 24, 2015

servo: Upgrade ipc-channel, and make the timeout value in WPT match that of Firefox in debug mode. #6719

Closed

script: Make the resource task communication use IPC channels. #6586

Merged

profile: Make the time and memory profilers run over IPC. #6629

Merged

pcwalton deleted the image-cache-shmem branch July 25, 2015 00:09

mbrubeck mentioned this pull request Oct 8, 2015

Switch to Piston's image library #3368

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

canvas: Move to shared memory for images and canvas backing stores. #6705

canvas: Move to shared memory for images and canvas backing stores. #6705

pcwalton commented Jul 23, 2015

hoppipolla-critic-bot commented Jul 23, 2015

pcwalton commented Jul 23, 2015

jdm commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

pcwalton commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

jdm commented Jul 23, 2015

pcwalton commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

jdm commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

bors-servo commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

bors-servo commented Jul 24, 2015

bors-servo commented Jul 24, 2015

canvas: Move to shared memory for images and canvas backing stores. #6705

canvas: Move to shared memory for images and canvas backing stores. #6705

Conversation

pcwalton commented Jul 23, 2015

hoppipolla-critic-bot commented Jul 23, 2015

pcwalton commented Jul 23, 2015

jdm commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

pcwalton commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

jdm commented Jul 23, 2015

pcwalton commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 23, 2015

bors-servo commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

jdm commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

bors-servo commented Jul 24, 2015

pcwalton commented Jul 24, 2015

bors-servo commented Jul 24, 2015

bors-servo commented Jul 24, 2015

bors-servo commented Jul 24, 2015