Reuse Puppeteer browser and page for text parser

[humphd]

In #383 we are moving to use Puppeteer. This uses a real Chromium browser to render content in a full DOM. We can reuse the browser and page objects, so we don't have more than one instance. We just have to use a Promise chain pattern, see puppeteer/puppeteer#486 (comment):

const browser = await puppeteer.launch();
const page = await browser.newPage();
let chain = Promise.resolve();
// Request handler 
app.get('/pdf', (req, res) => {
  // Handle request generation in the end of the chain
  chain = chain.then(async () => { 
    await page.goto(req.query.url);
    const pdf = await page.pdf();
    res.send(pdf);
  });
});

Depends on #383.

[sukhbeersingh]

I'll give this a go, once #383 is landed.

[humphd]

Turns out we're going to need this sooner rather than later. I'm using it in #408, and without it, it's going to use too many resources to run the server. Let's do this in class together tomorrow morning.

[humphd]

This might be overkill, but we could maybe use the puppeteer-cluster module, which does a bunch of this for free.

This is sort of what we're doing: https://github.com/thomasdondorf/puppeteer-cluster/blob/HEAD/examples/function-queuing-complex.js.

cc @giatuongtran9

[ragnarokatz]

@sukhbeersingh This is the updated code:

Current state:

module.exports = async function(htmlFragment) {
  let browser;
  try {
    browser = await puppeteer.launch();
    const page = await browser.newPage();
  }
...

Target state:

1. instantiate a browser instance to be used
2. use the same browser instance for all function calls
3. close the browser instance after all text parsing services have completed

[giatuongtran9]

@humphd sorry for my late reply. Do you still want to use puppeteer-cluster? I can have a look at it

[sukhbeersingh]

Hey @giatuongtran9 if you want to try this please go ahead. I would love to work on this but I have some assignments backlog for other courses so I won’t be able to touch this until next week.

[ragnarokatz]

@giatuongtran9 I can work with you on this issue.

[ragnarokatz]

@humphd can I take on this issue and assign it to myself? @giatuongtran9 seems to be busy and hasn't responded yet.

Don't worry about merging this PR in a hurry if you have other PRs to merge for Friday night/over the weekend.

[humphd]

@ragnarokatz go for it. I think others are slammed with other work. I'd love a PR for this.

[humphd]

This was fixed by #540.