Add cacheable lookup

[Amy0511]

Closes #

When our worker.flow processes execute HTTP tasks specifically on artifactory, Boomerang Essentials OCP cluster is configured to delegate these DNS lookup requests to our own DNS (named) service running on a VM (we do this because we need to override the default/public DNS IP address with a 10.x.x.x private IP address)]

During high load, i.e., large number of concurrently running Flow workflows executing HTTP tasks, we see some timeouts with this error:
parameters: {"response":"getaddrinfo EAI_AGAIN tools.servicesessentials.ibm.com"}

As Node.js does not cache DNS lookups by default, we add cacheable-lookup and may reduce/eliminate the timeouts.

Changelog

New

• {{new thing}}

Changed

• {{change thing}}

Removed

• {{removed thing}}

Testing / Reviewing

{{ Add descriptions, steps or a checklist for how reviewers can verify this PR works or not }}

[tlawrie]

Hi @Amy0511 can you provide some context to the PR? Thanks.

[Amy0511]

Sure. Updated in short description section. Thanks!

[gchickma]

Thx @Amy0511

[Amy0511]

@tlawrie @tlawrie @marcusdroy @florentinvintila Could you please help to review the PR?
As Node.js does not cache DNS lookups by default, we added cacheable-lookup to cache it. Thanks!

[gchickma]

@Amy0511 One additional thought, do you include any cache "time to live" type configuration so we ensure it refreshes the cache periodically? I know for most of our domains they won't change but this change will affect all public domain lookups.

Also, do we make this caching optional, i.e., adding a new cache enable/disable feature (with cache setting values) in the Flow helm chart?

I don't want to make this change bigger than it needs to be, however we need to consider all use cases here, not just our internal IBM one.

[Amy0511]

Thank you @gchickma ! I will think about questions. Fow now, I didn't add "time to live" and the switch for DNS cache.

[timrbula]

hey @gchickma @Amy0511 looks at the docs it seems that it uses the TTL from the response by default. The only configuration I see is setting the a max TTL so I guess you can ignore the TTL of a server above a threshold.

Besides that comment everything looks good to me.