Add support for app ids in spark

[ambud]

gprofiler doesn't pull the spark app ids which are necessary as a fallback mechanism if app name is unavailable.

Description

Adding a simple cli parser to pull app ids in case app names are not available

Related Issue

None

Motivation and Context

Without app ids we get no way to identify what app this is.

How Has This Been Tested?

Locally tested the snippet, added try catch in case the snippet fails.

[Jongy]

Hi @ambud ! I let the CI run, there are small lint errors that you can fix.

To e2e test the profiler, you can build it locally with the instructions here https://github.com/Granulate/gprofiler/blob/master/CONTRIBUTING.md#docker-build for a Docker image build, and ./scripts/build_x86_64_executable.sh for an executable build (depending on how you wish to deploy it). Note that the build can take 20-30 mins on an average server and requires at least 16GB of RAM (it builds all relevant components of the profiler). Those last few instructions are not included in the CONTRIBUTING doc, I'll make sure to add them :)

[Jongy]

Note that you might be able to achieve the same goal by using the --app-id-args-filter option. It will match an RE against each argument individually and will include all of them in the gProfiler appid for the application. (Using it you cannot match against the --app-id flag, however - only against the value)

[ambud]

Hi @ambud ! I let the CI run, there are small lint errors that you can fix.

To e2e test the profiler, you can build it locally with the instructions here https://github.com/Granulate/gprofiler/blob/master/CONTRIBUTING.md#docker-build for a Docker image build, and ./scripts/build_x86_64_executable.sh for an executable build (depending on how you wish to deploy it). Note that the build can take 20-30 mins on an average server and requires at least 16GB of RAM (it builds all relevant components of the profiler). Those last few instructions are not included in the CONTRIBUTING doc, I'll make sure to add them :)

I had already tried running the script however it failed due to Granulate utils dependency, is there something I need to install?

[ambud]

Note that you might be able to achieve the same goal by using the --app-id-args-filter option. It will match an RE against each argument individually and will include all of them in the gProfiler appid for the application. (Using it you cannot match against the --app-id flag, however - only against the value)

Thanks for this pointer, I don't think it will be as clean though since there are several places the application_XXXX is specified.

[ambud]

Updated the code and fixed the lint issue

[Jongy]

I had already tried running the script however it failed due to Granulate utils dependency, is there something I need to install?

Did you clone with submodules? (see as suggested here. granulate-utils is a submodule necessary for the build, yeah. Don't need to build it or anything, just clone

[Jongy]

Thanks for this pointer, I don't think it will be as clean though since there are several places the application_XXXX is specified.

I see, so a pattern like (application_[^ ]+) will catch many sites.

If the matching was done on the full commandline, the pattern could be --app-id[ =]([^ ]+) and it'd do the trick, am I right?

[Jongy]

Do you expect any particular exception type here? ValueError for idx+1? Please change to ValueError then (it's nicer compared to generic except Exception which might catch unexpected things)

[ambud]

I had already tried running the script however it failed due to Granulate utils dependency, is there something I need to install?

Did you clone with submodules? (see as suggested here. granulate-utils is a submodule necessary for the build, yeah. Don't need to build it or anything, just clone

Oh I didn't realize that's where the build instructions were. Let me try.

[Jongy]

We're currently investigating another approach, so putting this on hold.