[GCS]-Parse configurations from Kibana UI

[jignesh-crest]

Closes #466

This PR addresses the following changes:

• Parse the private-key present in the configurations passed from UI.
• Moves the configuration validation to a separate method.

Checklists

Pre-Review Checklist

• Covered the changes with automated tests
• Tested the changes locally by running make test and make ftest NAME=google_cloud_storage
• Added a label for each target release version (example: v7.13.2, v7.14.0, v8.0.0)
• Considered corresponding documentation changes
• Contributed any configuration settings changes to the configuration reference

[praveen-elastic]

buildkite test this

[timgrein]

Some comments around test specific code in connector implementations and a suggestion for narrowing an exception type.

[jignesh-crest]

Hey, I've rebased the branch on the latest main. on a side note, This PR needs to be backported to 8.7 as these changes will be required to make the Google Cloud Connector work with Kibana UI. But I'm not able to add or remove any labels, so can someone do that, would be a great help.

cc: @timgrein @tarekziade

[artem-shelkovnikov]

I'm not sure about approach taken here.

Ideally, when connector is initialised, it already has to have all fields initialised and ready to work. In this PR, however, values are initialised later with special private method - doesn't seem like a good pattern to me.

I want to discuss it with @tarekziade - ideally you validate config in initialiser and raise errors there, and framework will show errors in UI correctly.

[jignesh-crest]

@artem-shelkovnikov I've been validating the configurations at the time of initialization only, I've moved it to ping keeping this comment in mind.

[artem-shelkovnikov]

I understand, however it feels like this validation should happen in the initialiser OR we should actually introduce a method in the framework validate_configuration that each connector has to implement. Both of these changes require framework change that we need to discuss with @tarekziade.

Problem that I see with current approach is that connector becomes more fragile. If it happens so that we don't call ping before calling get_docs, then connector will break - configuration is populated only in ping, not in get_docs

[wangch079]

I agree with @artem-shelkovnikov that _validate_configurations() and _initialize_configurations() shouldn't happens in ping(). Any particular reason to move them out of __init__?

It also requires to be called in sequence, e.g. it will fail if you call _initialize_configurations() without calling _validate_configurations().

[moxarth-elastic]

buildkite test this

[moxarth-elastic]

buildkite test this

[moxarth-elastic]

buildkite test this

[moxarth-elastic]

buildkite test this

[moxarth-elastic]

buildkite test this

[timgrein]

LGTM

[jignesh-crest]

@artem-shelkovnikov, @wangch079, Can we get this merge if it looks good to you?

[moxarth-elastic]

buildkite test this

[praveen-elastic]

buildkite test this

[github-actions]

💔 Failed to create backport PR(s)

Status	Branch	Result
❌	8.8	The branch "8.8" is invalid or doesn't exist

To backport manually run:
backport --pr 493 --autoMerge --autoMergeMethod squash

[artem-shelkovnikov]

💚 All backports created successfully

Status	Branch	Result
✅	8.7

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation