Add support for workload identity federation and consolidate GCP/BigQuery APIs into seperate classes #120
Conversation
asatwal
force-pushed
the
workload-identity-federation-auth
branch
from
6b1eba7 to
6ddbc49
Compare
| end | ||
|
|
||
| context 'when authorization endoint returns OK response' do | ||
| it 'calls the expected big query apis' do |
| end | ||
|
|
||
| context 'when authorization endoint returns OK response' do | ||
| let(:events_client) { double(:events_client) } |
|
|
||
| azure_federated_auth: | ||
| description: | | ||
| Whether to use azure workload identity federation for authentication |
There was a problem hiding this comment.
@asatwal does this also switch between APIs? If so could we add that to the description?
There was a problem hiding this comment.
Yes this does switch APIs as the old APIs don't work with the new WIF Auth. Good point I'll add to this description.
|
|
||
| module DfE | ||
| module Analytics | ||
| # For use with for workload identity federeation |
There was a problem hiding this comment.
@asatwal would it be possible for a team to use this API without WIF? How much effort would it be to add that flexibility? In future I can see the old API being decommissioned and some teams needing to move to the new one but not having been able to get things set up in Azure such that they can use WIF.
There was a problem hiding this comment.
@asatwal I was wondering generally if it's a strong recommend that teams migrate to WIF?
There was a problem hiding this comment.
I think it is possible to use the new streaming APIs with the old authentication, but this would be effort though. I think we could consider this if there is a real need for this. As Erica points out above, we could insist on WIF initially and it's a strong recommendation.
asatwal
force-pushed
the
workload-identity-federation-auth
branch
from
b7b8206 to
7043e06
Compare
asatwal
force-pushed
the
workload-identity-federation-auth
branch
2 times, most recently
from
ed2384d to
3e0087c
Compare
asatwal
force-pushed
the
workload-identity-federation-auth
branch
3 times, most recently
from
36ae1eb to
e75f6d3
Compare
…uery APIs into seperate classes
asatwal
force-pushed
the
workload-identity-federation-auth
branch
from
e75f6d3 to
977c1ae
Compare
Use workload identity federation as an alternative authentication for GCP.
For more details please see GCP documentation here.
Configuration of workload identity federation with Azure and links to the azure documentation can be found here
The steps involved for authentication are shown in the diagram below:
From the diagram, Step 4 is what we have to today with the API key - We go straight to step 4 with the API Key.
So with WIF weeliminate the API Key, but go through Steps 1-3 to get an service account impersonation access token - This replaces the API Key.
This file:
lib/dfe/analytics/azure_federated_auth.rb
implements steps 1-3>
Also the streaming API used to send data to BQ need to be changed.
The old API doesn't work with the new Auth and the new Auth doesn't work with the old API, hence the new event client and the legacy event client.
All config need is from either Azure end variables setup by DevOps:
AZURE_CLIENT_ID
AZURE_FEDERATED_TOKEN_FILE
The rest of the config is from the ENV variable:
GOOGLE_CLOUD_CREDENTIALS
This comes from Google and is a downloadable cloud config.