Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[batch] Support deployment mode as a Terra on Azure app #13944

Merged
merged 34 commits into from
Mar 1, 2024
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
5abf4c7
[batch] Support deployment mode as a Terra on Azure app
daniel-goldstein Oct 21, 2022
65ddb0d
address comments
daniel-goldstein Nov 29, 2023
0e5658f
add chart template
daniel-goldstein Nov 29, 2023
d8e5a0d
fix inconsistencies between scala and python and assert or assume no …
daniel-goldstein Nov 29, 2023
e58f6b7
more cleanup
daniel-goldstein Nov 29, 2023
7a1d691
comment why no mount_tokens in terra
daniel-goldstein Nov 30, 2023
c1e4b7d
remove special handling of ConnectionRefusedError when talking to VMs
daniel-goldstein Dec 4, 2023
1f48af7
fxi
daniel-goldstein Dec 4, 2023
dd19a86
fix scala terra fs
daniel-goldstein Dec 4, 2023
d7c6427
Merge branch 'main' into azure-terra
daniel-goldstein Jan 8, 2024
8b56014
format
daniel-goldstein Jan 8, 2024
0fb82fa
Merge branch 'main' into azure-terra
daniel-goldstein Jan 23, 2024
d687af6
dont actually need a terra-specific deploy config in scala
daniel-goldstein Jan 23, 2024
479647a
lint
daniel-goldstein Jan 24, 2024
730db13
Merge branch 'main' into azure-terra
daniel-goldstein Jan 24, 2024
3916f11
fix
daniel-goldstein Jan 24, 2024
b9575a8
fix
daniel-goldstein Jan 24, 2024
394a933
Merge branch 'main' into azure-terra
daniel-goldstein Feb 6, 2024
5e199c6
fixes
daniel-goldstein Feb 6, 2024
53b0438
change a bit
daniel-goldstein Feb 9, 2024
1e33603
not necessary anymore
daniel-goldstein Feb 9, 2024
10df984
Merge branch 'main' into azure-terra
daniel-goldstein Feb 14, 2024
66d24cd
its not mandatory
daniel-goldstein Feb 20, 2024
082cfd0
get it compiling again
daniel-goldstein Feb 20, 2024
2807e4d
update
daniel-goldstein Feb 21, 2024
ad3ed31
refresh the azure client with a new terra sas token when it is close …
daniel-goldstein Feb 23, 2024
27a51b2
Merge branch 'main' into azure-terra
daniel-goldstein Feb 23, 2024
4f44742
remove unused import
daniel-goldstein Feb 23, 2024
b899fbe
fix
daniel-goldstein Feb 26, 2024
6e3d5cd
shell quote everything
daniel-goldstein Feb 29, 2024
305be51
rip out the extra disk
daniel-goldstein Feb 29, 2024
441e80b
handle other states
daniel-goldstein Feb 29, 2024
5b448d9
more shq and heredoc fix
daniel-goldstein Feb 29, 2024
0ec0ac3
run scalafix
daniel-goldstein Mar 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions auth/auth/auth.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@
from hailtop.auth import AzureFlow, Flow, GoogleFlow, IdentityProvider
from hailtop.config import get_deploy_config
from hailtop.hail_logging import AccessLogger
from hailtop.tls import internal_server_ssl_context
from hailtop.utils import secret_alnum_string
from web_common import render_template, set_message, setup_aiohttp_jinja2, setup_common_static_routes

Expand Down Expand Up @@ -900,5 +899,5 @@ def run():
host='0.0.0.0',
port=443,
access_log_class=AuthAccessLogger,
ssl_context=internal_server_ssl_context(),
ssl_context=deploy_config.server_ssl_context(),
)
5 changes: 5 additions & 0 deletions batch/batch/cloud/driver.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
import os

from gear import Database
from gear.cloud_config import get_global_config

from ..driver.driver import CloudDriver
from ..inst_coll_config import InstanceCollectionConfigs
from .azure.driver.driver import AzureDriver
from .gcp.driver.driver import GCPDriver
from .terra.azure.driver.driver import TerraAzureDriver


async def get_cloud_driver(
Expand All @@ -17,6 +20,8 @@ async def get_cloud_driver(
cloud = get_global_config()['cloud']

if cloud == 'azure':
if os.environ.get('HAIL_TERRA'):
return await TerraAzureDriver.create(app, db, machine_name_prefix, namespace, inst_coll_configs)
return await AzureDriver.create(app, db, machine_name_prefix, namespace, inst_coll_configs)

assert cloud == 'gcp', cloud
Expand Down
Empty file.
Empty file.
Empty file.
Loading