Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure monitor endpoint URL has changed format in v1.2.0 #677

Merged
merged 4 commits into from
Jun 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion conda/conda-reqs.txt
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ geoip2>=2.9.0
html5lib
httpx==0.24.0
ipython>=7.23.1
ipywidgets>=7.4.2, <8.0.0
ipywidgets>=7.4.2, <9.0.0
keyring>=13.2.1
lxml>=4.6.5
matplotlib>=3.0.0
Expand Down
8 changes: 5 additions & 3 deletions docs/source/data_acquisition/DataProv-Kusto-New.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,16 @@ Azure Data Explorer/Kusto Provider - New Implementation

This is a new implementation of the Azure Data Explorer/Kusto
QueryProvider using the
`azure-data-kusto SDK <https://learn.microsoft.com/azure/data-explorer/python-query-data>`__
`azure-kusto-data SDK <https://learn.microsoft.com/azure/data-explorer/python-query-data>`__
(the earlier implementation used
`Kqlmagic <https://github.com/microsoft/jupyter-Kqlmagic>`__).


.. warning:: This provider currently in beta and is available for testing.
It is available alongside the existing Kusto provider for you
to compare old and new.
to compare old and new. To use it you will need the ``azure-kusto-data``
package installed. You can install this with ``pip install azure-kusto-data``
or ``pip install msticpy[azure_query]``.
If you are using the existing implementation, see :doc:`./DataProv-Kusto`

Changes from the previous implementation
Expand Down Expand Up @@ -485,4 +487,4 @@ For examples of using the Kusto provider, see the samples
`Kusto Analysis Notebook <https://github.com/microsoft/msticpy/blob/master/docs/notebooks/Kusto-Analysis.ipynb>`__
and `Kusto Ingest Notebook <https://github.com/microsoft/msticpy/blob/master/docs/notebooks/Kusto-Ingest.ipynb>`__

:py:mod:`Kusto driver API documentation<msticpy.data.drivers.azure_kusto_driver>`
:py:mod:`Kusto driver API documentation<msticpy.data.drivers.azure_kusto_driver>`
4 changes: 3 additions & 1 deletion docs/source/data_acquisition/DataProv-MSSentinel-New.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ the

.. note:: This provider currently in beta and is available for testing.
It is available alongside the existing Sentinel provider for you
to compare old and new.
to compare old and new. To use it you will need the ``azure-monitor-query``
package installed. You can install this with ``pip install azure-monitor-query``
or ``pip install msticpy[azure_query]``.
If you are using the existing implementation, see :doc:`./DataProv-MSSentinel`

Changes from the previous implementation
Expand Down
9 changes: 8 additions & 1 deletion msticpy/data/drivers/azure_monitor_driver.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import pandas as pd
from azure.core.exceptions import HttpResponseError
from azure.core.pipeline.policies import UserAgentPolicy
from pkg_resources import parse_version

from ..._version import VERSION
from ...auth.azure_auth import AzureCloudConfig, az_connect
Expand All @@ -41,12 +42,14 @@

logger = logging.getLogger(__name__)

# pylint: disable=ungrouped-imports
try:
from azure.monitor.query import (
LogsQueryClient,
LogsQueryPartialResult,
LogsQueryResult,
)
from azure.monitor.query import __version__ as az_monitor_version
except ImportError as imp_err:
raise MsticpyMissingDependencyError(
"Cannot use this feature without Azure monitor client installed",
Expand Down Expand Up @@ -149,9 +152,13 @@ def __init__(self, connection_str: Optional[str] = None, **kwargs):
@property
def url_endpoint(self) -> str:
"""Return the current URL endpoint for Azure Monitor."""
return _LOGANALYTICS_URL_BY_CLOUD.get(
base_url = _LOGANALYTICS_URL_BY_CLOUD.get(
AzureCloudConfig().cloud, _LOGANALYTICS_URL_BY_CLOUD["global"]
)
# post v1.1.0 of azure-monitor-query, the API version requires a 'v1' suffix
if parse_version(az_monitor_version) > parse_version("1.1.0"):
return f"{base_url}v1"
return base_url

def connect(self, connection_str: Optional[str] = None, **kwargs):
"""
Expand Down
4 changes: 3 additions & 1 deletion msticpy/data/drivers/kql_driver.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ def __init__(self, connection_str: str = None, **kwargs):
self._ip = get_ipython()
self._debug = kwargs.get("debug", False)
super().__init__(**kwargs)

self.workspace_id: Optional[str] = None
self.set_driver_property(
DriverProps.FORMATTERS,
{"datetime": self._format_datetime, "list": self._format_list},
Expand Down Expand Up @@ -437,6 +437,8 @@ def _get_kql_current_connection():
"""Get the current connection Workspace ID from KQLMagic."""
connections = kql_exec("--conn")
current_connection = [conn for conn in connections if conn.startswith(" * ")]
if not current_connection:
return ""
return current_connection[0].strip(" * ").split("@")[0]

def _set_kql_cloud(self):
Expand Down
2 changes: 1 addition & 1 deletion requirements-all.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ httpx==0.24.0
html5lib
ipython >= 7.1.1; python_version < "3.8"
ipython >= 7.23.1; python_version >= "3.8"
ipywidgets>=7.4.2, <8.0.0
ipywidgets>=7.4.2, <9.0.0
keyring>=13.2.1
KqlmagicCustom[jupyter-basic,auth_code_clipboard]>=0.1.114.post22
KqlmagicCustom[jupyter-extended]>=0.1.114.post22
Expand Down
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ httpx==0.24.0
html5lib
ipython >= 7.1.1; python_version < "3.8"
ipython >= 7.23.1; python_version >= "3.8"
ipywidgets>=7.4.2, <8.0.0
ipywidgets>=7.4.2, <9.0.0
KqlmagicCustom[jupyter-basic,auth_code_clipboard]>=0.1.114.post22
lxml>=4.6.5
msal>=1.12.0
Expand Down