Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add preview label to HTS and MM notebooks and update data sources #2490

Merged
merged 4 commits into from
Jul 26, 2023
Merged

Add preview label to HTS and MM notebooks and update data sources #2490

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
36f7096
Addedd preview label to HTS and MM notebooks, removed data folder fro…
vlbejan Jul 25, 2023
4375ba7
fixed section reference links
vlbejan Jul 25, 2023
9de7a08
dropped pre-formatting
vlbejan Jul 26, 2023
426173b
Merge branch 'main' into vlbejan/hts-nb-dataset-update
vlbejan Jul 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 50 additions & 5 deletions ...imeseries-in-pipeline/automl-forecasting-demand-hierarchical-timeseries-in-pipeline.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,14 @@
"metadata": {},
"source": [
"# Automated Machine Learning\n",
"**Demand Forecasting Using HTS**\n",
"\n",
"\n",
"## Demand Forecasting Using HTS (preview)\n",
"\n",
"> [!IMPORTANT]\n",
"> Items marked (preview) in this article are currently in public preview.\n",
"> The preview version is provided without a service level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities.\n",
"> For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).\n",
"\n",
"## Contents\n",
"1. [Introduction](#Introduction)\n",
Expand Down Expand Up @@ -207,7 +214,20 @@
"source": [
"## <pre> 4. Data <a id=\"Data\"></pre>\n",
"\n",
"For illustration purposes we use the UCI electricity data ([link](https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014#)). The original dataset contains electricity consumption data for 370 consumers measured at 15 minute intervals. In the data set for this demonstrations, we have aggregated to an hourly frequency and convereted to the kilowatt hours (kWh) for 10 customers. Each customer is assigned to one of the two groups as denoted by the entries in the `group_id` column. The following cells read and print the first few rows of the training data as well as print the number of unique time series in the dataset."
"For illustration purposes we use the UCI electricity data ([link](https://archive.ics.uci.edu/ml/datasets/ElectricityLoadDiagrams20112014#)). The original dataset contains electricity consumption data for 370 consumers measured at 15 minute intervals. In the data set for this demonstrations, we have aggregated to an hourly frequency and convereted to the kilowatt hours (kWh) for 10 customers. Each customer is assigned to one of the two groups as denoted by the entries in the `group_id` column. \n",
"\n",
"The data for this notebook is located in the `automl-sample-notebook-data` container in the datastore and is publicly available. In the next few cells, we will download the train, test and inference datasets from the public datastore and store them locally."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_data_path = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/uci-demand-pipeline-data-hts/train/uci_electro_small_hts_train.parquet\"\n",
"test_data_path = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/uci-demand-pipeline-data-hts/test/uci_electro_small_hts_test.parquet\"\n",
"inference_data_path = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/uci-demand-pipeline-data-hts/inference/uci_electro_small_hts_inference.parquet\""
]
},
{
Expand All @@ -221,16 +241,41 @@
"hierarchy_column_names = [\"group_id\", \"customer_id\"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def create_folder_and_save_as_parquet(file_uri, output_folder):\n",
" os.makedirs(output_folder, exist_ok=True)\n",
" data_frame = pd.read_parquet(file_uri)\n",
" file_name = os.path.split(file_uri)[-1]\n",
" data_path = os.path.join(output_folder, file_name)\n",
" data_frame.to_parquet(data_path, index=False)\n",
" return None\n",
"\n",
"\n",
"create_folder_and_save_as_parquet(train_data_path, \"./data/train\")\n",
"create_folder_and_save_as_parquet(test_data_path, \"./data/test\")\n",
"create_folder_and_save_as_parquet(inference_data_path, \"./data/inference\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The following cells read and print the first few rows of the training data as well as the number of unique time series in the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dataset_type = \"train\"\n",
"df = pd.read_parquet(\n",
" f\"./data/{dataset_type}/uci_electro_small_mm_{dataset_type}.parquet\"\n",
")\n",
"df = pd.read_parquet(f\"./data/{dataset_type}\")\n",
"df.head(3)"
]
},
Expand Down
Binary file removed ...hierarchical-timeseries-in-pipeline/data/inference/uci_electro_small_mm_inference.parquet
View file
Open in desktop
Binary file not shown.
Binary file removed ...ng-demand-hierarchical-timeseries-in-pipeline/data/test/uci_electro_small_mm_test.parquet
View file
Open in desktop
Binary file not shown.
Binary file removed ...-demand-hierarchical-timeseries-in-pipeline/data/train/uci_electro_small_mm_train.parquet
View file
Open in desktop
Binary file not shown.
22 changes: 14 additions & 8 deletions ...ng-demand-many-models-in-pipeline/automl-forecasting-demand-many-models-in-pipeline.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,13 @@
"metadata": {},
"source": [
"# Automated Machine Learning\n",
"**Demand Forecasting Using Many Models**\n",
"\n",
"## Demand Forecasting Using Many Models (preview)\n",
"\n",
"> [!IMPORTANT]\n",
"> Items marked (preview) in this article are currently in public preview.\n",
"> The preview version is provided without a service level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities.\n",
"> For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).\n",
"\n",
"## Contents\n",
"1. [Introduction](#Introduction)\n",
Expand Down Expand Up @@ -218,9 +224,9 @@
"metadata": {},
"outputs": [],
"source": [
"train_data_path = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/uci_electro_small_public_mm_train/uci_electro_small_mm_train.csv\"\n",
"test_data_path = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/uci_electro_small_public_mm_test/uci_electro_small_mm_test.csv\"\n",
"inference_data_path = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/uci_electro_small_public_mm_infer/uci_electro_small_mm_inference.csv\""
"train_data_path = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/uci-demand-pipeline-data-mm/train/uci_electro_small_mm_train.parquet\"\n",
"test_data_path = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/uci-demand-pipeline-data-mm/test/uci_electro_small_mm_test.parquet\"\n",
"inference_data_path = \"https://automlsamplenotebookdata.blob.core.windows.net/automl-sample-notebook-data/uci-demand-pipeline-data-mm/inference/uci_electro_small_mm_inference.parquet\""
]
},
{
Expand All @@ -242,11 +248,11 @@
"source": [
"def create_folder_and_save_as_parquet(file_uri, output_folder):\n",
" os.makedirs(output_folder, exist_ok=True)\n",
" data_frame = pd.read_csv(file_uri, parse_dates=[time_column_name])\n",
" file_name = os.path.splitext(os.path.split(file_uri)[-1])[0] + \".parquet\"\n",
" data_frame = pd.read_parquet(file_uri)\n",
" file_name = os.path.split(file_uri)[-1]\n",
" data_path = os.path.join(output_folder, file_name)\n",
" data_frame.to_parquet(data_path, index=False)\n",
" return data_frame\n",
" return None\n",
"\n",
"\n",
"create_folder_and_save_as_parquet(train_data_path, \"./data/train\")\n",
Expand All @@ -258,7 +264,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The following cells read and print the first few rows of the training data as well as print the number of unique time series in the data."
"The following cells read and print the first few rows of the training data as well as the number of unique time series in the dataset."
]
},
{
Expand Down
Loading