Autogluon timeseries, addressed comments by sebhrusen #7

limpbot · 2022-10-05T16:25:31Z

Addressed all comments by sebhrusen. Except for the renaming of prediction length.

Innixma autogluon timeseries

Innixma

Nice updates! I'll merge but please consider addressing my comments in a small follow-up PR. I'm merging now to avoid asynchronous communication slowing our velocity.

Innixma · 2022-10-06T18:34:58Z

amlb/datasets/file.py

@@ -129,6 +138,38 @@ def __repr__(self):
        return repr_def(self)


+    def extend_dataset_with_timeseries_config(self, dataset, dataset_config):
+        if dataset_config['id_column'] is None:
+            log.warning("Warning: For timeseries task setting undefined itemid column to `item_id`.")


refer to it in the warning with the correct key 'id_column'

Innixma · 2022-10-06T18:36:42Z

amlb/datasets/file.py

@@ -129,6 +138,38 @@ def __repr__(self):
        return repr_def(self)


+    def extend_dataset_with_timeseries_config(self, dataset, dataset_config):


we are manipulating the outer context of dataset_config, maybe this is ok but want to mention that it is happening.

Same with manipulating outer context of dataset. Consider adding documentation stating that this is intended or otherwise do a deep copy on the objects.

Innixma · 2022-10-06T18:40:00Z

frameworks/AutoGluon/__init__.py

+    if dataset.type is not DatasetType.timeseries:

+        data = dict(
+            train=dict(path=dataset.train.data_path('parquet')),
+            test=dict(path=dataset.test.data_path('parquet')),
+            target=dict(
+                name=dataset.target.name,
+                classes=dataset.target.values
+            ),
+            problem_type=dataset.type.name  # AutoGluon problem_type is using same names as amlb.data.DatasetType
+        )
+        exec_file = "exec.py"
+
+    else:
+        dataset = deepcopy(dataset)
+        if not hasattr(dataset, 'timestamp_column'):
+            dataset.timestamp_column = None
+        if not hasattr(dataset, 'id_column'):
+            dataset.id_column = None
+        if not hasattr(dataset, 'forecast_range_in_steps'):
+            raise AttributeError("Unspecified `forecast_range_in_steps`.")
+
+        data = dict(
+            # train=dict(path=dataset.train.data_path('parquet')),
+            # test=dict(path=dataset.test.data_path('parquet')),
+            train=dict(path=dataset.train.path),
+            test=dict(path=dataset.test.path),
+            target=dict(
+                name=dataset.target.name,
+                classes=dataset.target.values
+            ),
+            problem_type=dataset.type.name,  # AutoGluon problem_type is using same names as amlb.data.DatasetType
+            timestamp_column=dataset.timestamp_column,
+            id_column=dataset.id_column,
+            forecast_range_in_steps=dataset.forecast_range_in_steps
+        )
+        exec_file = "exec_ts.py"


if/else could be better broken up into dedicated functions for each modality to avoid an overly long function with a bunch of if/elif/elif/elif/else in future

* Add AutoGluon TimeSeries Prototype * AutoMLBenchmark TimeSeries Prototype. (#6) * fixed loading test & train, changed pred.-l. 5->30 * ignore launch.json of vscode * ensuring timestamp parsing * pass config, save pred, add results * remove unused code * add readability, remove slice from timer * ensure autogluonts has required info * add comments for readability * setting defaults for timeseries task * remove outer context manipulation * corrected spelling error for quantiles * adding mape, correct available metrics * beautify config options * fixed config for public access * Update readme * Autogluon timeseries, addressed comments by sebhrusen (#7) * fixed loading test & train, changed pred.-l. 5->30 * ignore launch.json of vscode * ensuring timestamp parsing * pass config, save pred, add results * remove unused code * add readability, remove slice from timer * ensure autogluonts has required info * add comments for readability * setting defaults for timeseries task * remove outer context manipulation * corrected spelling error for quantiles * adding mape, correct available metrics * beautify config options * fixed config for public access * no outer context manipulation, add dataset subdir * add more datasets * include error raising for too large pred. length. * mergin AutoGluonTS framework folder into AutoGluon * renaming ts.yaml to timeseries.yaml, plus ext. * removing presets, correct latest config for AGTS * move dataset timeseries ext to datasets/file.py * dont bypass test mode * move quantiles and y_past_period_error to opt_cols * remove whitespaces * deleting merge artifacts * delete merge artifacts * renaming prediction_length to forecast_range_in_steps * use public dataset, reduced range to maximum * fix format string works * fix key error bug, remove magic time limit * Addressed minor comments, and fixed version call for tabular and timeseries modularities (#8) * fixed loading test & train, changed pred.-l. 5->30 * ignore launch.json of vscode * ensuring timestamp parsing * pass config, save pred, add results * remove unused code * add readability, remove slice from timer * ensure autogluonts has required info * add comments for readability * setting defaults for timeseries task * remove outer context manipulation * corrected spelling error for quantiles * adding mape, correct available metrics * beautify config options * fixed config for public access * no outer context manipulation, add dataset subdir * add more datasets * include error raising for too large pred. length. * mergin AutoGluonTS framework folder into AutoGluon * renaming ts.yaml to timeseries.yaml, plus ext. * removing presets, correct latest config for AGTS * move dataset timeseries ext to datasets/file.py * dont bypass test mode * move quantiles and y_past_period_error to opt_cols * remove whitespaces * deleting merge artifacts * delete merge artifacts * renaming prediction_length to forecast_range_in_steps * use public dataset, reduced range to maximum * fix format string works * fix key error bug, remove magic time limit * swapped timeseries and tabular to set version * make warning message more explicit * remove outer context manipulation * split timeseries / tabular into functions Co-authored-by: Leo <LeonhardSommer96@gmail.com>

limpbot and others added 30 commits September 14, 2022 13:49

fixed loading test & train, changed pred.-l. 5->30

fdac87d

ignore launch.json of vscode

acae465

ensuring timestamp parsing

b5723cf

pass config, save pred, add results

55c63e9

remove unused code

0f38986

add readability, remove slice from timer

f932669

ensure autogluonts has required info

16a165b

add comments for readability

758b92d

setting defaults for timeseries task

04872e7

remove outer context manipulation

888a1cb

corrected spelling error for quantiles

e15de3e

adding mape, correct available metrics

866492f

beautify config options

9252835

fixed config for public access

18cc6af

no outer context manipulation, add dataset subdir

3e8945a

add more datasets

4ca2118

include error raising for too large pred. length.

f7f21fc

mergin AutoGluonTS framework folder into AutoGluon

fb429c6

renaming ts.yaml to timeseries.yaml, plus ext.

23d057a

removing presets, correct latest config for AGTS

1396d20

move dataset timeseries ext to datasets/file.py

8332960

dont bypass test mode

d41f632

move quantiles and y_past_period_error to opt_cols

3935e9e

remove whitespaces

1f7c574

merge innxima into ours

537d9c7

deleting merge artifacts

79e54c9

delete merge artifacts

6a25170

Merge pull request #2 from limpbot/Innixma-autogluon_timeseries

5862ace

Innixma autogluon timeseries

renaming prediction_length to forecast_range_in_steps

928c2cf

use public dataset, reduced range to maximum

47d311c

limpbot added 2 commits October 6, 2022 11:00

fix format string works

b244e9c

fix key error bug, remove magic time limit

3074f42

Innixma approved these changes Oct 6, 2022

View reviewed changes

Innixma merged commit 53b816a into Innixma:autogluon_timeseries Oct 6, 2022

Innixma mentioned this pull request Oct 6, 2022

[PoC] AutoGluon TimeSeries Prototype openml/automlbenchmark#494

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Autogluon timeseries, addressed comments by sebhrusen #7

Autogluon timeseries, addressed comments by sebhrusen #7

limpbot commented Oct 5, 2022

Innixma left a comment

Innixma Oct 6, 2022

Innixma Oct 6, 2022

Innixma Oct 6, 2022

Innixma Oct 6, 2022

		@@ -129,6 +138,38 @@ def __repr__(self):
		return repr_def(self)


		def extend_dataset_with_timeseries_config(self, dataset, dataset_config):

Autogluon timeseries, addressed comments by sebhrusen #7

Autogluon timeseries, addressed comments by sebhrusen #7

Conversation

limpbot commented Oct 5, 2022

Innixma left a comment

Choose a reason for hiding this comment

Innixma Oct 6, 2022

Choose a reason for hiding this comment

Innixma Oct 6, 2022

Choose a reason for hiding this comment

Innixma Oct 6, 2022

Choose a reason for hiding this comment

Innixma Oct 6, 2022

Choose a reason for hiding this comment