Skip to content

Commit

Permalink
qs
Browse files Browse the repository at this point in the history
  • Loading branch information
lostmygithubaccount committed Aug 16, 2024
1 parent f442d05 commit 0c63bb0
Showing 1 changed file with 1 addition and 44 deletions.
45 changes: 1 addition & 44 deletions website/slides/synthetic-data/index.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -54,52 +54,9 @@ os.chdir(os.path.abspath(os.path.join(os.getcwd(), "..", "..", "..")))
Stored as a Python dictionary.

```{python}
from icarus.investments.dag.assets.seed import data
from icarus.synthetic_data.seed import data
print(data)
```

Note this could come from any Ibis backend. Of course, to use your own data swap out the seed and synthetic data with your own tables.

## synthetic data

Use table-valued UDFs to generate synthetic data.

```{.python}
from icarus.config import (
DATA_DIR,
BRONZE,
SILVER,
GOLD,
RAW_DATA_DIR,
RAW_BUY_SELL_TABLE,
RAW_SOCIAL_MEDIA_TABLE,
)
```

The data must:

- be random and infinitely generatable
- contain interesting statistical patterns for downstream use cases

## synthetic data

Use table-valued UDFs to generate synthetic data.

```{.python}
data_glob = os.path.join(DATA_DIR, RAW_DATA_DIR, RAW_BUY_SELL_TABLE, "*.parquet")
buy_sell_table = ibis.read_parquet(data_glob)
buy_sell_table
```

## synthetic data

Use table-valued UDFs to generate synthetic data.

```{.python}
data_glob = os.path.join(DATA_DIR, RAW_DATA_DIR, RAW_SOCIAL_MEDIA_TABLE, "*.parquet")
social_media_table = ibis.read_parquet(data_glob)
social_media_table
```

# the end

0 comments on commit 0c63bb0

Please sign in to comment.