Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature v34 #201

Merged
merged 7 commits into from
Apr 7, 2023
Merged

Feature v34 #201

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
aebbb06
wip
ronanstokes-db Mar 26, 2023
5f0ffc0
merge from origin
ronanstokes-db Mar 27, 2023
1eda552
wip
ronanstokes-db Apr 7, 2023
7de014c
Merge branch 'master' of https://github.com/databrickslabs/dbldatagen
ronanstokes-db Apr 7, 2023
cb4daca
prep for release
ronanstokes-db Apr 7, 2023
74d2ce3
prep for release
ronanstokes-db Apr 7, 2023
0775100
wip
ronanstokes-db Apr 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 12 additions & 5 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,30 @@
## Change History
All notable changes to the Databricks Labs Data Generator will be documented in this file.

### Version 0.3.4

#### Changed
* Modified option to allow for range when specifying `numFeatures` with `structType='array'` to allow generation
of varying number of columns
* When generating multi-column or array valued columns, compute random seed with different name for each column
* Additional build ordering enhancements to reduce circumstances where explicit base column must be specified

#### Added
* Added top level `random` attribute to data generator specification constructor


### Version 0.3.3post2

#### Changed
* Fixed use of logger in _version.py and in spark_singleton.py
* Fixed template issues
* Document reformatting and updates, related code comment changes
* Modified option to allow for range when specifying `numFeatures` with `structType='array'` to allow generation
of varying number of columns
* When generating multi-column or array valued columns, compute random seed with different name for each column

### Fixed
* Apply pandas optimizations when generating multiple columns using same `withColumn` or `withColumnSpec`

### Added
* Added use of prospector to build process to validate common code issues
* Added top level `random` attribute to data generator specification constructor



### Version 0.3.2
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ details of use and many examples.

Release notes and details of the latest changes for this specific release
can be found in the GitHub repository
[here](https://github.com/databrickslabs/dbldatagen/blob/release/v0.3.3post2/CHANGELOG.md)
[here](https://github.com/databrickslabs/dbldatagen/blob/release/v0.3.4/CHANGELOG.md)

# Installation

Expand Down
2 changes: 1 addition & 1 deletion dbldatagen/_version.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def get_version(version):
return version_info


__version__ = "0.3.3post2" # DO NOT EDIT THIS DIRECTLY! It is managed by bumpversion
__version__ = "0.3.4" # DO NOT EDIT THIS DIRECTLY! It is managed by bumpversion
__version_info__ = get_version(__version__)


Expand Down
2 changes: 1 addition & 1 deletion docs/source/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
author = 'Databricks Inc'

# The full version, including alpha/beta/rc tags
release = "0.3.3post2" # DO NOT EDIT THIS DIRECTLY! It is managed by bumpversion
release = "0.3.4" # DO NOT EDIT THIS DIRECTLY! It is managed by bumpversion


# -- General configuration ---------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion python/.bumpversion.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.3.3post2
current_version = 0.3.4
commit = False
tag = False
parse = (?P<major>\d+)\.(?P<minor>\d+)\.(?P<patch>\d+){0,1}(?P<release>\D*)(?P<build>\d*)
Expand Down
2 changes: 1 addition & 1 deletion setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@

setuptools.setup(
name="dbldatagen",
version="0.3.3post2",
version="0.3.4",
author="Ronan Stokes, Databricks",
description="Databricks Labs - PySpark Synthetic Data Generator",
long_description=long_description,
Expand Down