1.7.0 #37

dqops · 2024-08-21T09:32:48Z

1.7.0

Rule miner that proposes data quality rule configuration
Notifications can be filtered by various parameters, allowing to send notifications to the correct teams
Estimated word count checks added to support analyzing data for GenAI use cases
Some pattern validation checks were renamed to use a unified naming convention "invalid" in the check names

…pcode_percent

…one_percent, negated error sampling templates

…id_usa_phone_count and invalid_usa_zipcode_count

…ti.io/_git/dqoado into develop

… category duckdb tested only Related work items: #12685, #12699

…t regenerated to support new checks.

…uda phone percent

…ted test data

…t integration tests, fixed oracle zipcode sensors

… excluded from counts in patterns sensor duckdb and postgresql

…cumentation

…ti.io/_git/dqoado into feature/12732_Error_sampling_tab_should_have_a_download_as_CSV_button

…and zipcodes - integration tests for patterns usa phones and zip codes - excluded nulls from counts in duckdb and postgresql - fixed new sensors Related work items: #12809

…ocumentation

…ti.io/_git/dqoado into develop

…y it with the button name.

Related work items: #12912

word count sensors does not count rows with spaces only Related work items: #12914

aligned content table quality status top

…ltered notifications.

…ti.io/_git/dqoado into develop

…plete columns)

setting values correctly

minor changes IconButton

fixed offset rule mining

…rovide it adding .csv to dictionary name if user doesnt provide it

fixed parameters in creating filtered notification

…sues)

…t change the layout.

…eird.

Patryk Jatczak and others added 30 commits August 8, 2024 12:40

check specks for usa phone numbers and zip codes

e608116

sensor templates (.sql.jinja2 files) for column/patterns/valid_usa_zi…

f0c41bd

…pcode_percent

sensor templates (.sql.jinja2 files) for column/patterns/valid_usa_ph…

27df570

…one_percent, negated error sampling templates

copied valid_usa_phone_percent and valid_usa_zipcode_percent to inval…

a45e84d

…id_usa_phone_count and invalid_usa_zipcode_count

converted queries to use negation and count version of sensor

c2b7c98

Do not show older samples.

02929f5

Merge branch 'develop' of https://dev.azure.com/imagetemplates/documa…

35216b2

…ti.io/_git/dqoado into develop

Updated screens in Category of checks documents in the documentation

0482a21

duckdb zipcode and phone integration tests, bugfix on checkspecs

712d84d

Merged PR 2892: new checks usa phone numbers and zipcodes in patterns…

07eeb6e

… category duckdb tested only Related work items: #12685, #12699

Updated email and zipcode check names. Documentation and python clien…

1098f0e

…t regenerated to support new checks.

File synchronization uses parallel to perform unordered synchronization.

47bf87b

Currency and country code checks are suggested.

13f7c63

Accepted values for numeric values in set is proposed.

4dfc443

simplified count queries

dc8af12

Updated screens in Category of checks documents in the documentation

3986c6e

Merge remote-tracking branch 'origin/develop' into develop

a0e0e29

usa phone sample data improved, integration tests for patterns valid …

bc1bfe9

…uda phone percent

pii contains usa phone integration test assetions changed to the upda…

901b503

…ted test data

zip codes got better examples in test data, valid usa zipcodes percen…

4e37479

…t integration tests, fixed oracle zipcode sensors

patterns invalid usa phone and zipcode count integration tests, nulls…

ac260dd

… excluded from counts in patterns sensor duckdb and postgresql

Updated screens in Category of checks documents (anomalies) in the do…

a2780f0

…cumentation

fixed sqlserver phone and zipcode sensors

9607c86

repaired rendering tests with order by

572f9d9

strings are quoted for downloaded csv with error samples

c108ce4

Merge branch 'develop' of https://dev.azure.com/imagetemplates/documa…

bc9b0d9

…ti.io/_git/dqoado into feature/12732_Error_sampling_tab_should_have_a_download_as_CSV_button

Merged PR 2893: integration tests for pattern sensors for USA phones …

bb3176c

…and zipcodes - integration tests for patterns usa phones and zip codes - excluded nulls from counts in duckdb and postgresql - fixed new sensors Related work items: #12809

Updated screens in Category of checks documents (custom SQL) in the d…

e7dd036

…ocumentation

Check specification yaml files regenerated.

5a6a89c

Merge branch 'develop' of https://dev.azure.com/imagetemplates/documa…

57af7ea

…ti.io/_git/dqoado into develop

piotrczarnas and others added 29 commits August 19, 2024 20:30

Image on the incident notification resized and moved to git.

bb6f3e3

"Notification pattern" label changed to "Notification filter" to unif…

1860d31

…y it with the button name.

download button link bug fix

1c525cf

error samples downloaded file name fixed

0ebe57f

Merged PR 2933: error samples download link and file name fixed

56dca11

Related work items: #12912

word count sensors does not count rows with spaces only

ffced6e

Merged PR 2935: word count sensors does not count rows with spaces only

b44252d

word count sensors does not count rows with spaces only Related work items: #12914

aligned content table quality status to the top

217fec7

Merged PR 2936: 12891 aligned content table quality status top

0810930

aligned content table quality status top

Documentation and python client updated after small changes in the fi…

4854f29

…ltered notifications.

Merge branch 'develop' of https://dev.azure.com/imagetemplates/documa…

a34f741

…ti.io/_git/dqoado into develop

Updated screens on Categories of checks documentation (PII)

e1ba573

Version number changed to 1.7.0

c33f5cb

Updated screens on Categories of checks documentation (empty or incom…

af2ee78

…plete columns)

setting values correctly

e6e596f

Merged PR 2937: 12920 setting values correctly

127feb2

setting values correctly

minor changes IconButton

38d364b

Merged PR 2938: 12908 minor changes IconButton

8d2195a

minor changes IconButton

fixed offset rule mining

df39649

adding .csv to dictionary name if user doesnt provide it

5390106

fixed parameters in creating filtered notification

75753af

Merged PR 2939: 12918 fixed offset rule mining

18c25ab

fixed offset rule mining

Merged PR 2940: 12916 adding .csv to dictionary name if user doesnt p…

dcfcd44

…rovide it adding .csv to dictionary name if user doesnt provide it

Merged PR 2941: 12910 fixed parameters in creating filtered notification

79d7fbb

fixed parameters in creating filtered notification

Updated screens on Categories of checks documentation (data volume is…

d4a644a

…sues)

Minor updates on Categories of checks documentation (data volume issues)

2fc15e6

Fix the position of the table summary html table.

7f6f170

Clicking on columns without values in the table status screen does no…

deb020f

…t change the layout.

Paddings of a text area field decreased, so the regex does not look w…

73c233d

…eird.

dqops merged commit 388171c into main Aug 21, 2024
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

1.7.0 #37

1.7.0 #37

dqops commented Aug 21, 2024

1.7.0 #37

1.7.0 #37

Conversation

dqops commented Aug 21, 2024

1.7.0