Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/3.0.0 #27

Merged
merged 182 commits into from
Feb 19, 2025
Merged

Release/3.0.0 #27

merged 182 commits into from
Feb 19, 2025

Conversation

aarthy-dk
Copy link
Contributor

Features

  • scoring: display scoring for each table group in the overview page (fa05577)
  • ui: add sections to nav bar (09c80a5)
  • scoring: refresh data characteristics during test execution (acbe3e7)
  • scoring: add new score dashboard (fe5e36a)
  • scoring: add details page for score cards (c36ce70)
  • ui: display scores in Data Catalog, Profiling Runs & Test Runs (1ab9794)
  • test-suites: add "exclude from scoring" checkbox to add/edit dialog (2ddffcb)
  • components: improvements to button, checkbox and paginator (7f080fd)
  • score-issues: add pagination, multi-selection and issue reports in score drilldown (e178d13)
  • quality-dashboard: add profiling and testing scores to cards (3cc7ef8)
  • scores: support refreshing scores for runs and table groups (cd81a82)
  • scores: add score refresh buttons in hygiene issues and test results pages (ad79073)
  • data-catalog: consolidate, rename & improve tags (12eb738)
  • scoring: add support for custom score cards (92a6813)
  • scoring: refresh score cards results (4db34fb)
  • scoring: register a scorecard on table group creation (1b8c372)
  • ui: add component for column profiling results (afec021)
  • scoring: add column profiling to issues (938afa5)
  • project-dashboard: add sort by lowest score (1faa4a9)
  • cdes: add checkbox to table group for flagging cdes (a688763)
  • profiling: display bar for alpha case distribution (2b1d3b3)
  • Copy or Move test definitions TG-826 (0ba7fee)
  • sidebar: display project name above nav menu (3bf9612)
  • score-issues: add filters (6cf5e4d)
  • data-catalog: support drag to resize tree (aca99fa)

Bug Fixes

  • security: Switch the base image to Alpine to fix vulnerabilities (f95d10a)
  • functions: Eliminating the need of a trailing space on function calls (66d5408)
  • overview: minor fixes and styling improvements (491e6ed)
  • sql: special characters in pattern match test (3a15652)
  • overview: set default value for pandas null (00d267a)
  • overview: apply sort on first render (65d1766)
  • build: Pinning the matplotlib version (752b62f)
  • ui: use mutation observer to sync height (cc2a904)
  • test suites: add missing elements (320ac51)
  • ui: update highlight for client-side Selects (714d614)
  • test suites: dynamically display empty state (83f3136)
  • scoring: fixes + query and styling updates (8ddc483)
  • scoring: query fixes and styling updates (5a58672)
  • table groups: prevent unintentional edit to Explicit Table List (8de1510)
  • sql: bugs in data characteristics refresh and score dashboard queries (c3d6c4f)
  • sql: broken score details (da2d04a)
  • ui: make score cards compatible with dark mode (65e7fd5)
  • ui: display fixes for scoring, test suites and dashboard pages (32a8d4a)
  • project: handle project code in score queries (e05f974)
  • quality-dashboard: bugs in search (adce378)
  • profiling-results: broken function reference and missing filters (96449b4)
  • scoring: add missing labels and other suggestions (f61145a)
  • scoring: remove dq_dimension as a explorer filter (272cfc1)
  • scoring: rearrange the breadcrumbs in score explorer (987d43f)
  • scoring: minor css and text updates (008e197)
  • scoring: issues binding to query params (7a45396)
  • scoring: aggregate issues count for impact calc (d2d7a61)
  • broken link and incorrect date display on dashboard (4c363fd)
  • scoring: bug in latest_profile_record_ct (aa878d3)
  • router: exclude none query params when navigating (66d6ea3)
  • sql: bugs in upgrade script and test score calc (c48c2e9)
  • score-explorer: error on saving empty filter + responsive css (5c235c0)
  • sql: updates to impact calc (0fbc403)
  • data-chars: skip record count retrieval if no tables (5439b35)
  • scoring: reduce number of reruns in the explorer (03c8832)
  • scoring: update record count subquery (7723412)
  • scoring: load related objects for score definition (529b784)
  • scoring: send empty score card when no filters applied (e3313d0)
  • scoring: block saving of empty score cards (1ac2ab0)
  • scoring: initial display of empty card (9bae26e)
  • scoring: render initial state of score details in the explorer (02c808a)
  • upgrade: grant executor role rights to new scoring tables (c4d75ef)
  • test-execution: separate prevalence calc to prevent errors from affecting run (b458201)
  • column-profiling: align summary and percent bars (223fbba)
  • data-catalog: bug in editing tags (f0c02a5)
  • data-catalog: tweaks to column distribution, tooltips for tags (fe1ec09)
  • test-definitions: dropped tables and columns not displayed in filters (d9f8984)
  • test-validation: table tests for dropped tables not deactivated (4e1536c)
  • copy-tests: update to profiling and manual date fields (4a666f6)

Refactors

  • convert overview page to vanjs component (d0669a1)
  • ui: rename data hierarchy to data catalog (4a7e2b0)
  • forms: downgrade pydantic version for pypi release (1b968f5)
  • test suites: move page logic to vanjs (1c98c0e)
  • ui: support logo plugin (9c85f6b)
  • ui: rename "overview" and "score dashboard" pages (5cbcade)
  • scoring: split score details into components (e88dd94)
  • scores: make score displays consistent (6167060)
  • scoring: move queries to sqlalchemy model methods (0f48fb9)
  • scoring: enable cde score in score cards (0994dba)
  • components: add border property to Card (5e8ba8c)
  • data-catalog: make characteristics, distribution and issues cards reusable (5de4ac6)
  • data-catalog: make queries reusable and consistent (69ddd3e)
  • ui: replace column profiling with new component (ac46ffa)
  • remove dead code (f65acbc)
  • navigation: queue the navigation when receiving linkclicked event (36cb4d3)
  • tooltip-icon: add Icon component and use withTooltip (25a91d3)

Miscellaneous

  • functions: Code review feedback (93b2038)
  • add default ports to add connection form (6f68fa8)
  • ui: add empty state vanjs component (1b2d04e)
  • ui: add caption and metric reusable component (bc9e9dd)
  • ui: make some vanjs components more customizable (7060365)
  • Code review feedback (ed26f8d)
  • connections: change default max query chars (9e2a609)
  • pin scipy version (75aca92)
  • ui: update wording in score details page (2217061)
  • scores: add empty state message for score dashboard (43774ae)
  • scores: update styles and text (c709643)
  • linting: adress linting suggestions (e1d8b2f)
  • scoring: add new data_product column (bdf3ab2)
  • ui: remove cache clear when switching between scoring pages (b4386fc)
  • scoring: cache score card breakdown items (78d889f)
  • scoring: clear score list cache on delete and create (bab63a1)
  • scoring: display loading spinners (f025c0a)
  • components: support tooltips in input and attribute (45d8819)
  • scoring: uncomment score type dropdown in breakdown table (1c872ea)

rboni-dk and others added 30 commits November 12, 2024 12:10
fix(sql): special characters in pattern match test

See merge request dkinternal/testgen/dataops-testgen!120
fix(functions): Eliminating the need of a trailing space on function calls

See merge request dkinternal/testgen/dataops-testgen!114
fix(security): Switch the base image to Alpine to fix vulnerabilities

See merge request dkinternal/testgen/dataops-testgen!103
test: update max query chars

See merge request dkinternal/testgen/dataops-testgen!121
refactor: convert overview page to vanjs component

See merge request dkinternal/testgen/dataops-testgen!109
misc: sync from open source repo

See merge request dkinternal/testgen/dataops-testgen!122
feat(ui): add sections to nav bar

See merge request dkinternal/testgen/dataops-testgen!124
PyPI does not allow releases of packages referring to repository links
like was done for streamlit-pydantic package. In order to enable release
we downgraded to the latest released version and fixed pydantic
dependency to version 1.10.11. To keep the same functionality a
workaround for pydantic's computed_field decorator was implemented.
diogodk and others added 26 commits February 13, 2025 17:39
TG-826 Copy or Move test definitions

See merge request dkinternal/testgen/dataops-testgen!143
fix(data-catalog): bug in editing tags

See merge request dkinternal/testgen/dataops-testgen!158
misc(scoring): display loading spinners

See merge request dkinternal/testgen/dataops-testgen!160
refactor(navigation): queue the navigation when receiving linkclicked event

See merge request dkinternal/testgen/dataops-testgen!161
QA fixes + enhancements

See merge request dkinternal/testgen/dataops-testgen!162
refactor(test generation and scoring): improved test selection and dashboard query performance

See merge request dkinternal/testgen/dataops-testgen!163
misc(scoring): uncomment score type dropdown in breakdown table

See merge request dkinternal/testgen/dataops-testgen!164
fix(profiling): closes tg-885

See merge request dkinternal/testgen/dataops-testgen!165
@datakitchen-devops datakitchen-devops merged commit 0a4b4c7 into main Feb 19, 2025
2 checks passed
@datakitchen-devops datakitchen-devops deleted the release/3.0.0 branch February 19, 2025 16:07
Copy link

Coverage

Coverage Report •
FileStmtsMissCoverMissing
testgen/commands
   run_execute_cat_tests.py847212%17–23, 29–31, 36–39, 44–53, 58–59, 63–70, 76–144
   run_execute_tests.py998214%30–96, 100–115, 120–176
   run_launch_db_config.py24240%1–86
   run_profiling_bridge.py3002795%30, 35–41, 46–53, 58–118, 142–230, 234–243, 247–510, 514–521
   run_refresh_data_chars.py282122%18–75
   run_refresh_score_cards_results.py433613%10–54, 58, 91–107
   run_rollup_scores.py231626%11–20, 24–33
   run_test_parameter_validation.py74677%18–115
testgen/commands/queries
   execute_cat_tests_query.py855237%29–37, 40–43, 46–77, 81–82, 86–87, 91–92, 95–96, 99–100, 103–104, 107–108, 111–112, 115, 118, 121
   profiling_query.py24415032%78–>91, 94–97, 100–130, 134–135, 139–140, 144–145, 149–150, 154–155, 159–160, 164–165, 169–170, 174–175, 179, 183, 187–188, 192–212, 216–222, 226, 230–233, 237–240, 244–247, 255–344, 348–353, 357–360, 364–365, 369–372, 376–379
   refresh_data_chars_query.py59591%64, 72–76, 80, 84
   rollup_scores_query.py201045%9–10, 13–16, 20, 24, 28, 32
testgen/common
   read_file.py781377%22–>24, 24–>26, 29, 38–39, 42, 48–52, 58, 66–67, 69
testgen/common/database
   database_service.py37331213%56–70, 75, 97–111, 115–129, 135–148, 153–156, 162–200, 204–208, 213–221, 228–288, 293–350, 354–381, 385–414, 419–442, 447–449, 452–475, 479–521, 525–535, 539–550, 554–561, 565–575, 579–617, 621–623, 627–639
testgen/common/models
   scores.py1869543%60–69, 73–80, 89–98, 101–105, 108–111, 123–162, 188–223, 239–266, 269–276, 281, 340–348, 351–355
testgen/ui
   app.py44440%1–72
   bootstrap.py62620%1–105
   forms.py92920%1–136
testgen/ui/components/widgets
   button.py251915%32–56
   page.py482838%17–30, 34–38, 42, 46, 50, 54, 58, 62, 66, 70, 74, 78–81
   sidebar.py18755%32, 51–57
   testgen_component.py251819%40–69
testgen/ui/navigation
   menu.py461947%33–38, 41, 44–54, 57, 60
   page.py553330%25–30, 33–47, 50–57, 60–70, 74
   router.py725323%24–25, 28–62, 65, 72–74, 77, 80–104, 107–109, 113–119
testgen/ui/queries
   profiling_queries.py604124%64–77, 82–128, 133–185, 197–204, 217–222, 231–306, 311–362
   scoring_queries.py46460%1–165
   table_group_queries.py624524%9, 25–29, 34–40, 44–51, 56–74, 78–82, 87–92, 96–124, 128–182, 186–192, 196–213
   test_definition_queries.py695615%7–15, 20–84, 88–142, 146–237, 241–245, 249–255, 259–267, 271–282, 286–380, 384–394
   test_suite_queries.py473229%9–121, 126–145, 149–165, 169–191, 195–201, 205–211, 215–219, 223–234, 238–243, 247–255
testgen/ui/services
   hygiene_issues_service.py40400%1–89
   query_service.py938010%12–23, 27–41, 46–76, 80–86, 90–117, 121–136, 140–154, 158–169, 173–204, 208–257
   table_group_service.py947815%11–12, 16–17, 21–22, 26–30, 34–47, 51–54, 58–69, 73–77, 82–123, 127–135
   test_definition_service.py785821%11–13, 19–20, 26–43, 47–52, 56–58, 62–64, 68–70, 75–89, 93–95, 99–104, 108–125, 142–143, 148–149, 153–154
   test_results_service.py81729%11–36, 41–83, 88–186
testgen/ui/views
   data_catalog.py1041040%1–246
   hygiene_issues.py2122120%1–532
   login.py34340%1–62
   profiling_results.py79790%1–246
   profiling_runs.py76760%1–205
   project_dashboard.py42420%1–268
   project_settings.py40400%1–67
   quality_dashboard.py19190%1–43
   score_details.py72720%1–171
   score_explorer.py1091090%1–234
   test_definitions.py4483998%34–139, 144–187, 191–207, 222–610, 615, 620, 625–698, 701–710, 714–728, 733–752, 756–760, 767–913, 917–941, 947–948, 954–955, 960–967, 972–973, 977–987
   test_results.py31424917%59–201, 208–219, 235–236, 241–257, 262–264, 269–276, 288–289, 302–389, 394–406, 411–455, 466–467, 472–473, 478–479, 484–485, 489–552, 571–727, 731–805, 809–820, 825–852, 856–858, 862–870
   test_runs.py885334%36–90, 98–138, 142–146, 151–161, 166–167, 172–173, 178–247
   test_suites.py15211322%35–63, 101, 105–136, 141–142, 147, 152–153, 157–267, 272–318, 323–359
testgen/ui/views/connections
   forms.py1081080%2–283
   page.py1711710%1–334
testgen/ui/views/dialogs
   profiling_results_dialog.py13637%10–16, 21–24
testgen/ui/views/table_groups
   forms.py62620%2–191
   page.py1991990%1–466
testgen/utils
   __init__.py1017916%6, 20–22, 26–31, 36–37, 41–55, 59–75, 79–84, 88–90, 94–123, 144, 156–159, 166–179, 183–195
TOTAL8822700017% 

Tests Skipped Failures Errors Time
29 0 💤 0 ❌ 0 🔥 7.888s ⏱️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants