Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/dead letter queue core #29164

Merged
merged 82 commits into from
Dec 5, 2023
Merged

Feature/dead letter queue core #29164

merged 82 commits into from
Dec 5, 2023

Commits on Jul 5, 2023

  1. Update 2.50 release notes to include new Kafka topicPattern feature

    @johnjcasey
    johnjcasey committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    c22b8c1 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. Merge remote-tracking branch 'origin/master' 

    # Conflicts:
#	CHANGES.md
    @johnjcasey
    johnjcasey committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    f7cf5de View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2023

  1. Merge remote-tracking branch 'origin/master'

    @johnjcasey
    johnjcasey committed Aug 9, 2023
    Configuration menu
    Copy the full SHA
    ab68ecb View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2023

  1. Merge remote-tracking branch 'origin/master'

    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    40ad1d5 View commit details
    Browse the repository at this point in the history

  2. Create groovy class for io performance tests 

    Create gradle task and github actions config for GCS using this.
    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    6c9c28d View commit details
    Browse the repository at this point in the history

  3. delete unnecessary class

    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    520c9d1 View commit details
    Browse the repository at this point in the history

  4. fix env call

    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    062de23 View commit details
    Browse the repository at this point in the history

  5. Merge pull request #181 from johnjcasey/feature/automate-performance-… 

    …tests

Feature/automate performance tests
    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    01fc25b View commit details
    Browse the repository at this point in the history

  6. fix call to gradle

    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    9c9f86b View commit details
    Browse the repository at this point in the history

  7. Merge pull request #182 from johnjcasey/feature/automate-performance-… 

    …tests

fix call to gradle
    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    92306fa View commit details
    Browse the repository at this point in the history

  8. run on hosted runner for testing

    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    925ce55 View commit details
    Browse the repository at this point in the history

  9. Merge pull request #183 from johnjcasey/feature/automate-performance-… 

    …tests

run on hosted runner for testing
    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    2dcfb70 View commit details
    Browse the repository at this point in the history

  10. add additional checkout

    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    117ef8b View commit details
    Browse the repository at this point in the history

  11. Merge pull request #184 from johnjcasey/feature/automate-performance-… 

    …tests

add additional checkout
    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    1f73cda View commit details
    Browse the repository at this point in the history

  12. add destination for triggered tests

    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    cb6e01b View commit details
    Browse the repository at this point in the history

  13. Merge pull request #185 from johnjcasey/feature/automate-performance-… 

    …tests

add destination for triggered tests
    @johnjcasey
    johnjcasey committed Aug 31, 2023
    Configuration menu
    Copy the full SHA
    a9e86aa View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2023

  1. move env variables to correct location

    @johnjcasey
    johnjcasey committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    8ea6c51 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #186 from johnjcasey/feature/automate-performance-… 

    …tests

move env variables to correct location
    @johnjcasey
    johnjcasey committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    d8822d7 View commit details
    Browse the repository at this point in the history

  3. try uploading against separate dataset

    @johnjcasey
    johnjcasey committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    320a4cc View commit details
    Browse the repository at this point in the history

  4. Merge pull request #187 from johnjcasey/feature/automate-performance-… 

    …tests

try uploading against separate dataset
    @johnjcasey
    johnjcasey committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    e89b59e View commit details
    Browse the repository at this point in the history

  5. try without a user

    @johnjcasey
    johnjcasey committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    1cd4e55 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #188 from johnjcasey/feature/automate-performance-… 

    …tests

try without a user
    @johnjcasey
    johnjcasey committed Sep 1, 2023
    Configuration menu
    Copy the full SHA
    4473f17 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2023

  1. update branch checkout, try to view the failure log

    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    4fc5b8e View commit details
    Browse the repository at this point in the history

  2. Merge pull request #189 from johnjcasey/feature/automate-performance-… 

    …tests

update branch checkout, try to view the failure log
    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    706650d View commit details
    Browse the repository at this point in the history

  3. run on failure

    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    59069f2 View commit details
    Browse the repository at this point in the history

  4. Merge pull request #190 from johnjcasey/feature/automate-performance-… 

    …tests

run on failure
    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    7f79b62 View commit details
    Browse the repository at this point in the history

  5. update to use correct BigQuery instance

    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    6f51976 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #191 from johnjcasey/feature/automate-performance-… 

    …tests

update to use correct BigQuery instance
    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    e95d920 View commit details
    Browse the repository at this point in the history

  7. convert to matrix

    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    df716cb View commit details
    Browse the repository at this point in the history

  8. Merge pull request #192 from johnjcasey/feature/automate-performance-… 

    …tests

convert to matrix
    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    4d8eded View commit details
    Browse the repository at this point in the history

  9. add result reporting

    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    4bf0826 View commit details
    Browse the repository at this point in the history

  10. Merge pull request #193 from johnjcasey/feature/automate-performance-… 

    …tests

add result reporting
    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    403f054 View commit details
    Browse the repository at this point in the history

  11. add failure clause

    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    d40d04b View commit details
    Browse the repository at this point in the history

  12. Merge pull request #194 from johnjcasey/feature/automate-performance-… 

    …tests

add failure clause
    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    aca4b2e View commit details
    Browse the repository at this point in the history

  13. remove failure clause, update to run on self-hosted

    @johnjcasey
    johnjcasey committed Sep 5, 2023
    Configuration menu
    Copy the full SHA
    2739e92 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2023

  1. address comments, clean up build

    @johnjcasey
    johnjcasey committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    bd6efeb View commit details
    Browse the repository at this point in the history

  2. clarify branching

    @johnjcasey
    johnjcasey committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    226a655 View commit details
    Browse the repository at this point in the history

  3. Merge pull request #195 from johnjcasey/feature/automate-performance-… 

    …tests

Feature/automate performance tests
    @johnjcasey
    johnjcasey committed Sep 6, 2023
    Configuration menu
    Copy the full SHA
    9c7286b View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2023

  1. Merge branch 'apache:master' into master

    @johnjcasey
    johnjcasey committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    5b1b2c2 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2023

  1. Merge remote-tracking branch 'origin/master'

    @johnjcasey
    johnjcasey committed Oct 24, 2023
    Configuration menu
    Copy the full SHA
    f8c2b06 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. Add error handling base implementation & test DLQ enabled class

    @johnjcasey
    johnjcasey committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    d058ac9 View commit details
    Browse the repository at this point in the history

  2. Add test cases

    @johnjcasey
    johnjcasey committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    8c9dd94 View commit details
    Browse the repository at this point in the history

  3. apply spotless

    @johnjcasey
    johnjcasey committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    4d23fe8 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. Fix Checkstyles

    @johnjcasey
    johnjcasey committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    31432b7 View commit details
    Browse the repository at this point in the history

  2. Fix Checkstyles

    @johnjcasey
    johnjcasey committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    a9dae91 View commit details
    Browse the repository at this point in the history

  3. make DLH serializable

    @johnjcasey
    johnjcasey committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    64dde49 View commit details
    Browse the repository at this point in the history

  4. rename dead letter to bad record

    @johnjcasey
    johnjcasey committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    c82185d View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2023

  1. make DLH serializable

    @johnjcasey
    johnjcasey committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    78d45a8 View commit details
    Browse the repository at this point in the history

  2. Change bad record router name, and use multioutputreceiver instead of… 

    … process context
    @johnjcasey
    johnjcasey committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    6c36549 View commit details
    Browse the repository at this point in the history

  3. Refactor BadRecord to be nested

    @johnjcasey
    johnjcasey committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    44036be View commit details
    Browse the repository at this point in the history

  4. clean up checkstyle

    @johnjcasey
    johnjcasey committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    06ca166 View commit details
    Browse the repository at this point in the history

  5. Update error handler test

    @johnjcasey
    johnjcasey committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    a26d605 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2023

  1. Add metrics for counting error records, and for measuring feature usage

    @johnjcasey
    johnjcasey committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    5a6e8d0 View commit details
    Browse the repository at this point in the history

  2. apply spotless

    @johnjcasey
    johnjcasey committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    70c8991 View commit details
    Browse the repository at this point in the history

  3. fix checkstyle

    @johnjcasey
    johnjcasey committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    36baf98 View commit details
    Browse the repository at this point in the history

  4. make metric reporting static

    @johnjcasey
    johnjcasey committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    2119c76 View commit details
    Browse the repository at this point in the history

  5. spotless

    @johnjcasey
    johnjcasey committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    798cfc3 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2023

  1. Rework annotations to be an explicit label on a PTransform, instead o… 

    …f using java annotations
    @johnjcasey
    johnjcasey committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    c03bb2b View commit details
    Browse the repository at this point in the history

  2. fix checkstyle

    @johnjcasey
    johnjcasey committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    bf99363 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. Address comments

    @johnjcasey
    johnjcasey committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    881f9d8 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2023

  1. Address comments

    @johnjcasey
    johnjcasey committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    f8c6d8c View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2023

  1. Fix test cases, spotless

    @johnjcasey
    johnjcasey committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    a1b112c View commit details
    Browse the repository at this point in the history

  2. remove flatting without error collections

    @johnjcasey
    johnjcasey committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    ad1684a View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2023

  1. fix nullness

    @johnjcasey
    johnjcasey committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    074faf2 View commit details
    Browse the repository at this point in the history

  2. spotless + encoding issues

    @johnjcasey
    johnjcasey committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    17bf295 View commit details
    Browse the repository at this point in the history

  3. spotless

    @johnjcasey
    johnjcasey committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    e2ec57f View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2023

  1. throw error when error handler isn't used

    @johnjcasey
    johnjcasey committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    8b3f052 View commit details
    Browse the repository at this point in the history

  2. add concrete bad record error handler class

    @johnjcasey
    johnjcasey committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    525d912 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2023

  1. spotless, fix test category

    @johnjcasey
    johnjcasey committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    9b4a348 View commit details
    Browse the repository at this point in the history

  2. fix checkstyle

    @johnjcasey
    johnjcasey committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    d6f4097 View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. clean up comments

    @johnjcasey
    johnjcasey committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    a067238 View commit details
    Browse the repository at this point in the history

  2. fix test case

    @johnjcasey
    johnjcasey committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    408bc26 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'upstream/master' into feature/dead-lett… 

    …er-queue-core
    @johnjcasey
    johnjcasey committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    0246a80 View commit details
    Browse the repository at this point in the history

  4. remove "failing transform" field on bad record, add note to CHANGES.md

    @johnjcasey
    johnjcasey committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    ec50bff View commit details
    Browse the repository at this point in the history

  5. fix failing test cases

    @johnjcasey
    johnjcasey committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    b092709 View commit details
    Browse the repository at this point in the history

  6. fix failing test cases

    @johnjcasey
    johnjcasey committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    3f1e97c View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. apply spotless

    @johnjcasey
    johnjcasey committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    4356f27 View commit details
    Browse the repository at this point in the history

Commits on Dec 4, 2023

  1. Merge remote-tracking branch 'upstream/master' into feature/dead-lett… 

    …er-queue-core

# Conflicts:
#	CHANGES.md
    @johnjcasey
    johnjcasey committed Dec 4, 2023
    Configuration menu
    Copy the full SHA
    5067bd1 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2023

  1. apply final comments

    @johnjcasey
    johnjcasey committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    864c429 View commit details
    Browse the repository at this point in the history

  2. apply final comments

    @johnjcasey
    johnjcasey committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    8307b63 View commit details
    Browse the repository at this point in the history

  3. apply final comments

    @johnjcasey
    johnjcasey committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    fad9d56 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'upstream/master' into feature/dead-lett… 

    …er-queue-core
    @johnjcasey
    johnjcasey committed Dec 5, 2023
    Configuration menu
    Copy the full SHA
    c20c2d7 View commit details
    Browse the repository at this point in the history