Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds GPU implementation of JSON-token-stream to JSON-tree #11518

Merged
merged 36 commits into from
Sep 19, 2022

Commits on Aug 11, 2022

  1. pull changes from PR rapidsai#11291

    Squashed commit of the following:
    
    commit 6e1bc75
    Author: Karthikeyan Natarajan <karthikeyann@users.noreply.github.com>
    Date:   Fri Aug 12 03:06:30 2022 +0530
    
        remove debug print in logical stack
    
    commit 8e75645
    Author: Karthikeyan Natarajan <karthikeyann@users.noreply.github.com>
    Date:   Fri Aug 12 03:01:34 2022 +0530
    
        remove duplicate renamed header
    
    commit 3b2acb2
    Merge: 2b59b04 a67b718
    Author: Karthikeyan Natarajan <karthikeyann@users.noreply.github.com>
    Date:   Fri Aug 12 02:59:01 2022 +0530
    
        Merge branch 'branch-22.10' of https://github.com/rapidsai/cudf into json-tree
    
    commit 2b59b04
    Merge: 12cf0be 2d214ea
    Author: Karthikeyan Natarajan <karthikeyann@users.noreply.github.com>
    Date:   Tue Jul 26 13:40:41 2022 +0530
    
        Merge branch 'branch-22.08' of https://github.com/rapidsai/cudf into json-tree
    
    commit 12cf0be
    Author: Karthikeyan Natarajan <karthikeyann@users.noreply.github.com>
    Date:   Tue Jul 26 13:29:55 2022 +0530
    
        fix clang-format style fix
    
    commit 3e756bb
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Jul 18 08:17:03 2022 -0700
    
        replaces tree return type from tuple to struct
    
    commit bef4fb1
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon May 16 22:10:08 2022 -0700
    
        moved debug print to detail ns
    
    commit ff90528
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Fri May 13 09:52:20 2022 -0700
    
        squash & rebase on latest tokenizer version
    
    commit 987699f
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jun 2 05:19:53 2022 -0700
    
        fixes sg-count & uses rmm stream in fst tests
    
    commit 00a95eb
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 25 12:17:08 2022 -0700
    
        put lookup tables into their own cudf file
    
    commit a8ac5fa
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 25 09:59:37 2022 -0700
    
        refactored lookup tables
    
    commit f996ce9
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 11 12:17:55 2022 -0700
    
        squashed with bracket/brace test
    
    commit 671ce41
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Tue Apr 12 22:55:00 2022 -0700
    
        minor style changes addressing review comments
    
    commit f4ec994
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 4 07:35:33 2022 -0700
    
        device_span
    
    commit d18238f
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 4 02:28:30 2022 -0700
    
        renaming key-value store op to stack_op
    
    commit 62ddf66
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Mar 31 05:28:17 2022 -0700
    
        switched to using rmm also inside algorithm
    
    commit 2f7b254
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Mar 31 04:11:44 2022 -0700
    
        Added utility to debug print & instrumented code to use it
    
    commit 67f609d
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jul 14 04:15:11 2022 -0700
    
        renames enums & moving from device_span to ptr params
    
    commit 01aef44
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Wed Jul 13 07:22:52 2022 -0700
    
        wraps if with stream params into detail ns
    
    commit 4aaf595
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Wed Jul 13 05:45:49 2022 -0700
    
        fixes for breaking downstream interface changes
    
    commit 237456d
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jun 2 08:19:37 2022 -0700
    
        fixes breaking changes from dependent-FST-PR
    
    commit 7fc8619
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Tue May 3 07:05:44 2022 -0700
    
        rebase on latest FST
    
    commit 6d3eff2
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jun 2 05:19:53 2022 -0700
    
        fixes sg-count & uses rmm stream in fst tests
    
    commit 6548836
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 25 12:17:08 2022 -0700
    
        put lookup tables into their own cudf file
    
    commit 9dfd4ad
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 25 09:59:37 2022 -0700
    
        refactored lookup tables
    
    commit fe06f0b
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 11 12:17:55 2022 -0700
    
        squashed with bracket/brace test
    
    commit 36c8296
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Tue Apr 12 22:55:00 2022 -0700
    
        minor style changes addressing review comments
    
    commit 24dab9e
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 4 07:35:33 2022 -0700
    
        device_span
    
    commit 49fa996
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 4 02:28:30 2022 -0700
    
        renaming key-value store op to stack_op
    
    commit b260610
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Mar 31 05:28:17 2022 -0700
    
        switched to using rmm also inside algorithm
    
    commit 9b20d16
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Mar 31 04:11:44 2022 -0700
    
        Added utility to debug print & instrumented code to use it
    
    commit 78dd893
    Merge: 8a184e9 9627091
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Fri Jul 15 23:06:55 2022 -0700
    
        Merge remote-tracking branch 'upstream/branch-22.08' into feature/finite-state-transducer-trimmed
    
    commit 8a184e9
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Fri Jul 15 22:51:18 2022 -0700
    
        rephrases documentation on in-reg array
    
    commit bea2a02
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Fri Jul 15 01:54:20 2022 -0700
    
        replaces vanilla loop with iota
    
    commit cba1619
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jul 14 09:31:12 2022 -0700
    
        fixes style in dispatch dfa
    
    commit 3f47952
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jul 14 09:22:03 2022 -0700
    
        replaces gtest asserts with expects
    
    commit d351e5c
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jul 14 09:17:59 2022 -0700
    
        addresses style review comments & fixes a todo
    
    commit 3038058
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jul 14 09:17:09 2022 -0700
    
        adds excplitis error checking
    
    commit f52e614
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jul 14 09:16:18 2022 -0700
    
        replaces enum with typed constexpr
    
    commit eb24962
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Tue Jul 12 04:52:36 2022 -0700
    
        fixes logical stack test includes
    
    commit a798852
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Jul 11 11:00:22 2022 -0700
    
        adds check for state transition narrowing conversion
    
    commit e6f8def
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Jul 11 09:06:01 2022 -0700
    
        some west-const remainders & unifies StateIndexT
    
    commit 5f1c4b5
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Jul 11 06:26:47 2022 -0700
    
        removes state vector-wrapper in favor of vanilla array
    
    commit 485a1c6
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Fri Jul 8 22:49:57 2022 -0700
    
        adopts c++17 namespaces declarations
    
    commit f656f49
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jul 7 02:41:16 2022 -0700
    
        adopts device-side test data gen
    
    commit 694a365
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Wed Jun 15 04:28:51 2022 -0700
    
        adopts suggested fst test changes
    
    commit 9fe8e4b
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Tue Jun 14 03:12:35 2022 -0700
    
        minor doxygen fix
    
    commit eccf970
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Thu Jun 2 05:19:53 2022 -0700
    
        fixes sg-count & uses rmm stream in fst tests
    
    commit 6fdd24a
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon May 9 12:17:34 2022 -0700
    
        refactor lut sanity check
    
    commit 17dcbfd
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon May 9 10:33:00 2022 -0700
    
        making const vars const
    
    commit ea79a81
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon May 9 10:32:17 2022 -0700
    
        Adding hostdevice macros to in-reg array
    
    commit caf6195
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon May 9 10:24:51 2022 -0700
    
        unified usage of pragma unrolls
    
    commit e24a133
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Wed May 4 07:29:00 2022 -0700
    
        removing unused var post-cleanup
    
    commit 39cff80
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Wed Apr 27 04:42:31 2022 -0700
    
        Change interface for FST to not need temp storage
    
    commit 239f138
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 25 12:17:08 2022 -0700
    
        put lookup tables into their own cudf file
    
    commit 39a6b65
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 25 09:59:37 2022 -0700
    
        refactored lookup tables
    
    commit 355d1e4
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Wed Apr 20 05:11:32 2022 -0700
    
        clean up & addressing review comments
    
    commit 0557d41
    Author: Elias Stehle <3958403+elstehle@users.noreply.github.com>
    Date:   Mon Apr 11 12:17:55 2022 -0700
    
        squashed with bracket/brace test
    karthikeyann committed Aug 11, 2022
    Configuration menu
    Copy the full SHA
    cdc98d6 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d074579 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    f5287a5 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d694f21 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2022

  1. Configuration menu
    Copy the full SHA
    96eca61 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    34ba420 View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2022

  1. Configuration menu
    Copy the full SHA
    a083709 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2022

  1. Configuration menu
    Copy the full SHA
    1768fe7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4964826 View commit details
    Browse the repository at this point in the history
  3. cleanup, organize code

    karthikeyann committed Aug 24, 2022
    Configuration menu
    Copy the full SHA
    4007f6a View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2022

  1. Configuration menu
    Copy the full SHA
    0c092ea View commit details
    Browse the repository at this point in the history
  2. cleanup

    karthikeyann committed Aug 26, 2022
    Configuration menu
    Copy the full SHA
    8ff75a9 View commit details
    Browse the repository at this point in the history
  3. doc

    karthikeyann committed Aug 26, 2022
    Configuration menu
    Copy the full SHA
    f73a187 View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2022

  1. Configuration menu
    Copy the full SHA
    018a779 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e531cba View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    24d20e6 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2022

  1. cleanup tests

    karthikeyann committed Sep 5, 2022
    Configuration menu
    Copy the full SHA
    94bdd1f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    edb78ad View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ab1db5b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    0ba90db View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2022

  1. Configuration menu
    Copy the full SHA
    15570d4 View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2022

  1. Configuration menu
    Copy the full SHA
    b505e76 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ab4c7a1 View commit details
    Browse the repository at this point in the history
  3. add nvtx func ranges

    karthikeyann committed Sep 8, 2022
    Configuration menu
    Copy the full SHA
    a146936 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2022

  1. address review comments

    karthikeyann committed Sep 9, 2022
    Configuration menu
    Copy the full SHA
    6eb9bbe View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2022

  1. address review comments

    karthikeyann committed Sep 12, 2022
    Configuration menu
    Copy the full SHA
    38b99cf View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    285298a View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2022

  1. Configuration menu
    Copy the full SHA
    90cd2dc View commit details
    Browse the repository at this point in the history

Commits on Sep 17, 2022

  1. Configuration menu
    Copy the full SHA
    89188cd View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2022

  1. Configuration menu
    Copy the full SHA
    2bdf767 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7661669 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    e69038b View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    b8367a7 View commit details
    Browse the repository at this point in the history
  5. Revert "fix missing get_token_index in range_end"

    This reverts commit b8367a7.
    karthikeyann committed Sep 19, 2022
    Configuration menu
    Copy the full SHA
    7c4a490 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    d3725a8 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    2f34d3a View commit details
    Browse the repository at this point in the history