Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start to make Rust node API typed using arrow #353

Merged
merged 35 commits into from
Oct 26, 2023
Merged

Start to make Rust node API typed using arrow #353

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
7d96e03
Start to make Rust node API typed using arrow
phil-opp Sep 6, 2023
6154415
Update Rust APIs and rust-dataflow-example to use arrow types
phil-opp Sep 6, 2023
afae3b7
Merge branch 'main' into rust-typed-input
phil-opp Oct 16, 2023
b38c112
Merge branch 'main' into rust-typed-input
phil-opp Oct 17, 2023
ddf8035
Adjust multiple-daemons example
phil-opp Oct 17, 2023
89f980e
Simplify arrow array construction through new `IntoArrow` trait
phil-opp Oct 17, 2023
9a10280
Simplify reading of primitive and string types from arrow arrays
phil-opp Oct 17, 2023
8f9e2ca
Add more Rust<->arrow conversions
phil-opp Oct 17, 2023
a54f000
Update C++ operator API to use new conversions
phil-opp Oct 17, 2023
40b991d
Fix length call in benchmark example
phil-opp Oct 17, 2023
6140681
Rename and reexport `Data` enums
phil-opp Oct 19, 2023
df65eab
Adjust Python operator API tests
phil-opp Oct 19, 2023
631019f
Fix some clippy warnings
phil-opp Oct 19, 2023
14a5531
Fix doc test
phil-opp Oct 19, 2023
b78f913
Use 'value' instead of 'data' in Python node template
phil-opp Oct 19, 2023
96eceb8
Fix Python operator template too
phil-opp Oct 19, 2023
10c15d5
Fix: Remove invalid array length checks when reading byte arrays
phil-opp Oct 19, 2023
36a1f77
Provide accessor functions for input ID and data in C operator API
phil-opp Oct 19, 2023
94c0da6
Provide higher-level function to send output from C operator
phil-opp Oct 19, 2023
75b0513
Update C dataflow example to use new API functions
phil-opp Oct 19, 2023
4acc3c6
Fix name collision
phil-opp Oct 19, 2023
5dcaa9e
Adjust C++ dataflow example for new C operator API
phil-opp Oct 24, 2023
242513f
Start fixing Python dataflow example
phil-opp Oct 24, 2023
8cc5837
Update `arrow` and `pyo3` dependencies to latest versions
phil-opp Oct 24, 2023
c6ce457
Rework raw data to arrow array conversion
phil-opp Oct 24, 2023
ac7d174
Fix alignment errors when receiving empty arrays
phil-opp Oct 24, 2023
ad3f048
Avoid alignment errors by aligning raw data on deseralization
phil-opp Oct 24, 2023
905e9ee
Add more context to serialize_deserialize roundrip errors
phil-opp Oct 24, 2023
85338b6
Consider alignment when copying array into raw buffer
phil-opp Oct 24, 2023
99e175e
Link `CoreServices` etc when building C operator on macOS
phil-opp Oct 25, 2023
8099cb2
Link system libraries when building C operator on Windows
phil-opp Oct 25, 2023
27f621f
Link system libraries for C++ operator example too
phil-opp Oct 25, 2023
9830bb8
Move cargo target dir to `C:` drive on Windows CI job
phil-opp Oct 25, 2023
fcbe1d1
Print available disk space on Windows
phil-opp Oct 25, 2023
25ce08d
Cache `CARGO_TARGET_DIR` if set
phil-opp Oct 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,14 @@ jobs:
runs-on: ${{ matrix.platform }}
timeout-minutes: 60
steps:
- name: Print available space (Windows only)
run: Get-PSDrive
if: runner.os == 'Windows'
- name: Override cargo target dir (Windows only)
run: echo "CARGO_TARGET_DIR=C:\cargo-target" >> "$GITHUB_ENV"
shell: bash
if: runner.os == 'Windows'

- uses: actions/checkout@v3
- uses: r7kamura/rust-problem-matchers@v1.1.0
- run: cargo --version --verbose
Expand Down Expand Up @@ -50,6 +58,7 @@ jobs:
cache-on-failure: true
# only save caches for `main` branch
save-if: ${{ github.ref == 'refs/heads/main' }}
cache-directories: ${{ env.CARGO_TARGET_DIR }}

- name: "Check"
run: cargo check --all
Expand Down
Loading