Skip to content

Commit

Permalink
Add option to ignore errors in CSV parsing + improve error messages (k…
Browse files Browse the repository at this point in the history
…uzudb#4067)

* Copy exception handling concept

Change handling of copy exceptions to go through CSVErrorHandler

Add ignore errors csv reader option

Update error messages + account for header in line number calculation

Update expected error messages in tests

Print ... if line was not completely parsed when error occurred

Restart state machine if continuing after a parsing error

Set end of line for EOF + more test updates

Add more tests

Report warning in query result

Fix errors after rebase

* Populate warning messages for rel batch insert

* Avoid finalizing in aggregate scan operator

* Add expected warning messages

* Move warnings to factorized table + print in shell

* Limit total number of cached warnings

* Update tests with warnings

* Deal with case where warning table is null

* Avoid stack overflow caused by recursively calling parseCSV + add tests

* Fix failing tests caused by trimming whitespace at end of test checking'

* Add warnings to C API

* Add tests for query result warnings

* Code cleanup

* Update shell printings

* Improve test coverage

* Add shell tests to codecov

* Fix test failures

* Fix windows test failures

* Address review comments

* Run clang-format

* Avoid finalizing children for result collector

* Revert changes to factorized table util as it is no longer needed

* Revert shell test changes

* Remove warnings from query result (to add to CALL() function)

* Add function call show_warnings() to show warnings

* Populate query id in warning table

* Address review comments

* Address review comments 2

* Move warning limit to DB config

* Bump extension version

* Address review comments again

* Fix thread safety issue from progress func getting file offset

* More review comment addressing

* Address review comments

* Fix more error messages

* ResultCollector finalizes its children

* More review comment addressing

* Revert change to test_group.h

* Optimizations - cache errors locally before flushing to shared error handler + move file path from error to error handler

* Fix msvc compile error

* Move some shared fields into common csv error handler

* Hack around double-finalize issue

* Bump extension version

---------

Co-authored-by: CI Bot <royi-luo@users.noreply.github.com>

(cherry picked from commit b841014)
  • Loading branch information
royi-luo authored and wangqiang committed Aug 26, 2024
1 parent fc9d0cb commit b5cc60c
Show file tree
Hide file tree
Showing 67 changed files with 152,648 additions and 469 deletions.
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ add_subdirectory(third_party)
if(${BUILD_KUZU})
add_definitions(-DKUZU_ROOT_DIRECTORY="${PROJECT_SOURCE_DIR}")
add_definitions(-DKUZU_CMAKE_VERSION="${CMAKE_PROJECT_VERSION}")
add_definitions(-DKUZU_EXTENSION_VERSION="0.5.1.10")
add_definitions(-DKUZU_EXTENSION_VERSION="0.5.1.11")

include_directories(src/include)

Expand Down
Loading

0 comments on commit b5cc60c

Please sign in to comment.