wip: string overhaul #24439
Closed
wip: string overhaul #24439
Commits on Dec 8, 2017
-
-
iswellformed(c::Char) to test if
crepresents a code pointThis allows testing before attempting to decode `c` which we use in a number of places to handle malformed UTF-8 data correctly.
-
remove internal chomp! function
this function was always dangerous and has been equivalent to `chomp` since the `String` representation change introduced in Julia 0.6
-
convert(String, ::Vector{Char}): don't normalize surrogate pairs
we're in a new era where Julia tries as hard as it can to pass text data through intact rather than trying to guess what you meant. to that end, when converting a vector of characters to a `String` we no longer detect and combine surrogate pairs. there can be a utility function to do that kind of normalization on strings, but it's not the job of the basic string code.
-
-
-
iswellformed => !ismalformed; make test stricter
despite generally trying to avoid double negatives, it seems that "malformed" is the clearer and more common concept to test for here. it was also a bit confusing to have `MalformedCharError` but then have `iswellformed` as the predicate instead of `ismalformed`. testing for well-formedness more strictly doesn't seem to add too much complexity here; if performance becomes an issue, we'll need a `@inbounds`-like annotation anyway.
-
-
-
-
-
-
This reverts commit bc7f25d83165fe4be209d6485406604c762930dd.
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.