Skip to content

Commit

Permalink
[Bug] [seatunnel-formats] Allow the entry in the map to be null and a…
Browse files Browse the repository at this point in the history
…llow the key in the entry to be null (#5277)

* 🐛 "Add null check in text deserialization

This commit adds a null check while splitting the kv in the TextDeserializationSchema. Not having this check could potentially lead to a crash if 'kvs' has less than two elements. Now, the function will return null in this case, significantly improving reliability."

* 🐛 Refactor text deserialization, allow null value assignments

Modified the TextDeserializationSchema to allow key-value pairs with null values. This adjustment helps to handle data records with missing values more accurately. The code was also reformatted for improved readability.

* 🎨 Refactor Builder constructor in TextDeserializationSchema

Tidied up the Builder constructor in TextDeserializationSchema by removing unnecessary empty lines. This change improves code readability and consistency with the rest of the class?

* ✅ Add  test case where the value is empty and the key is empty in the map

---------

Co-authored-by: yangpeng <nickyooungp@gmail.com>
  • Loading branch information
NickCodeJourney and yangpeng authored Sep 11, 2023
1 parent 04ce22a commit 07105a5
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -187,9 +187,13 @@ private Object convert(String field, SeaTunnelDataType<?> fieldType, int level)
String[] kvs = field.split(separators[level + 1]);
for (String kv : kvs) {
String[] splits = kv.split(separators[level + 2]);
objectMap.put(
convert(splits[0], keyType, level + 1),
convert(splits[1], valueType, level + 1));
if (splits.length < 2) {
objectMap.put(convert(splits[0], keyType, level + 1), null);
} else {
objectMap.put(
convert(splits[0], keyType, level + 1),
convert(splits[1], valueType, level + 1));
}
}
return objectMap;
case STRING:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,14 @@ public class TextFormatSchemaTest {
+ '\002'
+ "Kris"
+ '\003'
+ "21\001"
+ "21"
+ '\002'
+ "nullValueKey"
+ '\003'
+ '\002'
+ '\003'
+ "1231"
+ "\001"
+ "tyrantlucifer\001"
+ "true\001"
+ "1\001"
Expand Down

0 comments on commit 07105a5

Please sign in to comment.