Support partition pruning during file listing

[xushiyan]

Provide basic table API to accept predicates like foo > 10 or bar != A. The partition loading for file system view should process the predicates and load relevant partitions. Just consider supporting AND for multiple predicate string expressions.

Follow up work #159 #160

[KnightChess]

Hi, I'd like to work for this feature if no one plan work for it.

[xushiyan]

@KnightChess great! can you please describe a high-level approach for the implementation here?

[KnightChess]

support hudi internal filter, engine like datafusion or other need cover it expression to hudi partition filter.
something like this code,

let filter_one = PartitionFilter::try_from(("shortField", "=", "100", short_field_data_type)).unwrap();
let filter_two = PartitionFilter::try_from(("shortField", ">", "100", short_field_data_type)).unwrap();
hudi_table.partition_filter_replace(vec![filter_one, filter_two]);

the core struct filter define, key is partirion field, value is expression-value

pub struct PartitionFilter {
    /// The key of the PartitionFilter
    pub key: String,
    /// The value of the PartitionFilter
    pub value: PartitionValue
}

expression-value: reuse datafusion ScalarValue.

pub enum PartitionValue {
    /// The partition value with the equal operator
    Equal(ScalarValue),
    /// The partition value with the not equal operator
    NotEqual(ScalarValue),
    /// The partition value with the greater than operator
    GreaterThan(ScalarValue),
    /// The partition value with the greater than or equal operator
    GreaterThanOrEqual(ScalarValue),
    /// The partition value with the less than operator
    LessThan(ScalarValue),
    /// The partition value with the less than or equal operator
    LessThanOrEqual(ScalarValue),
    /// The partition values with the in operator
    In(Vec<ScalarValue>),
    /// The partition values with the not in operator
    NotIn(Vec<ScalarValue>),
}