[SPARK-49507][SQL] Fix Expected only partition pruning predicates exception

[TongWei1105]

What changes were proposed in this pull request?

bug fix

Why are the changes needed?

How to reproduce:

CREATE TABLE t (ID BIGINT, DT STRING) USING parquet PARTITIONED BY (DT);
set spark.sql.hive.metastorePartitionPruningFastFallback=true;
select * from t where dt=20240820;

Error msg:

org.apache.spark.sql.AnalysisException: Expected only partition pruning predicates: List(isnotnull(DT#21), (cast(DT#21 as bigint) = 20240820)).
  at org.apache.spark.sql.errors.QueryCompilationErrors$.nonPartitionPruningPredicatesNotExpectedError(QueryCompilationErrors.scala:2414)
  at org.apache.spark.sql.catalyst.catalog.ExternalCatalogUtils$.generatePartitionPredicateByFilter(ExternalCatalo

Does this PR introduce any user-facing change?

No

How was this patch tested?

add ut

Was this patch authored or co-authored using generative AI tooling?

No

[TongWei1105]

cc @wangyum

[wangyum]

cc @cxzl25 @cloud-fan

[cxzl25]

Because the catalogTable here is loaded directly from HMS, the partition fields are in lowercase instead of being restored to uppercase partition fields, which leads to this problem.

Because the catalog of conf.metastorePartitionPruning=false has partition field names in uppercase, there is no such problem.

spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/ExternalCatalogUtils.scala

Lines 163 to 173 in 319e7cc

	def listPartitionsByFilter(
	conf: SQLConf,
	catalog: SessionCatalog,
	table: CatalogTable,
	partitionFilters: Seq[Expression]): Seq[CatalogTablePartition] = {
	if (conf.metastorePartitionPruning) {
	catalog.listPartitionsByFilter(table.identifier, partitionFilters)
	} else {
	ExternalCatalogUtils.prunePartitionsByFilter(table, catalog.listPartitions(table.identifier),
	partitionFilters, conf.sessionLocalTimeZone)
	}