Code

pull changes to forked master#7

Code

Merged

ahshahid merged 30 commits intoahshahid:masterahshahid/spark:masterfrom apache:masterapache/spark:masterCopy head branch name to clipboardNov 23, 2020

+2,127-576

Commits

Commits on Nov 19, 2020

• [SPARK-33045][SQL][FOLLOWUP] Fix build failure with Scala 2.13

  Show description for 6da8ade
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  6da8ade

  Copy full SHA for 6da8ade

• [MINOR] Structured Streaming statistics page indent fix

  Show description for 883a213
  

  gaborgsomogyi

  authored and

  dongjoon-hyun

  committed
  883a213

  Copy full SHA for 883a213

Commits on Nov 20, 2020

• [MINOR][DOCS] Document 'without' value for HADOOP_VERSION in pip installation

  Show description for 02d410a
  HyukjinKwon

  committed
  02d410a

  Copy full SHA for 02d410a

• [SPARK-32919][SHUFFLE][TEST-MAVEN][TEST-HADOOP2.7] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

  Show description for 8218b48
  

  2 peopleauthored and

  Mridul Muralidharan

  committed
  8218b48

  Copy full SHA for 8218b48

• [SPARK-33441][BUILD][FOLLOWUP] Make unused-imports check for SBT specific

  Show description for 2289389
  

  LuciferYang

  authored and

  HyukjinKwon

  committed
  2289389

  Copy full SHA for 2289389

• [SPARK-32512][SQL][TESTS][FOLLOWUP] Remove duplicate tests for ALTER TABLE .. PARTITIONS from DataSourceV2SQLSuite

  Show description for 870d409
  

  MaxGekk

  authored and

  cloud-fan

  committed
  870d409

  Copy full SHA for 870d409

• [SPARK-33422][DOC] Fix the correct display of left menu item

  Show description for cbc8be2
  

  liucht-inspur

  authored and

  HyukjinKwon

  committed
  cbc8be2

  Copy full SHA for cbc8be2

• [SPARK-33468][SQL] ParseUrl in ANSI mode should fail if input string is not a valid url

  Show description for 3384bda
  

  ulysses-you

  authored and

  cloud-fan

  committed
  3384bda

  Copy full SHA for 3384bda

• [SPARK-28704][SQL][TEST] Add back Skiped HiveExternalCatalogVersionsSuite in HiveSparkSubmitSuite at JDK9+

  Show description for 47326ac
  

  AngersZhuuuu

  authored and

  dongjoon-hyun

  committed
  47326ac

  Copy full SHA for 47326ac

• [SPARK-33466][ML][PYTHON] Imputer support mode(most_frequent) strategy

  Show description for 116b7b7
  

  zhengruifeng

  authored and

  srowen

  committed
  116b7b7

  Copy full SHA for 116b7b7

• [MINOR][TESTS][DOCS] Use fully-qualified class name in docker integration test

  Show description for a1a3d5c
  

  huaxingao

  authored and

  dongjoon-hyun

  committed
  a1a3d5c

  Copy full SHA for a1a3d5c

• [SPARK-33492][SQL] DSv2: Append/Overwrite/ReplaceTable should invalidate cache

  Show description for 2479778
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  2479778

  Copy full SHA for 2479778

• [SPARK-32670][SQL] Group exception messages in Catalyst Analyzer in one file

  Show description for de0f50a
  

  anchovYu

  authored and

  HyukjinKwon

  committed
  de0f50a

  Copy full SHA for de0f50a

Commits on Nov 21, 2020

• [SPARK-33223][SS][FOLLOWUP] Clarify the meaning of "number of rows dropped by watermark" in SS UI page

  Show description for 67c6ed9
  HeartSaVioR

  committed
  67c6ed9

  Copy full SHA for 67c6ed9

• [SPARK-33505][SQL][TESTS] Fix adding new partitions by INSERT INTO InMemoryPartitionTable

  Show description for 530c0a8
  

  MaxGekk

  authored and

  dongjoon-hyun

  committed
  530c0a8

  Copy full SHA for 530c0a8

• [SPARK-32381][CORE][FOLLOWUP][TEST-HADOOP2.7] Don't remove SerializableFileStatus and SerializableBlockLocation for Hadoop 2.7

  Show description for b623c03
  

  sunchao

  authored and

  dongjoon-hyun

  committed
  b623c03

  Copy full SHA for b623c03

• Revert "[SPARK-28704][SQL][TEST] Add back Skiped HiveExternalCatalogVersionsSuite in HiveSparkSubmitSuite at JDK9+"

  Show description for cf74901
  dongjoon-hyun

  committed
  cf74901

  Copy full SHA for cf74901

• [SPARK-33463][SQL] Keep Job Id during incremental collect in Spark Thrift Server

  Show description for 517b810
  

  gumartinm

  authored and

  dongjoon-hyun

  committed
  517b810

  Copy full SHA for 517b810

Commits on Nov 22, 2020

• [SPARK-28704][SQL][TEST] Add back Skiped HiveExternalCatalogVersionsSuite in HiveSparkSubmitSuite at JDK9+

  Show description for d7f4b2a
  

  AngersZhuuuu

  authored and

  dongjoon-hyun

  committed
  d7f4b2a

  Copy full SHA for d7f4b2a

• [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

  Show description for d338af3
  

  cchighman

  and

  HeartSaVioR

  committed
  d338af3

  Copy full SHA for d338af3

• [SPARK-33469][SQL] Add current_timezone function

  Show description for 6d625cc
  

  ulysses-you

  authored and

  dongjoon-hyun

  committed
  6d625cc

  Copy full SHA for 6d625cc

Commits on Nov 23, 2020

• [SPARK-33512][BUILD] Upgrade test libraries

  Show description for df4a1c2
  dongjoon-hyun

  committed
  df4a1c2

  Copy full SHA for df4a1c2

• [MINOR][INFRA] Suppress warning in check-license

  Show description for a459238
  

  williamhyun

  authored and

  HyukjinKwon

  committed
  a459238

  Copy full SHA for a459238

• [SPARK-33427][SQL][FOLLOWUP] Put key and value into IdentityHashMap sequantially

  Show description for aa78c05
  

  viirya

  authored and

  HyukjinKwon

  committed
  aa78c05

  Copy full SHA for aa78c05

• [SPARK-33143][PYTHON] Add configurable timeout to python server and client

  Show description for 0bb911d
  

  gaborgsomogyi

  and

  HyukjinKwon

  committed
  0bb911d

  Copy full SHA for 0bb911d

• [SPARK-33510][BUILD] Update SBT to 1.4.4

  Show description for 84e7036
  

  williamhyun

  authored and

  dongjoon-hyun

  committed
  84e7036

  Copy full SHA for 84e7036

• Revert "[SPARK-32481][CORE][SQL] Support truncate table to move data to trash"

  Show description for c891e02
  

  gatorsmile

  authored and

  HyukjinKwon

  committed
  c891e02

  Copy full SHA for c891e02

• [SPARK-33515][SQL] Improve exception messages while handling UnresolvedTable

  Show description for 60f3a73
  

  imback82

  authored and

  cloud-fan

  committed
  60f3a73

  Copy full SHA for 60f3a73

• [SPARK-33511][SQL] Respect case sensitivity while resolving V2 partition specs

  Show description for 23e9920
  

  MaxGekk

  authored and

  cloud-fan

  committed
  23e9920

  Copy full SHA for 23e9920

• [SPARK-33278][SQL][FOLLOWUP] Improve OptimizeWindowFunctions to avoid transfer first to nth_value

  Show description for f83fcb1
  

  beliefer

  authored and

  cloud-fan

  committed
  f83fcb1

  Copy full SHA for f83fcb1