[SPARK-45179][PYTHON] Increase Numpy minimum version to 1.21

[zhengruifeng]

What changes were proposed in this pull request?

Increase Numpy minimum version to 1.21

Why are the changes needed?

• according to the release history, Numpy 1.15 was released about 5 years ago, while the last maintenance release in 1.21 was released 1 year ago;
• with 1.21 as the minimum version, we can discard all version checking in PySpark;
• pandas==1.4.4 just depends on numpy>=1.21.0;

Does this PR introduce any user-facing change?

No

How was this patch tested?

search with ag

(spark_dev_310) ➜  spark git:(master) ag --py 'numpy\.__version' python
(spark_dev_310) ➜  spark git:(master)
(spark_dev_310) ➜  spark git:(master) ag --py 'np\.__version' python
python/pyspark/ml/image.py
231:        if LooseVersion(np.__version__) >= LooseVersion("1.9"):

python/pyspark/pandas/typedef/typehints.py
152:    if sys.version_info >= (3, 8) and LooseVersion(np.__version__) >= LooseVersion("1.21"):

python/pyspark/pandas/tests/test_typedef.py
365:            if sys.version_info >= (3, 8) and LooseVersion(np.__version__) >= LooseVersion("1.21"):

python/pyspark/pandas/tests/computation/test_apply_func.py
257:        if sys.version_info >= (3, 8) and LooseVersion(np.__version__) >= LooseVersion("1.21"):

Was this patch authored or co-authored using generative AI tooling?

no

[zhengruifeng]

thanks, merged to master