-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathpytest_final.log
112 lines (99 loc) · 9.53 KB
/
pytest_final.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
C:\Users\admin\.conda\envs\main\python.exe "C:/Program Files/JetBrains/PyCharm 2023.1.1/plugins/python/helpers/pycharm/_jb_pytest_runner.py" --path C:\datadev\work\perbak\src\tests -- --jb-show-summary
Testing started at 23:16 ...
Launching pytest with arguments C:\datadev\work\perbak\src\tests in C:\datadev\work\perbak\src\tests
============================= test session starts =============================
platform win32 -- Python 3.10.9, pytest-7.1.2, pluggy-1.0.0 -- C:\Users\admin\.conda\envs\main\python.exe
cachedir: .pytest_cache
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: C:\datadev\work\perbak\src\tests
plugins: anyio-3.6.2, benchmark-3.4.1
collecting ... collected 13 items
test_p1funsOnMainData.py::test_initiate PASSED [ 7%]random data (mkt and stock mins): -0.029736141768715967 -0.0647430743060547
fixed data (mkt and stock mins): -0.028345506444168257 -0.06909354203029808
test_p1funsOnMainData.py::test_conversion[fixed_data]
test_p1funsOnMainData.py::test_conversion[rndm_data]
test_p1funsOnMainData.py::test_overflow[fixed_data] PASSED [ 15%]PASSED [ 23%]
test_p1funsOnMainData.py::test_overflow[rndm_data]
test_p2_p4_corr.py::test_corr_pdres[fixed_df] PASSED [ 30%]PASSED [ 38%]
test_p2_p4_corr.py::test_corr_pdres[rndm_df]
test_p2_p4_corr.py::test_corr_pdnans[fixed_df-100] PASSED [ 46%]PASSED [ 53%]PASSED [ 61%]
test_p2_p4_corr.py::test_corr_pdnans[rndm_df-100] PASSED [ 69%]
test_p2_p4_corr.py::test_corr_my_np_onecol PASSED [ 76%]
test_p2_p4_corr.py::test_corr_my_np_roll_upd PASSED [ 84%]
test_p2_p4_corr.py::test_corr_my_numba_roll_upd PASSED [ 92%]
test_p3_hd_corr_spa.py::test_corr_spa_res 23/06/01 23:19:12 WARN Shell: Did not find winutils.exe: {}
java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see https://wiki.apache.org/hadoop/WindowsProblems
at org.apache.hadoop.util.Shell.fileNotFoundException(Shell.java:548)
at org.apache.hadoop.util.Shell.getHadoopHomeDir(Shell.java:569)
at org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:592)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:689)
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
at org.apache.hadoop.conf.Configuration.getTimeDurationHelper(Configuration.java:1886)
at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1846)
at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1819)
at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
at org.apache.hadoop.util.ShutdownHookManager$HookEntry.<init>(ShutdownHookManager.java:207)
at org.apache.hadoop.util.ShutdownHookManager.addShutdownHook(ShutdownHookManager.java:304)
at org.apache.spark.util.SparkShutdownHookManager.install(ShutdownHookManager.scala:181)
at org.apache.spark.util.ShutdownHookManager$.shutdownHooks$lzycompute(ShutdownHookManager.scala:50)
at org.apache.spark.util.ShutdownHookManager$.shutdownHooks(ShutdownHookManager.scala:48)
at org.apache.spark.util.ShutdownHookManager$.addShutdownHook(ShutdownHookManager.scala:153)
at org.apache.spark.util.ShutdownHookManager$.<init>(ShutdownHookManager.scala:58)
at org.apache.spark.util.ShutdownHookManager$.<clinit>(ShutdownHookManager.scala)
at org.apache.spark.util.Utils$.createTempDir(Utils.scala:335)
at org.apache.spark.deploy.SparkSubmit.prepareSubmitEnvironment(SparkSubmit.scala:344)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:898)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset.
at org.apache.hadoop.util.Shell.checkHadoopHomeInner(Shell.java:468)
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:439)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:516)
... 22 more
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
23/06/01 23:19:13 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
23/06/01 23:19:13 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.
23/06/01 23:19:13 WARN Utils: Service 'SparkUI' could not bind on port 4041. Attempting port 4042.
23/06/01 23:19:13 WARN Utils: Service 'SparkUI' could not bind on port 4042. Attempting port 4043.
23/06/01 23:19:13 WARN Utils: Service 'SparkUI' could not bind on port 4043. Attempting port 4044.
PASSED [100%]23/06/01 23:19:25 WARN ProcfsMetricsGetter: Exception when trying to compute pagesize, as a result reporting of ProcessTree metrics is stopped
============================== warnings summary ===============================
test_p1funsOnMainData.py::test_conversion[fixed_data]
test_p1funsOnMainData.py::test_conversion[rndm_data]
test_p1funsOnMainData.py:35: PytestBenchmarkWarning: Benchmark fixture was not used at all in this test!
@pytest.mark.parametrize('data', ['fixed_data', 'rndm_data'])
test_p2_p4_corr.py::test_corr_pdres[fixed_df]
test_p2_p4_corr.py::test_corr_pdres[rndm_df]
test_p2_p4_corr.py:36: PytestBenchmarkWarning: Benchmark fixture was not used at all in this test!
@pytest.mark.parametrize('df', ['fixed_df', 'rndm_df'])
test_p3_hd_corr_spa.py: 14 warnings
C:\Users\admin\.conda\envs\main\lib\site-packages\pyspark\sql\pandas\utils.py:35: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
if LooseVersion(pandas.__version__) < LooseVersion(minimum_pandas_version):
test_p3_hd_corr_spa.py::test_corr_spa_res
C:\Users\admin\.conda\envs\main\lib\site-packages\pyspark\sql\pandas\conversion.py:371: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.
for column, series in pdf.iteritems():
test_p3_hd_corr_spa.py::test_corr_spa_res
C:\Users\admin\.conda\envs\main\lib\site-packages\pyspark\sql\pandas\conversion.py:194: FutureWarning: Passing unit-less datetime64 dtype to .astype is deprecated and will raise in a future version. Pass 'datetime64[ns]' instead
series = series.astype(t, copy=False)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
-------------------------------------------------------------------------------------------------------- benchmark: 6 tests --------------------------------------------------------------------------------------------------------
Name (time in us) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_corr_my_numba_roll_upd 21.2998 (1.0) 24.8002 (1.0) 22.2000 (1.0) 1.4647 (1.0) 21.6002 (1.0) 1.1000 (1.0) 1;1 45,044.9855 (1.0) 5 1
test_corr_my_np_roll_upd 15,234.9002 (715.26) 17,502.9002 (705.76) 15,590.8461 (702.29) 254.0253 (173.43) 15,572.3998 (720.94) 59.5505 (54.14) 2;9 64.1402 (0.00) 65 1
test_corr_spa_res 19,568.1998 (918.70) 98,054.3997 (>1000.0) 49,097.5332 (>1000.0) 42,697.6894 (>1000.0) 29,670.0001 (>1000.0) 58,864.6500 (>1000.0) 1;0 20.3676 (0.00) 3 1
test_corr_pdnans[fixed_df-100] 96,261.2000 (>1000.0) 98,871.2003 (>1000.0) 97,332.1364 (>1000.0) 902.9720 (616.50) 97,117.2997 (>1000.0) 1,584.9500 (>1000.0) 5;0 10.2741 (0.00) 11 1
test_corr_pdnans[rndm_df-100] 98,317.4001 (>1000.0) 100,692.8999 (>1000.0) 99,297.6909 (>1000.0) 663.1202 (452.74) 99,075.6000 (>1000.0) 800.6505 (727.86) 3;0 10.0707 (0.00) 11 1
test_corr_my_np_onecol 179,455.1001 (>1000.0) 181,290.0999 (>1000.0) 180,313.6833 (>1000.0) 672.0159 (458.82) 180,423.8001 (>1000.0) 966.3003 (878.45) 2;0 5.5459 (0.00) 6 1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
================= 13 passed, 20 warnings in 254.96s (0:04:14) =================
Process finished with exit code 0