Skip to content

Ska3 (2025.1 with Python 3.12) testing

Jump to bottom Edit New page
Jean Connelly edited this page Jan 30, 2025 · 16 revisions

Summary

This document describes testing plans and results for the transition to 2025.1, with Python 3.12 and updates to most of the core packages such as NumPy and Astropy.

Core functionality (Ska3 team responsibility)

Core testing is done via the Ska3 integration testing package which consists of running available package unit tests and regression test scripts using machinery in the testr package. This is done with the run_testr command, where some tests may be skipped on standalone platforms.

Current results for 2025.1

See the test outputs are shown in:

Load Review Checks

JC confirmed interactive plotting with macosx backend on OSX arm64 and tkagg backend on Linux

Ska3 cron processing

acdc

Date: 2025-Jan-23

Run with

c3po-v$ cd /tmp
c3po-v$ mkdir -p acdc/regress_twelve/VC2
# Get enough data to make the last two reports
c3po-v$ cp /dsops/GOT/input/2024*VC2* acdc/regress_twelve/VC2
c3po-v$ cp /dsops/GOT/input/2025*VC2* acdc/regress_twelve/VC2
c3po-v$ acdc_process_vc2_to_l0 --vc2-root=acdc/regress_twelve/VC2 --data-root=acdc/regress_twelve
c3po-v$ acdc_process_l0_to_quads --data-root=acdc/regress_twelve --start 2024:301
# At this step manually edit vc2_index.dat to just have entries after 2024:301
# and mark any cals FAIL to match flight
c3po-v$ acdc_process_quads_to_cals --data-root=acdc/regress_twelve
c3po-v$ acdc_make_reports --data-root=acdc/regress_twelve
mkdir /proj/sot/ska/www/ASPECT_ICXC/test_review_outputs/acdc/twelve
cp -Ruva acdc /proj/sot/ska/www/ASPECT_ICXC/test_review_outputs/acdc/twelve
cd /proj/sot/ska/www/ASPECT_ICXC/test_review_outputs/acdc/twelve
rm -R acdc/regress_twelve/VC2
rm -R acdc/regress_twelve/2024/
rm -R acdc/regress_twelve/2025/

Checks:

Note on comparing outputs:

The temperature values assigned to cals varied in this process by up to 0.08 C compared to flight. This is likely because flight processing is done soon after the cal and can be using available MAUDE telemetry from realtime. This will generate differences in temperatures and mask values because the temperatures are not available during the VC2 readout interval.

I'm not actually sure about the mask values with differences up to 3 values, but these appear benign.

Diffs:

For the 2024:345 cal the diffs are:


In [1]: from astropy.io import fits
In [2]: dat1 = fits.open('/proj/sot/ska/data/acdc/cals/flight/2024/345/cal_2024_345_10_58_21.fits.gz')[0].data
In [5]: dat2 = fits.open('/proj/sot/ska/www/ASPECT_ICXC/test_review_outputs/acdc/twelve/acdc/regress_twelve/cals/flight/2024/345/cal_2024_345_10_58_21.fits.gz')[0].data
In [6]: diff = dat1 - dat2
In [8]: import numpy as np
In [9]: argmax = np.argmax(np.abs(diff)).flatten()[0]

In [10]: print(
    ...:     f"Mean diff: {np.mean(diff)}",
    ...:     f"Diff std dev: {np.std(diff)}",
    ...:     f"Max diff: {np.max(np.abs(diff))} ({dat1.ravel()[argmax]}, {dat2.ravel()[argmax]})",
    ...: )
Mean diff: 0.003968886565417051 Diff std dev: 0.36400797963142395 Max diff: 26.3343505859375 (692.5942993164062, 718.9286499023438)

acis_taco

Date: 2025-Jan-23 JC

Reprocess current week (JAN2025.pkl) with Ska3-flight and 2025.1:

cd /tmp
mkdir acis_taco
cd acis_taco
mkdir flight
mkdir twelve
/proj/sot/ska3/flight/bin/skare /proj/sot/ska3/flight/share/acis_taco/make_esaview_data.py --nweeks=1 --data-root=flight
/proj/sot/ska3/test/bin/skare /proj/sot/ska3/test/share/acis_taco/make_esaview_data.py --nweeks=1 --data-root=twelve

Then from ipython

import pickle
import numpy as np                                                                                                 
dat1 = pickle.load(open('/tmp/acis_taco/flight/JAN2025.pkl', 'rb'))                                         
dat2 = pickle.load(open('/tmp/acis_taco/twelve/JAN2025.pkl', 'rb')) 
np.all(dat1['illums'] == dat2['illums'])  # True
np.all(dat1['times'] == dat2['times'])  # True

Result:

  • OK

arc

Date: 2025-Jan-23

Setup:

# 

rm /proj/sot/ska3/test/data/arc3
# Make a new dir for speedy testing
mkdir /proj/sot/ska3/test/data/arc3
# Copied the /proj/sot/ska/data/arc3 contents into that directory
rsync -aruvz /proj/sot/ska/data/arc3/* /proj/sot/ska3/test/data/arc3/

ln -s /proj/sot/ska/www/ASPECT/arc3_twelve_test /proj/sot/ska3/test/www/ASPECT/arc3

cd ~/git/arc
git checkout 4.9.0
source /proj/sot/ska3/test/bin/ska_envs.csh
make -n install # check paths 
make install

Then ran the task pieces by hand to look for errors at the console:

aca-c3po-v% /proj/sot/ska3/test/share/arc3/get_iFOT_events.pl
aca-c3po-v% /proj/sot/ska3/test/share/arc3/get_web_content.pl
aca-c3po-v% /proj/sot/ska3/test/share/arc3/get_goes_x.py --h5=/proj/sot/ska3/test/data/arc3/GOES_X.h5
aca-c3po-v% /proj/sot/ska3/test/share/arc3/plot_goes_x.py --h5=/proj/sot/ska3/test/data/arc3/GOES_X.h5 --out=/proj/sot/ska3/test/www/ASPECT/arc3/goes_x.png
Matplotlib is building the font cache; this may take a moment.
aca-c3po-v% /proj/sot/ska3/test/share/arc3/get_ace.py --h5=/proj/sot/ska3/test/data/arc3/ACE.h5
aca-c3po-v% /proj/sot/ska3/test/share/arc3/get_hrc.py --h5=/proj/sot/ska3/test/data/arc3/hrc_shield.h5 --data-dir=/proj/sot/ska3/test/data/arc3
aca-c3po-v% /proj/sot/ska3/test/share/arc3/plot_hrc.py --h5=/proj/sot/ska3/test/data/arc3/hrc_shield.h5 --out=/proj/sot/ska3/test/www/ASPECT/arc3/hrc_shield.png
aca-c3po-v% /proj/sot/ska3/test/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/test/data/arc3
aca-c3po-v% /proj/sot/ska3/test/share/arc3/arc.pl 
aca-c3po-v% /proj/sot/ska3/test/share/arc3/arc.pl -config arc3:arc_ops
aca-c3po-v% /proj/sot/ska3/test/share/arc3/arc_time_machine.pl

Examined output at https://cxc.cfa.harvard.edu/mta/ASPECT/arc3_twelve_test/ and checked:

  • HTML has right dates
  • snapshot content appears correct
  • plots match flight bye eye

cheta

Date: 2025-Jan-27

  • cheta_sync from 2025.1 ran with no errors.

Manual run of regression test script 2025-Jan-24 JC

jeanconn-fido> sh regression_test_eng_archive.sh 
Updating archive, do 'tail -f test/make_eng_archive.log' in another window
Fetch file is /proj/sot/ska3/test/lib/python3.12/site-packages/cheta/fetch.py
ENG_ARCHIVE is None
Getting 1crat
Getting fptemp_11
Getting orbitephem0_x
Getting sim_z
Getting tephin
Getting cvcductr
Getting dp_dpa_power
fetch: using ENG_ARCHIVE=/proj/sot/ska/jeanproj/git/cheta/test/eng_archive for archive path
Fetch file is /proj/sot/ska/jeanproj/git/cheta/test/../cheta/fetch.py
ENG_ARCHIVE is /proj/sot/ska/jeanproj/git/cheta/test/eng_archive
Getting 1crat
Getting fptemp_11
Getting orbitephem0_x
Getting sim_z
Getting tephin
Getting cvcductr
Getting dp_dpa_power
[OK] 1crat dat
[OK] 1crat dat
[OK] 1crat dat5
[OK] 1crat dat5
[OK] 1crat dat5
[OK] 1crat dat5
[OK] 1crat dat5
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] fptemp_11 dat
[OK] fptemp_11 dat
[OK] fptemp_11 dat5
[OK] fptemp_11 dat5
[OK] fptemp_11 dat5
[OK] fptemp_11 dat5
[OK] fptemp_11 dat5
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] orbitephem0_x dat
[OK] orbitephem0_x dat
[OK] sim_z dat
[OK] sim_z dat
[OK] sim_z dat5
[OK] sim_z dat5
[OK] sim_z dat5
[OK] sim_z dat5
[OK] sim_z dat5
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] tephin dat
[OK] tephin dat
[OK] tephin dat5
[OK] tephin dat5
[OK] tephin dat5
[OK] tephin dat5
[OK] tephin dat5
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] cvcductr dat
[OK] cvcductr dat
[OK] cvcductr dat5
[OK] cvcductr dat5
[OK] cvcductr dat5
[OK] cvcductr dat5
[OK] cvcductr dat5
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
All OK: True
jeanconn-fido> echo $SKA
/proj/sot/ska3/test
jeanconn-fido> which python
/proj/sot/ska3/test/bin/python

Regression and unit tests provide full coverage of package user functionality and daily cron updates.

kadi

No extra tests. Regression and unit tests provide full coverage of package user functionality and daily cron updates.

arc5gl

Date: 2025-Jan-23 JC

  • confirm cmdline works fine in test environment to fetch files

mica

No extra tests. Regression and unit tests provide full coverage of package user functionality and daily cron updates.

skawatch3

Date: 2025-Jan-23

  • jobwatch unit tests pass
  • skawatch.py and hourly_watch.py run without error in the test environment.
  • html outputs look reasonable to visual inspection
  • OK

task_schedule

Date: 2025-Jan-27

  • task_schedule3.pl ran without errors

I ran task schedule to run the test arc3 job and confirmed the test page (https://cxc.cfa.harvard.edu/mta/ASPECT/arc3_twelve_test/) updated and the logs looked reasonable.

jeanconn-fido> /proj/sot/ska3/test/bin/skare task_schedule3.pl -config arc3/task_schedule.cfg

xija

Date: 2025-Jan-30

xija_gui_fit at xija version 4.33.1 was confirmed to successfully start and run a fit.

xija_gui_fit ~/ska/data/chandra_models/chandra_models/xija/acisfp/acisfp_spec_matlab.json
WARNING: imaging routines will not be available, 
failed to import sherpa.image.ds9_backend due to 
'RuntimeErr: DS9Win unusable: Could not find ds9 on your PATH'
Fetching msid: fptemp_11 over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
INFO:numexpr.utils:NumExpr defaulting to 16 threads.
Fetching msid: 1cbat over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: pitch over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: roll over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: 1dahtbon over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: aoeclips over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: sim_z over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Getting kadi commanded states over 2025:005:00:02:46.816 to 2025:020:19:36:54.816
Fetching msid: dp_dpa_power over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: orbitephem0_x over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: orbitephem0_y over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: orbitephem0_z over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: aoattqt1 over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: aoattqt2 over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: aoattqt3 over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: aoattqt4 over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: solarephem0_x over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: solarephem0_y over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: solarephem0_z over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Fetching msid: 215pcast over 2025:004:23:35:26.816 to 2025:020:20:04:14.816
Adding plot  fptemp data__time
Adding plot  solarheat__sim_px solar_heat__pitch
WARNING: imaging routines will not be available, 
failed to import sherpa.image.ds9_backend due to 
'RuntimeErr: DS9Win unusable: Could not find ds9 on your PATH'
INFO:numexpr.utils:NumExpr defaulting to 16 threads.
INFO:fit:Fit started 
...
INFO:fit:Fit finished normally

*********************************
  FIT FINISHED
*********************************
Add a custom footer

For Users

For Maintainers

Continuous Integration

Old Pages

Clone this wiki locally