Skip to content

Ska Overview

Jump to bottom Edit New page
Tom Aldcroft edited this page Jan 21, 2025 · 4 revisions

Ska 101

The Ska runtime environment is an integrated suite of Python and Perl packages and associated applications. This environment is used in many facets of Chandra operations:

  • Mission critical functions:
    • Mission planning
    • Load review
    • SOT MTA (Monitoring, trends, alerts)
    • Find attitude from full-field search stars
  • SOT and FOT spacecraft and science instrument analysis
  • SOT Aspect / ACA
    • Trending
    • Operations tools (find attitude, ACA image viewer)

Ska is supported on linux-64, Windows-64, and MacOS ARM-64. The SOT Ska team has primary responsibility for the management, release, and deployment of the Ska runtime environment:

  • Flight Ska (linux) maintained on HEAD and GRETA (chimchim and cheru) by the Ska team.
  • MATLAB Ska (Windows) created by Ska team and is included in the MATLAB tools by the FOT.
  • Standalone installations (linux, Mac, Windows) are managed by users.

Managing Ska requires substantial resources. The Ska team release manager maintains numerous scripts and GitHub workflow actions for automated nightly integration testing of all packages (in development), automated package building on target architectures, documentation build and deployment, release documentation and deployment, and more.

Package development

Testing

  • Most packages have unit tests that cover functionality.
  • Changes require new unit tests or documented functional testing.
  • Each night, master or main branch of all Ska packages are deployed to a masters Ska test environment. All unit tests are run along with addition functional integration tests.
  • Results of integration testing are available in a web page app and failures reported in email.

What's in Ska?

At the highest level Ska includes the following:

  • Packages developed and maintained by Chandra operations (e.g. xija or kadi).
    • Known as the ska3-flight or ska3-matlab packages.
    • About 50 distinct packages.
    • Updated and released about 10 times per year.
  • Off-the-shelf Packages from the open-source community (e.g. numpy or matplotlib).
    • Known as the ska3-core packages.
    • Nearly 500 distinct packages.
    • Normally updated once per year, including an update of the major Python version.
  • Optional components:
    • Perl interpreter and various Perl packages.
    • "Non-FSDS" packages for processing, trending, etc. These get installed to the HEAD production environment but have different control authority.

Key flight packages

  • chandra_maneuver: Compute Chandra maneuver profile.
  • cheta: Fast (20 M points / s) telemetry archive, plus an interface to MAUDE telemetry.
    • Fun fact: cheta code was forked (copied) and adapted for use by JWST, with 30000+ MSIDs and higher data rates. Many mods and a fancy web telemetry viewer, but the basic architecture and code remains!
  • cxotime: Represent times with CxoTime class and convert between formats.
  • xija: Thermal modeling infrastructure.
  • proseco: ACA star catalog selection.
  • sparkles: ACA star catalog review.
  • kadi:
    • events: Key events from telemetry or iFOT such as maneuvers, momentum dumps, SIM moves, Safe Modes, CAPs, etc.
    • commands: Archive of most commands run by Chandra since 2002.
    • commands.states: Spacecraft states such as SIM position from commands.
    • occweb: Read OCCweb pages or list directories.
  • parse_cm: Parse load product files (backstop, DOT, maneuver summary)
  • ska_sun: Sun ephemeris.
  • Quaternion: Represent and manipulate quaternions.

Key core packages

  • astropy: Tables, time, coordinates, I/O (CSV, FITS, HDF5).
  • beautifulsoup4: HTML parsing.
  • ipython: Interactive Python.
  • jplephem: Planets and other JPL ephemeris handling.
  • jupyter: Jupyter notebooks (with jupyterlab as well).
  • matplotlib: Plotting (most common).
  • numba: Make Python code run at C speed.
  • numexpr: Make Python expressions run faster.
  • numpy: Core numerical library for Python.
  • pandas: Provide data structures for working with labeled data.
  • plotly: Make interactive HTML plots.
  • pyqt: GUI applications.
  • requests: HTTP requests for humans.
  • scikit-learn: Machine learning in Python.
  • scipy: Fundamental algorithms for scientific Python.
  • sqlalchemy: Python SQL toolkit and Object Relational Mapper.

Generally useful flight packages

  • agasc: ACA star catalog access and maintenance.
  • chandra_aca: Collection of sub-packages related to ACA - focus on functions.
    • planets: Get planet positions using JPL DE432s or Horizons.
  • chandra_limits: Chandra planning limits management.
  • maude: Low-level interface to MAUDE telemetry server.
  • mica: Collection of sub-packages related to ACA.
  • ska_dbi: Access databases.
  • ska_ftp: FTP access.
  • ska_helpers: Common utilities for Ska packages. Most generally useful:
    • chandra_models: Access different versions of chandra_models
    • logging: Wrapper around Python logging to make it easier to use.
  • ska_matplotlib: Make time-based plot using CXC format for dates.
  • ska_numpy: Performant functions for filtering and smoothing numpy arrays or record arrays.
  • ska_tdb: Access and search the Telemetry Database (TDB).
  • testr: Utilities related to unit testing including some handy helpers.

Domain specific flight packages

  • aca_view: ACA image viewer application (real-time and archival).
  • acis_taco: Compute Earth illumination on ACIS radiator.
  • acis_thermal_check: Thermal load review for ACIS and HRC.
  • acispy: Utilities for ACIS.
  • backstop_history: ACIS load review backstop history.
  • fot-matlab: Python interface scripts for MATLAB tools.
  • ska_arc5gl: Access to CXC archive.
  • ska_sync: Sync Ska data to another machine.
  • starcheck: ACA load review.

Legacy - Do NOT use in new code

  • chandra_time: Time package providing DateTime class.
  • hopper: Orphaned package still used for ACA load review.
  • pyyaks: Logging and context-dependent variables.
  • ska-sphinx-theme: Documenation theme.
  • ska_astro: Astro functions (?)
  • ska_file: File functions. (Note: contextlib.chdir() since Python 3.11).
  • ska_parsecm: Original version of parse_cm, deprecated now.
  • ska_path: Ska-specific path library that fizzled.
  • ska_shell: Run system commands - mostly replaced by subprocess.
  • ska_quatutil: Functions that are now available in Quaternion or chandra_aca.

Getting help

Documentation

Most flight Ska packages have online documentation that can be found at: https://sot.github.io/.

Updates to cheta and kadi docs

Documentation for two key packages have been recently overhauled for both content and theme. The goal is making the content more focused and accessible.

Community support

Writing good documentation is difficult and time-consuming. If you have a question that is not quickly answered by the online docs then ASK THE COMMUNITY!!.

The best approach is to ask on Slack in the #analysis-coding channel:

  • Do not spend too much time struggling to figure something out.
  • No question is too basic.
  • Community "experts" genuinely enjoy answering code questions.
  • If you are starting to write some basic utility, it may have already been written. Ask first!

Asking questions also helps us focus energy where needed on improving the docs.

Use the Source, Luke

All the source code for Ska flight packages is available on GitHub at https://github.com/sot. Caveat: Some packages (e.g. parse_cm, maude, chandra_maneuver, chandra_limits) are private and visible only to the 20 sot organization members.

This can be useful:

  • Navigate to https://github.com/sot and then Search for a function.
  • This should show both the function definition and examples of other code that uses it.

Ska data latency

Ska depends on data products that are maintained largely via automated cron jobs that run on HEAD linux machines. These data are then synced to GRETA (chimchim) on a daily basis. Here we describe the timing of key cron jobs.

Cheta telemetry archive

Cron processing to ingest CXC L0 telemetry starts at 4am local each morning and generally finished by 6:30am. Typically, data from all previous comms will be available. Processing finishes by updating the cheta sync archive which is used for maintaining a local copy of the cheta archive.

When there is safe mode or other issue that significantly impacts the clock correlation, then CXCDS processing waits for a new clock correlation. This can take several days, so during that time the CXC L0 telemetry processing halts and cheta telemetry stops getting updated.

Kadi

Events

Kadi events depend on telemetry. Processing starts at 7am and finishes in a minute or two.

COMING SOON!! -- kadi events will be updated on HEAD and GRETA within an hour of data being available in MAUDE: https://github.com/sot/kadi/pull/343

Commands

The Kadi commands archive files are udpated on HEAD every 10 minutes. However, the architecture of kadi commands is that commanding within the last 30 days is dynamically generated based on the Command Events sheet, so it should always be up to date.

Syncing from HEAD to GRETA

Ska data are synced from HEAD to GRETA each morning at 7:30am local. This finishes within 15 minutes.

How to use Ska

HEAD or GRETA linux

This is simple since everything is pre-installed:

source /proj/sot/ska3/flight/bin/ska_envs.csh

It is highly recommended that you create a ska3 alias to also update your prompt to indicate that Ska3 is active.

You can quickly check that things are working in the Testing Ska3 section.

Remote access

You can use Ska on your laptop via a remote SSH connection to chimchim or a HEAD linux machine with a VS code remote SSH session.

  • VS code provides a terminal and Jupyter notebook interface.
  • Feels like editing and running in your native OS, but with full access to Ska.
  • FOT MP are successfully using this approach.

Mac or linux laptop

The user guide for Ska gives details for using, installing, and maintaining the Ska runtime environment.

FOT Windows laptop

Ska is bundled in the FOT MATLAB tools and is used extensively from the MATLAB code. Using that bundled Ska directly is possible with the right incantations.

Questions you might have

Ska3 vs Ska

  • You often see "Ska3", which refers to current Ska with Python 3.
  • Legacy Ska using Python 2 installed on HEAD and chimchim for old production scripts.
  • For common use, "Ska" is the same as "Ska3".

ska_matplotlib vs Ska.Matplotlib (package names)

  • In older code you will see package names like Ska.Matplotlib or Chandra.Time.
  • These "namespace" packages (with the dot) are a legacy of Perl conventions.
  • They cause all sorts of problems.
  • We have migrated to all lower case with underscores for new code.
  • Special case: Ska.engarchive => cheta. ALWAYS use cheta for new code.

CxoTime vs DateTime

  • Both classes provide time representation.
  • DateTime is legacy, based on old C++ code that is not actively maintained.
  • CxoTime is faster, better, and maintained. ALWAYS use this.
  • Simply substituting DateTime -> CxoTime in older code will generally work, with some caveats.

Astropy Table vs pandas DataFrame

  • Ska flight packages use only astropy Table for tables -> consistency.
  • Astropy Table supports several key features that are not in DataFrame:
    • Multi-dimensional column data, e.g. to store 8x8 pixel ACA images in a table.
    • Metadata in columns and table.
    • Native support for CxoTime as a table column.
  • Transforming to DataFrame is easy with tbl.to_pandas().

Cheta vs MAUDE

These two telemetry servers are complementary.

Cheta MAUDE
On-disk data (~225 Gb circa 2025) Web server backed by data archvie (~Tb's)
Back-orbit only, updated daily Realtime + back-orbit soon after each comm
Fetch size limited only by memory 200,000 points or 3.2MB, per MSID query
20M points/sec ~200k points/sec
Python-only Python, Javascript (any language really)
State codes mostly match TDB (3TSCMOV) State codes strictly match TDB
Trailing spaces like "ON ", "OFF" No trailing spaces "ON", "OFF"
Includes certain SI telemetry (ACIS FP, HRC SS) Does not include SI telemetry
Time stamp at beginning of sample interval Time stamp at beginning of minor frame
Dwell mode data ?? Dwell mode data supported

You can access MAUDE from cheta. Cheta can even automatically stitch together data from the fast on-archive plus fill in the most recent available data from MAUDE.

kadi events vs kadi commands vs kadi states:

Kadi events are from telemetry and web resources:

  • Events in telemetry such as maneuvers, NPM dwells, obsids, mech movements, momentum dumps, orbit events, and so forth.
  • Events from web resources, for instance CAPs, DSN passes, Load segments, and Chandra major events.

Kadi commands:

  • Every load command run on-board or approved to run, with a link to source load products.
  • Non-load commands which result from either autonomous on-board commanding (e.g. SCS-107) or real-time ground commanding (e.g. anomaly recovery).
  • Includes predictive commands (expected to happen).
  • Kadi commands is always fresh by using a combination of data:
    • Archive HDF5 files (synced from HEAD/chimchim at least once / 4 weeks)
    • Live web-accessible resources (Command Events Google sheet + OCCweb for loads)
    • Command Events sheet maintained by FOT MP and normally kept up to date in a timely manner.

Kadi states:

  • State(s) of Chandra derived from kadi commands.
  • Examples include Sun pitch angle, SIM position, ACIS configuration.
  • Also predictive.

Cool things you can do with Ska

Cheta quaternion comps

This defines the following computed MSIDs based on the corresponding TDB MSIDs:

  • quat_aoattqt = AOATTQT[1-4]
  • quat_aoatupq = AOATUPQ[1-3]
  • quat_aocmdqt = AOCMDQT[1-3]
  • quat_aotarqt = AOTARQT[1-3]

Example:

>>> from cheta import fetch
>>> qatt = fetch.Msid('quat_aoattqt', '2022:001:00:00:00', '2022:001:00:00:04')
>>> qatt.vals
Quat(array([[-0.07434856, -0.55918674, -0.80432653,  0.18665828],
            [-0.07434854, -0.55918679, -0.8043265 ,  0.18665825],
            [-0.07434849, -0.55918674, -0.80432653,  0.18665829],
            [-0.07434849, -0.55918667, -0.80432653,  0.18665852]]))
>>> qatt.vals.equatorial
array([[193.28905806,  19.16894296,  67.36207683],
        [193.28905485,  19.1689407 ,  67.36208471],
        [193.28906329,  19.16893787,  67.36207699],
        [193.28908839,  19.16895134,  67.36206404]])

Make your own kadi states and continuity

import kadi.commands.states as kcs

class ERObsidTransition(kcs.ObsidTransition):
    state_keys = ["er_obsid"]
    transition_key = "er_obsid"

    @classmethod
    def get_state_changing_commands(cls, cmds):
        cmds = super().get_state_changing_commands(cmds)
        ok = (cmds["id"] >= 38000) & (cmds["id"] < 60000)
        return cmds[ok]

safing_time = CxoTime("2025:012:14:37:04.000")
er_continuity = kcs.get_continuity(safing_time, state_keys=["er_obsid"])
print(f"continuity er_obsid = {er_continuity['er_obsid']}")
print()

states = kcs.get_states("2025:010", "2025:016", state_keys=["obsid", "er_obsid"])
states["datestart", "obsid", "er_obsid", "trans_keys"].pprint_all()

This outputs the following:

continuity er_obsid = 42780

      datestart       obsid er_obsid   trans_keys
--------------------- ----- -------- --------------
2025:010:00:00:00.000 30716    42788
2025:010:01:20:02.635 30725    42788          obsid
2025:010:04:44:25.700 30717    42788          obsid
2025:010:08:08:48.765 30726    42788          obsid
2025:010:11:34:02.363 42787    42787 er_obsid,obsid
2025:010:12:48:32.952 42786    42786 er_obsid,obsid
2025:010:14:16:45.094 42785    42785 er_obsid,obsid
...
2025:010:19:00:30.000 42782    42782 er_obsid,obsid
2025:010:19:44:14.532 42781    42781 er_obsid,obsid
2025:010:22:17:39.791 42780    42780 er_obsid,obsid
2025:011:00:05:43.510 30712    42780          obsid
...
2025:015:15:20:15.528 29963    42780          obsid
2025:015:18:34:22.826 42779    42779 er_obsid,obsid
2025:015:19:52:40.485 42778    42778 er_obsid,obsid

Grab stuff from OCCweb

The kadi.occweb module includes functions for reading OCCweb directories and pages, as well as iFOT tables.

>>> from kadi.occweb import get_ifot, get_occweb_dir, get_occweb_page

>>> get_ifot("ECLIPSE", "2025:001", "2025:030")
<Table length=6>
linenum   id   type_desc         tstart
  str1  int64     str7           str21
------- ------ --------- ---------------------
        947559   Eclipse 2025:016:01:42:50.818
        947560   Eclipse 2025:018:17:10:06.238
        947561   Eclipse 2025:021:08:37:22.600
        947562   Eclipse 2025:024:00:04:42.099
        947563   Eclipse 2025:026:15:32:28.871
        947564   Eclipse 2025:029:07:00:36.082

We can get the directory contents of the approved loads in January 2025:

>>> get_occweb_dir("FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JAN")
<Table length=6>
      Name        Last modified   Size
     str16            str16       str1
---------------- ---------------- ----
Parent Directory               --    -
       JAN0325A/ 2025-01-02 14:42    -
       JAN1325A/ 2025-01-10 11:15    -
       JAN1425A/ 2025-01-13 14:52    -
       JAN1525A/ 2025-01-14 11:47    -
       JAN2025A/ 2025-01-17 10:15    -

Now let's read the JAN1425A backstop file into a kadi commands table:

>>> from kadi.commands import read_backstop
>>> backstop_file = "FOT/mission_planning/PRODUCTS/APPR_LOADS/2025/JAN/JAN1425A/CR014_0305.backstop"
>>> backstop_text = get_occweb_page(backstop_file)
>>> cmds = read_backstop(backstop_text.splitlines())

We can print the first 5 commands in a format that looks like backstop:

>>> cmds[:5].pprint_like_backstop()
2025:014:02:55:00.000 | LOAD_EVENT       | None       |        0 | event_type=RUNNING_LOAD_TERMINATION_TIME, scs=0
2025:014:03:02:00.000 | ACISPKT          | AA00000000 |        0 | cmds=3, words=3, scs=133
2025:014:03:02:03.000 | ACISPKT          | AA00000000 |        0 | cmds=3, words=3, scs=133
2025:014:03:02:33.000 | COMMAND_SW       | COENASX    |        0 | hex=8448600, msid=COENASX, coenas1=134 , scs=133
2025:014:03:02:33.000 | COMMAND_SW       | CODISASX   |        0 | hex=8458700, msid=CODISASX, codisas1=135 , scs=133
Add a custom footer

For Users

For Maintainers

Continuous Integration

Old Pages

Clone this wiki locally