Skip to content

This is a python package that helps keep track of both Raw Binary and CDF Files in a Relational Database.

License

Notifications You must be signed in to change notification settings

HERMES-SOC/CDFTracker

Repository files navigation

CDFTracker

GitHub issues Build status codecov Commit activity License

This is a python package that helps keep track of both Raw Binary and CDF Files in a Relational Database.

Features

  • Track both Raw Binary and CDF Files
  • Supports multiple instrument configurations
  • Support for multiple RDBMS via SQLAlchemy connection strings

Requirements

  • Python 3.8 or higher
  • SQL Database (via connection string)
  • Make
  • Poetry (optional)

Installation and Usage

To install the library and its dependencies, follow these steps:

  1. Install with dependency manager of your choice:

    # Install with pip
    pip install git+https://github.com/HERMES-SOC/CDFTracker.git
    
    # Install with poetry
    poetry add git+https://github.com/HERMES-SOC/CDFTracker.git
  2. Create both the engine and the session, with your RDBMS of choice connection string. For example, to create a session with a SQLite database, you can do the following:

    from cdftracker.database import create_engine, create_session
    
    # Create a database engine and session
    engine = create_engine("sqlite:///test.db")
    session = create_session(engine)
  3. If this is your first time using the library, you will need to create the database tables. To do so, run the following command:

    from cdftracker.database.tables import set_up_tables
    
    # Set up tables
    set_up_tables(engine, session)
  4. Define a science file name parser function which parses the file Path object and returns the following information in a dictionary. This is the formart the dictionary outputted by the function should have:

    # def science_file_name_parser():
    #    return {
      #      "instrument": str,
      #      "mode": str,
      #      "test": bool,
      #      "time": str,
      #      "level": str,
      #      "version": str,
      #      "descriptor": str,
    #    }
    
    # For this example we will be using a predefined science file parser defined in the 
    # hermes_core package (https://github.com/HERMES-SOC/hermes_core.git).
    from hermes_core.util import util
    
    parser = util.parse_science_filename
  5. Now you can instantiate a CDFTracker object with the engine and science file parser function you defined:

    from cdftracker.tracker import tracker
    from pathlib import Path
    
    # Instantiate the tracker
    tracker = tracker.CDFTracker(engine, parser)
    
    # Path of the science file to be tracked (for example)
    file = Path("./hermes_MAG_l0_2022259-030002_v01.bin")
  6. You can also track the file which adds the appropriate entries to the database. To do so, run the following command:

    # Track the file
    tracker.track(file)

Database Schema

This is the database schema for the CDFTracker database. The database schema is defined in the cdftracker.database.tables module.

Contributing

How to set-up Development Environment

This project makes use of Poetry to manage dependencies and virtual environments. Also included is a Make file to set-up your development environment. To set-up your development environment, follow these steps:

  1. Clone the repository

    git clone https://github.com/HERMES-SOC/CDFTracker.git
    
    cd CDFTracker
  2. Set-up your development environment

    make install

How to run tests

This project uses pytest to run tests and exports an HTML report of the code coverage. To run tests, follow these steps:

  1. Inside the project directory, run the following command:

    make test

How to run linter

This project uses black and ruff to lint the code. To run the linter, follow these steps:

  1. Inside the project directory, run the following command:

    make check

License

This project is licensed under the terms of the MIT license.

About

This is a python package that helps keep track of both Raw Binary and CDF Files in a Relational Database.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published