Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added fastq glue services #899

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Added fastq glue services #899

wants to merge 3 commits into from

Conversation

alexiswl
Copy link
Member

@alexiswl alexiswl commented Mar 12, 2025

Just added readme so far

Fastq Glue

The fastq glue service is a simple tool that connects the fastq manager to the automation pipeline processes.

Starting BCLConvert

The first interaction the fastq glue service has with the automation is when the sequence run manager registers a samplesheet.

The fastq glue service then collects the samplesheet and generates fastq sets and fastq list rows for each sample in the samplesheet.

The fastq glue service then also creates a final event with 'FastqsCreated' with an instrument run id in the detail.

Workflow glue services can then query the fastq manager for any fastq sets of interest an 'initiate' the fastq process

Post BSSH Fastq Copy

Once the bssh fastq copy service has completed and the fastq data is now in our BYOB bucket, the fastq glue service then performs the following.

  1. Adds read sets to the fastq list rows by querying the fastq_list.csv in the instrument run id output directory.
  2. Triggers qc processes endpoint for each fastq list row in the instrument run.
  3. Triggers the file compression information endpoint for each fastq list row.
  4. Triggers the ntsm information endpoint for each fastq list row

Resolves #870

TODO

* SDK versions from 2.177 to 2.182
* Added vpc contexts to cdk.context.json
* Upgraded yarn version to 4.7.0
* Prefix prettier with yarn prettier (takes longer but prettier was breaking for me)
Updated toolkits (and typehints) for the following services:
* workflow manager
* metadata manager
* file manager
* fastq manager

Also updated the dynamodb partition table to use pointInTimeRecoverySpecification since pointInTimeRecovery parameter is deprecated
@alexiswl alexiswl self-assigned this Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add 'fastq-glue' service
1 participant