Slack Test Results Notification

Send Slack message with test results from popular testing frameworks

A Slack test reporting tool that supports all major testing frameworks. Generate, publish and alert your team with detailed test results, including summaries, in-depth reports, failed test analyses, flaky test detection and AI analyses directly to your chosen Slack channel.

💚

CTRF tooling is open source and free to use

You can support the project with a follow and a star

Maintained by Matthew Thomas
Contributions are very welcome!
Explore more integrations

Features

• Send Test Results to Slack: Automatically send test results to a Slack channel.
• Send Flaky Test Details to Slack: Automatically send flaky test details to a Slack channel.
• Send AI Test Summary to Slack: Automatically send AI test summary to a Slack channel.
• Tagging: Tag users, channels and groups in the message.
• Conditional Notifications: Use the --onFailOnly option to send notifications only if tests fail.

Setup

You'll need a CTRF report generated by your testing framework. CTRF reporters are available for most testing frameworks and easy to install.

No CTRF reporter? No problem!

Use junit-to-ctrf to convert a JUnit report to CTRF.

Create a Slack Incoming Webhook

1. Go to the Slack API: Incoming Webhooks page.
2. Click on "Create a Slack App" or use an existing app.
3. Add the "Incoming Webhooks" feature to your app.
4. Activate the Incoming Webhook and add a new webhook to your workspace.
5. Copy the webhook URL provided.

Set the Environment Variable

Set the webhook URL as an environment variable in your shell or CI environment:

export SLACK_WEBHOOK_URL='https://hooks.slack.com/services/your/webhook/url'

Make sure to replace 'https://hooks.slack.com/services/your/webhook/url' with your actual webhook URL.

You might want to store the webhook URL as a secret.

Usage

You can use a glob pattern or a single file path to send the test results summary to Slack.

Results

To send the test results summary to Slack:

npx slack-ctrf results /path/to/ctrf-report.json

You can use a glob pattern with multiple files which will be merged together:

npx slack-ctrf results "ctrf/*.json"

AI Summary

To send AI failed test summary to Slack:

npx slack-ctrf ai /path/to/ctrf-report.json

Use --consolidated argumement to send a single message with all AI summaries

npx slack-ctrf ai /path/to/ctrf-report.json --consolidated

See the AI Test Reporter to add AI summaries to your CTRF report

Flaky

To send flaky test report to Slack:

npx slack-ctrf flaky /path/to/ctrf-report.json

Failed

To send failed test report to Slack:

npx slack-ctrf failed /path/to/ctrf-report.json

Use --consolidated argumement to send a single message with all AI summaries

npx slack-ctrf failed /path/to/ctrf-report.json --consolidated

Send Only on Failures

To send the test results summary to Slack only if there are failed tests, use the --onFailOnly option:

npx slack-ctrf results /path/to/ctrf-file.json --onFailOnly

or using the alias:

npx slack-ctrf results /path/to/ctrf-file.json -f

Custom Notification Title

You can choose a custom title for your notification, use the --title option:

npx slack-ctrf results /path/to/ctrf-file.json --title "Custom Title"

or using the alias:

npx slack-ctrf results /path/to/ctrf-file.json -t "Custom Title"

Add Prefix and Suffix

Add custom markdown text to as a prefix, or suffix to the message:

npx slack-ctrf results /path/to/ctrf-file.json -p "prefix" -s "suffix"

Read about compatible markdown

Tagging a user, channel or group

You can tag users, channels or groups in the message by using the --prefix or --suffix options.

You need to a Slack user ID, channel ID or group ID.

You can find your user ID by:

• Clicking on your profile picture in Slack
• Clicking the three dots (...)
• Selecting "Copy member ID"

Once you have your user ID (which will look something like U01ABC123DE), use that in your command instead of your display name:

npx slack-ctrf results /path/to/ctrf-file.json -s "<@U0123456789> please review the results"

You can also tag a channel by using the # symbol with the channel ID:

npx slack-ctrf results /path/to/ctrf-file.json -s "<#C0123456789> please review the results"

And finally, you can tag a group by using the \!subteam^ symbol with the group ID:

npx slack-ctrf results /path/to/ctrf-file.json -s "<\!subteam^0123456789> please review the results"

Options

• --onFailOnly, -f: Send notification only if there are failed tests.
• --title, -t: Title of the notification.
• --prefix, -p: Custom markdown text to add as a prefix to the message
• --suffix, -p: Custom markdown text to add as a prefix to the message

Merge reports

You can merge reports if your chosen reporter generates multiple reports through design, parallelisation or otherwise.

If you use a glob pattern, the reports will be merged automatically.

Otherwise, the ctrf-cli package provides a method to merge multiple ctrf json files into a single file.

After executing your tests, use the following command:

npx ctrf merge <directory>

Replace directory with the path to the directory containing the CTRF reports you want to merge.

Programmatic Usage

You can use the package programmatically to send notifications to Slack. To install the package, run:

npm install slack-ctrf

The package exports the following functions:

• sendTestResultsToSlack
• sendFlakyResultsToSlack
• sendFailedResultsToSlack
• sendAISummaryToSlack

import { sendTestResultsToSlack } from 'slack-ctrf';

sendTestResultsToSlack(report);

What is CTRF?

CTRF is a universal JSON test report schema that addresses the lack of a standardized format for JSON test reports.

Consistency Across Tools: Different testing tools and frameworks often produce reports in varied formats. CTRF ensures a uniform structure, making it easier to understand and compare reports, regardless of the testing tool used.

Language and Framework Agnostic: It provides a universal reporting schema that works seamlessly with any programming language and testing framework.

Facilitates Better Analysis: With a standardized format, programatically analyzing test outcomes across multiple platforms becomes more straightforward.

Support Us

If you find this project useful, consider giving it a GitHub star ⭐ It means a lot to us.