Skip to content

Commit

Permalink
Add support for Sentry monitoring
Browse files Browse the repository at this point in the history
Includes addition to documentation.
  • Loading branch information
pvannierop committed Oct 4, 2024
1 parent dfa8058 commit 0e5e0eb
Show file tree
Hide file tree
Showing 7 changed files with 173 additions and 8 deletions.
46 changes: 38 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
# Kafka Connect REST Source and Fitbit Source

This project contains a Kafka Connect source connector for a general REST API, and one for
Fitbit in particular. The documentation of the Kafka Connect REST source still needs to be done.
This project contains a Kafka Connect source connector for a general REST API, for
specific Fitbit and Oura devices. The documentation of the Kafka Connect REST source still needs to
be done.

<!-- TOC -->

* [Kafka Connect REST Source and Fitbit Source](#kafka-connect-rest-source-and-fitbit-source)
* [Fitbit source connector](#fitbit-source-connector)
* [Installation](#installation)
* [Usage](#usage)
* [Sentry monitoring](#sentry-monitoring)
* [Contributing](#contributing)

<!-- TOC -->

## Fitbit source connector

Expand All @@ -12,7 +24,9 @@ of Java 17 or later.

### Usage

Generally, this component is installed with [RADAR-Kubernetes](https://github.com/RADAR-base/RADAR-Kubernetes). It uses Docker image [radarbase/kafka-connect-rest-fitbit-source](https://hub.docker.com/r/radarbase/kafka-connect-rest-fitbit-source).
Generally, this component is installed
with [RADAR-Kubernetes](https://github.com/RADAR-base/RADAR-Kubernetes). It uses Docker
image [radarbase/kafka-connect-rest-fitbit-source](https://hub.docker.com/r/radarbase/kafka-connect-rest-fitbit-source).

First, [register a Fitbit App](https://dev.fitbit.com/apps) with Fitbit. It should be either a
server app, for multiple users, or a personal app for a single user. With the server app, you need
Expand All @@ -22,7 +36,8 @@ For every Fitbit user you want access to, copy `docker/fitbit-user.yml.template`
`docker/users/`. Get an access token and refresh token for the user using for example the
[Fitbit OAuth 2.0 tutorial page](https://dev.fitbit.com/apps/oauthinteractivetutorial).

For automatic configuration for multiple users, please take a look at `scripts/REDCAP-FITBIT-AUTH-AUTO/README.md`.
For automatic configuration for multiple users, please take a look at
`scripts/REDCAP-FITBIT-AUTH-AUTO/README.md`.

Copy `docker/source-fitbit.properties.template` to `docker/source-fitbit.properties` and enter
your Fitbit App client ID and client secret. The following tables shows the possible properties.
Expand Down Expand Up @@ -90,7 +105,8 @@ your Fitbit App client ID and client secret. The following tables shows the poss
<td>fitbit.user.firebase.collection.user.name</td><td>Firestore Collection for retrieving User details. Only used when a Firebase based user repository is used.</td><td>string</td><td>users</td><td></td><td>low</td></tr>
</tbody></table>

If the ManagementPortal is used to authenticate against the user repository, please add an OAuth client to ManagementPortal with the following properties:
If the ManagementPortal is used to authenticate against the user repository, please add an OAuth
client to ManagementPortal with the following properties:

```
Client ID: fitbit.user.repository.client.id
Expand All @@ -102,7 +118,8 @@ Access Token validity: 600
Refresh Token validity: 0
```

Finally set the `fitbit.user.repository.oauth.token.url` to `http://managementportal-app:8080/managementportal/oauth/token`.
Finally set the `fitbit.user.repository.oauth.token.url` to
`http://managementportal-app:8080/managementportal/oauth/token`.

Now you can run a full Kafka stack using

Expand Down Expand Up @@ -159,7 +176,20 @@ sequenceDiagram
connector ->> connector: Update offset times
```

## Sentry monitoring

To enable Sentry monitoring for the generic REST, Fitbit, or Oura source connector service:

1. Set a `SENTRY_DSN` environment variable that points to the desired Sentry DSN.
2. (Optional) Set the `SENTRY_LOG_LEVEL` environment variable to control the minimum log level of
events sent to Sentry.
The default log level for Sentry is `WARN`. Possible values are `TRACE`, `DEBUG`, `INFO`, `WARN`,
and `ERROR`.

## Contributing

Code should be formatted using the [Google Java Code Style Guide](https://google.github.io/styleguide/javaguide.html).
If you want to contribute a feature or fix browse our [issues](https://github.com/RADAR-base/RADAR-REST-Connector/issues), and please make a pull request.
Code should be formatted using
the [Google Java Code Style Guide](https://google.github.io/styleguide/javaguide.html).
If you want to contribute a feature or fix browse
our [issues](https://github.com/RADAR-base/RADAR-REST-Connector/issues), and please make a pull
request.
6 changes: 6 additions & 0 deletions kafka-connect-fitbit-source/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
plugins {
// TODO Remove this when new release of radar-commons is available and used in this project.
// This version has Sentry support built in for radar-kotlin plugin.
id("io.sentry.jvm.gradle") version "4.11.0"
}

description = "Kafka connector for Fitbit API source"

dependencies {
Expand Down
6 changes: 6 additions & 0 deletions kafka-connect-oura-source/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
plugins {
// TODO Remove this when new release of radar-commons is available and used in this project.
// This version has Sentry support built in for radar-kotlin plugin.
id("io.sentry.jvm.gradle") version "4.11.0"
}

description = "Kafka connector for Oura API source"

dependencies {
Expand Down
39 changes: 39 additions & 0 deletions kafka-connect-oura-source/src/main/resources/log4j2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8" ?>
<!--
~ /*
~ * Copyright 2024 The Hyve
~ *
~ * Licensed under the Apache License, Version 2.0 (the "License");
~ * you may not use this file except in compliance with the License.
~ * You may obtain a copy of the License at
~ *
~ * http://www.apache.org/licenses/LICENSE-2.0
~ *
~ * Unless required by applicable law or agreed to in writing, software
~ * distributed under the License is distributed on an "AS IS" BASIS,
~ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ * See the License for the specific language governing permissions and
~ * limitations under the License.
~ */
-->

<configuration status="INFO">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
/>
</Console>
<!-- For Sentry to work the DSN must be set via SENTRY_DSN environment variable
When SENTRY_DSN is empty string, the Sentry SDK is disabled -->
<Sentry name="Sentry" debug="false"/>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="Console" />
<!-- Note that the Sentry logging threshold is at WARN level by default -->
<appender-ref ref="Sentry" level="${env:SENTRY_LOG_LEVEL:-WARN}" />
</root>
</loggers>
</configuration>
6 changes: 6 additions & 0 deletions kafka-connect-rest-source/build.gradle.kts
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
plugins {
// TODO Remove this when new release of radar-commons is available and used in this project.
// This version has Sentry support built in for radar-kotlin plugin.
id("io.sentry.jvm.gradle") version "4.11.0"
}

description = "Kafka connector for generic REST API sources"

dependencies {
Expand Down
39 changes: 39 additions & 0 deletions kafka-connect-rest-source/src/main/resources/log4j2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8" ?>
<!--
~ /*
~ * Copyright 2024 The Hyve
~ *
~ * Licensed under the Apache License, Version 2.0 (the "License");
~ * you may not use this file except in compliance with the License.
~ * You may obtain a copy of the License at
~ *
~ * http://www.apache.org/licenses/LICENSE-2.0
~ *
~ * Unless required by applicable law or agreed to in writing, software
~ * distributed under the License is distributed on an "AS IS" BASIS,
~ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ * See the License for the specific language governing permissions and
~ * limitations under the License.
~ */
-->

<configuration status="INFO">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
/>
</Console>
<!-- For Sentry to work the DSN must be set via SENTRY_DSN environment variable
When SENTRY_DSN is empty string, the Sentry SDK is disabled -->
<Sentry name="Sentry" debug="false"/>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="Console" />
<!-- Note that the Sentry logging threshold is at WARN level by default -->
<appender-ref ref="Sentry" level="${env:SENTRY_LOG_LEVEL:-WARN}" />
</root>
</loggers>
</configuration>
39 changes: 39 additions & 0 deletions oura-library/src/main/resources/log4j2.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
<?xml version="1.0" encoding="UTF-8" ?>
<!--
~ /*
~ * Copyright 2024 The Hyve
~ *
~ * Licensed under the Apache License, Version 2.0 (the "License");
~ * you may not use this file except in compliance with the License.
~ * You may obtain a copy of the License at
~ *
~ * http://www.apache.org/licenses/LICENSE-2.0
~ *
~ * Unless required by applicable law or agreed to in writing, software
~ * distributed under the License is distributed on an "AS IS" BASIS,
~ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ * See the License for the specific language governing permissions and
~ * limitations under the License.
~ */
-->

<configuration status="INFO">
<appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout
pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"
/>
</Console>
<!-- For Sentry to work the DSN must be set via SENTRY_DSN environment variable
When SENTRY_DSN is empty string, the Sentry SDK is disabled -->
<Sentry name="Sentry" debug="false"/>
</appenders>

<loggers>
<root level="INFO">
<appender-ref ref="Console" />
<!-- Note that the Sentry logging threshold is at WARN level by default -->
<appender-ref ref="Sentry" level="${env:SENTRY_LOG_LEVEL:-WARN}" />
</root>
</loggers>
</configuration>

0 comments on commit 0e5e0eb

Please sign in to comment.