Skip to content

Commit

Permalink
Issue 302: Add Logging information to the Contributing guide (Qbeast-…
Browse files Browse the repository at this point in the history
…io#308)

Adds logging documentation with examples into the CONTRIBUTING.md file.
  • Loading branch information
osopardo1 authored May 2, 2024
1 parent 6a8e1ca commit 6075355
Show file tree
Hide file tree
Showing 2 changed files with 79 additions and 3 deletions.
1 change: 1 addition & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Describe the change you're making: how it affects the API, user experience...
Here is the list of things you should do before submitting this pull request:

- [ ] New feature / bug fix has been committed following the [Contribution guide](https://github.com/Qbeast-io/qbeast-spark/blob/main/CONTRIBUTING.md).
- [ ] Add logging to the code following the [Contribution guide](https://github.com/Qbeast-io/qbeast-spark/blob/main/CONTRIBUTING.md).
- [ ] Add comments to the code (make it easier for the community!).
- [ ] Change the documentation.
- [ ] Add tests.
Expand Down
81 changes: 78 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,81 @@ All contributed code, docs, and other materials are considered licensed under th
```bash
sbt doc
```

## Logging Documentation

- We use the [Spark Logging interface](https://spark.apache.org/docs/1.5.2/api/java/org/apache/spark/Logging.html#:~:text=Interface%20Logging&text=Creates%20a%20SLF4J%20logger%20for,intended%20as%20an%20internal%20utility).
- Spark uses `log4j` for logging. You can configure it by adding a `log4j2.properties` file in the conf directory. One way to start is to copy the existing `log4j2.properties.template` located there.

An example of using logging on a class is:

```scala
import org.apache.spark.internal.Logging

case class MyClass() extends Logging {

def myMethod(): Unit = {
logInfo("This is an info message")
logWarn("This is a warning message")
logError("This is an error message")
logTrace("This is a trace message")
logDebug("This is a debug message")
}
}

```

The following log levels are used to track code behaviour:
- `WARN` level is supposed to be critical and actionable. If the user sees a WARN, then something bad happened and it might require user intervention.

**Example on `DeltaMetadataWriter` class:**
```scala
def writeWithTransaction(writer: => (TableChanges, Seq[FileAction])): Unit = {
// [...] Code to write the transaction [...]
if (txn.appId == appId && version <= txn.version) {
val message = s"Transaction ${version} from application ${appId} is already completed," +
" the requested write is ignored"
logWarn(message)
return
}
}
```
- `INFO` level provides information about the execution, but not necessarily actionable and it avoids being verbose. It is not uncommon to see INFO level on in production, so it is expected to be lightweight with respect to the volume of messages generated.

**Example on `BlockWriter` class:**
```scala
def writeRow(rows: Iterator[InternalRow]): Iterator[(AddFile, TaskStats)] = {
// [...] Code to write the rows [...]
logInfo(s"Adding file ${file.path}")
}
```
- `DEBUG` provides debug level info when debugging the code. It can be verbose as it is not expected to be on in production.

**Example on `IndexedTable` class:**
```scala
if (isNewRevision(options)) {
// Merging revisions code
logDebug(
s"Merging transformations for table ${tableID} with cubeSize=${newRevisionCubeSize}")
// Code to merge revisions
}
```
- `TRACE` provides further detail to DEBUG on execution paths, and in particular, it indicates the execution of critical methods.

**Example on `IndexedTable` class:**
```scala
def doWrite(
data: DataFrame,
indexStatus: IndexStatus,
options: QbeastOptions,
append: Boolean): Unit = {
logTrace(s"Begin: Writing data to table ${tableID}")
// [...] Code to write the data [...]
logTrace(s"End: Writing data to table ${tableID}")
}
```

> We should enforce **all the Pull Request**, specially those containing critical code, **to have logging messages** that are meaningful and informative.

## Step-by-step guide
#### 1 - Click `Fork` on Github, and name it as `yourname/projectname`
Expand Down Expand Up @@ -161,14 +236,14 @@ If the current version is 0.y.z, these rules must be applied for a new release:

## post 1.0.0 release

We'll apply Semantic Versioning rules as defined at (semver.org)[https://semver.org].
We'll apply Semantic Versioning rules as defined at [semver.org](https://semver.org).

## Snapshots

Snapshots can be made available for internal and test purposes.
In this case the versioning to be applied is: <new-version-number>-<short-commit-sha>-SNAPSHOT
In this case the versioning to be applied is: `<new-version-number>-<short-commit-sha>-SNAPSHOT`

Example: 0.6.2-badfbadf-SNAPSHOT
Example: `0.6.2-badfbadf-SNAPSHOT`


# Community Values
Expand Down

0 comments on commit 6075355

Please sign in to comment.