Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate embedded kafka support #1136

Merged
merged 3 commits into from
Jul 6, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 49 additions & 11 deletions docs/src/main/paradox/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,15 @@ project.description: Alpakka Kafka provides a Testkit with support for running l

To simplify testing of streaming integrations with Alpakka Kafka, it provides the **Alpakka Kafka testkit**. It provides help for

* @ref:[Using an embedded Kafka broker](#testing-with-an-embedded-kafka-server)
@@@ note

**Embedded Kafka testkit support has been deprecated since 2.0.4 and will be removed in the next minor release.**

**Use @ref:[testcontainers (Docker)](testing-testcontainers.md) instead.**

@@@

* ~~@ref:[Using an embedded Kafka broker](#testing-with-an-embedded-kafka-server)~~
* @ref:[Using Docker to launch a local Kafka cluster with testcontainers](testing-testcontainers.md)
* @ref:[Mocking the Alpakka Kafka Consumers and Producers](#mocking-the-consumer-or-producer)

Expand Down Expand Up @@ -33,27 +41,39 @@ We've overriden the `commons-compress` library to use a version with [fewer know

## Running Kafka with your tests

@@@ note

**Embedded Kafka testkit support has been deprecated since 2.0.4**

@@@

The Testkit provides a variety of ways to test your application against a real Kafka broker or cluster. There are two main options:

1. @ref:[Embedded Kafka](#testing-with-an-embedded-kafka-server)
1. ~~@ref:[Embedded Kafka](#testing-with-an-embedded-kafka-server)~~
2. @ref:[Testcontainers (Docker)](testing-testcontainers.md)

The table below helps guide you to the right Testkit implementation depending on your programming language, testing framework, and use (or not) of Docker containers.
You must mix in or implement these types into your test classes to use them.
See the documentation for each for more details.

| Type | Test Framework | Runtime Mode | Cluster | Schema Registry | Lang | Lifetime |
|-----------------------------------------------------------------------------------------------------------------------------------------------------|----------------|-----------------|---------|-----------------|----------|----------------------|
| @ref:[`akka.kafka.testkit.javadsl.EmbeddedKafkaTest`](#testing-with-avro-and-schema-registry-from-java-code) | JUnit 5 | Embedded Kafka | No | Yes | Java | All tests, Per class |
| @ref:[`akka.kafka.testkit.javadsl.EmbeddedKafkaJunit4Test`](#testing-with-avro-and-schema-registry-from-java-code) | JUnit 4 | Embedded Kafka | No | Yes | Java | All tests, Per class |
| @ref:[`akka.kafka.testkit.scaladsl.EmbeddedKafkaLike`](#testing-with-avro-and-schema-registry-from-scala-code) | ScalaTest | Embedded Kafka | No | Yes | Scala | Per class |
| @ref:[`akka.kafka.testkit.javadsl.TestcontainersKafkaJunit4Test`](testing-testcontainers.md#testing-with-a-docker-kafka-cluster-from-java-code) | JUnit 5 | Testcontainers | Yes | Yes | Java | All tests, Per class |
| @ref:[`akka.kafka.testkit.javadsl.TestcontainersKafkaTest`](testing-testcontainers.md#testing-with-a-docker-kafka-cluster-from-java-code) | JUnit 4 | Testcontainers | Yes | Yes | Java | All tests, Per class |
| @ref:[`akka.kafka.testkit.scaladsl.TestcontainersKafkaLike`](testing-testcontainers.md#testing-with-a-docker-kafka-cluster-from-scala-code) | ScalaTest | Testcontainers | Yes | Yes | Scala | All tests |
| @ref:[`akka.kafka.testkit.scaladsl.TestcontainersKafkaPerClassLike`](testing-testcontainers.md#testing-with-a-docker-kafka-cluster-from-scala-code) | ScalaTest | Testcontainers | Yes | Yes | Scala | Per class |
| Type | Test Framework | Runtime Mode | Cluster | Schema Registry | Lang | Lifetime |
|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|---------------------|-------------|---------------------|--------------|--------------------------|
| ~~@ref:[`akka.kafka.testkit.javadsl.EmbeddedKafkaTest`](#testing-with-avro-and-schema-registry-from-java-code)~~ | ~~JUnit 5~~ | ~~Embedded Kafka~~ | ~~No~~ | ~~Yes~~ | ~~Java~~ | ~~All tests, Per class~~ |
| ~~@ref:[`akka.kafka.testkit.javadsl.EmbeddedKafkaJunit4Test`](#testing-with-avro-and-schema-registry-from-java-code)~~ | ~~JUnit 4~~ | ~~Embedded Kafka~~ | ~~No~~ | ~~Yes~~ | ~~Java~~ | ~~All tests, Per class~~ |
| ~~@ref:[`akka.kafka.testkit.scaladsl.EmbeddedKafkaLike`](#testing-with-avro-and-schema-registry-from-scala-code)~~ | ~~ScalaTest~~ | ~~Embedded Kafka~~ | ~~No~~ | ~~Yes~~ | ~~Scala~~ | ~~Per class~~ |
| @ref:[`akka.kafka.testkit.javadsl.TestcontainersKafkaJunit4Test`](testing-testcontainers.md#testing-with-a-docker-kafka-cluster-from-java-code) | JUnit 5 | Testcontainers | Yes | Yes | Java | All tests, Per class |
| @ref:[`akka.kafka.testkit.javadsl.TestcontainersKafkaTest`](testing-testcontainers.md#testing-with-a-docker-kafka-cluster-from-java-code) | JUnit 4 | Testcontainers | Yes | Yes | Java | All tests, Per class |
| @ref:[`akka.kafka.testkit.scaladsl.TestcontainersKafkaLike`](testing-testcontainers.md#testing-with-a-docker-kafka-cluster-from-scala-code) | ScalaTest | Testcontainers | Yes | Yes | Scala | All tests |
| @ref:[`akka.kafka.testkit.scaladsl.TestcontainersKafkaPerClassLike`](testing-testcontainers.md#testing-with-a-docker-kafka-cluster-from-scala-code) | ScalaTest | Testcontainers | Yes | Yes | Scala | Per class |

## Testing with an embedded Kafka server

@@@ note

**Embedded Kafka testkit support has been deprecated since 2.0.4**

@@@

To test the Alpakka Kafka connector the [Embedded Kafka library](https://github.com/embeddedkafka/embedded-kafka) is an important tool as it helps to easily start and stop Kafka brokers from test cases.

Add the Embedded Kafka to your test dependencies:
Expand All @@ -78,11 +98,23 @@ The testkit contains helper classes used by the tests in the Alpakka Kafka conne

### Testing with Avro and Schema Registry

@@@ note

**Embedded Kafka testkit support has been deprecated since 2.0.4**

@@@

If you need to run tests using [Confluent's Schema Registry](https://docs.confluent.io/current/schema-registry/docs/index.html), you might include [embedded-kafka-schema-registry](https://github.com/embeddedkafka/embedded-kafka-schema-registry) instead.


### Testing with Avro and Schema Registry from Java code

@@@ note

**Embedded Kafka testkit support has been deprecated since 2.0.4**

@@@

Test classes may extend @scaladoc[EmbeddedKafkaTest](akka.kafka.testkit.javadsl.EmbeddedKafkaTest) (JUnit 5) or @scaladoc[EmbeddedKafkaJunit4Test](akka.kafka.testkit.javadsl.EmbeddedKafkaJunit4Test) (JUnit 4) to automatically start and stop an embedded Kafka broker.

Furthermore it provides
Expand All @@ -105,6 +137,12 @@ The JUnit test base classes run the @javadoc[assertAllStagesStopped](akka.stream

### Testing with Avro and Schema Registry from Scala code

@@@ note

**Embedded Kafka testkit support has been deprecated since 2.0.4**

@@@

The @scaladoc[KafkaSpec](akka.kafka.testkit.scaladsl.KafkaSpec) class offers access to

* preconfigured consumer settings (`consumerDefaults: ConsumerSettings[String, String]`),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@
/**
* JUnit 4 base-class with some convenience for creating an embedded Kafka broker before running the
* tests.
*
* @deprecated Use testcontainers instead. Since 2.0.4.
*/
@Deprecated
public abstract class EmbeddedKafkaJunit4Test extends KafkaJunit4Test {

private static EmbeddedKafkaConfig embeddedKafkaConfig(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,10 @@
* running the tests. Extending classes must be annotated with `@TestInstance(Lifecycle.PER_CLASS)`
* to create a single instance of the test class with `@BeforeAll` and `@AfterAll` annotated methods
* called by the test framework.
*
* @deprecated Use testcontainers instead. Since 2.0.4.
*/
@Deprecated
public abstract class EmbeddedKafkaTest extends KafkaTest {

private static EmbeddedKafkaConfig embeddedKafkaConfig(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ package akka.kafka.testkit.scaladsl

import net.manub.embeddedkafka.{EmbeddedKafka, EmbeddedKafkaConfig}

@deprecated("Use testcontainers instead", "2.0.4")
trait EmbeddedKafkaLike extends KafkaSpec {

lazy implicit val embeddedKafkaConfig: EmbeddedKafkaConfig = createKafkaConfig
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@ package akka.kafka.scaladsl

// #embeddedkafka
import akka.kafka.testkit.scaladsl.EmbeddedKafkaLike
import com.github.ghik.silencer.silent
import net.manub.embeddedkafka.EmbeddedKafkaConfig

@silent
class EmbeddedKafkaSampleSpec extends SpecBase(kafkaPort = 1234) with EmbeddedKafkaLike {

// if a specific Kafka broker configuration is desired
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,14 @@ import akka.Done
import akka.kafka.KafkaPorts
import akka.kafka.testkit.scaladsl.EmbeddedKafkaLike
import akka.stream.testkit.scaladsl.StreamTestKit.assertAllStagesStopped
import com.github.ghik.silencer.silent
import net.manub.embeddedkafka.EmbeddedKafkaConfig

import scala.collection.immutable
import scala.concurrent.duration._
import scala.concurrent.{Await, Future}

@silent
class MultiConsumerSpec extends SpecBase(kafkaPort = KafkaPorts.MultiConsumerSpec) with EmbeddedKafkaLike {

override def createKafkaConfig: EmbeddedKafkaConfig =
Expand Down
2 changes: 2 additions & 0 deletions tests/src/test/scala/akka/kafka/scaladsl/ReconnectSpec.scala
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,14 @@ import akka.kafka.testkit.scaladsl.EmbeddedKafkaLike
import akka.stream.scaladsl.{Keep, Sink, Source, SourceQueueWithComplete, Tcp}
import akka.stream.testkit.scaladsl.StreamTestKit.assertAllStagesStopped
import akka.stream.{KillSwitches, OverflowStrategy, UniqueKillSwitch}
import com.github.ghik.silencer.silent
import net.manub.embeddedkafka.EmbeddedKafka
import org.apache.kafka.clients.producer.ProducerRecord

import scala.concurrent.duration._
import scala.concurrent.{Await, Future}

@silent
class ReconnectSpec extends SpecBase(KafkaPorts.ReconnectSpec) with EmbeddedKafkaLike {

val proxyPort = KafkaPorts.ReconnectSpecProxy
Expand Down
2 changes: 2 additions & 0 deletions tests/src/test/scala/docs/scaladsl/PartitionExamples.scala
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,14 @@ import akka.kafka.testkit.scaladsl.EmbeddedKafkaLike
import akka.kafka.{KafkaConsumerActor, KafkaPorts, Subscriptions}
import akka.stream.scaladsl.{Keep, Sink}
import akka.stream.testkit.scaladsl.StreamTestKit.assertAllStagesStopped
import com.github.ghik.silencer.silent
import net.manub.embeddedkafka.EmbeddedKafkaConfig
import org.apache.kafka.common.{Metric, MetricName, TopicPartition}

import scala.concurrent.{Await, Future}
import scala.concurrent.duration._

@silent
class PartitionExamples extends DocsSpecBase(KafkaPorts.ScalaPartitionExamples) with EmbeddedKafkaLike {

override def createKafkaConfig: EmbeddedKafkaConfig =
Expand Down