Add Examples

[valencik]

Building a Pipe that uses a tokenizer:

  otherStream => {
    val tokenizer = AnalyzerBuilder.default
      .withLowerCasing
      .withASCIIFolding
      .withPorterStemmer
      .tokenizer[IO]
    Stream.resource(tokenizer).flatMap(f => otherStream.evalMap(x => f(x)))
  }

Note: Try to avoid calling .tokenizer in a loop