doc(kafka): Add documentation #140
Conversation
* Document common types * Document Consumer types and functions * Add module documentation (full example from readme) * Fix links to replacements for deprecated functions
| module Kafka.Consumer | ||
| ( module X | ||
| ( KafkaConsumer |
There was a problem hiding this comment.
Note I moved this up in the import list, as it was close to the bottom of the module, when I find it's one of the most important ones. I am totally fine reverting this change if you disagree
| @@ -48,7 +97,7 @@ import Kafka.Types as X | |||
|
|
|||
| -- | Runs high-level kafka consumer. | |||
| -- A callback provided is expected to call 'pollMessage' when convenient. | |||
| {-# DEPRECATED runConsumer "Use newConsumer/closeConsumer instead" #-} | |||
| {-# DEPRECATED runConsumer "Use 'newConsumer'/'closeConsumer' instead" #-} | |||
There was a problem hiding this comment.
An example where I added quotes to get correct links in Haddock, as such:
| -- consumerSub :: 'Subscription' | ||
| -- consumerSub = 'topics' ['TopicName' "kafka-client-example-topic"] | ||
| -- <> 'offsetReset' 'Earliest' | ||
| -- <> 'extraSubscriptionProps' (fromList [("prop1", "value 1"), ("prop2", "value 2")]) |
There was a problem hiding this comment.
I added a use of extraSubscriptionProps but I wasn't sure which other properties were configurable here vs to be configured in ConsumerProperties
| -- import Kafka.Consumer | ||
| -- | ||
| -- -- Global consumer properties | ||
| -- consumerProps :: 'ConsumerProperties' |
There was a problem hiding this comment.
I reused a technique I found in Aeson documentation where code examples also link all their types/functions. It's more verbose in the Haddock, but it provides links to everything provided by this library, which I think is a great way to discover and understand it:
| @@ -30,14 +30,17 @@ where | |||
|
|
|||
| import Data.Bifoldable (Bifoldable (..)) | |||
| import Data.Bifunctor (Bifunctor (..)) | |||
| import Data.Bitraversable (Bitraversable (..), bimapM, bisequenceA) | |||
| import Data.Bitraversable (Bitraversable (..), bimapM, bisequence) | |||
There was a problem hiding this comment.
bisequenceA is just an alias to bisequence so I directly used bisequence
| import Data.Int (Int64) | ||
| import Data.Text (Text) | ||
| import Data.Typeable (Typeable) | ||
| import GHC.Generics (Generic) | ||
| import Kafka.Internal.Setup (HasKafka (..), HasKafkaConf (..), Kafka (..), KafkaConf (..)) | ||
| import Kafka.Types (Millis (..), PartitionId (..), TopicName (..)) | ||
|
|
||
| -- | The main type for Kafka consumption, used e.g. to poll and commit messages. | ||
| -- | ||
| -- Its constructor is intentionally not exposed, instead, one should used 'newConsumer' to acquire such a value. |
There was a problem hiding this comment.
I felt this was useful in understanding the module mechanics to explain the constructor is hidden + link to how to create such a value
| @@ -67,6 +79,7 @@ data RebalanceEvent = | |||
| | RebalanceRevoke [(TopicName, PartitionId)] | |||
| deriving (Eq, Show, Generic) | |||
|
|
|||
| -- | The partition offset | |||
| data PartitionOffset = | |||
There was a problem hiding this comment.
The difference between a PartitionOffset and an Offset didn't strike me (of course PartitionOffset has other constructors, but I failed to find a good explanation of why both types exist)
| @@ -148,53 +163,56 @@ instance Bitraversable ConsumerRecord where | |||
| bitraverse f g r = (\k v -> bimap (const k) (const v) r) <$> f (crKey r) <*> g (crValue r) | |||
| {-# INLINE bitraverse #-} | |||
|
|
|||
| {-# DEPRECATED crMapKey "Isn't concern of this library. Use 'first'" #-} | |||
There was a problem hiding this comment.
I deprecated those 3 functions as they are aliases to Bifunctor module from base, what do you think?
| data TopicType = | ||
| User -- ^ Normal topics that are created by user. | ||
| | System -- ^ Topics starting with "__" (__consumer_offsets, __confluent.support.metrics) are considered "system" topics | ||
| | System -- ^ Topics starting with a double underscore "\__" (@__consumer_offsets@, @__confluent.support.metrics@, etc.) are considered "system" topics |
There was a problem hiding this comment.
Display was broken because double underscore is bold in Haddock, I fixed it
|
Awesome, thanks! |
Hi @AlexeyRaga
Having used this library for a while (and it works pretty much flawlessly), I think it's time I contribute back to it!
What better way to contribute to a Haskell library than to (try to) improve documentation for future users 😄
Here's a first PR where I added various documentation, mainly for Consumer module (and common types which appear in Consumer).
Note that I wasn't always sure what was from Kafka, what was from
librdkafkaand what was specific to this library, I did my best but please check and let me know if I made mistakes.When pure Kafka stuff, I tried to add links to Kafka documentation to help users know how to configure (e.g. the
ResetOffset)When
librdkafkastuff, I did my best to understand what's going on (I've improved on reading the C-Haskell binding file but I can't claim I'm fluent yet 😄 ) and explain accordingly. I would be surprised if I didn't do any mistake, or misexplained (is that even a word?) some things.If you're ok with this PR, I will find some time to add documentation for Producer and other modules in another PR.
Again, thank you for this library, it's awesome and It Just Works™, we've been using it for 1 year in production. I hope this PR serves this library well!