While implementing it in our backend, we found a few plugins that were missing, especially when dealing with asynchronous communication.
One of them was KafkaJS.
We took this opportunity to give back to the community and developed it ourselves. This plugin allows you to track all Kafka interactions in your collected traces, which means, you get a more comprehensive view of your application behavior when using Kafka as a message broker.
This article describes this plugin for the
kafkajs plugin captures producer and consumer operations and creates spans according to the semantic conventions for Messaging Systems.
- Each message being produced and consumed is represented by a span with attributes such as
- Context is propagated from producers to consumers. When a message is sent to Kafka, the trace will reveal which services consume it and what other cascading operations happen down the pipe.
- Batch operations can aggregate multiple messages into a single batch and receive\process them together. This is handled in the plugin according to the specification.
- The plugin can be extended with hooks, which enable users to run custom logic for adding span attributes depending on the Kafka message.
The above screenshot shows an example of a producer application named
kafka-producer, that exposes an HTTP endpoint (first line), route it in express (second line), and produce two messages to a Kafka topic named
test, which are then consumed by another application called
As mentioned above,
kafkajs was one of the missing plugins we found and as you are reading this we are working to add more plugins.
You can find the OpenTelemetry kafkajs Instrumentation for Node.js here.
Feel free to reach out to us with any questions as we are very much invested in OpenTelemetry and the OpenTelementry community.
Community contributions and involvement are more than welcome ?
As always, this extension (and many others) was developed by Aspecto with ❤️