Introduction
The iGrafx Kafka Modules are open-source applications designed to enhance your data streaming and integration workflows. These modules enable real-time data processing and transformation, allowing you to connect, enrich, and analyze data and send it to the iGrafx Mining platforms.
Using these modules, you can configure kafka connectors, define custom UDFs (User-Defined Functions), and enable live connections for seamless data streaming and analysis.
The Docker Compose module provides a Kafka infrastructure setup located in the docker-compose/ subdirectory. It includes essential components for managing and interacting with Kafka.
In the iGrafx Connectors module, you will find 2 connectors.
- The iGrafx Aggregation Connector allows you to aggregate multiple records originating from the same partition into a single, structured array.
- The iGrafx Aggregation and Sink Connector allows you to aggregate and send data from Kafka to the iGrafx Mining platform.
In the iGrafx UDFs module, you will find 3 UDFs.
- The iGrafx Case Events UDF retrieves detailed information related to specific case IDs within Druid.
- The iGrafx Sessions UDF takes a collection of lines and organizes them into separate sessions.Each session groups related events.
- The iGrafx Transposition UDF transforms a single row containing multiple dates, each associated with specific activities, into multiple rows—each with a single date (or period) linked to its corresponding activity.
Please note that an iGrafx account is required to fully utilize these modules. For account setup, please contact iGrafx support.
This version of the iGrafx Mining Kafka Modules is compatible with platforms of version 2.32.0 and above.
Find the home page of the GitHub repository for the iGrafx Kafka Modules here.
To begin with ksqlDB and Kafka, you may follow these links: