Mandatory Properties

Below are examples of values for required properties. The following properties, however, should remain unchanged:

  • connector.class (String)
  • key.converter (String)
  • value.converter (String)

You may modify the following properties as needed:

  • tasks.max (Int): Specifies the number of tasks to create for the connector.
  • api.url (String): API URL of the iGrafx Mining API for file transfer.
  • api.authUrl (String): URL for authentication to obtain a connection token.
  • workGroupId (String): ID of the workgroup associated with the iGrafx project.
  • workGroupKey (String): Key of the workgroup associated with the iGrafx project.
  • projectId (String): ID of the iGrafx project.
  • csv.encoding (String): Encoding for the file (generally UTF-8).
  • csv.separator (String): Field separator character in the CSV file (only one character).
  • csv.quote (String): Quote character (only one character).
  • csv.fieldsNumber (Int): Number of fields per line (must be >= 3).
  • csv.header (Boolean): Indicates whether the CSV file includes a header (true/false).
  • csv.defaultTextValue (String): Default value for missing fields in the file.
  • retentionTimeInDay (Int): Archive file retention time in days (must be > 0).
  • threshold.elementNumber (Int): Maximum number of elements in one aggregation; aggregation is sent to the iGrafx Mining API when this number is reached.
  • threshold.valuePattern (String): Optional—triggers an aggregation flush to the iGrafx Mining API if an incoming sink record matches the regex pattern in threshold.valuePattern (applies even if threshold.elementNumber is not met). The pattern applies to the entire string value of a SinkRecord, and may need adjustments based on the data structure (e.g., accounting for {} in JSON format).
  • threshold.timeoutInSeconds (Int): Maximum time in seconds since the last aggregation result was sent. If exceeded, the aggregation is sent to the iGrafx Mining API (even if threshold.elementNumber is not met).
  • bootstrap.servers (String): List of Kafka brokers.
  • value.converter.schema.registry.url (String): URL of the Confluent Schema Registry.

For more information on regex (used with the threshold.valuePattern property): Regex Cheat Sheet