productName variable is missing Metrics

The productName variable is missing provides a following set of metrics to monitor the archiving process:

Add User-Defined Metric

While the productName variable is missing provides a comprehensive set of the archiving process metrics, the user has the option to define their own metrics using the standard Flink APIs. To define a new custom metric:

• Add your custom metric as an instance variable in your UDF implementation

  public class MyExampleImplementation implements UDF {
  private Counter customMetric;
  ...
  }
  

• Override the open(Configuration parameters, RuntimeContext runtimeContext) API and register the custom metric which you added in the previous step with the MetricGroup of the provided RuntimeContext. For example:

  ...
  @Override
  public void open(Configuration parameters, RuntimeContext runtimeContext) {
    customMetric = runtimeContext.getMetricGroup().counter("myCustomMetric");
  }
  

• Record the metric in your UDF implementation:

  customMetric.inc();
  

• The following dependencies should be added with provided scope. Note that this example is for maven build tool.

  <dependency>
   <groupId>org.apache.flink</groupId>
   <artifactId>flink-core</artifactId>
   <scope>provided</scope>
  </dependency>
  <dependency>
   <groupId>org.apache.flink</groupId>
   <artifactId>flink-metrics-core</artifactId>
   <scope>provided</scope>
  </dependency>
  

Refer to com/here/platform/data/archive/example/AvroSimpleKeyExample.java in the Data Archive Avro Example for an example implementation.

Monitoring Metrics

Metrics registered with the Flink runtime context will be available in Grafana prefixed with flink_taskmanager_job_task_operator_. For example:

flink_taskmanager_job_task_operator_messageCounter
flink_taskmanager_job_task_operator_myCustomMetric

Refer to the Flink documention to learn more about the scope and naming of metrics.