

API Docs - v1.0.3

Sink

prometheus (Sink)

This sink publishes events processed by Siddhi into Prometheus metrics and exposes them to the Prometheus server at the specified URL. The created metrics can be published to Prometheus via 'server' or 'pushGateway', depending on your preference.
 The metric types that are supported by the Prometheus sink are 'counter', 'gauge', 'histogram', and 'summary'. The values and labels of the Prometheus metrics can be updated through the events.

Syntax

@sink(type="prometheus", job="<STRING>", publish.mode="<STRING>", push.url="<STRING>", server.url="<STRING>", metric.type="<STRING>", metric.help="<STRING>", metric.name="<STRING>", buckets="<STRING>", quantiles="<STRING>", quantile.error="<DOUBLE>", value.attribute="<STRING>", push.operation="<STRING>", grouping.key="<STRING>", @map(...)))

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
job	This parameter specifies the job name of the metric. This must be the same job name that is defined in the Prometheus configuration file.	siddhiJob	STRING	Yes	No
publish.mode	The mode in which the metrics need to be exposed to the Prometheus server.The possible publishing modes are 'server' and 'pushgateway'.The server mode exposes the metrics through an HTTP server at the specified URL, and the 'pushGateway' mode pushes the metrics to the pushGateway that needs to be running at the specified URL.	server	STRING	Yes	No
push.url	This parameter specifies the target URL of the Prometheus pushGateway. This is the URL at which the pushGateway must be listening. This URL needs to be defined in the Prometheus configuration file as a target before it can be used here.	http://localhost:9091	STRING	Yes	No
server.url	This parameter specifies the URL where the HTTP server is initiated to expose metrics in the 'server' publish mode. This URL needs to be defined in the Prometheus configuration file as a target before it can be used here.	http://localhost:9080	STRING	Yes	No
metric.type	The type of Prometheus metric that needs to be created at the sink.  The supported metric types are 'counter', 'gauge',c'histogram' and 'summary'.		STRING	No	No
metric.help	A brief description of the metric and its purpose.		STRING	Yes	No
metric.name	This parameter allows you to assign a preferred name for the metric. The metric name must match the regex format, i.e., [a-zA-Z_:][a-zA-Z0-9_:]*.		STRING	Yes	No
buckets	The bucket values preferred by the user for histogram metrics. The bucket values must be in the 'string' format with each bucket value separated by a comma as shown in the example below. "2,4,6,8"	null	STRING	Yes	No
quantiles	This parameter allows you to specify quantile values for summary metrics as preferred. The quantile values must be in the 'string' format with each quantile value separated by a comma as shown in the example below. "0.5,0.75,0.95"	null	STRING	Yes	No
quantile.error	The error tolerance value for calculating quantiles in summary metrics. This must be a positive value, but less than 1.	0.001	DOUBLE	Yes	No
value.attribute	The name of the attribute in the stream definition that specifies the metric value. The defined 'value' attribute must be included in the stream definition. The system increases the metric value for the counter and gauge metric types by the value of the 'value attribute. The system observes the value of the 'value' attribute for the calculations of 'summary' and 'histogram' metric types.	value	STRING	Yes	No
push.operation	This parameter defines the mode for pushing metrics to the pushGateway. The available push operations are 'push' and 'pushadd'. The operations differ according to the existing metrics in pushGateway where 'push' operation replaces the existing metrics, and 'pushadd' operation only updates the newly created metrics.	pushadd	STRING	Yes	No
grouping.key	This parameter specifies the grouping key of created metrics in key-value pairs. The grouping key is used only in pushGateway mode in order to distinguish the metrics from already existing metrics. The expected format of the grouping key is as follows:  "'key1:value1','key2:value2'"		STRING	Yes	No

System Parameters

Name	Description	Default Value	Possible Parameters
jobName	This property specifies the default job name for the metric. This job name must be the same as the job name defined in the Prometheus configuration file.	siddhiJob	Any string
publishMode	The default publish mode for the Prometheus sink for exposing metrics to the Prometheus server. The mode can be either 'server' or 'pushgateway'.	server	server or pushgateway
serverURL	This property configures the URL where the HTTP server is initiated to expose metrics. This URL needs to be defined in the Prometheus configuration file as a target to be identified by Prometheus before it can be used here. By default, the HTTP server is initiated at 'http://localhost:9080'.	http://localhost:9080	Any valid URL
pushURL	This property configures the target URL of the Prometheus pushGateway (where the pushGateway needs to listen). This URL needs to be defined in the Prometheus configuration file as a target to be identified by Prometheus before it can be used here.	http://localhost:9091	Any valid URL
groupingKey	This property configures the grouping key of created metrics in key-value pairs. Grouping key is used only in pushGateway mode in order to distinguish these metrics from already existing metrics under the same job. The expected format of the grouping key is as follows: "'key1:value1','key2:value2'" .	null	Any key value pairs in the supported format

Examples EXAMPLE 1

@sink(type='prometheus',job='fooOrderCount', server.url ='http://localhost:9080', publish.mode='server', metric.type='counter', metric.help= 'Number of foo orders', @map(type='keyvalue'))
define stream FooCountStream (Name String, quantity int, value int);

In the above example, the Prometheus-sink creates a counter metric with the stream name and defined attributes as labels. The metric is exposed through an HTTP server at the target URL.

EXAMPLE 2

@sink(type='prometheus',job='inventoryLevel', push.url='http://localhost:9080', publish.mode='pushGateway', metric.type='gauge', metric.help= 'Current level of inventory', @map(type='keyvalue'))
define stream InventoryLevelStream (Name String, value int);

In the above example, the Prometheus-sink creates a gauge metric with the stream name and defined attributes as labels.The metric is pushed to the Prometheus pushGateway at the target URL.

Source

prometheus (Source)

This source consumes Prometheus metrics that are exported from a specified URL as Siddhi events by sending HTTP requests to the URL. Based on the source configuration, it analyzes metrics from the text response and sends them as Siddhi events through key-value mapping.The user can retrieve metrics of the 'including', 'counter', 'gauge', 'histogram', and 'summary' types. The source retrieves the metrics from a text response of the target. Therefore, it is you need to use 'string' as the attribute type for the attributes that correspond with the Prometheus metric labels. Further, the Prometheus metric value is passed through the event as 'value'. This requires you to include an attribute named 'value' in the stream definition.
The supported types for the 'value' attribute are 'INT', 'LONG', 'FLOAT', and 'DOUBLE'.

Syntax

@source(type="prometheus", target.url="<STRING>", scrape.interval="<INT>", scrape.timeout="<INT>", scheme="<STRING>", metric.name="<STRING>", metric.type="<STRING>", username="<STRING>", password="<STRING>", client.truststore.file="<STRING>", client.truststore.password="<STRING>", headers="<STRING>", job="<STRING>", instance="<STRING>", grouping.key="<STRING>", @map(...)))

QUERY PARAMETERS

Name	Description	Default Value	Possible Data Types	Optional	Dynamic
target.url	This property specifies the target URL to which the Prometheus metrics are exported in the 'TEXT' format.		STRING	No	No
scrape.interval	This property specifies the time interval in seconds within which the source should send an HTTP request to the specified target URL.	60	INT	Yes	No
scrape.timeout	This property is the time duration in seconds for a scrape request to get timed-out if the server at the URL does not respond.	10	INT	Yes	No
scheme	This property specifies the scheme of the target URL.  The supported schemes are 'HTTP' and 'HTTPS'.	HTTP	STRING	Yes	No
metric.name	This property specifies the name of the metrics that are to be fetched. The metric name must match the regex format, i.e., '[a-zA-Z_:][a-zA-Z0-9_:]* '.	Stream name	STRING	Yes	No
metric.type	This property specifies the type of the Prometheus metric that is required to be fetched.  The supported metric types are 'counter', 'gauge'," 'histogram', and 'summary'.		STRING	No	No
username	This property specifies the username that needs to be added in the authorization header of the HTTP request if basic authentication is enabled at the target. It is required to specify both the username and the password to enable basic authentication. If you do not provide a value for one or both of these parameters, an error is logged in the console.		STRING	Yes	No
password	This property specifies the password that needs to be added in the authorization header of the HTTP request if basic authentication is enabled at the target. It is required to specify both the username and the password to enable basic authentication. If you do not provide a value for one or both of these parameters, an error is logged in the console.		STRING	Yes	No
client.truststore.file	The file path to the location of the truststore to which the client needs to send HTTPS requests via the 'HTTPS' protocol.		STRING	Yes	No
client.truststore.password	The password for the client-truststore. This is required to send HTTPS requests. A custom password can be specified if required.		STRING	Yes	No
headers	Headers that need to be included as HTTP request headers in the request. The format of the supported input is as follows, "'header1:value1','header2:value2'"		STRING	Yes	No
job	This property defines the job name of the exported Prometheus metrics that needs to be fetched.		STRING	Yes	No
instance	This property defines the instance of the exported Prometheus metrics that needs to be fetched.		STRING	Yes	No
grouping.key	This parameter specifies the grouping key of the required metrics in key-value pairs. The grouping key is used if the metrics are exported by Prometheus 'pushGateway' in order to distinguish those metrics from already existing metrics.  The expected format of the grouping key is as follows: "'key1:value1','key2:value2'"		STRING	Yes	No

System Parameters

Name	Description	Default Value	Possible Parameters
scrapeInterval	The default time interval in seconds for the Prometheus source to send HTTP requests to the target URL.	60	Any integer value
scrapeTimeout	The default time duration (in seconds) for an HTTP request to time-out if the server at the URL does not respond.	10	Any integer value
scheme	The scheme of the target for the Prometheus source to send HTTP requests. The supported schemes are 'HTTP' and 'HTTPS'.	HTTP	HTTP or HTTPS
username	The username that needs to be added in the authorization header of the HTTP request if basic authentication is enabled at the target. It is required to specify both the username and password to enable basic authentication. If you do not specify a value for one or both of these parameters, an error is logged in the console.		Any string
password	The password that needs to be added in the authorization header of the HTTP request if basic authentication is enabled at the target. It is required to specify both the username and password to enable basic authentication. If you do not specify a value for one or both of these parameters, an error is logged in the console.		Any string
trustStoreFile	The default file path to the location of truststore that the client needs to access in order to send HTTPS requests through 'HTTPS' protocol.	${carbon.home}/resources/security/client-truststore.jks	Any valid path for the truststore file
trustStorePassword	The default password for the client-truststore that the client needs to access in order to send HTTPS requests through 'HTTPS' protocol.	wso2carbon	Any string
headers	The headers that need to be included as HTTP request headers in the scrape request. The format of the supported input is as follows, "'header1:value1','header2:value2'"		Any valid http headers
job	The default job name of the exported Prometheus metrics that needs to be fetched.		Any valid job name
instance	The default instance of the exported Prometheus metrics that needs to be fetched.		Any valid instance name
groupingKey	The default grouping key of the required Prometheus metrics in key-value pairs. The grouping key is used if the metrics are exported by the Prometheus pushGateway in order to distinguish these metrics from already existing metrics. The expected format of the grouping key is as follows: "'key1:value1','key2:value2'"		Any valid grouping key pairs

Examples EXAMPLE 1

@source(type= 'prometheus', target.url= 'http://localhost:9080/metrics', metric.type= 'counter', metric.name= 'sweet_production_counter', @map(type= 'keyvalue'))
define stream FooStream1(metric_name string, metric_type string, help string, subtype string, name string, quantity string, value double);

In this example, the Prometheus source sends an HTTP request to the 'target.url' and analyzes the response. From the analyzed response, the source retrieves the Prometheus counter metrics with the 'sweet_production_counter' nameand converts the filtered metrics into Siddhi events using the key-value mapper.
The generated maps have keys and values as follows:
  metric_name -> sweet_production_counter
  metric_type -> counter
  help -> <help_string_of_metric>
  subtype -> null
  name -> <value_of_label_name>
  quantity -> <value_of_label_quantity>
  value -> <value_of_metric>

EXAMPLE 2

@source(type= 'prometheus', target.url= 'http://localhost:9080/metrics', metric.type= 'summary', metric.name= 'sweet_production_summary', @map(type= 'keyvalue'))
 define stream FooStream2(metric_name string, metric_type string, help string, subtype string, name string, quantity string, quantile string, value double);

In this example, the Prometheus source sends an HTTP request to the 'target.url' and analyzes the response. From the analysed response, the source retrieves the Prometheus summary metrics with the 'sweet_production_summary' nameand converts the filtered metrics into Siddhi events using the key-value mapper.
The generated maps have keys and values as follows:
  metric_name -> sweet_production_summary
  metric_type -> summary
  help -> <help_string_of_metric>
  subtype -> <'sum'/'count'/'null'>
  name -> <value_of_label_name>
  quantity -> <value_of_label_quantity>
  quantile -> <value of the quantile>
  value -> <value_of_metric>

EXAMPLE 3

@source(type= 'prometheus', target.url= 'http://localhost:9080/metrics', metric.type= 'histogram', metric.name= 'sweet_production_histogram', @map(type= 'keyvalue'))
define stream FooStream3(metric_name string, metric_type string, help string, subtype string, name string, quantity string, le string, value double);

In this example, the prometheus source sends an HTTP request to the 'target.url' and analyzes the response. From the analyzed response, the source retrieves the Prometheus histogram metrics with the 'sweet_production_histogram' name and converts the filtered metrics into Siddhi events using the key-value mapper.
The generated maps have keys and values as follows,
  metric_name -> sweet_production_histogram
  metric_type -> histogram
  help -> <help_string_of_metric>
  subtype -> <'sum'/'count'/'bucket'>
  name -> <value_of_label_name>
  quantity -> <value_of_label_quantity>
  le -> <value of the bucket>
  value -> <value_of_metric>