“” is supposed to be automatically resolved to the IP address of the local host. In addition, make sure $HOST environment variable is set (it should be defined if you use Marathon), or replace it with your hostname, of course. I randomly chose ports 80 – choose your own.
Using “spark-submit”, this is done as follows: spark-submit. In Spark, these should be passed to Spark driver by setting “”. 2.1 Expose JMX portĪs usual in Java, this is done by setting the “.*” parameters of the JVM. This step is necessary in order to learn what and how to configure Prometheus JMX Exporter. The shortest way (if you know what to do…) to viewing them now is just to open the JMX remote port, and to connect through some JMX client. So, Spark’s metrics are now exposed as MBeans, but are still unreachable from outside the JVM. Expose JMX port and browse through VisualVM (optional) To enable exposing metrics as JMX MBeans, you should edit “$SPARK_HOME/conf/metrics.properties” file.Īdd (or uncomment) the row: metrics.properties *.=.sink.JmxSink 2. By default, it doesn’t expose its metrics, but only through the web UI, as mentioned above.
It exposes MBeans of a JMX target (either locally as a Java Agent, or a remote JVM) via an HTTP endpoint, in Prometheus format, to be scraped by Prometheus server. Prometheus JMX Exporter is a JMX to Prometheus bridge. I was about to write an “exporter”, but then Brian Brazil one of Prometheus contributors and the creator of Prometheus JMX Exporter, opened my eyes and suggested to take advantage of Spark’s configurable metrics system and Prometheus’ jmx_exporter, rather than creating a new exporter. However, we need to find a way to scrape it into Prometheus’ DB, where our monitoring metrics are collected and analyzed. Spark already has its own monitoring capabilities, including a very nice web UI and a REST API.
Finding no evidence of how to do that using Prometheus online (shouting a little bit on Twitter, talking with some guys in the IRC channel…), I decided to write this post / walk-through guide. In this case, we need to monitor our Spark application. It is very modular, and lets you easily hook into your existing monitoring/instrumentation systems. It is a relatively young project, but it’s quickly gaining popularity, already adopted by some big players (e.g Outbrain). Prometheus is an “open-source service monitoring system and time series database”, created by SoundCloud. Lately, we decided to give a try to Prometheus. As part of our efforts of establishing a robust back-end, one important part of it is monitoring the environment, hosts and services.
0 kommentar(er)
