Skip to main content

Circonus Logwatch

Circonus Logwatch is a small utility for extracting metrics from log files and forwarding to Circonus.


  1. mkdir -p /opt/circonus/{sbin,etc,etc/log.d}
  2. Download latest release from repository (or build manually)
  3. Extract archive into /opt/circonus
  4. Create config in /opt/circonus/etc and log configs in /opt/circonus/etc/log.d


$ /opt/circonus/sbin/circonus-logwatchd -h
--api-app string [ENV: CLW_API_APP] Circonus API Token app (default "circonus-logwatch")
--api-ca-file string [ENV: CLW_API_CA_FILE] Circonus API CA certificate file (optional, Inside API server not using public certs)
--api-key string [ENV: CLW_API_KEY] Circonus API Token key or 'cosi' to use COSI config
--api-url string [ENV: CLW_API_URL] Circonus API URL (default "")
-c, --config string config file (default is /opt/circonus/etc/circonus-logwatch.(json|toml|yaml)
-d, --debug [ENV: CLW_DEBUG] Enable debug messages
--debug-cgm [ENV: CLW_DEBUG_CGM] Enable CGM & API debug messages
--debug-metric [ENV: CLW_DEBUG_METRIC] Enable metric rule evaluation tracing debug messages
--debug-tail [ENV: CLW_DEBUG_TAIL] Enable log tailing messages
--dest string [ENV: CLW_DESTINATION] Destination[agent|check|log|statsd] type for metrics (default "log")
--dest-agent-interval string [ENV: CLW_DEST_AGENT_INTERVAL] Destination[agent] Interval for metric submission to agent (default "60s")
--dest-cid string [ENV: CLW_DEST_CID] Destination[check] Check ID (not check bundle)
--dest-id string [ENV: CLW_DEST_ID] Destination[statsd|agent] metric group ID (default "circonus-logwatch")
--dest-instance-id string [ENV: CLW_DEST_INSTANCE_ID] Destination[check] Check Instance ID
--dest-port string [ENV: CLW_DEST_PORT] Destination[agent|statsd] port (agent=2609, statsd=8125)
--dest-statsd-prefix string [ENV: CLW_DEST_STATSD_PREFIX] Destination[statsd] Prefix prepended to every metric sent to StatsD (default "host.")
--dest-tag string [ENV: CLW_DEST_TAG] Destination[check] Check search tag
--dest-target string [ENV: CLW_DEST_TARGET] Destination[check] Check target (default hostname)
--dest-url string [ENV: CLW_DEST_URL] Destination[check] Check Submission URL
-h, --help help for circonus-logwatch
-l, --log-conf-dir string [ENV: CLW_PLUGIN_DIR] Log configuration directory (default "/opt/circonus/etc/log.d")
--log-level string [ENV: CLW_LOG_LEVEL] Log level [(panic|fatal|error|warn|info|debug|disabled)] (default "info")
--log-pretty [ENV: CLW_LOG_PRETTY] Output formatted/colored log lines
--show-config Show config (json|toml|yaml) and exit
--stat-port string [ENV: CLW_STAT_PORT] Exposes app stats while running (default "33284")
-V, --version Show version and exit


  • --dest check metrics are sent directly to the circonus broker (will create a check if --dest-cid not provided). --dest-instance-id, --dest-target, and --dest-tag can be used to customize the check created.
  • --dest agent metrics are sent to /write endpoint of local circonus-agent (http://localhost:2609/write/id) uses --dest-id to categorize the metrics. --dest-port controls the agent port (default 2609)
  • --dest statsd metrics sent to statsd listener of local circonus-agent (localhost:8125) uses --statsd-prefix for each metric name, followed by --dest-id (--dest-statsd-prefix should match circonus-agent --statsd-host-prefix to ensure metrics are routed to correct destination by the agent). --dest-port controls the agent statsd port (default 8125)


Create a JSON, YAML, or TOML config in /opt/circonus/etc/circonus-logwatch.(json|yaml|toml). Or, use environment variables and/or command line parameters.

YAML with a StatsD destination (send metrics to local circonus-agent statsd listener):

app_stat_port: "33284"
debug: false
debug_cgm: false
debug_metric: false
debug_tail: false
log_conf_dir: /opt/circonus/etc/log.d
key: circonus api token key
app: circonus-logwatch
url: ''
ca_file: path to api ca file, if needed
type: statsd
id: circonus-logwatch
port: "8125"
statsd_prefix: "host."

JSON with a Check destination (send metrics directly to a Circonus check, a check will be created if not supplied via cid or url config settings):

NOTE: use cid or url to identify an existing check. Or, to search for a check a combination of target, search_tag and/or instance_id.

"api": {
"app": "circonus-logwatch",
"ca_file": "path to api ca file, if needed",
"key": "circonus api token key",
"url": ""
"app_stat_port": "33284",
"debug": false,
"debug_cgm": false,
"debug_metric": false,
"debug_tail": false,
"destination": {
"type": "check",
"config": {
"cid": "check id (of existing check)",
"url": "submission url of existing check",
"target": "to find|create a check",
"search_tag": "to find|create a check",
"instance_id": "to find|create a check"
"log": {
"level": "info",
"pretty": false
"log_conf_dir": "/opt/circonus/etc/log.d"

TOML with an Agent destination (send metrics to local circonus-agent):

app_stat_port = "33284"
debug = false
debug_cgm = false
debug_metric = false
debug_tail = false
log_conf_dir = "/opt/circonus/etc/log.d"

key = "circonus api token key"
app = "circonus-logwatch"
url = ""
ca_file = "path to api ca file, if needed"

type = "agent"

id = "circonus-logwatch"
port = "2609"

level = "info"
pretty = false

Log Configs

Create one config (JSON, YAML, or TOML) in --log-conf-dir for each distinct log. See examples in the repository.

  1. id of the log, short identifier - optional, the base file name will be used if omitted
  2. log_file path to the log
  3. metrics a list of:
    1. match regular expression to identify lines and optionally extract named subexpressions for value and metric name
    2. name a static string to use as the metric name or a template for naming the metric if named subexpressions were used in match regex
    3. type what type of metric (all numbers are 64bit)
      • c counter int
      • g gauge int or float
      • ms timing (treated as histogram) value can be a float (3.12) or duration (25ms, 3.2s, 1m10s, etc.) durations are converted to milliseconds, floats are assumed to already represent milliseconds
      • h histogram float
      • s set (ala statsd set metrics) unique string to count
      • t text string


  • any metric which does not have a type will be treated as a counter.
  • any metric which does not have a subexpression named 'Value' (case insensitive) will be treated as a counter.
  • named subexpressions can be used in the name template with the following syntax {{.id}} where id is the name given to a named subexpression in the match regex
  • metric names will be prefixed with the log id

Manual Build

  1. Clone repo (outside of GOPATH) git clone && cd circonus-logwatch
  2. Build go build -o circonus-logwatchd
  3. Ensure target directories exist mkdir -p /opt/circonus/logwatch/{sbin,etc,etc/log.d}
  4. Install cp circonus-logwatchd /opt/circonus/logwatch/sbin