Skip to main content

Agent Manager

Agent Manager is the ultimate sidekick for your configuration files. It’s always on the lookout for new files and triggers the appropriate restart/reload functionality of the supported agents. With its simple installation and infrequent updates, you can rest assured that your configuration files are always up-to-date and running smoothly.

Once you install the agent manager on a host where your collection agents are located, you can easily manage everything through the intuitive Passport UI.

agent-managers-overview

Installation​

The Circonus Agent Manager is supported on both Linux and macOS operating systems with Windows soon to come. The following installation instructions will guide you through installing Agent Manager on your host(s) and then being able to create/assign configurations to them from the Passport UI.

Pro Tip

Circonus recommends installing one or more supported agents before installing the Agent Manager. If an agent is installed after the Agent Manager's installation then you will need to stop, re-inventory and restart the Agent Manager for it to detect the new agents.

Supported Platforms​

info

If the following guided installation doesn't fit your environment, please review the available Agent Manager packages on the GitHub packages release page.


Step 1 - Add the Manager Repo and Install
  • Note: Regardless of the Ubuntu or Debian version or name, use the following commands to install the Manager.
amd64
sudo wget -O /etc/apt/trusted.gpg.d/circonus.asc https://keybase.io/circonuspkg/pgp_keys.asc?fingerprint=14ff6826503494d85e62d2f22dd15eba6d4fa648 &&
echo "deb http://updates.circonus.net/ubuntu/ jammy main" | sudo tee -a /etc/apt/sources.list.d/circonus.list &&
sudo apt update && sudo apt install circonus-am
Step 2 - Register, restart and view the status

  • Navigate to Passport > Agent Management > Registration to retrieve a valid registration token secret and replace <registrationTokenSecret> with your account registration token secret and then run the following terminal command.

    • Note: This secret can not be retrieved again once the window is closed and a new one will need to be created, so keep this for future use.

  • Agent Manager Tagging - Optional

    • Tags can be added only during registration times by using the --tags flag either before or after the --register flag.
    • If tags need to be added after the initial registration, re-register your agent manager again with the desired tags.
    • Ex: CLI tags: --tags="foo:bar,baz:qux" with , separating the key:val entries.
    • Ex: Environment variables: CAM_TAGS="foo:bar baz:qux" with spaces separating the key:val entries.
sudo /opt/circonus/am/sbin/circonus-am --register="<registrationTokenSecret>" &&
sudo systemctl restart circonus-am &&
sudo systemctl status circonus-am
SUCCESS!

If the registration is successful, then you will see "message": "registration complete" from the Agent Manager and also the status of its service as Active: active (running).

Example - Successful installation

Linux Ubuntu
# Step 1
circonus@ubuntu-host:/etc/apt/sources.list.d$ sudo wget -O /etc/apt/trusted.gpg.d/circonus.asc https://keybase.io/circonuspkg/pgp_keys.asc?fingerprint=14ff6826503494d85e62d2f22dd15eba6d4fa648 &&
echo "deb http://updates.circonus.net/ubuntu/ jammy main" | sudo tee -a /etc/apt/sources.list.d/circonus.list &&
sudo apt update && sudo apt install circonus-am
--2023-09-22 14:14:42--  https://keybase.io/circonuspkg/pgp_keys.asc?fingerprint=14ff6826503494d85e62d2f22dd15eba6d4fa648
Resolving keybase.io (keybase.io)... 52.3.107.250, 52.72.123.132
Connecting to keybase.io (keybase.io)|52.3.107.250|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3122 (3.0K) [text/plain]
Saving to: ‘/etc/apt/trusted.gpg.d/circonus.asc’

/etc/apt/trusted.gpg.d/circonus.asc 100%[===================================================================>]   3.05K  --.-KB/s    in 0s

2023-09-22 14:14:42 (773 MB/s) - ‘/etc/apt/trusted.gpg.d/circonus.asc’ saved [3122/3122]

deb http://updates.circonus.net/ubuntu/ jammy main
Hit:1 http://us-central1.gce.archive.ubuntu.com/ubuntu jammy InRelease
Hit:2 http://us-central1.gce.archive.ubuntu.com/ubuntu jammy-updates InRelease
Hit:3 http://us-central1.gce.archive.ubuntu.com/ubuntu jammy-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease
Hit:5 https://download.docker.com/linux/ubuntu jammy InRelease
Hit:6 http://updates.circonus.net/ubuntu jammy InRelease
Hit:7 https://packages.fluentbit.io/ubuntu/jammy jammy InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
78 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  circonus-am
0 upgraded, 1 newly installed, 0 to remove and 78 not upgraded.
Need to get 4281 kB of archives.
After this operation, 10.7 MB of additional disk space will be used.
Get:1 http://updates.circonus.net/ubuntu jammy/main amd64 circonus-am amd64 0.2.9 [4281 kB]
Fetched 4281 kB in 1s (5300 kB/s)
Selecting previously unselected package circonus-am.
(Reading database ... 124909 files and directories currently installed.)
Preparing to unpack .../circonus-am_0.2.9_amd64.deb ...
Unpacking circonus-am (0.2.9) ...
Setting up circonus-am (0.2.9) ...
Created symlink /etc/systemd/system/multi-user.target.wants/circonus-am.service → /lib/systemd/system/circonus-am.service.
Scanning processes...
Scanning candidates...
Scanning linux images...

Restarting services...
Service restarts being deferred:
 /etc/needrestart/restart.d/dbus.service
 systemctl restart docker.service
 systemctl restart networkd-dispatcher.service
 systemctl restart unattended-upgrades.service

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

# Step 2
circonus@ubuntu-host:/etc/apt/sources.list.d$ sudo /opt/circonus/am/sbin/circonus-am --register="eyJhbGciOiJIUzUxMiIsImtpZCI6IjIzMzAyNjE2ODMwMTgyMDY3NSIsInR5cCI6IkpXVCJ9.eyJhdWQiOlsiY29uZmlndXJhdGlvbnMtYXBpIl0sImNpcmNvbnVzOmFjY291bnQiOiJwcm9kdGVzdGluZyIsImNpcmNvbnVzOmdyYW50cyI6WyJjdXN0b21lcl9wYXNzcG9ydHJlZ2lzdGVyIl0sImNpcmNvbnVzOm9yZ2lkIjoiMjMzMDI2MTY4MzAxODIwNjc1IiwiY2lyY29udXM6dG9rZW4iOiIxYjMwNDViMC1lZjA4LTQ0NDMtOGYwYi0wMDZmNTNjOTdmMzQiLCJleHAiOjIwMTA5NzA3NjQsImlhdCI6MTY5NTM5NDc2NSwiaXNzIjoiY29uZmlndXJhdGlvbnMtYXBpIiwibmJmIjoxNjk1Mzk0NzY1LCJzdWIiOiIwIn0.BtN3jnlv78Ib0Dd5M3wQExlvokiobj3NBEkajRLQ6F9PgugmpxkCVgr2bGHXV6Kav-DopSFjVV31jOxjgz0oCg" --tags="os:linux_deb" && sudo systemctl restart c
irconus-am && sudo systemctl status circonus-am
{"level":"info","cfg_file":"/opt/circonus/am/etc/circonus-am.yaml","time":1695395093,"message":"config file found/used"}
{"level":"info","name":"circonus-am","version":"0.2.9","time":1695395093,"message":"starting"}
{"level":"info","time":1695395093,"message":"starting registration"}
{"level":"warn","file":"/usr/bin/newrelic-infra-service","time":1695395093,"message":"agent binary not found, skipping"}
{"level":"warn","file":"/usr/bin/otelcol","time":1695395093,"message":"agent binary not found, skipping"}
{"level":"info","agent":"telegraf","time":1695395093,"message":"found"}
{"level":"info","agent":"cua","time":1695395093,"message":"found"}
{"level":"warn","file":"/usr/bin/datadog-agent","time":1695395093,"message":"agent binary not found, skipping"}
{"level":"warn","file":"/usr/share/filebeat/bin/filebeat","time":1695395093,"message":"agent binary not found, skipping"}
{"level":"info","agent":"fluent-bit","time":1695395093,"message":"found"}
{"level":"warn","file":"/usr/share/metricbeat/bin/metricbeat","time":1695395093,"message":"agent binary not found, skipping"}
{"level":"warn","file":"/usr/bin/vector","time":1695395093,"message":"agent binary not found, skipping"}
{"level":"info","pkg":"manager","time":1695395093,"message":"registration complete"}
● circonus-am.service - Circonus Agent Manager
     Loaded: loaded (/lib/systemd/system/circonus-am.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2023-09-22 15:04:53 UTC; 23ms ago
       Docs: https://github.com/circonus/agent-manager
   Main PID: 1249575 (circonus-am)
      Tasks: 6 (limit: 9525)
     Memory: 1.4M
        CPU: 4ms
     CGroup: /system.slice/circonus-am.service
             └─1249575 /opt/circonus/am/sbin/circonus-am --config=/opt/circonus/am/etc/circonus-am.yaml

Video Tutorial​


Managing additional collection agents​

If additional agents have been added to the host where Agent Manager is running, then you will need to run a few CMDs for them to become discovered.

Take an inventory of local collection agents
Example: stop, re-inventory, start and view the Agent Manager's status
sudo systemctl stop circonus-am &&
sudo /opt/circonus/am/sbin/circonus-am --inventory &&
sudo systemctl start circonus-am &&
sudo systemctl status circonus-am
Success

Example of what the output will look like when a telegraf agent has been found.

{"level":"info","agent":"telegraf","time":1692044346,"message":"found"}
{"level":"info","pkg":"manager","time":1692044346,"message":"inventory complete"}
Example - Successful re-inventory

Linux Ubuntu
ubuntu-host:~$ sudo systemctl stop circonus-am &&
sudo /opt/circonus/am/sbin/circonus-am --inventory &&
sudo systemctl start circonus-am &&
sudo systemctl status circonus-am
{"level":"info","name":"circonus-am","version":"0.2.3","time":1694466173,"message":"starting"}
{"level":"warn","error":"exit status 127","agent":"cua","time":1694466174,"message":"getting agent version"}
{"level":"info","agent":"cua","time":1694466174,"message":"found"}
{"level":"warn","file":"/usr/share/filebeat/bin/filebeat","time":1694466174,"message":"agent binary not found, skipping"}
{"level":"warn","file":"/usr/bin/otelcol","time":1694466174,"message":"agent binary not found, skipping"}
{"level":"warn","file":"/usr/bin/vector","time":1694466174,"message":"agent binary not found, skipping"}
{"level":"warn","file":"/usr/sbin/datadog-agent","time":1694466174,"message":"agent binary not found, skipping"}
{"level":"info","agent":"fluent-bit","time":1694466174,"message":"found"}
{"level":"warn","file":"/usr/share/metricbeat/bin/metricbeat","time":1694466174,"message":"agent binary not found, skipping"}
{"level":"warn","file":"/usr/bin/newrelic-infra-service","time":1694466174,"message":"agent binary not found, skipping"}
{"level":"info","agent":"telegraf","time":1694466175,"message":"found"}
{"level":"info","pkg":"manager","time":1694466175,"message":"invetory complete"}
● circonus-am.service - Circonus Agent Manager
     Loaded: loaded (/lib/systemd/system/circonus-am.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-09-11 21:02:55 UTC; 20ms ago
       Docs: https://github.com/circonus/agent-manager
   Main PID: 3323503 (circonus-am)
      Tasks: 7 (limit: 9525)
     Memory: 2.6M
        CPU: 4ms
     CGroup: /system.slice/circonus-am.service
             └─3323503 /opt/circonus/am/sbin/circonus-am --config=/opt/circonus/am/etc/circonus-am.yaml


Uninstalling​

To uninstall Agent Manage, run the following CMD.

sudo /opt/circonus/am/sbin/circonus-am --decommission &&
sudo apt remove -y circonus-am &&
sudo rm -r /opt/circonus/am

Troubleshooting​

Agent Manager usage flags

Linux Ubuntu
# Check all available usage flags for the Agent Manager
ubuntu-testing-dev-box:/opt/circonus/am/sbin$ sudo ./circonus-am -h
Manager for local agents (metrics, logs, etc.)

Usage:
  circonus-am [flags]

Flags:
      --agents strings         [ENV: CAM_AGENTS] List of agents (Docker specific)
      --apiurl string          [ENV: CAM_API_URL] Circonus API URL (default "https://agents-api.circonus.app/configurations/v1")
      --aws-ec2-tags strings   [ENV: CAM_AWS_EC2_TAGS] AWS EC2 tags for registration meta data
  -c, --config string          config file (default: /opt/circonus/am/etc/circonus-am.yaml|.json|.toml)
  -d, --debug                  [ENV: CAM_DEBUG] Enable debug messages
      --decommission           Decommission agent manager and exit
      --force-register         [ENV: CAM_FORCE_REGISTER] Force registration attempt, even if manager is already registered
  -h, --help                   help for circonus-am
      --instance-id string     [ENV: CAM_INSTANCE_ID] Instance ID (Docker specific)
      --inventory              Inventory installed agents and exit
      --log-level string       [ENV: CAM_LOG_LEVEL] Log level [(panic|fatal|error|warn|info|debug|disabled)] (default "info")
      --log-pretty             Output formatted/colored log lines [ignored on windows]
      --poll-interval string   [ENV: CAM_POLL_INTERVAL] Polling interval for actions (default "60s")
      --register string        [ENV: CAM_REGISTER] Registration token -- register agent manager, inventory installed agents and exit
      --tags strings           [ENV: CAM_TAGS] Custom key:value tags for registration meta data
  -V, --version                Show version and exit

Check to see if the Agent Manager is running

Linux Ubuntu
# Check the status of the Agent Manager
ubuntu-testing-dev-box:/opt/circonus/am/sbin$ sudo systemctl status circonus-am
● circonus-am.service - Circonus Agent Manager
     Loaded: loaded (/lib/systemd/system/circonus-am.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-08-14 19:34:30 UTC; 2min 23s ago
       Docs: https://github.com/circonus/agent-manager
   Main PID: 1359376 (circonus-am)
      Tasks: 8 (limit: 4680)
     Memory: 8.0M
        CPU: 27ms
     CGroup: /system.slice/circonus-am.service
             └─1359376 /opt/circonus/am/sbin/circonus-am --config=/opt/circonus/am/etc/circonus-am.yaml

Aug 14 19:34:30 ubuntu-testing-dev-box systemd[1]: Started Circonus Agent Manager.
Aug 14 19:34:30 ubuntu-testing-dev-box circonus-am[1359376]: {"level":"info","name":"circonus-am","version":"0.1.3","time":1692041670,"message":"starting"}
Aug 14 19:34:30 ubuntu-testing-dev-box circonus-am[1359376]: {"level":"info","interval":"1m0s","time":1692041670,"message":"starting poller"}

View the logs of Agent Manager while it is running

Linux Ubuntu
# View the logs of the running service
ubuntu-host:/opt/circonus/am/etc$ sudo journalctl -u circonus-am.service
Aug 15 18:07:28 ubuntu-qa21-1-observability systemd[1]: Started Circonus Agent Manager.
Aug 15 18:07:28 ubuntu-qa21-1-observability circonus-am[4079997]: {"level":"info","name":"circonus-am","version":"0.1.3","time":1692122848,"message":"starting"}
...