Skip to main content

Release Notes 2021

2021-12-13

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20211213/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

caql_broker

Version: 0.1.1635874390 (6b64b31)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1637102250 (07c7370)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

fault_detection

Version: 0.1.1637776368 (60a3653)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.10.7

  • No user-facing changes.

Hooper

Version: 0.1.1639170233 (99fe533)

  • Operators may now select from the list of previously hard-coded NNTBS rollup periods to set which ones should be used. The available rollup periods are 1m, 5m, and 3h. If unspecified, all three will be active. To select specific rollups, add a rollups attribute in the data_storage role, with an array of periods to activate:
    "rollups": [ "1m", "5m" ]

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.3

notification

Version: 0.1.1640026848 (b149263)

  • Removed Log4j, replaced with built-in Java logging.
    • The active log file was previously bert.log, now it will be bert.log.0.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1635524746 (2832691)

Web UI/API

Version: 0.1.1639167406 (cb2c2bd)

  • The /graph API endpoints are now forwarded to GoAPI in all environments.
  • Add a database migration to include a primary key for the noit_stratcon table.

web_stream

Version: 0.1.1631023129 (445f7a4)

2021-11-29

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20211129/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

caql_broker

Version: 0.1.1635874390 (6b64b31)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1637102250 (07c7370)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Fix /find/X/active_count to respect query querystring parameter and to force activity_start_secs and activity_end_secs into activity window alignment.

fault_detection

Version: 0.1.1637776368 (60a3653)

  • Fix a race condition that could cause old ruleset thresholds to stay resident until the next restart.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.10.2

  • The /metric endpoint supports string time expressions for start and end times of metric searches.

Hooper

Version: 0.1.1636032514 (567b79b)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.3

notification

Version: 0.1.1627584408 (d5ce67a)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1635524746 (2832691)

Web UI/API

Version: 0.1.1637682545 (d8de073)

  • Fixed a bug with the "Single" dashboard widget when trying to fetch low-frequency CAQL data.
  • Updated integrations list and documentation to become and remain synchronized with the docs portal.

web_stream

Version: 0.1.1631023129 (445f7a4)

2021-11-15

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20211115/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

caql_broker

Version: 0.1.1635874390 (6b64b31)

  • No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1636552998 (4ef7b97)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Allow Graphite find to accept an optional irondb_tag_filter query string parameter.
    • This parameter will take a Circonus-style tag-query and filter the metrics down to only include matching metrics.
  • Fix bug causing nodes to fail to start in ephemeral mode.

fault_detection

Version: 0.1.1635527091 (1ec2c49)

  • Fix crash on unparsable tag queries.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.10.1

  • Improve the error message returned for invalid IRONdb tag queries.
  • To improve readability, JSON data contained in log entries written to log files and log entry data sent to fq will no longer be escaped for HTML display.

Hooper

Version: 0.1.1636032514 (567b79b)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.3

notification

Version: 0.1.1627584408 (d5ce67a)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1635524746 (2832691)

  • Add missing libmtev memory-safety transaction on show check console command.

Web UI/API

Version: 0.1.1636143522 (64474a4)

  • Removed the /admin/broker/new page.
  • Bugfix: rule_set_group API failed to save changes to name attribute.
  • Bugfix: API now returns 404 instead of 403 when object access is not permitted, to not reveal that the id exists.

web_stream

Version: 0.1.1631023129 (445f7a4)

2021-11-01

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20211101/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

caql_broker

Version: 0.1.1634922187 (4a2510c)

  • No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1635284666 (7a20721)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Fix bug in rollup that could cause rollups to get scheduled to run multiple times.
  • Improve reconstitute stability.
  • Fix incorrectly formatted error message when receiving invalid graphite input.
  • Fix potential crash on bad fetches from CAQL.
  • Fix bug in reconstitute that caused unnecessary replaying of data, slowing down the recovery process after a node finishes reconstituting.

fault_detection

Version: 0.1.1631023167 (52dbbd9)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.24

  • To improve readability, JSON data contained in log entries written to log files and log entry data sent to FQ will no longer be escaped for HTML display.

Hooper

Version: 0.1.1632422042 (75ce913)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.3

  • Document time duration strings.
  • Allow the fallback in ENV:fallback:{VAR} and consul:fallback:VAR config statements to include colons.

notification

Version: 0.1.1627584408 (d5ce67a)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1633472520 (f06391a)

Web UI/API

Version: 0.1.1634773060 (0cba474)

  • UI: Alphabetize the integrations list case-insensitively.

web_stream

Version: 0.1.1631023129 (445f7a4)

2021-10-18

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20211018/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

caql_broker

Version: 0.1.1633472520 (45e173c)

  • No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1634253618 (aac2aea)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Propagate /fetch errors and warnings into CAQL intelligently.

fault_detection

Version: 0.1.1631023167 (52dbbd9)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.23

  • No user-facing changes.

Hooper

Version: 0.1.1632422042 (75ce913)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.2

  • Allow spaces between numbers and units in mtev_confstr_parse_duration. Currently the parsing allows spaces between time units, such as "1hr 2min". After this change, a string like "1 hour 2 minutes" will also be allowed.
  • Fix alignment of mtev_hash.
  • Use const for JSON strings in mtev_json_object_from_{fd,file}.

notification

Version: 0.1.1627584408 (d5ce67a)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1633472520 (f06391a)

  • Safe-memory fixes in check handling.

Web UI/API

Version: 0.1.1634241546 (ce032ee)

  • New service dashboard for HAProxy integration.
  • New service dashboard for Memcached Mcrouter integration.

web_stream

Version: 0.1.1631023129 (445f7a4)

2021-10-04

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20211004/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

caql_broker

Version: 0.1.1632755093 (e7358b4)

  • No user-facing changes

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1633012120 (e639f42)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Align fetch remote client-side timeout with expressed server-side timeout.
  • Clip off the right number of leading branch names in graphite results when using prefixes with wildcards.
  • Add filter:not CAQL namespace for any,all,min,max, and mean that inverts the stream selection outcome.
  • GET requests to the /rollup/<check_uuid>/<metric> endpoint that request data for which no rollup exists now return a 400: Bad Request error response.
  • Remove unused /find/<account_id>/all endpoint.
  • Expand CAQL durations to support all mtev duration units.
  • Fix potential crash on tags key/value pairs that are exactly 256 characters long.
  • Fix assertion failure in timeshard transaction.
  • Remove deprecated /raw/<uuid> and /full/<uuid> DELETE endpoints.
  • Remove deprecated /histogram_raw check and metric DELETE endpoints.
  • Remove unused /admin GET and PUT endpoints.
  • Remove deprecated /raw/reconstitute endpoint.
  • GET requests to the /metafeed/check/set endpoint that do not provide the required checkpoint and subscriber query parameters now returns a 400: Bad Request error response.
  • Add new logging utility, debug/old_data, that will log any metric coming into the system that are a configurable amount of time older than the current time. This time value is configurable via the old_data_logging/@metric_age_threshold value in the configuration file.

fault_detection

Version: 0.1.1631023167 (52dbbd9)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.22

  • No user-facing changes

Hooper

Version: 0.1.1632422042 (75ce913)

  • Add hub cron job to clean up notification error logs from Postgres after 90 days.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.1

notification

Version: 0.1.1627584408 (d5ce67a)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1632421276 (51edbd2)

  • Fix bare / in tag searches. For example, and(foo:/) should exactly match / and not be a regex.
  • Restore compiler optimizations.
  • Numerous stack overflow and memory-handling fixes for increased stability.
  • Make metric search parsing more flexible:
    • Allow spaces.
    • Treat // as a greedier match (allowing for more reserved characters).
    • Allow for double-quoted strings as an alternative to base64-encoded b"". Double-quoted strings accept all printable ASCII characters other than " and \, which must be escaped as \" and \\, respectively. See Tag Search Quoting.

Web UI/API

Version: 0.1.1632854309 (c92e4de)

  • Bugfix: newly-invited users are auto-confirmed, so API tokens will work properly for them.
  • Bugfix: fixed hit rate calculations on Memcached dashboards.

web_stream

Version: 0.1.1631023129 (445f7a4)

2021-09-20

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210920/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

caql_broker

Version: 0.1.1631806205 (e02b97a)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1631810865 (48bb33c)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Add helper tool to accelerate Whisper file scanning.
  • Fixed bug with caql_info extension that would cause syntax errors on queries using VIEW_PERIOD or VIEW_RANGE.

fault_detection

Version: 0.1.1631023167 (52dbbd9)

  • No user-facing changes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.20

Hooper

Version: 0.1.1630680277 (c62ebb0)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.1

  • Fix crash when parsing certain unreadable JSON strings.

notification

Version: 0.1.1627584408 (d5ce67a)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1631806205 (927dd0b)

  • Memory usage fixes in numerous check modules.
  • Fix race conditions when accessing the check's stat array pointers.
  • Double-free in filterset handling (crash fix).
  • Reduce build optimization on targeted objects to avoid compiler bug.
  • Fix incorrect reference counts in several check modules.

Web UI/API

Version: 0.1.1631904942 (9e90ed9)

  • Update: removed obsolete bin/checks/automatic_metric_collection.pl, which predates current collect-all-by-default data ingestion.
  • UI: Added service dashboards for 11 Google Cloud services.

web_stream

Version: 0.1.1631023129 (445f7a4)

  • No user-facing changes.

2021-09-06

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210906/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

  • Updated CRL handling. Move expired certificates to a separate directory, causing them to be omitted from the CRL.

caql_broker

Version: 0.1.1630353905 (e02b97a)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1630363120 (49e5340)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Remove deprecated /raw/reconstitute/metrics endpoint.
  • Fix issue in /fetch where absent/null values could be returned as zeroes.
  • Add locking to consolidate_ingestion_logs.pl so that we don't run multiple copies.
  • Fixed CAQL histogram:percentile(x,y,z) to maintain the user-specified ordering.
  • Deprecate <pools> configuration. Jobq settings are now done via mtev eventer configuration.

fault_detection

Version: 0.1.1629814098 (b75a556)

  • No user-facing changes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.20

Hooper

Version: 0.1.1630680277 (c62ebb0)

  • Remove obsolete resource configuration from snowth.conf template.
  • Update circonus-snowth systemd config with better ExecStop method.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.0

notification

Version: 0.1.1627584408 (d5ce67a)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1630353905 (2c1c762)

  • Broker: Obsolete C versions of MySQL and PostgreSQL check modules are no longer part of the broker package.
  • Broker: Reference-counting fixes in the ping_icmp check module.
  • Broker: Memory leak fixes in check object handling.
  • Broker: Memory leaks, races, and crash fixes in various check modules.

Web UI/API

Version: 0.1.1630602413 (ba5005d)

  • Add failed notification warnings to the alert detail page
  • Inside: Selfchecks for web/api HTTPS services will use the correct FQDN to avoid certificate errors.

web_stream

Version: 0.1.1629816536 (33fe800)

  • Performance improvements from moving some frequent work into job queues.

2021-08-23

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210823/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1628184123 (e02b97a)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1629420828 (8d826ad)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Fix bug where raw rollups would occasionally start prematurely.
  • Fetch API transform 'none' will now show an error on numeric streams.
  • Fix bug causing occasional NNTBS data corruption on raw numeric rollups.
  • Remove outlier reports.
  • Consolidate DELETE endpoints around /tag versions.

fault_detection

Version: 0.1.1628516584 (7748081)

  • No user-facing changes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.20

Hooper

Version: 0.1.1627305461 (a981b16)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.0

  • Update jQuery (web UI) to 3.6.0.
  • Add convenience function for getting the backlog value of a jobq.

notification

Version: 0.1.1627584408 (d5ce67a)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1629147533 (fc0a18e)

  • No user-facing changes.

Web UI/API

Version: 0.1.1629384892 (f8b9cf3)

  • Inside: Update external monitoring script with new stream-tagged names and fix submission URL.
  • Inside: Safeguard on application CSR upload script to ensure the file is a CSR.
  • Bugfix: Login would redirect to /user/profile instead of fully-qualified URL.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-08-09

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210809/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1628184123 (e02b97a)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1628178960 (76a4a95)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Fix crash that can occur when shutting down the raw database at the end of a reconstitute.

fault_detection

Version: 0.1.1628086660 (7451661)

  • Fix "unknown numeric predicate" message in the Rules Explorer UI for numeric equals/not-equals rules.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.20

Hooper

Version: 0.1.1627305461 (a981b16)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.0.4

notification

Version: 0.1.1627584408 (d5ce67a)

  • Bugfix: Possible indefinite delay of enterprise broker outage or ruleset group alerts.
  • Bugfix: Possible duplicate alerts sent when a case comes out of maintenance.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1628184123 (b0af5c8)

  • No user-facing changes.

Web UI/API

Version: 0.1.1628111632 (dfa80f4)

  • Feature: Docker dashboard.
  • UI: Menu options to force clear alerts for multi-metric (aka pattern-based) rulesets.
  • Bugfix: Editing ruleset group rule expression could get browser stuck in a loop.
  • Bugfix: CAQL graph data formula applied twice in list mode.
  • Feature: Dashboard template variable queries may reference other variables.
  • Bugfix: Corrected datepicker hour/minute dropdowns.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-07-26

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210726/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1625158873 (e02b97a)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1627057335 (cbc6dfb)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • If corruption is detected in an NNTBS shard, offline it instead of exiting with a fatal error.
    • Operators should be on the lookout for errorlog messages matching one of these patterns:
      Error committing txn: <return code> (error description)
      Error putting to lmdb cursor: <return code> (error description)
      If these logs appear, contact Circonus Support for help in remediating the issue.
  • Fix race condition in single-shard NNTBS reconstitute.
  • Fix crashes when trying to use graphite find on a node that is not participating in any topology.
  • Fix memory leaks when fetching graphite data.
  • Enable LMDB metadata sync for NNTBS shards.
  • Allow NNTBS shards to close properly on shutdown.
  • Fix memory leaks in the find path
  • Switch /rollup//<metric_name> endpoint to accept type=<derivative|derivative_stddev|derivative2|derivative2_stddev>
  • The utility snowth_lmdb_tool now supports a new "dump" sieve that can dump an entire NNTBS shard as text with human-readable surrogate id and timestamp fields.
  • Add Swagger documentation to be served directly out of IRONdb on /api/index.html.
  • Performance improvements to the reconstitute process - startup is now considerably faster.
  • Clean up internal status files when a reconstitute completes.
  • Node IDs in the Topology UI tab are now clickable, and lead to that node's UI.

fault_detection

Version: 0.1.1624976101 (1b75360)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.20

Hooper

Version: 0.1.1627305461 (a981b16)

  • Remove the circonus-field-broker-crashreporter package on caql-broker hosts. This package was previously used just to pull in the necessary dependencies for crash reporting, but is now obsolete. The dependencies are now pulled in directly by the crash_reporting recipe.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.0.4

  • Fix race in usage of SSL_get_ex_new_index.
  • mtev.thread_self() now also returns thread name.
  • Fix error checking in mtev_reverse.
  • Fix illegal write in string prepend initializer.

notification

Version: 0.1.1627063853 (edf1169)

  • Bugfix: possible duplicate alerts for broker outage (present in 2021-07-12 release).
  • Bugfix: correct possible concurrency issues in timer thread.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1625158873 (0e71f6e)

Web UI/API

Version: 0.1.1627058121 (c9ea987)

  • Improvement: added check_bundle._check_status to get data from enterprise brokers via API.
  • Improvement: support unlimited Just-Send-Data check filtersets via auto_add_multiplier=0.
  • Feature: Template variables for custom dashboards.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-07-12

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210712/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1625158873 (e02b97a)

  • No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1625779910 (0684edc)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Remove debug message that would spam the errorlog when running a reconstitute.
  • During a rebalance, sending data for an NNTBS shard the destination node is not configured to handle no longer crashes the destination node.
  • Remove /hist_shard/reconstitute/surrogate and /hist_shard/reconstitute/metrics endpoints.
  • Fix bug where extraneous data would be filled in when populating values for tag categories.
  • Speed up NNTBS reconstitute/rebalance by eliminating unpacking and repacking of nntbs_block_t.
  • Fix bug in rollups where data ingested after a shard has previously rolled up could get erroneously deleted.
  • Rename /raw_only_delete to /clean_rollup_suppressed_metrics for clarity. This endpoint is primarily for manual debugging and should not typically be used by operators.

fault_detection

Version: 0.1.1624976101 (1b75360)

  • No user-facing changes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.20

  • If a graph is posted with a duplicate title to an existing graph, the existing graph will now be updated with the posted values, rather than returning an error.
  • Graph types will now allow numeric and string values for datapoint alpha values.
  • When proxy errors occur the actual error message will now be returned to the caller, not just a message about too many proxy retries.

Hooper

Version: 0.1.1625251835 (039d05e)

  • Remove obsolete tokens from circonus.conf.
  • Only update crash-reporting packages if not in maintenance mode.
  • Fix first-time setup issue with snowth.conf template.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.0.3

notification

Version: 0.1.1625692628 (22304b4)

  • Bugfix: ruleset group alert could be left open+empty if members cleared during processing.
  • Improvement: Java 16 update and GC settings.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1625158873 (0e71f6e)

  • Broker: Update Jezebel GC settings for Java 16

Web UI/API

Version: 0.1.1625772640 (8419b52)

  • Bugfix: Trim trailing whitespace when auto-copying code in the UI.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-06-28

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210628/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1624476875 (e02b97a)

  • No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1624543869 (f65d728)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • There is a known bug discovered post-release that affects the display of tag values in the UI. If possible, this release should be skipped. Please contact Circonus Support if you need to update to this release.
  • Implement exclude:label(<re>) in CAQL
  • Implement exclude(<tag search>) in CAQL.
  • Fix potential deadlock if surrogates fail to write while using the surrogate2 surrogate backend by asserting.
  • Implement graphite:tagbynode("tag", p1, ...) in CAQL.
  • Snowth tag_cats and tag_vals should work with __name.
  • Snowth tag_cats and tag_vals should work with __check_uuid.
  • Make X-Snowth-Proxied header values of '0' or 'off' mean that same thing as if no header is set.
  • Fix wrong response code on invalid query responses.
  • Allow trailing whitespace in JSON documents POSTed to lua extensions.
  • Remove "sweep delete" API endpoints, since this function was for the old NNT backing store and is made redundant by setting retention policies on data.
  • Remove NNT backing store support.
  • Remove support for parts elements when inserting NNTBS data directly.

fault_detection

Version: 0.1.1624381548 (8aa7d47)

  • No user-facing changes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.18

Hooper

Version: 0.1.1624627280 (f7c8f4c)

  • With the removal of NNT support in IRONdb, the site.json config options backing_store and one_minute_rollup_since are no longer relevant and will be ignored. The <rollups> configuration stanza is also no longer configured. Numeric rollups are now exclusively controlled via <nntbs>.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.0.3

notification

Version: 0.1.1622737432 (57d2094)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1624476875 (57f39eb)

  • The circonus-field-broker package now installs automated crash reporting by default. To avoid package upgrade errors, please ensure that each broker's package repository configuration includes the crash-reporting entry, as documented in the User Manual. If local network policy prohibits uploading crash reports, they may be disabled with an environment variable in /opt/noit/prod/etc/noit.local.env.

Web UI/API

Version: 0.1.1624494876 (c47edb2)

  • Support modules' dark mode logos on service dashboards.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-06-14

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210614/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1622603286 (e02b97a)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1623350037 (e168803)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Add connect_timeout, speed_timeout, and speed_limit options to the check tag replicator with reasonable defaults.
  • Add normalize CAQL function.
  • Update /find/<account>/* endpoints to be /find/* with X-Snowth-Account-Id header.
  • Make anomaly_detection in CAQL apply to all slots.
  • Fix crash on null topology.

fault_detection

Version: 0.1.1621261402 (e90a961)

  • No user-facing changes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.18

  • Correct an issue in the graph service to allow additional date formats in the access key lock ranges.
  • Limit retries of IRONdb proxy requests returning 500 errors to one retry. This avoids a breaking call spreading across a cluster.
  • Fix a bug so that type and module can now be used as aliases for check_module in the order parameter when searching using the /alert endpoint.

Hooper

Version: 0.1.1623436562 (1c6d49c)

  • (hub) Add script for populating stratcon UUIDs in Postgres.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.0.3

  • Fix race condition in eventer.

notification

Version: 0.1.1622737432 (57d2094)

  • Output "FORMAT ERROR" into message where template contains malformed tag or similar.
  • Reduce logging on ruleset group case intake.
  • Better internal tracking and error recording for notification attempts.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1622603286 (ec033d2)

  • Fix crash bug in message decoding.

Web UI/API

Version: 0.1.1623434140 (dd9b532)

  • Support for new PagerDuty service key install flow.
  • Add a unique constraint to the noit_reconstitute_queue table.
  • Fix empty contact group selector in Dashboard alert widget.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-05-31

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210531/x86_64/

Note: The installation/upgrade role ordering has been updated to explicitly include the api role. This ensures that the API will be updated before any of its consumers.

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1619710957 (e02b97a)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1620933777 (30a664a)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

fault_detection

Version: 0.1.1621004832 (de9ce42)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.16

  • Remove the deprecated link, notes, and tags fields from results returned by the /metric endpoint.

Hooper

Version: 0.1.1621017371 (4613e70)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.0.1

notification

Version: 0.1.1620071084 (1888555)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1619710957 (f13d405)

Web UI/API

Version: 0.1.1622065261 (eecf075)

  • UI: add service dashboards for Unified Agent installations on MacOS/Darwin and FreeBSD.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-05-17

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210517/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1619710957 (e02b97a)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1620933777 (30a664a)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Fix reconstitute issue where it was possible to try to write to a transaction after the transaction was committed, leading to potential data corruption.
  • Allow for whitespace before (...) and {...} in CAQL function invocations.
  • Allow forcing the reconstitute process to skip specific nodes.
  • Add a graphite translate endpoint to assist graphite -> CAQL translation.
  • Add accounting stats for metrics_db cache.

fault_detection

Version: 0.1.1621004832 (de9ce42)

  • Increase the size of the negative-match (miss) cache.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

GoAPI

Related roles: api

Version: 0.9.15

  • Correct a bug so that name and check_name can now be used as aliases for display_name in the order parameter when searching using the /check endpoint.
  • Requests to IRONdb will now include a X-Snowth-Timeout header. The value will be 1 second less than the total request timeout configuration value. This will prevent IRONdb from continuing to process a query after GoAPI has given up.
  • Fix an error causing custom dashboards to only return with the first 21 widgets associated with a dashboard.

Hooper

Version: 0.1.1621017371 (4613e70)

  • Remove checks for obsolete packages, services, and cron jobs that all deployments have now migrated past.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.0.1

  • Fix race in mtev_memory_gc that could potentially cause crashes.

notification

Version: 0.1.1620071084 (1888555)

  • Reduce backend messaging on deleted ruleset for handling of active alerts (performance improvement).

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1619710957 (f13d405)

  • No user-facing changes.

Web UI/API

Version: 0.1.1620937081 (8d8e1e9)

  • UI: updated Integrations library so it's easier to find the Integration you're looking for; also separated out Agents into their own area.
  • Bugfix: ruleset parent change (via UI) would not notify alert service.
  • Internals: improved error handling and retry for database migrations.
  • Internals: prevent deletion of an active check (ensures related cases+rulesets transition correctly.)
  • Bugfix: clearing open cases (for all-checks rulesets) in alert service when check is disabled.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-05-03

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210503/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1619108717 (e02b97a)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1619647223 (da66549)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Increase default rollup concurrency for raw numeric and histogram shards from 1 to 4 jobq threads.
  • Add op:mod, each:mod, and % to CAQL.
  • Do not coarsen fetches windows in window:mean or window:sum when period kwarg is provided.
  • Fix memory leak in histogram ingestion.
  • Add find:sum(...) to CAQL that will return count * average.
  • Allow implicit type shifts in CAQL: op:div(){pass(){1,2} | vector(), 2}.
  • Implement derivative() and counter() CAQL functions that perform per-second calculations.
  • Implement filter:values:{quantile,percentile}:<op> and filter:values:not:{quantile,percentile}:<op> in CAQL to remove stream outliers.

fault_detection

Version: 0.1.1619110760 (8e18adf)

  • Fix memory leaks in ruleset handling.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

  • Fix race condition in client connection status.

GoAPI

Related roles: api

Version: 0.9.12

  • Add the counter_cum value to the values accepted for custom dashboard metric query transforms.
  • Correct an error that caused incorrect decoding of the URL %2F escape character when looking up metrics by ID.
  • Correct an error that could cause a not-found result when looking up a metric by cid with a name containing mixed base64 and non-base64 Stream Tag values.

Hooper

Version: 0.1.1619793087 (3980149)

  • Webstream config updated to better utilize available CPUs for fq processing.
  • FQ package now restarts fqd on update if necessary, so it no longer needs to be restarted by Hooper.
  • Update Circonus corporate mailing address.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.0.0

  • Update aggregate logging construction to work with C++. This breaks the ABI from 1.x versions.

notification

Version: 0.1.1617737913 (c4fd4dc)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1619108717 (2420e5f)

  • No user-facing changes.

Web UI/API

Version: 0.1.1619798450 (7ae8a95)

  • Fix team page issue where a team member's role label was not updated immediately when their role was changed.
  • Fix display of base64-encoded tags in tag lists throughout the app.
  • Optimize ruleset change notice to alert/notification services.
  • Fix security vulnerability where a user could delete a ruleset which did not belong to them.
  • Make ruleset view return 404 for id not found/not yours.
  • Make new v2 dashboard builder available in the UI.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-04-19

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210419/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1616788161 (e02b97a)

Pre-upgrade Notice

The 2021-03-22 release changed the way CAQL metrics are labeled. Previously, if the label function was not used, resulting metrics would be named output[N]. As of the 2021-03-22 release, CAQL metrics are labeled based on the computation that led to the particular output.

This can cause the names of existing metrics to change, if no explicit label was given in the CAQL expression. To avoid this, run the following script on your "hub" node prior to updating if you are on a release earlier than 2021-03-22:

/www/bin/inside/label_caql_metrics.pl

This will locate active CAQL checks across all accounts that do not have explicit labels, and configure them to preserve the existing output[N] names. It is idempotent, so it is safe to run it multiple times if necessary.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1618349681 (fd5134c)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • No user-facing changes.

fault_detection

Version: 0.1.1618585059 (cee7e74)

  • Fix potential memory leaks and double-free crashes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.9

GoAPI

Related roles: api

Version: 0.9.9

  • No longer trigger a database (500) error if an invalid custom dashboard CID or user CID is sent to the /custom_dashboard/user_favorites endpoint.

Hooper

Version: 0.1.1616172883 (5ba95b0)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.18

notification

Version: 0.1.1617737913 (c4fd4dc)

  • Performance optimizations to backend messaging and internal handling of ruleset change notifications.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1618250506 (d6990a1)

  • (broker) Fix XML entity encoding issue that can cause configuration sync issues in broker clusters.

Web UI/API

Version: 0.1.1617904514 (b7295d0)

  • UI: Fix query builder tag processing bug, where a tag value of / was misinterpreted as a regular expression.
  • Quiet uninitialized warning for customers that don't have Slack installed.
  • UI: Fix for incorrect "latest value" selection for metrics that have been collected as both numeric and histogram.
  • UI: Only show broker decommission button on the account that owns the broker, for cases where a broker is shared between accounts.

web_stream

Version: 0.1.1616613015 (fbff434)

2021-04-05

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210405/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1616788161 (e02b97a)

Pre-upgrade Notice

The 2021-03-22 release changed the way CAQL metrics are labeled. Previously, if the label function was not used, resulting metrics would be named output[N]. As of the 2021-03-22 release, CAQL metrics are labeled based on the computation that led to the particular output.

This can cause the names of existing metrics to change, if no explicit label was given in the CAQL expression. To avoid this, run the following script on your "hub" node prior to updating if you are on a release earlier than 2021-03-22:

/www/bin/inside/label_caql_metrics.pl

This will locate active CAQL checks across all accounts that do not have explicit labels, and configure them to preserve the existing output[N] names. It is idempotent, so it is safe to run it multiple times if necessary.

  • No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1617130292 (8d2b0ad)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Fix possible deadlock on histogram fetch errors.
  • Add site-local extensions config file, /opt/circonus/etc/irondb-extensions-site.conf.
  • Make rollup and delete timing behavior more accurate, especially after crash or restart.
  • Replication Latency UI tab bugfix: now each node's latency is still calculated even if its sub-list isn't expanded (regression in 2021-03-22).
  • Fix potential crash when fetching metrics with very large names.
  • Fix potential deadlock in raw database rollups.
  • Update artmap file version from 1 to 2. Metric artmap files (cached tag-search indexes) will regenerate upon updating to this version, increasing search accuracy. This will cause the first service restart after upgrading to this version to be slower than normal. Take care during update not to have too many nodes restart at the same time.

fault_detection

Version: 0.1.1616595856 (2948129)

  • Fix race condition when setting up timed events.
  • Properly decrement reference count for removed eventer objects (memory leak).

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.9

GoAPI

Related roles: api

Version: 0.9.8

  • Fix an error causing the wrong status code to return after a call to POST /acknowledgement.
  • Change the alert force clear endpoint to POST /alert/{id}/clear and expose it to the API for use.
  • Include the search terms severities and severity, in addition to the already existing _severity, when searching for Alerts.
  • Several fixes to make the /maintenance endpoint match documented behavior:
    • Fix parsing of shorthand string notation for maintenance severities.
    • Allow JSON arrays of ints to be used for maintenance severities.
    • Use a default value of [1,2,3,4,5] for maintenance severities.

Hooper

Version: 0.1.1616172883 (5ba95b0)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.18

notification

Version: 0.1.1617220746 (3a888a6)

  • Add resmon counters for alert type, reduce logging of clear-to-clear message noise.
  • Correct rare case in sev transition that could leave the first sev in alert.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1616788161 (52b599c)

  • (broker, caql_broker) Move check storage into a non-default jobq. At high frequency, check updating could cause watchdog timeouts.

Web UI/API

Version: 0.1.1617375010 (79ff9a1)

  • Reduce the maximum number of Apache child processes for the web_frontend configuration, to avoid exceeding the default number of allowed database connections. To restore the previous values, create a file with a .conf extension in the /www/etc/local-apache/ directory with these lines:
    ServerLimit 128
    MaxClients 128
    but note that the max_connections limit for Postgres may need to be raised. See web_db tuning for details.

web_stream

Version: 0.1.1616613015 (fbff434)

  • No user-facing changes.

2021-03-22

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210322/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1615828651 (14a4222)

Pre-upgrade Notice

This release changes the way CAQL metrics are labeled. Previously, if the label function was not used, resulting metrics would be named output[N]. As of this release, CAQL metrics will be labeled based on the computation that led to the particular output.

This can cause the names of existing metrics to change, if no explicit label was given in the CAQL expression. To avoid this, run the following script on your "hub" node prior to updating to this release:

/www/bin/inside/label_caql_metrics.pl

This will locate active CAQL checks across all accounts that do not have explicit labels, and configure them to preserve the existing output[N] names. It is idempotent, so it is safe to run it multiple times if necessary.

  • Increase the size of the negative-match (miss) cache. This tracks metrics that are not part of any CAQL checks, to reduce CPU load for evaluation of the incoming metric flow.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1616097284 (0872ea8)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Improved web UI performance: the Replication Latency tab now won't update unless it's visible.
  • Fixes to recovering after an incomplete NNTBS live reconstitute and added automatic backup that can be overridden using the optional (backup=0) parameter.
  • Improved error checking and logging for jlog read/write errors.

fault_detection

Version: 0.1.1616159406 (b6955ca)

  • Increase the size of the negative-match (miss) cache. This tracks metrics that are not part of any rulesets, to reduce CPU load for evaluation of the incoming metric flow.
  • Crash fixes in ruleset handling.
  • Performance improvements.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.9

  • Fix file descriptor leak on connection error.
  • Correct usage of pthread_* return values. Remove invalid use of volatile.
  • Add missing volatile in Java client, and replace LinkedList with ArrayDeque.

GoAPI

Related roles: api

Version: 0.9.5

  • Correct an error that caused failures for some searches using the order query parameter.
  • Update SQL queries to be able to handle far future dates. (Y2038)

Hooper

Version: 0.1.1616172883 (5ba95b0)

  • The web_frontend Apache configuration file is now provided with the package, and as such will be kept in sync with the configuration used in the Circonus SaaS environment. Changes with this sync:
    • If the web_frontend tier is behind a load balancer that adds the X-Forwarded-For HTTP header, the requesting client's IP will be logged. Previously only the load balancer's IP would have been logged.
    • For additional security against click-jacking, the X-Frame-Options and Content-Security-Policy headers will be added, to prevent the site from being rendered in a frame or other embedded context
    • Cache-control headers added for additional static assets.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.18

  • More protections from NPE in eventer SSL code.
  • Add console-accessible statistics for the mtev_fq module via show fq.
  • (fault_detection) Fix assertion failure in AMQP code.

notification

Version: 0.1.1614804524 (2b9910e)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1615907911 (a4f87d1)

  • (caql_broker, fault_detection) Performance improvement in metric director.
  • Fix inverted probability for metric director miss cache replacement.

Web UI/API

Version: 0.1.1616186886 (4d7c104)

  • Optimize ruleset storage and handling.
  • Add 'equals' and 'does not equal' criteria support for numeric rulesets.

web_stream

Version: 0.1.1614106164 (7bb508e)

2021-03-08

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210308/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1614713390 (b325186)

  • Add site-local config file for overriding eventer configuration (/opt/noit/prod/etc/circonus-eventer-site.conf). Changes to this file will be preserved across package updates.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1614794509 (c3a5701)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Improve logging on data journaling errors and fix some logic holes that could lead to infinite loops.
  • Add requirement to single-shard NNTBS live reconstitute to specify if the shard should be replaced with data from other nodes (merge=0) or if data from other nodes should be merged into the preexisting shard data (merge=1).

fault_detection

Version: 0.1.1612819317 (7806f93)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.8

GoAPI

Related roles: api

Version: 0.9.4

  • Allow alerts to be searchable by acknowledged status.
  • Fix problems with historical searches for alerts.
  • Correct a bug in the CORS request header processing.
  • The /maintenance endpoint will now accept quoted values for numeric fields.
  • Add support for new b64 syntax in the v3 query parser.

Hooper

Version: 0.1.1614953790 (974f434)

  • Fix resource dependency issues that could interfere with service restarts on changes.
  • Remove obsolete IfDefine directives from web_frontend and api Apache configuration templates.
  • Add optional support for ZFS storage to the web_db and long_tail_storage service roles. ZFS support may only be configured during initial deployment of a host in one of these roles. There is no support for migrating a deployed host to ZFS storage.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.17

notification

Version: 0.1.1614804524 (2b9910e)

  • Correct updating of maintenance windows where a previous definition could mask an alert.
  • Add resmon counters for incoming message type.
  • Add support for ruleset_user_json webhook attribute.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1614713390 (d8a9ca2)

  • (broker) Fix SNMP v2c batched mode OID filling and reqid assignment.
  • (broker) Fix null-pointer exception in snmp module.

Web UI/API

Version: 0.1.1614791445 (67c6194)

  • (UI) Add links from ruleset page to contact groups and checks.

web_stream

Version: 0.1.1614106164 (7bb508e)

  • No user-facing changes.

2021-02-22

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210222/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

caql_broker

Version: 0.1.1613508844 (81bec98)

  • No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1614026691 (89a6713)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Fix numerous races dealing with time shard manipulation
  • Fix race condition when setting a single shard into maintenance mode
  • Fix bug that could leak an entire time shard.
  • Fix bug that could leak an LMDB transaction leading to database corruption.
  • Fix race conditions when handling time shards.
  • Fix bug where extremely old raw data could be inserted into the database via flatbuffer.
  • Fix bug where attempting to roll up an NNTBS shard with a floor value of 0 would cause a crash.
  • Fix crash in Graphite find with x-snowth-adivisory-limit header.
  • /fetch histogram sum, count_above, count_below should normalize per minute.
  • Add VIEW_{PERIOD,RANGE}_{MINUTES,SECONDS} CAQL variables.
  • Add CAQL /fetch optimizations for graphite:find:*.
  • Allow histogram transforms to work in the reduce phase for /fetch using each_ prefix.
  • Fix groupby_* default label construction in fetch.
  • Make /fetch append transaction pipelines to labels.
  • Fix transaction pipeline appends to occur between the untagged metric name and any stream tags.
  • Add collapsed /fetch optimizations for group_byin CAQL.

fault_detection

Version: 0.1.1612819317 (7806f93)

  • No user-facing changes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.8

  • Upgrade jquery to 3.5.1 for management UI

GoAPI

Related roles: api

Version: 0.9.0

Hooper

Version: 0.1.1613139929 (7ac583c)

  • Configuration check will ensure that the master attribute is set for web_db if multiple hosts are in this role.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.17

notification

Version: 0.1.1611166949 (3f20f18)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1613508844 (71c55f4)

  • (broker) Trim spaces and quotation marks out of string values before attempting to coerce them to numeric.
  • Crash fixes related to Lua coroutine cleanup.
  • (libnoit) Make tag helper APIs public.

Web UI/API

Version: 0.1.1613674902 (48aa85a)

  • Check secrets will now default to a random string.
  • Add Strict-Transport-Security (HSTS) Header to responses to force HTTPS on all secure installs.

web_stream

Version: 0.1.1591797218 (3c0de1a)

2021-02-08

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210208/x86_64/

ca

Version: 0.1.1612472455 (ff6ddf6)

  • The circonus-ca_processor service will now automatically detect when its PostgreSQL connection goes away, and reconnect. This removes the need to restart the service when the web_db Postgres service is restarted.
  • Update service from SysV-init to systemd.

caql_broker

Version: 0.1.1611672414 (6d33887)

  • Show check cluster selection based on CAQLbroker's custom method of assigning checks to nodes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1612556757 (a2acce9)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Log NNTBS timeshard being referenced at the time of an LMDB-related crash, to aid in repair.
  • Use external numlua.
  • Added a new FIFO to streamline raw fetch, reducing latency and memory requirements.
  • Fix CAQL optimizer and allow push-down of more chains into direct /fetch expressions.
  • Implement sort:label(reverse=0) and sort:{mean,min,max}(reverse=0) CAQL functions.
  • Implement filter:limit(N), filter:label(regex), and filter:not:label(regex) CAQL functions.
  • Implement filter:{mean,min,max}(X) and filter:not:{mean,min,max}(X) CAQL functions.
  • Implement histogram:random() to synthesize histograms for common density functions.
  • Implement randomwalk(max,min,change,seed=) CAQL function for generating random data.
  • Implement graphite:aliassub in CAQL to emulate Graphite's aliasSub function.
  • Implement stats:ratio(of=1) in CAQL to calculate each input stream over the sum of streams.
  • Added optional verbose rollup/delete debug logging.
  • Fix race in raw data rollups where we could attempt to roll data up before opening the db.
  • Support vectorized operations across most CAQL functions (stats and ops).
  • Add several Graphite-style functions to assist those migrating from Graphite.
  • Allow CAQL #pragmas to have values.
  • Allow CAQL #pragmas to be one-line (no trailing line feed required).
  • Support #min_period=Xs as a granular replacement for hf:find().
  • Leverage level_index for tag searches on __name that are explicitly using the [graphite] matching engine (e.g. and(__name:[graphite]*.prod.**.count)).
  • Update to new libnoit metric search APIs.
  • Further improvements to activity tracking accuracy.

fault_detection

Version: 0.1.1611592814 (6c443eb)

  • Adopt new libnoit search APIs.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.7

  • Queue drops are tracked as dropped_to.
  • Web UI updated to display queue drops/rate.
  • -b option deprecated, -B added, and Backtrace integration (libbcd) is now disabled by default.

GoAPI

Related roles: api

Version: 0.9.0

  • Add support for new b64 syntax in the v3 query parser.
  • Correct a bug in origin verification for API CORS headers.
  • Add a default X-Snowth-Advisory-Limit header to all proxy requests.
  • Correct the implementation of query limits on IRONdb tag searches.
  • Correct a metric search error that caused some base64 encoded regex searches to fail.

Hooper

Version: 0.1.1612476339 (ddf57af)

  • Accommodate CA service move to systemd.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.17

  • Decode + to space in querystring decoding.

notification

Version: 0.1.1611166949 (3f20f18)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1611619043 (b9b5cbd)

  • Fix tag search NPEs.
  • Fix memory issues with tag search.
  • Register noit_check_t * ctype with mtev lua hooks.
  • (broker) Write a whole line at a time for console command show timing_slots.
  • Update web console to latest libmtev web bits.
  • Introduce API for metric search with opaque types.

Web UI/API

Version: 0.1.1612802676 (40816cf)

  • Bugfix: API check_bundle._reverse_connection_urls address:port values were incorrect when reverse-brokers did not have an ipaddress assigned.
  • New broker slots will no longer have .noit.circonus.net appended to their CNs.
  • Update: The /custom_dashboard API endpoints have been forwarded to GoAPI for all installations.

web_stream

Version: 0.1.1591797218 (3c0de1a)

2021-01-25

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210125/x86_64/

caql_broker

Version: 0.1.1608059216 (bdaa081)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1610656849 (865c4d0)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Improvements to activity tracking accuracy.
  • Added optional logging and increased error reporting for raw shard rollups and deletes.
  • Improve accuracy when compacting metricsdb by accounting for out-of-order surrogates arriving.
  • histogram:count* and histogram:rate* CAQL functions now work on sub-minute periods.
  • histogram:count* CAQL functions now return raw counts instead of per-minute values.
  • histogram:rate* CAQL functions now take an option named period to specify the denominator units (default: per second).
  • histogram:count_bucket and histogram:rate_bucket have been renamed histogram:count_bin and histogram:rate_bin, respectively.
  • [raw_ingestor] Performance improvements when sending data to jlog.
  • Reconstitute process now pulls activity data from the surrogate db rather than relying on raw data to fill it in. This will prevent the possible loss of activity data.

fault_detection

Version: 0.1.1610649467 (13af349)

  • Fix erroneously doubled initial absence window on pattern-based rules.
  • Support check tags in notifications. Fault will now include check-level tags in messages to the Notification system, enabling them to be included in alerts.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.5

GoAPI

Related roles: api

Version: 0.8.20

  • Correct a bug that caused the _cannonical_metric_name and _signature fields not to populate when looking up an individual /alert.
  • Remove redundant cache invalidations that could cause a race condition in the API authentication token application approval process.
  • Correct an error that could cause certain values for a search query order parameter to return a 500 error.

Hooper

Version: 0.1.1610404291 (7f14c8b)

  • If the external_host API attribute is defined, and the certificate type is set to internal, use the external_host value. Previously in this situation, the certificate would always be issued for api.[domain].

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.16

  • (broker, caql_broker, fault_detection) Fix potential off-by-one error in clustering.
  • (broker, caql_broker, fault_detection) Fix stack smash on cluster hash table lookups on clusters of more than 16 nodes.
  • (broker, caql_broker, fault_detection) Increase maximum cluster size to 256 nodes.

notification

Version: 0.1.1611166949 (3f20f18)

  • Shortened and reduced debug log output.
  • Optimized message formatting, limit message detail/size.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1610653240 (c71a100)

  • (broker) Fix crash in SNMP check module.
  • (broker) Performance improvements in the StatsD check module.
  • (broker) Add available and ssl_available metrics to the TCP check module.
  • Allow metric names with empty stream tags (foo|ST[]) as valid metrics.
  • (broker) If a check's XML node name isn't valid, use name/value encoding.

Web UI/API

Version: 0.1.1611416884 (df0b650)

  • Add automatic service dashboards for Apache, NGINX, and Memcached checks.

web_stream

Version: 0.1.1591797218 (3c0de1a)

2021-01-11

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20210111/x86_64/

caql_broker

Version: 0.1.1608059216 (bdaa081)

  • No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1608133589 (a449354)

Function Removal Notice

Metric Search v2 and Metric Clusters have been deprecated for some time. We plan to remove these deprecated functions soon for the SaaS site, and in the subsequent Inside release. This will affect CAQL checks as well as CAQL Datapoints on graphs. The UI will now show users a warning when one of the deprecated functions is used. Circonus offers a more powerful tag-search feature, exposed as find() in CAQL.

  • Add -R flag to snowthsurrogatecontrol tool that will allow repairing corrupt surrogate databases.
  • Better error reporting and handling for various find calls.
  • The shard compactor script now checks the shard's status just prior to replacement, to make sure it is still offline.
  • Support metric_name, display_name, and multiple tag values in v2 search conversion.
  • Increased logging for LMDB errors and resize, detect resize failures, and improve LMDB and RocksDB retries.
  • Improved logging for ingestion flatbuffer errors.
  • Fix stack overflow when fetching many histograms in sequence.

fault_detection

Version: 0.1.1606942529 (f49a224)

  • Send broker information to the Notification system on every alert. Addresses failures to suppress notifications when a broker is unavailable.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.5

  • Force disconnect on message read/write error.
  • Reuse listener threads.
  • Various code cleanups.
  • Better bounds checking on auth handshake (allow full size).
  • Fix BCD integration.

GoAPI

Related roles: api

Version: 0.8.17

  • Changes the way metric search handles category-only v3 search queries, such as: and(component). Previously these were sent to IRONdb as: and(component:*), now the query will be sent unchanged.
  • Corrects an error in metric lookup by ID query creation causing searches for metrics containing the " character to fail.
  • Changes the behavior of CORS headers for the API and proxy servers to match the behavior of the Perl API.
  • Corrects an error in metric lookup by ID url parsing which caused the % character to be handled incorrectly.
  • Corrects how /metric endpoint handles requests with the count_only query parameter.
  • Corrects an issue causing IRONdb proxy error messages to contain unreadable messages.
  • Fix errors that occur when memcache operations fail.
  • Corrects a bug that could cause server panics if no memcache servers were assigned in the configuration.
  • Fixes an error that would allow certain values to break db queries with the order parameter on API search endpoints.
  • Corrects an error that occurs when metric search operation contain multiple terms containing only tag categories and not tag values.
  • Corrects how the db.Stats() wait_duration metric data is reported.
  • Corrects an error with metric search query parsing of search terms containing a tag category but not a tag value.
  • Allows the parse_only query parameter to work using the regular /metric endpoint.
  • Contains goapi implementation of the graph service.
  • Query parser errors will now return a 400 status code instead of an incorrect 500 status code.

Hooper

Version: 0.1.1609887173 (4921f87)

  • (Breaking change) Replace the obsolete nad host agent with the current version of circonus-agent on all systems managed by Hooper. The format of metrics emitted by circonus-agent has changed, and now uses Stream Tags. Any existing checks/rules against nad on these hosts may need to be adjusted or replaced.
  • Fix bug in config-check with the ca role primary attribute.
  • stratcon: Enable tuning of timeouts for ingestion into IRONdb. This includes both TCP connection timeouts (connecttimeout) and overall transaction timeout (timeout). Values are in milliseconds. Deprecate the previous irondb_put_concurrency attribute in favor of a new irondb_tuning object, which can contain any of three keys:
    "irondb_tuning": {
    "timeout": 8000,
    "connecttimeout": 1000,
    "put_concurrency": 50
    }
    See site.json stratcon attributes for more details. Reducing IRONdb timeouts may improve ingestion throughput by allowing stratcon to move on more quickly from poorly-performing IRONdb nodes. However, setting timeouts too low could negatively impact ingestion as well.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.14

  • Fix missing top of crash stacktrace when libunwind is being used.
  • Set log message 'SSL layer X not understood' to debug-only if SSL version X is being explicitly disabled. This reduces logging noise with OpenSSL 1.1.x and later.
  • Fix segment_size and precommit config processing for jlog logs. Previously, application requests to set these parameters could sometimes be silently ignored.

notification

Version: 0.1.1609956725 (b151580)

  • Bugfix: Correct handling of maintenance windows, expiration, incoming updates.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1608135246 (4b1e25d)

  • (broker) Reopen log files with dropped privileges. This addresses a corner case at startup where, if there is a JLog precommit buffer and the last available segment is full, a newly created segment will be owned by the wrong user.
  • (broker) Improved locking around filterset manipulation (crash fix).
  • (stratcon) Crash fix in the ingestion pipeline.
  • (stratcon) Fix crash when removing a broker that has data submissions in flight.

Web UI/API

Version: 0.1.1610123864 (30c290c)

  • Bugfix: send maintenance window changes to the Notification service.
  • Bugfix: correct the Apache proxy forwarding rules to accommodate GoAPI proxy requests to Lua CAQL extensions.
  • Bugfix: when searching on the Alert History page, it would remove any date range filter you had previously selected.
  • Bugfix: support for streamtags and non-alphanumeric characters in metric names for SNMP and other checks. Also requires broker update to support this case.

web_stream

Version: 0.1.1591797218 (3c0de1a)