Release Notes 2022

2022-11-14

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20221114/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1665503401 (06b870f)

No changes since 2022-08-22

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1667338493 (4f811e4)

Function Removal Notice

Discard incoming metrics that match the configured tag value. (CIRC-9298)
- To use the tag __storage:false, configure <discard_data_filter tag="__storage:false"/>
Fix ingestion of multiple OpenTSDB records. (CIRC-9339)

fault_detection

Version: 0.1.1658772502 (e96e66d)

No changes since 2022-09-19

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.14.2

Add a caching layer to the external services /stats endpoints, to prevent possible overwhelming of back end services by stats requests. (CIRC-9366)
Update documentation to improve default parameter values when the API documentation interface is used for testing requests. (CIRC-9346)
Fix a bug causing two server memory metrics not to initialize with the correct unit tags. (CIRC-9326)
Correct a confusing log entry when the /broker endpoint returns a "404 Not Found" result. (CIRC-9326)
Fix stratcon CN verification used by the reconnoiter package. (CIRC-9275)
Fix for __account check tag not being set properly when metrics are updated in IRONdb. (CIRC-2491)

Hooper

Version: 0.1.1666013286 (212c56d)

No changes since 2022-10-17

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.3.7

No changes since 2022-10-17

notification

Version: 0.1.1667848992 (9e0ce8a)

Set case signature to "noit_maintenance" for noit cases (was blank.) (CIRC-9235)
Eliminate duplicate insert attempts with ruleset group alerts from before a restart. (CIRC-9233)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1666616114 (fa91e95)

(broker) Fix crash in filterset handling.

Web UI/API

Version: 0.1.1668117788 (13a445f)

No user-facing changes.

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-10-17

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20221017/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1665503401 (06b870f)

No changes since 2022-08-22

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1665174809 (b568bfb)

Function Removal Notice

Update irondb-eventer.conf with default settings for new find jobqs. (CIRC-8892)
Default to no find limit for graphite find queries. (CIRC-9224)
Update old data logging to log old metrics received by the /graphite, /raw, and /journal endpoints, as well as the pickle listener. (CIRC-9204)
Write histogram journal data in batches instead of adding one entry to the
jobq at a time. (CIRC-9144)

fault_detection

Version: 0.1.1658772502 (e96e66d)

No changes since 2022-09-19

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.13.12

Set server shutdown timeout to be the same as server request timeout. This reduces the chance of dropped requests during server restarts. (CIRC-9275)
Add support for correct zero-initialization of histogram metrics, upon process restart, in addition to counter metrics. (CIRC-9196)
Correct a regression in a previous release preventing GoAPI from loading configuration settings from environment variables correctly. (CIRC-9196)
Initialize all metrics emitted at process startup to prevent gaps in metric data between process restarts. (CIRC-9196)
Change the name of backlog_error metrics to log_backlog_error and backlog_access metrics to log_backlog_access to better reflect the meaning of their contents. (CIRC-9196)
Corrects a bug that would result in an error if a CAQL query contained an error response with a JSON message field that could not be decoded into a string value. Any values will now be accepted in the message field. (CIRC-9196)
The X-Snowth-Timeout header will be limited for proxy requests to a duration no longer than the configured maximum IRONdb request timeout. Requests containing longer values in this header will be dropped to this duration before being passed to IRONdb. (CIRC-9196)

Hooper

Version: 0.1.1666013286 (212c56d)

Add slow find query logging configuration to snowth.conf. It is disabled by default and is intended to be activated temporarily during troubleshooting. (CIRC-8882)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.3.7

Fix use-after-free in reverse socket code.
Allow for an addext field for lua CSR generation (requires at least openssl 1.1): addext = { subjectAltName = { "DNS:name1", "DNS:name2" } }

notification

Version: 0.1.1663876285 (1baeb75)

No user-facing changes.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1665503401 (860304e)

(broker, stratcon) Add libmtev version to backtraces on crash.

Web UI/API

Version: 0.1.1665693152 (ec1ebe3)

Add indexes to checks and notification_cases sql tables. (CIRC-9250)
The /check_bundle API endpoint has been forwarded to GoAPI in all environments. (CIRC-9196)
Fix bug with editing Graph Zoom Levels preference where minutes were changed to months. (CIRC-9189)
Custom Dashboard TileMaps now support arbitrary value rules for status-based tile coloring and text. (CIRC-8565)
Graph Overlays have been marked as deprecated and will be removed in December 2022. (CIRC-9162)

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-09-19

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220919/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1662038757 (06b870f)

No changes since 2022-08-22

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1662736239 (8107538)

Function Removal Notice

Update art indexing storage to use regular mutex instead of spin lock to avoid excessive CPU usage. (CIRC-9136)
Avoid RoaringBitmap copies in queries on level index. (CIRC-9137)
Audit ephemeral node support and reduce overhead. (CIRC-8968)
Fix local assertion failure in fetch when remote node errors out. (CIRC-9040)
LIFO and FIFO Job Queue modes are now color coded in the admin UI. (CIRC-8879)
Fix deadlock in raw shard post-rollup deletion that can result in a node getting hung and not accepting new data. (CIRC-8030)
Add graphite_rewrite module. (CIRC-8974)
Use the cheaper search AST construction APIs. (CIRC-8967)
Add hook for find jobq assignment suggestion. (CIRC-8969)

fault_detection

Version: 0.1.1658772502 (e96e66d)

Fix race condition with cull events that could lead to double free. (CIRC-8811)
Fix race condition with absence events that could lead to double free. (CIRC-8811)
Fix atomic operations around events. (CIRC-8811)
Plug leaks due to incorrect atomic operations and assumptions. (CIRC-8811)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.13.7

Log check-not-found errors at the correct logging level. (CIRC-9082)
Update timeout settings used by the reconnoiter package clients. (CIRC-9082)
Fix an error in the return payload of the /provision_broker endpoint when changing the IP address of a broker. The error was due to invalid formation of the broker host name while updating the broker self check. (CIRC-9082)
Correct an issue causing the /metric endpoint to return base64 encoded check tags in the wrong format. The beginning b" and trailing " were being removed from the base64 encoded sequences during data processing. These base64 encoded check tags will now be returned using the same format as IRONdb. (CIRC-9068)
Enable processing of check move operations. This obsoletes the hub cron job that formerly performed this work. (CIRC-3357)
Changes the behavior of /alert force clear requests to return success (status code 200) when the alert being force cleared has already been cleared. (CIRC-9046, CIRC-9047)

Hooper

Version: 0.1.1661893514 (fe96cfe)

Remove the move_checks_new_noit.pl script from the hub cron jobs, as this processing is now handled by GoAPI. (CIRC-3357)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.3.5

The HTTP system should only write so much data before returning control to the event loop. It now limits to 512k (programmatically changeable).
LIFO and FIFO Job Queue modes are now color coded in the admin UI. (CIRC-8879)
Add mtev_lfu_release_f for releasing a cached item from a potentially destroyed cache.

notification

Version: 0.1.1654017522 (6607199)

No changes since 2022-06-27

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1662038757 (45b8a11)

(broker) Adopt default behavior of units:1 in OpenTelemetry module for dimensionless metrics.
(broker) Support both endpoint and metrics_endpoint for OpenTelemetry check configuration.
(libnoit) Add function noit_metric_tag_search_unparse_limit_size, which allows setting a size limit to ensure that we don't go overboard with extremely large queries, and limit recursion to avoid stack overflow.
(broker) Avoid crashes when encountering a null metric object. (CIRC-9039)
(broker) Fix mtev_conf lock acquisition.
(libnoit) Automatically optimize PCRE matching during metric tag searches. (CIRC-8958)
- Add parse_lazy and clone_lazy functions to avoid PCRE compilation during parsing. Do not study PCREs by default; track executions and once we have 50 calls to pcre_exec then study the PCRE. Rework the LFU cache such that the token is tracked with the returned object and is released once the node is freed.

Web UI/API

Version: 0.1.1663182509 (4f59c94)

Individual widgets on Custom Dashboards may now be cloned. (CIRC-7121)
The /check_bundle_metrics API endpoint is now handled by GoAPI. (CIRC-9104)
The /check_move API endpoint is now handled by GoAPI. (CIRC-9050)
Absence rule's number of seconds was not displayed in the UI view of text rulesets. (CIRC-8979)
The reconstitute_noit_config.pl command would cause some check XML configuration data to be lost. (CIRC-9083)

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-08-22

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220822/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1660253810 (06b870f)

No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1660795138 (18ded6f)

Function Removal Notice

CAQL sort:min/max/mean no longer errors on false inputs. (CIRC-8961)
Be smarter about CAQL fetch limitations when optimizations are used. (CIRC_8965)
Add additional error handling when decoding flatbuffer messages. (CIRC-8944)
Make assertion in fetch map collision non-fatal. (CIRC-8937)
Fix assertion failure assertion on fdlc1_set_result. (CIRC-8937)
Fix issue that would cause nntbs shard to increase in size too frequently. (CIRC-8933)
Fix min/max statistics in CAQL for windows with missing values. (CIRC-8902)
Defer allocation in index manager to improve find performance. (CIRC-8849)
Fix crash bug in /fetch group_by stats. (CIRC-8859)
Fix hex parsing bug in prometheus module.
Proper maintenance deletes of histograms. (CIRC-8868)
Only a single raw deletion maintenance task at a time. (CIRC-8857)
Fix memory corruption and races in find. (CIRC-8854)
/fetch numeric transforms now support a fill=<X|forward kwarg. (CIRC-8850)
CAQL optimizations for numeric find() | fill() and find | fill:foward(). (CIRC-8850)
Introduce a /find query classification and isolation system. (CIRC-8849)
- Consolidate /find (related) endpoints to use three new classes of jobq called find_{local,remote}, find_fast_{local,remote}, and find_slow_{local,remote}.
- Add a performance profiling system to track and classify /find to place them automatically into each of these three new classes.
Add jobq mode (LIFO/FIFO) to snowth console web UI.
Expand /find calls to pull from all nodes if the user specifies that they want the latest datapoints to be included. (CIRC-8832)
Compare cached last 2 numeric values when deduping results from find calls to make sure we're always using the latest data. (CIRC-8832)
Disallow adding __activity, __check_uuid, __name, and __type as stream tags (CIRC-8819)
Add support for histogram shards to rebalance. (CIRC-8167)
Implement filter:whence() and filter:not:whence() CAQL functions. (CIRC-8767)
graphite_translate additions/fixes to better match graphite behavior:
- More Graphite functions implemented in translator - stddevSeries(), stdev() and unique(). (CIRC-6904)
- time() added as alias to timeFunction() (CIRC-8766)
- timeSlice() translation (CIRC-8767)
Add filter:label:unique and stats:stddev to CAQL. (CIRC-6904)
Disallow adding __type or __activity tags in check tags and ignore them if they are already there. (CIRC-8815)

fault_detection

Version: 0.1.1658435961 (0431bdd)

No changes since 2022-07-25

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.12.18

Correct a bug resulting in CAQL query cache data not being updated correctly when context timeout errors occurred. (CIRC-8743)
Fix a bug resulting in errors during log file rotation causing the log to revert to outputting to stderr. (CIRC-8743)
Bring v2 search syntax into parity with Perl search syntax. Search terms will be combined using OR only if they contain the same term. Otherwise, all search terms are combined using AND. (CIRC-8654)
- For example, the query (active:1)(active:0)(name:test) will result in a search where (active = 1 OR active = 0) AND name = test.
When the /alert endpoint finds an alert for a rule set group it will now correctly return a /rule_set_group/{id} CID in the rule_set field, and no longer return an incorrect /rule_set/{id} CID. (CIRC-8814)
Add a search term called related to the /alert endpoint. When provided with an alert CID or ID, it will return all alerts related to the provided CID (i.e. members of a rule set group alert.) (CIRC-8814)
Update the active search term for the /alert endpoint, so that a search of (active:*) will return both active and inactive alerts. (CIRC-8814)
Fix a bug that was causing context errors during IRONdb operations to sometimes get incorrectly wrapped in a 500 Database error, resulting in incorrect reporting of error statistics. (CIRC-8830)
Update gosnowth library to v1.11.0. Uses the type definitions for DF4 from the gosnowth library exclusively, and eliminates redundant DF4 types in GoAPI. This adds the missing head.error and head.warning fields to results that return data in DF4 format when they are available. And, this fixes several small bugs in the DF4 Go implementation. (CIRC-8800)

Hooper

Version: 0.1.1657286925 (ed9db35)

No changes since 2022-07-25

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.3.3

Fix forensic HTTP logging in both HTTP/1 and HTTP/2. (CIRC-8915)
Fix ALPN/NPN on new TLS connections from cached contexts. (CIRC-8915)
Fix HTTP/2 integration such that sessions correctly resume on EAGAIN. (CIRC-8915)
Add jobq "mode" (LIFO/FIFO) to web UI.
Track REST requests as a counter by endpoint.
Support OPTIONS requests in REST.
Fix regression in http forensic logging where an extra 0 latency was tracked on each HTTP request.

notification

Version: 0.1.1654017522 (6607199)

No changes since 2022-06-27

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1660253810 (1f05f4b)

(broker) Speed up cluster replication by avoiding work if the check sequence number has not increased.
(broker) Use HTTP/1.1 for cluster replication.
(broker) Use a jobq for building check replication XML to avoid stalling the event loop.
(broker) LMDB check/filterset storage changes:
- Use a larger starting map size, aligned with page size.
- Resize databases by a factor of 2x, rather than 1.5x.
- Better log messages when resizing.
- Avoid unnecessary resize operations.
(broker) Improve logging in collectd's failure scenarios.
(libnoit) Cache pcre_study results to avoid watchdogs. (CIRC-8835)
(libnoit) Allow passing a custom validation function to drop select tags during tag compaction.

Web UI/API

Version: 0.1.1660657692 (19c9e39)

Revamped the Kubernetes service dashboard. The new version requires significantly fewer metrics. (CIRC-6811)
- Minimum Kubernetes version: 1.20
- Minimum Kubernetes Agent version: 0.13.0
Now CAQL warnings are shown in the Metrics Explorer. (CIRC-8801)
Added context menus to Custom Dashboard widgets to provide "View Queries in Metrics Explorer" links. (CIRC-8680)
Bugfix: in certain places where ruleset tag filters are displayed in the UI, any tag in the filter which has been base64 decoded will not erroneously have base64 delimiters wrapping around it. (CIRC-8757)
Update: new graphs will now default to "interpolated line" style instead of "stepped area" style. (CIRC-8725)

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-07-25

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220725/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1658256585 (2033494)

Checks on clustered nodes will interoperate better with the Circonus UI. (CIRC-8693)
Add support for find:sum(). (CIRC-8729)

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1658332819 (5d6d5ee)

Function Removal Notice

Disallow adding __check_uuid or __name tags in check tags and ignore them if they are already there. (CIRC-8774)
Floating-point latest values in /find calls will now round less and be more accurate. (CIRC-8615)
Add min mechanism arg to top reduction and add bottom reduction to /fetch. (CIRC-8732)
Add method="min" to top() and bottom() CAQL functions. (CIRC-8732)
Implement bottom() CAQL function. (CIRC-8732)
More user-errors in CAQL are reported as 4xx HTTP status codes. (CIRC-7874)
Fix error where when writing a copy of the last raw data point in a shard into the next shard, we could write a bad data point if no additional data had been written to that shard since the previous rollup. (CIRC-8228)
Fix crash in /fetch groupby in uncommon scenario when replication is stuck and the cluster is unstable. (CIRC-8683)
Revert graphite SUM treatment as AVG and add a graphite module config to support the old behavior <coerce_sum>avg</corece_sum>. (CIRC-8573)
Improve raw reconstitute performance by iterating and sending data from all column families in parallel. (CIRC-8649)
Nodes that are being reconstituted will no longer serve fetch requests. (CIRC-8662)

fault_detection

Version: 0.1.1658435961 (0431bdd)

Only replace in-memory rulesets when they're modified. This is an optimization to reduce unnecessary AMQP messages to the notification system about the status of rulesets. (CIRC-8789)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.12.13

Correct a bug that was causing /alert endpoint results to contain invalid maintenance window data. (CIRC-8716)
Change the status code of context canceled errors from 500 to 499. This prevents a server error from being logged and recorded in metrics when a caller closes their connection early. (CIRC-8716)
Correct an issue that could cause /provision_broker calls to fail, if updating check tags data in IRONdb is needed, and many checks are affected. (CIRC-8803)

Hooper

Version: 0.1.1657286925 (ed9db35)

Assign only one RabbitMQ host per fault_detection node. RabbitMQ is itself clustered, so this reduces unnecessary duplication of messages between fault_detection and notification hosts. (CIRC-8694)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.3.2

Change heartbeats in eventer to be per-callback instead of per-cycle. This is an optimization to reduce the chances of a watchdog timeout.
Add REST API http/access/forensic receive logging to preexisting logging on completion. (CIRC-8612)

notification

Version: 0.1.1654017522 (6607199)

No changes since 2022-06-27

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1658171414 (aa5fe8c)

(stratcon) Fix a crash bug in stratcon when removing a broker. (CIRC-8705)

Web UI/API

Version: 0.1.1657059396 (80882d4)

For Inside deployments, force all graph data series to be shown, always. (CIRC-8606)

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-06-27

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220627/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1653328103 (bbae29b)

No changes since 2022-02-21

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1655918547 (248d285)

Function Removal Notice

Improve default labels for CAQL delay(...)
Use less memory from tag queries. (CIRC-8593)
Use less memory for tag index compaction. (CIRC-8578)
Find timeouts will now respect either the default timeout, or values set via the X-Snowth-Timeout header. Queries will stop evaluating once they detect that they are past the query deadline rather than running to completion. (CIRC-8556)
Change default find timeout from five minutes to one minute. Make the timeout value configurable via find_optimization/@default_find_timeout in configuration. (CIRC-8556)
Fix race conditions in fetches when encountering errors. (CIRC-8397)
Fix find timeouts so nodes will break connections to remote nodes during proxy calls if the timeout is reached. (CIRC-8556)
Fix reconstitute bug where surrogate databases would get re-downloaded even if they had already finished being reconstituted. (CIRC-8555)
Use separate jobqs for tag_cats and tag_vals find queries. (CIRC-8544)
Fix possible DoS in reconstitute. (CIRC-8515)
Reduce memory footprint and improve performance for tag_vals find queries. (CIRC-8493)
Fix incorrect level indexing delete. (CIRC-8009)
Reduce memory footprint and improve performance for tag_cats find queries. (CIRC-8442)
graphite_translate additions/fixes to better match graphite behavior:
- More Graphite functions implemented in translator (CIRC-6904) - aliasByTags, groupByTags, logit, sigmoid, sinFunction, substr, squareRoot, timeFunction, useSeriesAbove.
Add math:logit and math:sigmoid to CAQL. (CIRC-6904)
Change the surrogate format for reconstitutes to use surrogate3. From this point on, reconstitute and rebalance are not backward compatible with prior releases. (CIRC-8461)
Fix a double regression in graphite:aliasbynode where the name was incorrectly set.

fault_detection

Version: 0.1.1655252254 (43c93e9)

Fix double-free crash in rulestream handling. (CIRC-8572)

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.12.11

Search v2 terms containing only a category will now work correctly. (CIRC-8718)
Correct an error that defaulted existing graph datapoints to hidden when those datapoints did not contain a lines object in the original graph definition. (CIRC-8606)
Fix query builder bug that caused search results to return one more than the number specified in the size parameter. (CIRC-8627)
Increase efficiency of authentication by removing one database query from the cookie authentication process. (CIRC-8601)
Add improved validation for all contact group contact methods. For methods that accept JSON strings for the contact_info field, the JSON is now also validated. (CIRC-8533)
Fix unnecessary retries on certain IRONdb request errors that should not be retried. (CIRC-8570)
Update the OpenAPI documentation for the /contact_group endpoint to include documentation of the supported external contact methods. (CIRC-8524)
Fix a bug that was preventing IRONdb connection retries when a node is down. (CIRC-8551)
Correct a bug that could result in PUT or DELETE requests to the /maintenance endpoint changing the start time of the maintenance window by a fractional second. (CIRC-8505)
Preserve the ordering of datapoints submitted to the /graph endpoint. (CIRC-8463)
Correct a bug preventing the /maintenance endpoint from accepting a value of "metric" for the type field. (CIRC-8463)
Add API updates to support dashboard defaults for all dashboard types, for user and account defaults. (CIRC-8413)

Hooper

Version: 0.1.1654534949 (0ad3f78)

Restart the circonus-bert service only when the package is updated, rather than on every run. (CIRC-8457)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.3.0

Report LIFO/FIFO mode for jobqs via console.
Allow changing of jobq mode online via console.
Add POST/PUT payload to backtrace reports.
Always print double-free/corruption detection in mtev_memory_safe_free.

notification

Version: 0.1.1654017522 (6607199)

Ruleset Group alert state will now respond to member maintenance start/stop. (CIRC-8187)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1655134932 (ef4e780)

(broker) Fix cluster replication when check configuration uses the extended name/value format.
(broker) Fix crash in ping_icmp check module.

Web UI/API

Version: 0.1.1655930000 (f1dffa8)

"Force Clear" and "Related Alerts" options are now available in the UI for all ruleset alerts. (CIRC-5485)
Alert pre-check was blocking "Enterprise Broker Disconnected" cases. (CIRC-8507)
Update database schema to support custom and service dashboard defaulting for users and accounts. (CIRC-8413)
rule_set_group API incorrectly returned wait value in seconds unless it was under 60 minutes. It will now always convert PUT/POST values to minutes, yielding the same result as from GET. (CIRC-7233)

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-05-30

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220530/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1653328103 (bbae29b)

No changes since 2022-02-21

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1653417917 (aff861f)

Function Removal Notice

Watchdog would occur when taking too long to set a check's metadata. (CIRC-8412)
Fix shard reporting issues in stats.json. (CIRC-8326)
Fix crash if pulling surrogate data during a reconstitute fails. (CIRC-8347)
Enable live single-shard reconstitute for raw numeric and histogram shards. (CIRC-7279)
graphite_translate additions/fixes to better match graphite behavior:
- More Graphite functions implemented in translator (CIRC-6904) - constantLine, currentAbove, currentBelow, highest, legendValue, lowest, lowestAverage, nPercentile, percentileOfSeries, setXFilesFactor, sortBy, sortByMaxima, sortByMinima, sortByName.
Fix deadlock when trying to delete a rolled-up raw shard. (CIRC-8288)
/extension/lua/caql_info endpoint now accepts query in addition to q.
/extension/lua/caql_info endpoint now returns post-optimized query.
CAQL forecasting:regression now better supports streams with gaps. (CIRC-8274)
Fix bugs in side-aware finds where count-only results would be incorrect and proxied requests went to the wrong side. (CIRC-8252)
Implement label:replace(...) CAQL function for regex based label manipulation. (CIRC-7903)
Add CAQL trigonometry functions (sin, cos, etc.).
Add configuration option controlling the size of batches handled by clean_rollup_suppressed_metrics when rollup-suppressed metrics are culled. It is suppressed_rollup_delete_batch_size in the <raw_database> element. (CIRC-5892)

fault_detection

Version: 0.1.1652989390 (09e5157)

Remove measurement tags from incoming metrics. (CIRC-8342)
Update /ruleset endpoint to return the tag filter used for the ruleset. (CIRC-5107)
- Update rules explorer page in the management UI to display this newly-available information.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.12.4

Correct a bug preventing shared legacy dashboards from being able to be favorited if they were not owned by the favoriting user. (CIRC-8443)
Correct a broker provisioning bug that caused incorrect handling of broker provisioning when a new broker is being configured to join an existing cluster by cluster name. (CIRC-8418)
Add documentation for the provision broker make_public field.
Modify the /custom_dashboard/user_favorite/{user_id} endpoint to filter favorites returned to be only those available to the account associated with the request credentials. (CIRC-8438)
Update OpenAPI documentation to include information about the selector type in metric filters for checks. (CIRC-7937)
Correct an error that was preventing deletion of custom dashboards if they had been favorited. (CIRC-8379)
Modify the /custom_dashboard/user_favorite endpoints to support favoriting of legacy dashboards. (CIRC-8232)
Add a snowth_cluster_id field to errors logged. (CIRC-8366)
Modify the /custom_dashboard/user_favorite endpoints to accept the value current anywhere a user ID is required. Also, the endpoints are limited to only being able to retrieve or modify favorites data for the user associated with the credentials passed to the request, unless the credentials have Admin privileges and the user ID has a role in the account to which the credentials belong. (CIRC-8351)
Correct a validation error message from the /custom_dashboard/user_favorite endpoint to be more clear that the data submitted to the endpoint must contain either a custom_dashboard CID or a service_dashboard_check_uuid (UUID). (CIRC-8324)
The /caql endpoint will now return an X-Circonus-Cached-Result header whenever the result returned (data or error) is pulled from the cache. (CIRC-8317)
Correct an issue causing problems looking up metrics by ID containing a % character in their name. (CIRC-8285)
Allow the /graph endpoint to handle datapoint derive values set to an empty string "" without returning a validation error. (CIRC-8284)
Implement favoriting of service dashboards through modifications to the /custom_dashboard/user_favorite endpoints. (CIRC-8269)
Update status_code metrics to include a snowth_cluster_id tag. This will be set to the snowth_cluster_id when generated by endpoints that interact with IRONdb and will be set to none when generated by endpoints that do not. (CIRC-8254)
Remove the web/disable_local_auth configuration option since the only use case for this option no longer exists. (CIRC-8254)
Correct a bug that was causing the /maintenance endpoint to return expired maintenance windows during searches. (CIRC-8266)
Modify stats recorded by CAQL query caching, so that when queries fetched from the cache return that the query is still running in another process, it is recorded as a cache_notready metric. (CIRC-8257)

Hooper

Version: 0.1.1648834383 (c9ccbca)

No changes since 2022-04-04

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.2.7

Avoid assertion in internal membuf bounds checking.
Add support for building with OpenSSL 3.0.

notification

Version: 0.1.1652466360 (ad89d10)

Use correct metric_name (consistently without streamtags) for all rulesets. (CIRC-8311)
Avoid possible lost UUID problems with maintenance transitions. (CIRC-7395)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1653328103 (ab01c41)

(libnoit) Add debug information when message decoding fails.
(libnoit) Reduce size of metric_t struct. (CIRC-8356)
(libnoit) Fix off-by-one error in noit_metric_canonicalize.
(broker) Fix memory leak in Prometheus check module. (CIRC-8268)

Web UI/API

Version: 0.1.1653584286 (0434f7d)

The Dashboard Listing page has been overhauled to allow for easier finding of dashboards; this now includes the ability to have "favorite" dashboards. (CIRC-8098)
Pre-check to prevent alert creation for deactivated checks or rulesets. (CIRC-7959)
Correct metric name field for alerts to consistently omit streamtags. (CIRC-8311)
Fix ruleset group alert members not showing for some long-running group alerts. (CIRC-8305)

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-05-02

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220502/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1650309546 (bbae29b)

No changes since 2022-02-21

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1650654785 (c18f4e9)

Function Removal Notice

Graphite find queries with .. now correctly match nothing. (CIRC-8025)
Add /suggest/<accountid>/tags auto-completion assistance endpoint. (CIRC-8209)
The find mgr will auto-expand searches when a side-informed query falls short. (CIRC-8047)
The find mgr will complete before all nodes have responded if it knows the answer to be complete. (CIRC-8047)
Fix graphite_translate test for asPercent(). (CIRC-8166)

fault_detection

Version: 0.1.1650305680 (5648f44)

No user-facing changes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.11.6

Correct a problem where the /broker endpoint was incorrectly counting for results pagination, potentially causing some broker details to be missing. (CIRC-8249)
Fix an error that caused the /stats endpoint to emit unnecessary warnings about missing ID's into the error log. (CIRC-8191)
The /caql endpoint now supports caching of IRONdb CAQL query results. Caching will only be used when the query runtime is greater than a configured limit. (CIRC-8191)
Implement logic to ensure a minimum of 5 unprovisioned broker slots are available whenever a GET request is made to the /broker endpoint. (CIRC-8202)
Change the logging of remote addresses to use the X-Forwarded-For header if it is available. (CIRC-8190)
Include user_id in the tags added to log entries, where available, to support cookie authentication when tokens are not used. (CIRC-8190)
Add access logging functionality to the proxy server. (CIRC-8190)
Update to latest version of gosnowth (v1.10.4) to gain support for context termination of requests during the initialization process of gosnowth clients. (CIRC-8169)
Add account_id and auth_token_id tags to log entries when this context data is available. (CIRC-8112)
Adds better error handling for responses from IRONdb fetch requests. (CIRC-8117)

Hooper

Version: 0.1.1648834383 (c9ccbca)

No changes since 2022-04-04

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.2.5

Support lua coroutine killing and cancellation via console.
Fix bug in preemption resumption where it was calling a non-implementation-specific resume.
Zipkin
- fix binary annotations of type double
- fix log attachment
- add lua helpers for logging and tagging to spans

notification

Version: 0.1.1649971612 (b2cc34c)

No changes since 2022-04-18

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1650309546 (88550c0)

(libnoit) Fix crash due to bad tag search regular expressions.

Web UI/API

Version: 0.1.1651002002 (73bcf67)

Add columns and constraints to the custom_dashboard_user_favorites table to support service dashboard favorites. (CIRC-8269)
Now when you create a new custom dashboard, you will be taken to view the dashboard when exiting the dashboard editor. (CIRC-8212)
Remove "Add Broker" button on global admin page. (CIRC-8203)

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-04-18

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220418/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1649886155 (bbae29b)

No changes since 2022-02-21

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1649955505 (8f92668)

Function Removal Notice

CAQL queries on a single time point now return HTTP 4xx instead of 5xx. (CIRC-8099)
Removed usage of legacy fetch api (snowth.get). (CIRC-7987)
Fix artificially limited finds due to limit being pushed down, then further limited by a filter. (CIRC-8067)
find optimization: flatten nested and(and()). (CIRC-8067)
find optimization: push limit context predicates down through hints. (CIRC-8067)
CAQL find() | group_by:merge() is now a user-facing error with explanation. (CIRC-8059)
Optimize __type and __activity finds. (CIRC-8050)
Fix side-aware failure tolerances for search. (CIRC-8047)
Allow passing an engine_mode_hint query parameter to find calls - on (which utilized hint nodes that are explicitly defined), off (which will ignore hint nodes), and auto (which will make determinations for optimizations using internal criteria) (CIRC-7654)
User friendly time ranges now support omission of start and end times on the __activity tag when using the [epoch] - [epoch] format (colon format does not allow omission). Corrected an issue where singular epoch times were allowed but should be an error. (CIRC-8024)
Allow setting of journal batch size and timeout via console configure set
Fix CAQL optimization that could artificially limit find results if the pattern matched streams of a different data type. (CIRC-8035)
Make a variety of find queries utilize a single side (preferring local) when running a sided topology and a sufficient number of nodes are up. (CIRC-8013)
Fix heap use after free when removing RocksDB-based raw shards. (CIRC-7881)
graphite_translate additions/fixes to better match graphite behavior:
- Fix asPercent() translation (CIRC-7754)
Make best-node selection side aware. (CIRC-8013)
Allow CAQL find and friends to support a single argument with a tag search expression. (CIRC-8007)
Executor metrics updated to be more Metrics 2.0 style. They are now tagged with the thread ID as thread-name, queue_<#>_num_jobs is now size with units: messages, and queue_<#>_wait_time_hist_ms and queue_<#>_run_time_hist_ms are now simply wait_time and run_time with the additional information tagged as stated above, units: seconds and "hist" removed from the name entirely. Metrics 1.0 changes include simply grouping by thread name rather than the superfluous internal details in place before. (CIRC-7972)

fault_detection

Version: 0.1.1639089770 (90cc626)

No changes since 2022-01-10

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.11.1

Most metric data retrieved from the /stats endpoint will no longer be reset every time the /stats endpoint is requested. This will change the some previous gauge metrics into counters in Circonus. Histogram metrics will still be reset on each /stats request. (CIRC-8102)
Add support for searching alerts using a string of multiple severity values. So, /alert?search(severities:123) will now find alerts with severity 1, 2, or 3. This matches the functionality of the Perl API and the UI. (CIRC-8113)
Add an idle timeout to the API and proxy servers preventing keep-alive connections from remaining open indefinitely. (CIRC-8118)
Update to latest version of gosnowth v1.10.3. (CIRC-8083)
Fix a bug that caused failures creating /alert endpoint responses. (CIRC-8094)
Add routing support for a /v2/ URL prefix to all endpoints. This will work whether or not it is used in conjunction with the /api-proxy/ prefix. (CIRC-8088)
Correct an issue with /graph datapoints deduplication logic that caused search datapoints to be incorrectly deduplicated. (CIRC-8054)
Improve account cache operations to cache by account ID and account URL. (CIRC-2545)

Hooper

Version: 0.1.1648834383 (c9ccbca)

No changes since 2022-04-04

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.2.3

Avoid stack overflow in mtev_hash_retrieve and mtev_hash_delete when the key size (minus its null terminator) is the same as ONSTACK_KEY_SIZE. (CIRC-8046)
Fix two luajit runtime use errors that could break garbage collection:
- lua_call during resume when coming back from mtev.sleep
- nl_resume return value off-by-one.
Default show lua now shows all lua states in all loops.
Add mtev.runtime_defunct(<cb>) and mtev.runtime_reload().
Add /module/lua/bump.json to cause on-demand reloads.
Make preemptive interruption an option for lua_web and lua_general.
Change lua mtev.log to report the lua file/line in debug context.
Support refreshing lua_web lua_States.
- Add a dev_mode config option that forces a state refresh before each web request.
- Trigger a refresh when the interrupt timer fires within a state.
Add /<app>/@workingdir config attribute (default '/') to control the chdir(2) when running managed.

notification

Version: 0.1.1649971612 (b2cc34c)

Fix possible wedged ruleset group when the incoming message has an all-zeroes check UUID. (CIRC-8072)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1649886155 (894df00)

(broker) Support units extraction and histogram coercion in prometheus module.
(broker) Fix NPE in external check module. (CIRC-8043)
(libnoit) Reduce memory growth in tag search results.
(libnoit) Add tag search swap function.
(broker) Additional thread safety in prometheus module.

Web UI/API

Version: 0.1.1649863064 (f6bb1a2)

No user-facing changes.

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-04-04

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220404/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1648060046 (bbae29b)

No changes since 2022-02-21

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1648500976 (a421792)

Function Removal Notice

Make tag index searching release memory as it executes queries. (CIRC-8000)
graphite:{aliasbynode,removenode,tagbynode} now all support negative indexes. (CIRC-7985)
Update internal version of the search index database (metric_name_db), which will cause the first restart after updating to this version to take longer than normal, to allow the database to generate. (CIRC-7999)
Remove falsely reported corruption message from reconstitute caused by empty shard files. (CIRC-7961)
Added -s [<period>,]<floor>,<ceiling> commandline option for offline single shard reconstitute. (CIRC-7958)
CAQL graphite:removenode removing duplicate arguments instead of error:
- *WithWildards() functions graphite translation allow duplicated arguments (CIRC-7962)
graphite_translate additions/fixes to better match graphite behavior:
- Fix hitcount() translation (CIRC-7953)
graphite_translate additions/fixes to better match graphite behavior:
- minSeries() and grep() are implemented. (CIRC-7957)
Canonicalize inbound metrics without their measurement tags. (CIRC-7919)
Fix CAQL graphite:aliasbynode regression wherein label was unset. (CIRC-7922)
Fix crash on removal of members from search index. (CIRC-7898)

fault_detection

Version: 0.1.1639089770 (90cc626)

No changes since 2022-01-10

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.10.29

Replace the cache_writes and cache_reads metrics with cache_sets and cache_hits, tagged with units:bytes. The existing cache_sets and cache_hits metrics are now tagged with units:operations. (CIRC-3356)
Add support for units tags to all emitted metrics. (CIRC-3356)
Improve memory cache client handling of no available servers. (CIRC-3356)
Add support for handling redirects returned from noit servers. (CIRC-3356)

Hooper

Version: 0.1.1648834383 (c9ccbca)

RabbitMQ maintenance (ensuring the necessary exchanges and users are present) is now done internally by the service itself, at start. No longer need to run the external maintenance script. (CIRC-5845)

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.2.2

No changes since 2022-03-07

notification

Version: 0.1.1644263161 (1e0f82e)

No changes since 2022-02-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1648060046 (88f7dad)

(broker) Various fixes to the prometheus check module. (CIRC-7933)

Web UI/API

Version: 0.1.1647962411 (3b743c2)

No user-facing changes.

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-03-21

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220321/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1647441048 (bbae29b)

No changes since 2022-02-21

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1647381305 (ef39c46)

Function Removal Notice

Add support for find hinting. This allows adding hints such as and(hint(__check_uuid:<a uuid>,index:none)) that will make the search evaluate against an existing set rather than using the full metric index. (CIRC-7654)
- Note: this is an experimental feature. It is not yet used by the product, but provides a basis for improved query optimizations in the future.
Fix egress_function field updates in surrogate entries. (CIRC-7873)
Make histogram:sum() not normalize by default, period= behavior is unchanged. This is a change in the default. (CIRC-7869)
Make find:sum() not normalize by default and add a period= kwarg to support normalization. (CIRC-7869)
Add op:div2 to CAQL strictly requiring two arguments. Make / use op:div2. (CIRC-7864)
Fix /fetch reductions of sum and count to return a stream of zeroes when zero metrics are requested. (CIRC-7863)
Change logic for finding best node to be smarter and find valid data more frequently. (CIRC-7848)
Add math:ceil and math:round to CAQL. (CIRC-7637)
Allow operator to set jlog precommit buffer size on metadata feeds via //metadata/@feed_precommit_size. (CIRC-7852)
Add histogram merge pipeline reordering when trailing pipeline functions are commutative. (CIRC-7846)
Add CAQL /fetch optimization for histogram:count(), histogram:count_above(), histogram:count_below(), histogram:rate_above(), and histogram:rate_below(). (CIRC-7844)
Unoptimized histogram:rate() math fixed to match (correct) optimized version. (CIRC-7845)
Add filter:sum:* and filter:not:sum:* functions for graphite translation assistance. (CIRC-7842)

fault_detection

Version: 0.1.1639089770 (90cc626)

No changes since 2022-01-10

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.10.28

Statistics for the proxy server will now be returned along with the statistics for the API server by the /stats endpoint. (CIRC-7923)
Better error messages will be returned when executing CAQL queries that contain user-facing errors. (CIRC-7874)
Better error messages will be returned when metric tag search queries contain whitespace errors or other parsing errors. (CIRC-7839)

Hooper

Version: 0.1.1643896828 (a06c0b8)

No changes since 2022-02-07

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.2.2

No changes since 2022-03-07

notification

Version: 0.1.1644263161 (1e0f82e)

No changes since 2022-02-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1647544876 (0acc2af)

(libnoit) Fix double-free in tag search. (CIRC-7920)
(broker) Fix Prometheus module not accepting traffic from remote_write-type tooling. (CIRC-7900)

Web UI/API

Version: 0.1.1647877197 (3499973)

Added support for heatmap histograms to Custom Dashboard "Metric Trends" and "Single Metric" widgets. (CIRC-5493)

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-03-07

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220307/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1644523407 (bbae29b)

No changes since 2022-02-21

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1646431984 (5593c04)

Function Removal Notice

Shard status for raw and raw-hist now reporting to /stats.json.
Add config setting max_udp_payload_size_bytes to set limit on gossip UDP packet sizes (default=1432).
Add group_by:prod to CAQL.
CAQL stats:ratio() now accepts partition or partition_fmt to calculate partition-relative ratios.
CAQL filter:latest:* and filter:not:latest:* with configurable lookback.
Fixed a potential crash when manually triggering rollups.
Reduce upper bound of default gossip interval from 2 seconds to 1.8 seconds. This avoids the occasional appearance of excess gossip latency in the Replication UI.
Fix compatibility issues with CAQL graphite:aliassub.
Improve check tag find search performance.
Improve find search performance when using the *:* wildcard tag search.
Add shard maintenance for raw numeric and raw/rollup histogram shards.
When /fetch upgrades numerics to histograms, it should exclude NaNs.

fault_detection

Version: 0.1.1639089770 (90cc626)

No changes since 2022-01-10

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.10.25

Correct an error in the CA certificate lookup for noit operations.
Correct an error in IRONdb tag query formatting when using match impl options and b64 encoding.
Fix a bug causing broker cluster assignment not to work correctly during broker provisioning.
Correct a bug in the provision broker process resulting in errors when updating tag data.

Hooper

Version: 0.1.1643896828 (a06c0b8)

No changes since 2022-02-07

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.2.2

Add offsets = true option for mtev.pcre functions to return a table of paired offsets for matches instead of a true or false as the first return value.
Fix inflight accounting issue on eventer jobq subqueues that would cause job lockup after UINT32_MAX jobs were run on the static (default) subqueue. Notably, this affected stratcon and was the root cause of brokers appearing to be in maintenance when they were not.

notification

Version: 0.1.1644263161 (1e0f82e)

No changes since 2022-02-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1646772565 (f6ae3e9)

(libnoit) Make tag search argument arrays grow geometrically. This drastically reduces the number of memory allocation and copy operations, as well as memory usage for tag searches.

Web UI/API

Version: 0.1.1646428139 (69e310e)

No user-facing changes.

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-02-21

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220221/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1644523407 (bbae29b)

No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1645029602 (6896be6)

Function Removal Notice

Graphite find endpoint will no longer return a 500 on a bad request, favoring the more accurate 400 error.
Fix crash when attempting to fetch histogram shard data from the /histogram_raw endpoint when histogram shards are not enabled.
Fix crash when enabling the monitor module and using the legacy histogram database.
Gossip messages modified to be larger and fewer in number.
DELETE on /full/tags now accepts X-Snowth-Advisory-Limit properly to limit the number of deletes that will occur in one operation.
- There is also a config-based maximum that this value may be, and beyond which the limit set by X-Snowth-Advisory-Limit will not be respected. The default is 10,000 metrics per batch. To change the advisory limit maximum, add the following line of configuration, within the <rest> stanza:
```
<rest>
  ...
  <delete max_advisory_limit="20000"/>
</rest>
```
- Additionally, this endpoint now returns the header X-Snowth-Incomplete-Results: "true" if the requested delete operation requires more deletes than allowed by X-Snowth-Advisory-Limit or the server-configured maximum. This helps ensure that very large deletes will not consume memory ad infinitum. NOTE: This is a change in behavior. If utilizing the full delete API in scripting, use a loop to maintain the previous behavior, e.g.
```
do {
   response = call-delete-api()
} while( response->header->'X-Snowth-Incomplete-Results: "true"' )
```
Add explicit histogram:random: functions for each supported CDF.
Fix bug where jlog subscribers could hang around for too long, causing journal data that should have been removed to remain on disk until the next time the node restarts.

fault_detection

Version: 0.1.1639089770 (90cc626)

No changes since 2022-01-10

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.10.23

Add re-connection support for IRONdb proxy dial attempts.
Add support for metric search match impl options.
Correct an error causing metric search graphs not to save correctly.
Add the _check_module and _check_target fields to the response data returned by the /alert endpoint.
Correct an error in the /graph endpoint handling of stacks.
Improve error messaging when proxy requires an account ID.
Add support for future broker check module type support to the /check_bundle endpoint.

Hooper

Version: 0.1.1643896828 (a06c0b8)

No changes since 2022-02-07

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.2.0

Deprecate mtev_atomic functionality.
Added MTEV_MAYBE_REALLOC_WITH_TYPE for C++ compatibility. The older version is maintained for backwards compatibility.

notification

Version: 0.1.1644263161 (1e0f82e)

Create and send alerts when Postgres database is unavailable, store them when Postgres is back.
Configurable log size and count via resources.conf.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1644523407 (c08cd36)

(libnoit) Tag searches can be (much) longer than tags.
- A Graphite-style tag search could have thousands of {opt1,opt2,opt3} expansion internals making the search expression very long while preserving a reasonable final matching tag cat or name. Allow the AST to parse them arbitrarily long.

Web UI/API

Version: 0.1.1644424598 (5edec6a)

Remove the view-password toggle from all password entry form fields.

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-02-07

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220207/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1643128765 (31f5da2)

No changes since 2022-01-10

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1643924105 (c7a9d48)

Function Removal Notice

Change user-facing CAQL errors to 400 (from 520) HTTP codes.
Implement each:coalesce(X) in CAQL.
NNTBS shard states are now being reported to stats.json.
vector:pack() now takes a meta option that passed tag/name stream metadata through. The default (1) is to pass through metadata. This changes the default naming of unpacked vectors.
vector:pack() now takes a sloppy option that allows colliding streams by vector index. The default (0) will throw a run-time error when two streams map to the same vector index.
Finding an unconfigured shard on disk during boot will no longer result in a FATAL exit.
CAQL histogram:sum() now supports a period kwarg defaulting to 1M.
/fetch sums now normalize to 1s ranges and support override with period kwarg.
Support period=X transform and reduce arguments for histogram:rate functions.
Add rate_below and rate_above transform operations for /fetch.
Add each_rate_below and each_rate_above reduce operations for /fetch.
Update reporting of syntax errors in find tag queries to offer more in-depth information to users.
Fix CAQL histogram:rate(period=X) where X is not 1s when optimization is on.
Support smart quotes in CAQL for strings.
CAQL is now strict for duration-typed arguments.
Allow CAQL non-aggregate-safe CAQL functions to work on sub-minute periods.
Fix table header on #internals tab in web console.
Prevent NNTBS timeshard manager from accidentally opening a shard in maintenance.
Completely deprecate all system configuration via the <pools> stanza. New configuration will need to be done via the <eventer> stanza. Any existing <pools> values will be ignored and a warning message will print to the error log.
Improve rollup efficiency by no longer rolling up raw numeric shards that fall entirely outside of all NNTBS retention windows. Instead, mark them as completed so they will be removed.

fault_detection

Version: 0.1.1639089770 (90cc626)

No changes since 2022-01-10

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.10.21

Coerce numeric JSON output of graph access key width and height fields.
Fix error that allowed saving worksheets containing no JSON data.
Fix a bug causing the /broker endpoint to only return details for a single member of a clustered broker in certain cases.
Fix alpha field values on graph datapoints being returned as floating point values in some cases. These will now always be returned as string values.
Correct an error causing contact group escalations not to be correctly returned as a 5-item array with empty items padded to null. Also change contact group searching so that deleted contact groups are no longer searchable.

Hooper

Version: 0.1.1643896828 (a06c0b8)

Add logging config to Bert's resources.conf file. Hard-coded for now, and matches in-code defaults. Could be made site-configurable in the future.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.7

Expose observer_note in the lua http context.
Add utils/mtev_curl API for doing non-blocking curl work within the eventer including ACO bindings.
Improve consul support adding programmatic access and lua integration.
Add mtev_reverse_proxy_changed hook on reverse socket proxy setup and teardown.

notification

Version: 0.1.1640026848 (b149263)

No changes since 2021-12-13

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1643128765 (1affc8f)

(broker) Create the external module checks hash in a thread-safe manner.

Web UI/API

Version: 0.1.1643820545 (5e11ce8)

Change how whereTo redirection parameter is handled to disallow redirection to any domain except valid Circonus or configured app hostnames.
Additional fix for CAQLbroker provision script and FQDN hostnames.

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-01-24

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220124/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1641824017 (31f5da2)

No changes since 2022-01-10

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1642171555 (5a29dc7)

Function Removal Notice

Make time shard resizes less frequent.
Add graphite:removenode to CAQL.
Modified the usage of x-snowth-delete-rollups to be accepted via the histogram_rollup and no other raw deletion endpoint.
Add a backlog table heading to the job queues admin interface.

fault_detection

Version: 0.1.1639089770 (90cc626)

No changes since 2022-01-10

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.10.18

Correct an error causing extended permissions not to get applied correctly to superuser logins when using cookie authentication.
Fix a search query builder bug that was causing errors when searching using the /graph or /worksheet endpoints.

Hooper

Version: 0.1.1642430216 (01612bb)

Convert web_frontend and bert-bot services to native Systemd.
Install circonus-agent from Circonus yum repo rather than from setup.circonus.com.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.5

No changes since 2022-01-10

notification

Version: 0.1.1640026848 (b149263)

No changes since 2021-12-13

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1640890694 (21a574a)

No changes since 2022-01-10

Web UI/API

Version: 0.1.1642784326 (a242e89)

CAQL support has been added to the Metrics Explorer.
Bugfix: UI role selectors were missing from the tokens page and account settings SAML dialog.
CAQLbroker provision script can handle the case where the system hostname is a FQDN but the short name is used in site.json.

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

2022-01-10

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20220110/x86_64/

ca

Version: 0.1.1630083710 (f5879e1)

No changes since 2021-09-06

caql_broker

Version: 0.1.1641486310 (dbb4c57)

Disable TLS versions less than 1.2 in listener configuration.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1641426910 (70d35c3)

Function Removal Notice

Remove deprecated nnt field from /state output.
Fix potential crash when bad data points are found during raw data rollups.
Fix tag_cats and tag_vals endpoints to respect the X-Snowth-Advisory-Limit header.
Do not overwrite the stack when receiving garbage data from a shard.
Improve CAQL error messages.
Fix bug where find requests were not respecting the default limit if no limit was manually specified.
graphite_translate additions/fixes to better match graphite behavior:
- new functions - add(), movingMin(), randomWalk(), stddevSeries(), pow() and powSeries()
- pieMin(), pieAverage() and pieMax() implemented as min(), avg() and max()
- all sorting and pictire manipulation functions implemented as pass()
Fix race condition on old shard removal that can cause corruption.
Improve find query performance in some pathological cases.
Fix possible crash when deleting a timeshard while it's also being resized.
Fix graphite irondb_tag_filter search bug that prevented query proxying.
Fix potential crash on invalid histogram journal data.
Fix races in arts and arts compaction.
Reconstitute recovery from errors on the sending nodes has been partially automated. It now automatically skips shards on errors from the sending node after a configurable number of retries and will retry them once more after finishing all the well-behaving shards. Upon completion, a final list of shards that could not be successfully reconstituted will be recorded in the errorlog, allowing further diagnosis and actions to be taken by the operator as needed. Shards which are in maintenance will also be skipped and retried in a similar way.

fault_detection

Version: 0.1.1639089770 (90cc626)

No user-facing changes.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.10

No changes since 2021-05-03

GoAPI

Related roles: api

Version: 0.10.16

Prevent provisioning brokers from receiving empty certificate data.
Add a case for handling /broker searches using ?f__type=enterprise.

Hooper

Version: 0.1.1641325247 (9c49ef4)

Change product statistics survey cron job to run as the root user.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 2.1.5

More usage of const types for strings in DSO modules.
Expand TLS version options and default 1.2 as the minimum.

notification

Version: 0.1.1640026848 (b149263)

No changes since 2021-12-13

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1640890694 (21a574a)

Remove alloca() usage.
Remove Log4j.

Web UI/API

Version: 0.1.1641403825 (668c047)

Bugfix: some UI page ordering options were ignored due to params validation.

web_stream

Version: 0.1.1631023129 (445f7a4)

No changes since 2021-09-20

Release Notes 2022

2022-11-14​

ca​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2022-10-17​

ca​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2022-09-19​

ca​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2022-08-22​

ca​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2022-07-25​

ca​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2022-06-27​

ca​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2022-05-30​

2022-11-14

ca

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2022-10-17

ca

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2022-09-19

ca

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2022-08-22

ca

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2022-07-25

ca

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2022-06-27

ca

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2022-05-30

ca