Release Notes 2020

2020-11-16

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20201116/x86_64/

caql_broker

Version: 0.1.1600709045 (265d3aa)

No changes since 2020-10-05

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1605212210 (ad1d295)

Function Removal Notice

(stratcon) Fix raw_ingestor bug that could result in incorrect error messages.
Improve raw_only_delete efficiency and memory usage, add /raw_only_delete REST API.
Ensure tag-search indexes are correctly updated when deleting metrics.
Restrict batch size in raw-only delete, in case the find set is too large to fit in memory.
Performance improvements to the active_count query.

fault_detection

Version: 0.1.1604429515 (5d19ba9)

If a ruleset would trigger due to a rule number change but the severity would remain at RULE_SEV_NONE, do not send a notification. This eliminates work for the notification system when there is no effective change of severity.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.3

No changes since 2020-10-19

GoAPI

Related roles: api

Version: 0.8.11

Removes reference to an obsolete database column that can cause errors for broker provisioning.

Hooper

Version: 0.1.1605017577 (e941604)

Update /www/etc/circonus.conf with information about the secondary IRONdb cluster in multi-datacenter deployments. This is used to create a special database view for queries from the secondary datacenter.
Remove support for obsolete additional_clusters attribute in the data_storage role.
The master attribute on the ca role is deprecated, and replaced with primary. The value remains unchanged (the hostname of the primary CA host). A notice will be issued at the end of the run if the old attribute is in use.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.13

No changes since 2020-10-19

notification

Version: 0.1.1603745331 (f700be6)

No changes since 2020-11-02

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1604518842 (dc38d64)

(broker, libnoit) Fix decoding of DD-style statsd tags in multi-line mode. Allow decoding of bare tag names as categories with no values (foo →foo:)

Web UI/API

Version: 0.1.1605196537 (2edda85)

Feature: add support for new Ruleset "Minimum Window Duration" parameter to the UI.
Feature: add "export data" menu item to the metric display page.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-11-02

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20201102/x86_64/

caql_broker

Version: 0.1.1600709045 (265d3aa)

No changes since 2020-10-05

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1603910579 (5c50080)

Function Removal Notice

Improve selection criteria for non-rollup metrics to increase performance of surrogate db removal.
Use localstate directory instead of /var/tmp for temporary proxy data used in inter-node data fetches.
Add tag:synth CAQL function.
Fixed several file descriptor leaks related to improper handling of proxy buffers.

fault_detection

Version: 0.1.1602882979 (e4420e3)

Clarify curl failure messages in API calls.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.3

No changes since 2020-10-19

GoAPI

Related roles: api

Version: 0.8.10

No changes since 2020-10-05

Hooper

Version: 0.1.1603920911 (564f555)

(hub) Remove obsolete enzo_sync.pl cron job.
Add circonus.conf key Inside::PrimaryDC to indicate the active datacenter.
In backup datacenters, build PostgreSQL database replicas from the local primary, i.e., the host that will become the primary database when this datacenter becomes active. This removes the need to do multiple replica rebuilds when failing over.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.13

No changes since 2020-10-19

notification

Version: 0.1.1603745331 (f700be6)

Bugfix: Correct log/error messages about connect/retry with multiple mq hosts.
Bugfix: Eliminate possible contention between threads on mq connection.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1603464075 (2daf7f6)

(broker) Fix lingering sockets on listener-based checks that could lose data when the check is modified.
(broker) Fix off-by-one error that affects modules that listen for inbound traffic, such as graphite.

Web UI/API

Version: 0.1.1603992643 (0b42696)

Improvement: new HTTPtrap checks created in the UI will default to asynchronous mode.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-10-19

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20201019/x86_64/

caql_broker

Version: 0.1.1600709045 (265d3aa)

No changes since 2020-10-05

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1602868761 (bc6f8b7)

Function Removal Notice

Fix tag search index compaction bug where entries could be skipped, leading to incomplete find results.
Fix error in tag index iteration that could cause incomplete find results.

fault_detection

Version: 0.1.1601906111 (b41e705)

No user-facing changes.

Composite Checks Removal Notice

This release will remove the "Ernie" component completely, disabling all composite checks and removing all composite brokers.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.3

Set SO_REUSEPORT = 1 for listener.
Add -b to disable BCD/backtrace integration.
Name threads on Linux to aid debugging.
Prevent abort when queue removal fails.

GoAPI

Related roles: api

Version: 0.8.10

No changes since 2020-10-05

Hooper

Version: 0.1.1602875577 (ae1405e)

Bert's MQ heartbeat value is in seconds, not milliseconds, so use the correct value.
Keep fault_detection and notification services off in backup datacenters.
Remove the composite broker (Ernie) completely. All active composite checks will be deactivated, and all composite brokers will be decommissioned.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.13

Ensure enough bchain space for over-sized headers. Large headers would previously cause an assertion.

notification

Version: 0.1.1602708622 (3de2ee3)

Bugfix: Correct MQ connection heartbeat value to seconds, not ms.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1600709045 (04fc977)

No changes since 2020-10-05

Web UI/API

Version: 0.1.1602787689 (f3efdcf)

Cleanup: Removal of database support for a number of no longer used features, such as pending_accounts, audit_*, and oncall.
Improvement: Rulesets page now allows sorting by Last Modified date.
Improve error handling in product survey script.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-10-05

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20201005/x86_64/

caql_broker

Version: 0.1.1600709045 (265d3aa)

No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1601578609 (c9eb730)

Function Removal Notice

Add explain=1 option to /find//tags endpoint.
(stratcon) Add 'backoff' capability to allow stratcon to detect when a node is overloaded and defer sending more data to that node.
Restore eventer site config file. Eventer customization can now be placed in /opt/circonus/etc/irondb-eventer-site.conf and will be safe from package updates.
Fix memory leak when the eventer rejects raw journal data for having too many jobs on the backlog.
Fix memory leak when compacting tag-search indexes.

fault_detection

Version (faultd): 0.1.1600893778 (dbd6888)

Fix crash from race condition on absence scheduling.
Add support for "equals" and "does not equal" to numeric rulesets. This is an upcoming feature that is not yet available via the UI/API.

Version (ernie): 0.1.1593201733 (d1e0e7b)

Composite Checks Removal Notice

Since the 2019-05-06 release, the circonus-ernie service has remained, serving its secondary role as a broker for Composite checks. This functionality has been replaced and extended by CAQL checks. The Composite check type, and the supporting service, will be removed in a future release. Operators and users of composite checks should immediately begin converting to CAQL checks. Please contact Circonus Support (support@circonus.com) with any questions.

No changes since 2020-06-29

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.1

Add Backtrace (crash reporting) support for the FQ daemon.

GoAPI

Related roles: api

Version: 0.8.10

Fixes a blocking issue with fq tracing span data recording that could cause a goroutine leak and cause the goapi process to hang.
Fixes a mutex lock issue in goapi fq logging that could cause the goapi process to hang.
Corrects an issue that could cause duplicate token applications to attempt to add, resulting in a false authentication error.
Updates OAuth2 get user information API calls to allow multiple types to be possible for the OAuth2 response error property.
Returns an empty object for /user endpoint contact_info responses when no information exists, rather than omitting the property.
Corrects the error code returned for unauthorized /user requests.

Hooper

Version: 0.1.1601401182 (971bc71)

Enable FQ crash reporting in the mq role.
Add an attribute to the stratcon role to configure its concurrency for sending data into IRONdb. The irondb_put_concurrency value is the desired number of threads that should be devoted to sending metrics into IRONdb. If not specified, it defaults to 50. The actual number of active threads may drop lower than this value, depending on the volume of metrics coming from brokers, but it will not exceed the configured concurrency. Raising the concurrency can help if the storage feeds from brokers show delay, and the stratcon host still has CPU and network resources to spare.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.12

Allow <config> in <eventer> to be a deep descendent.

notification

Version: 0.1.1601405635 (ce8fcd1)

Send alerts for public brokers to account ID 1.
Lower system-wide alert suppression threshold to 20% of public brokers (was 50% i.e. 1 of 2.)

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1600709045 (04fc977)

(broker) Support CORS via httptrap.

Web UI/API

Version: 0.1.1601580062 (a2f6e95)

The /worksheet API endpoint is now handled by GoAPI.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-09-21

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200921/x86_64/

caql_broker

Version: 0.1.1597935486 (f91cde9)

No changes since 2020-08-24

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1600205584 (587454e)

Function Removal Notice

Fix graphite module crash due to bad FlatBuffer identifier.
Block size for raw reconstitute by shard is now based on number of datapoints instead of number of one-hour blocks for better consistency and resilience with variable-frequency data.
Improve inter-node communication failure error messages.
Fix memory leak in stratcon raw_ingestor module.

fault_detection

Version (faultd): 0.1.1599572381 (a7d1f13)

No user-facing changes.

Version (ernie): 0.1.1593201733 (d1e0e7b)

Composite Checks Removal Notice

No changes since 2020-06-29

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api

Version: 0.8.6

No changes since 2020-09-07

Hooper

Version: 0.1.1600699570 (f7f35e5)

Make data_storage rollup suppression filter configurable. This is a tag query whose matching metrics will be skipped during rollup, and removed from the raw database as shards expire. It was hard-coded at introduction in the 2020-04-06 release, but it is an unnecessary operation if not needed. Operators wishing to preserve the previous value should set the rollup_suppression_filter attribute to and(__rollup:false) in site.json. See data_storage attributes in the Installation manual for details.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.11

Allow OPTIONS method in REST layer.

notification

Version: 0.1.1600113814 (436df91)

New macro support for metric stream tags and tag values in custom alert format. See Custom Alert Formats in the User Manual for details.
Better accounting of cleared cases to avoid buildup in Postgres.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1599663111 (78f6e9a)

No user-facing changes.

Web UI/API

Version: 0.1.1600284339 (eac1098)

No user-facing changes.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-09-07

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200907/x86_64/

caql_broker

Version: 0.1.1597935486 (f91cde9)

No changes since 2020-08-24

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1599158541 (cddac2b)

Function Removal Notice

Allow raw numeric reconstitute to go by shard instead of by metric. This will significantly increase the speed of the raw_db portion of the reconstitute process.
Compaction tool for removing fully-deleted metrics from numeric rollups (beta feature).

fault_detection

Version (faultd): 0.1.1598630616 (459a74a)

Fix use-after-free crash in rulestream handling.

Version (ernie): 0.1.1593201733 (d1e0e7b)

Composite Checks Removal Notice

No changes since 2020-06-29

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api

Version: 0.8.6

Remove host remapping feature of graph copying.

Hooper

Version: 0.1.1598448150 (bdc8405)

(data_storage, web_frontend) Treat the absence of one_minute_rollup_since as equivalent to 0. In the common case of always having had one-minute data, this attribute can be omitted from site.json.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.10

Fix uninitialized read in mtev_b32_decode and mtev_b64_decode in the slow path.

notification

Version: 0.1.1598890278 (ba1c5c5)

Improvement: Limit alert summary emails to a maximum of 30 alerts, preventing the application from stalling on building unreasonably large payloads when there are a large number of active alerts.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1598412125 (668b758)

(broker) Fix memory leak and data loss in httptrap check module. This can occur when a trap check is set to async mode and the payload happens to not be fully consumed in a single read.

Web UI/API

Version: 0.1.1599254196 (3eeee5f)

Bugfix: "metric 'x' not found in Check" error in dashboard create/update via API.
Bugfix: Validating tag query "filter" in rule_set creation failed to require closing parenthesis.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-08-24

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200824/x86_64/

caql_broker

Version: 0.1.1597935486 (f91cde9)

Change search:metric:...() implementation to leverage the IRONdb /find//tags API directly.
find and search in CAQL now both support internal and external expansion via the impl keyword. Pre-existing approach (search: external, find: internal) remains unchanged.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1598018876 (9487717)

Function Removal Notice

Allow retrying on failures to write to the LMDB NNTBS shards during reconstitute. Improve error messages related to the NNTBS reconstitute process.

fault_detection

Version (faultd): 0.1.1597438234 (d5d9d7c)

Remove null pointer access on missing ruleset when handling metric message.
Fix for race condition in rule set culling.
Don't allow windowed rules to leave "booting" stage until windowing_min_duration is met.
Disable autocull on absence-triggered rulesets.

Version (ernie): 0.1.1593201733 (d1e0e7b)

Composite Checks Removal Notice

No changes since 2020-06-29

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api

Version: 0.8.5

Fix an authentication token cache issue.
Add a format query option to metric search requests. If set to format=min, only a minimal subset of metric data will be retrieved.
Fix an error that was causing log entry data over 1k not to be truncated correctly for structured types.

Hooper

Version: 0.1.1597857834 (52d370a)

(caql_broker) Install crash reporting infrastructure, if enabled.
(web_db) Provide for alternate PostgreSQL database usernames for standby datacenters.
(data_storage) Override sysctl configuration for vm.max_map_count if system memory is large enough.
(notification) Removed all reference to XMPP/Jabber from Notification templates and sample site.json. These settings were never used in the Inside product and have not been supported in public SaaS for years.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.9

No user-visible changes.

notification

Version: 0.1.1597767422 (7b777bf)

Increase Java max heap size to better cope with large-scale influxes of events.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1597935486

(libnoit) Add statistics for the metric_director miss cache.
(libnoit) Fix startup issues with metric_director module load and hook registration.

Web UI/API

Version: 0.1.1598275702 (56472e6)

Improvement: Inside selfchecks have been updated to take advantage of metric filters and pattern-based rulesets. Additional checks have also been added. See the installation manual for details.
Deprecated XMPP support removed. XMPP removed from user profile.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-08-10

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200810/x86_64/

caql_broker

Version: 0.1.1594921008 (db6e6bc)

No changes since 2020-07-27

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1596753483 (02d1b57)

Function Removal Notice

Various use-after-free fixes.
Various memory leak fixes.
Implement vector:pack() and vector:unpack() in CAQL. These are experimental.

fault_detection

Version (faultd): 0.1.1596642818 (20855a6)

Crash fixes related to ruleset deletion.

Version (ernie): 0.1.1593201733 (d1e0e7b)

Composite Checks Removal Notice

No changes since 2020-06-29

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api

Version: 0.8.3

Three new fields have been added to the data returned by the /metric endpoint: _check_broker, _check_module, and _check_target.
Fix for worksheet graphs to be returned with incomplete CID values.
Fix an error that would cause all worksheets returned by the API to be tagged as favorite.
Fix an error that could cause the process to hang on API servers without a local fq installation.
Split the goapi log into separate access and error logs. These logs by default will be located at /var/log/circonus/goapi_access.log and /var/log/circonus/goapi_error.log.
Truncate log entry data fields at 2 KB. This prevents very long individual log entry data fields from obscuring the log files.
Fix a noisy error about trace information from the proxy server.
Change the log level of redundant errors to debug. This allows only one error level log entry to be reported for any single error, while extra detail will still be available at the debug level if needed.
Failures to authenticate are no longer logged at the error level. They are instead logged at the warning level.
Correct an issue with LDAP authentication causing cookies not to be parsed correctly, resulting in failed authentications.
Correct a config load issue that would cause errors during OAuth authentication attempts.

Hooper

Version: 0.1.1596827857 (9228394)

Stratcon: The uuid attribute is deprecated in favor of a node_ids object containing a key for each stratcon host, whose value is a UUID string. All stratcon hosts must be included. Previously, if multiple stratcon hosts were present, they would all have the same UUID. This does not cause issues for stratcon functionality, but does affect the metric names of its internal statistics. If node_ids is not present and there are multiple stratcon nodes, a warning will be issued at the end of the run.
- Example:
```
"stratcon": {
    "_machlist": [ "host1", "host2" ],
    "node_ids": {
        "host1": "<a_uuid>",
        "host2": "<another_uuid>"
    }
}
```

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.8

Revert some internal changes to selection of the active zipkin span. These are believed to lead to some hard to track use-after-free issues.
Fix reference counting bug in mtev_zipkin_aco_swap_span.
Skip a lua stack attempt on failed resumption.
Fix compile issue with clang and eventer spec instantiation.
Improve the zipkin implementation to be more aware and friendly in ACO.

notification

Version: 0.1.1594401421 (97de27d)

No changes since 2020-07-13

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1596462854 (9112bc5)

(libnoit) Use rwlock on feeds iteration and manipulation.
(libnoit) Improve time encoding on B* records and IEP filtering.

Web UI/API

Version: 0.1.1596832425 (219d51f)

The /tag API endpoint is now handled by GoAPI.
Improvement: search now supports multiple values per field, e.g., ruleset_id:123,124,125.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-07-27

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200727/x86_64/

caql_broker

Version: 0.1.1594921008 (db6e6bc)

Convert check and filterset storage from XML to LMDB. This improves initial load times as well as configuration update times.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1595881924 (3430646)

Function Removal Notice

Fix race condition in search index management.
Various use-after-free fixes.
Default timeout for latency_sensitive event loop increased to 10 seconds.
Move parsing of raw data into separate jobq to process asynchronously.
Update CAQL search:metric:...() to up-convert to a tag search and leverage the find:...() processing unit.
Alter both search:metric:...() and find:...() to support external (out-of-band) expansion replacement. Within IRONdb, default both to internal index-based expansion. When not in IRONdb (CAQL-broker) maintain the pre-existing default (search: external, find: internal). Allow config-based override of these extra-IRONdb defaults.
Add the impl keyword arg to both search:metric:...() and find:...() to allow statement-based selection of internal or external search expansion.

fault_detection

Version (faultd): 0.1.1595266722 (dc5b850)

No user-facing changes.

Version (ernie): 0.1.1593201733 (d1e0e7b)

Composite Checks Removal Notice

No changes since 2020-06-29

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.7.24

Add support for authentication using LDAP cookies.

Hooper

Version: 0.1.1595336617 (126e230)

API: Fix regex match of origin URL in Apache configuration to allow for deployments where the URL host is the base site domain.
Add optional top-level site.json attribute, cookie_domain. This is the domain name common to all client-facing endpoints, which will be used for session cookies. If absent, the domain value is used, and it is assumed that all endpoints are either the same as, or subdomains of, the site domain. This option may be useful if your web frontend and API hostnames are not both derived from the site domain. For example, if your web and API hosts are "circonus.example.com" and "circonusapi.example.com", respectively, then you would set cookie_domain to example.com.
Add optional site.json attribute for the API role, external_host. This may be used in deployments where the client-facing API host does not follow the convention of api.<site_domain>. It is used by the web UI to construct requests from client browsers directly to the API for features such as metric name autocomplete in Metrics Explorer.
Add hub cron job to gather product usage statistics on a monthly basis.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.5

Fix bug in mtev_plock_try_rtos in MTEV_PLOCK_HEAVY mode where multiple threads could enter the S critical section.
Support -DMTEV_MEMORY_DEBUG to track alloc/free stacks for mtev_memory_safe operations.
Introduce new version global symbols to assist debuggers.
Add mtev_log_backtrace that will print a previously captured backtrace.
MTEV_RDTSC_ENABLE now replaces MTEV_RDTSC_DISABLE (defaults to off.)
Add flexible conf interpolation system supporting {{</*impl>:<fallback>:{<term*/>}}}.
Add a conf interpolation implementation for hwloc supporting numanodes, packages, cores, and pus.
Add a conf interpolation implementation for ENV.

notification

Version: 0.1.1594401421 (97de27d)

No changes since 2020-07-13

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1594926202 (99f8170)

No user-facing changes.

Web UI/API

Version: 0.1.1595525197 (af3fac0)

Bugfix: Do not deactivate checks when retiring a node from a broker cluster (unless it is the last one.)

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-07-13

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200713/x86_64/

caql_broker

Version: 0.1.1593725670 (419b941)

Adopt updated noit_metric_director API for registering interest in metric streams.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1594313779 (28acd7a)

Function Removal Notice

Memory leak fixes.
Add a header, X-Snowth-Verify-Owner, for all find calls that will verify that the node being queried owns the metric in question before reporting it. This will make counts more accurate on clusters where a rebalance has been performed and there are extraneous surrogate database entries on nodes.

fault_detection

Version (faultd): 0.1.1593542161 (53ad2a2)

No user-facing changes.

Version (ernie): 0.1.1593201733 (d1e0e7b)

Composite Checks Removal Notice

No changes since 2020-06-29

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.7.23

Fixes a bug where metric searches for a term equivalent to a single forward slash were not being processed correctly.
Modifies metric search so that base64-encoded terms are not decoded, but passed straight through to IRONdb.

Hooper

Version: 0.1.1594230879 (e688475)

Add on_first_install script to web_db initial setup.
Sync circonus.conf template with recent changes in Circonus SaaS.
Reorganize circonus.conf template to keep it sorted, making future audits easier.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.4

Add mtev_memory_regular_ck_malloc for unsafe/regular memory allocations.

notification

Version: 0.1.1594401421 (97de27d)

Bugfix: added cleared_value macro (previously only had clear_value).
Improvement: added ruleset_notes and ruleset_link macros.
Bugfix: support numeric_max/min rule_desc macro expansion.
Bugfix: switch from deleted event_criteria_numeric table to event_criteria_lookup.

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1593022540 (2254dfe)

No changes since 2020-06-29

Web UI/API

Version: 0.1.1594402895 (e1dd82d)

Improvement: internal cleanup of rule_set storage and indexing.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-06-29

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200629/x86_64/

caql_broker

Version: 0.1.1593022540 (07bbb3a)

No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1593189054 (bb85dcd)

Function Removal Notice

Fix bug causing crashes when resuming a reconstitute and the debug/surr_load debug log is enabled.
Remove the concept of source/category and check_name. These were metadata recorded in the surrogate database, and used only for Graphite searching. Their removal reduces complexity and saves memory, while Graphite search compatility is preserved through other means. This change modifies the content of replicated surrogate data, which will cause an increase in surrogate update activity on nodes that have not yet been upgraded. Operators are advised to complete a rolling upgrade as quickly as possible to minimize potential disruptions.

fault_detection

Version (faultd): 0.1.1593025381 (72ec5dc)

Auto-cull cleared rule streams for patterned-based rules.
Cull on clear (API and AMQP).
Change internal absence triggering to use METRIC_ABSENT to differentiate from a value-present-but-null.
Change windowing_min_duration to default to 0.
Adopt new noit_metric_director APIs to delegate matching.
Use miss cache for better performance.

Version (ernie): 0.1.1593201733 (d1e0e7b)

Composite Checks Removal Notice

Configuration changes in PostgreSQL queries run at startup. Has no effect for Inside deployments since Ernie no longer does fault detection. The update is being done for consistency.

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.7.21

Fixes a memory leak in the GoAPI logging package that was leading to periodic out-of-memory crashes.
A new field has been added to data results returned by the /metric endpoint. The _display_name field contains the display name of the check which collects the metric described in the results.
Internal logger will now submit log entry data to fq if configured.
The check ID data cache will now only automatically update data for active checks, improviding query efficiency.
No longer returns a 500 error if IRONdb fetch or caql data results contain inf or NaN data values. It will still log the event as a warning.

Hooper

Version: 0.1.1593436753 (1f1ceee)

(data_storage) No longer need to explicitly install/update jemalloc, as it is now directly depended upon by the snowth package.
(web_frontend) Use only the site domain value for Web::CookieDomain in /www/etc/circonus.conf. If the optional url_host attribute is in use, operators are advised to have users log out and clear their browser cookies, then log back in.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.3

Various fixes from Coverity static analysis.
Support the new MLKV() macro instead of the MLKV{} mistake if MTEV_USE_MLKV_2 is defined. This will become the default in the next major release and users can get their code ready now by adding -DMTEV_USE_MLKV_2 to their CPPFLAGS. This is not ABI breaking, but the next major release will break the ABI such that C++ can be supported.
Fix string-matching in mtev_logic.
Add functional probability for mtev_frrh via mtev_frrh_set_prob_function.
Remove noitedit/strlcpy.h header and provide that via mtev_str.h.
Add mtev_strlcpy and mtev_strlcat.
Runtime detection of ASAN to avoid stack-buffer-underflow failures when apps use ACO (runtime avoids memcpy for stack replacement).
Allow basic use of logging immediately at application start (before init).
Add mtev_log_hexdump_ex.
Add documentation for the logging facilities.

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1593022540 (2254dfe)

(libnoit) Performance and thread-safety fixes to noit_metric_director.

Support base64-encoded histograms as string values, e.g.:

{ "metric": { "_type": "[hH]", "_value": "<b64encoded hist>" } }

Web UI/API

Version: 0.1.1593203059 (ef7d482)

Fixed graph rendering bug: when viewing high-resolution (typically sub-1m period) data, gaps in the data were not rendered as wide as they should have been.
windowing_min_duration has been added to the RuleSet API as a new field for numeric rulesets. This is the minimum amount of time a windowing method must collect before it is evaluated for the purpose of alerting.
Ruleset support for metric_name + filter combinations for matching with tag filters on a specific metric name over one or more checks.
New simplified Ruleset creation modal.

web_stream

Version: 0.1.1591797218 (3c0de1a)

No changes since 2020-06-15

2020-06-15

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200615/x86_64/

caql_broker

Version: 0.1.1591810140 (84eef8c)

Fix: Only keep compiled CAQL statements in the register. When using disabled tag_search, we were keeping half-initialized search objects around. Subsequent pull() calls were failing on them.
Enable mtev log stats for error/notice/warn.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1591905048 (97b8283)

Function Removal Notice

Remove source and check name from graphite tree.
Replace check name with explicitly configured aliases.
Implement on-disk persisted ART maps for tag search (improve boot-time index construction by up to 100%)
Fix stuck set-crdt (metadata) replication to third-parties: automatic feed (jlog) repair when corruption is detected.
Fix possible memcmp crash when surrogate keys are smaller than uint64_t in size.

fault_detection

Version (faultd): 0.1.1591214787 (4d1c531)

Allow null text values to update rule states.

Version (ernie): 0.1.1544639627 (db11a23)

Composite Checks Removal Notice

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.7.19

Allow alert searches to return inactive alerts only when the search contains the term (active:false) or (active:0).
Incorporates all of the documented search terms supported for alerts.
Fixes an issue that could cause problems parsing V2 search queries.
Adds a ruleset_pattern search term to /alert searches.

Hooper

Version: 0.1.1591907623 (f3b4eaa)

Remove the search-only GoAPI instance from web_frontend nodes. Searching will now be performed directly against API hosts.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.12.0

Fix more json-lib brokeness (bad pointers).
Only install public mtev_json headers.
Fix an issue where valid SSL connections that do not handshake in one non-blocking action would register a spurious certificate error.
Inform lua bindings of TLS1.3.
Change mtev_plock atomic mode to pause instead of repnop for stall.

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1591810140 (1a0ba6b)

Adopt libmtev public JSON APIs.
Broker: Update default CA trust store file. Some newer root certificate chains were not being recognized, leading to spurious validation errors.
Crash fixes related to thread safety.

Web UI/API

Version: 0.1.1591995987 (67e3bdf)

Fixed security vulnerability: Prevent XSS being passed to the Integrations Module page via the type parameter.
Fixed default metric type on the Metric Details page (so if a metric has its type changed, it will default to the latest type).
Deprecated manual histogram collection toggling on static metrics. It will be unavailable for Checks and Accounts created on or after 2020-07-01.
Fixed histogram collection toggling on static metrics (on the Check Details page).
Renamed Account Profile to Account Settings.
Deprecated Worksheets and hid the Worksheets menu item on accounts which don't have any Worksheets.
Removed the Tags page from the UI.
Fixed security vulnerability: Prevent XSS being passed in as pagination parameters.
When listing static check metrics on the Check Details page, list them alphabetically, with the disabled metrics at the bottom.
When listing static check metrics on the Check Details page, don't use placeholder values which cause the metric type to not be detected properly.
Fixed security vulnerability: Prevent XSS coming from modified page URLs being output unencoded.

web_stream

Version: 0.1.1591797218 (3c0de1a)

Correct some mtev_json usage.

2020-06-01

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200601/x86_64/

caql_broker

Version: 0.1.1590667248 (f4a03a2)

Optional config variable to disable tag_search completely.
Add configurable queue-size limit to metric_director. Incoming messages will be dropped when the queue is full.
Add caql-broker-start script for managed startup.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1590590608 (494fa3c)

Function Removal Notice

Load libmtev ptrace modules for better-annotated crash reporting.
Change the /find//active_count API to use the count_only methodology from find which is faster and more memory efficient, but yields estimates when nodes are down.
Support __type:(numeric|histogram|text) in tag searching. Does not support pattern matching.
Add optional lmdb_flags attribute to nntbs and metadata nodes in config to allow for better control. Change to drop MDB_NOMEMINIT by default as the performance implications aren't significant. Drop MDB_NOSYNC from the default metadata flags.
Allow nntbs timeshards to recover when bad data is encountered and add .db.nntbs.errors (errors|ST[db-type:nntbs,...]) statistic.
Move module initialization startup messages out of the error log and into the startup log.

fault_detection

Version (faultd): 0.1.1590695961 (0b8b8e1)

Fix incorrect assessment of parent rulesets sev 1 state.

Version (ernie): 0.1.1544639627 (db11a23)

Composite Checks Removal Notice

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.7.17

No changes since 2020-05-18

Hooper

Version: 0.1.1590068866 (07c29e4)

Integration of tooling to help assess customer experience for on-premises deployments.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.11.2

Fix race condition that can cause dead events at boot.
Fix crashes by correctly using SMR in http_observer.
Ship builtin 1024 and 2048 DH params for older OpenSSL.
Eventer and HTTP improvements:
- Add eventer_ssl_ctx_new_ex which takes a mtev_hash_table as a more future-proof parameterization.
- Support for inlining key/cert/ca in config (alternative to file path)
- sslconfig changes:
  - certificate as certificate_file
  - key as key_file
  - ca_verify as ca_chain
  - Add ca_accept (for advertising acceptable CA signers to clients)
  - Add dhparam_bits for forcing a DH parameter bit length (0 disables)
- Update ssl_dhparams eventer options to use NIST defaults when omitted or left blank.
- Improve readability of SSL/crypto error messages.
- NPN upgrades could reference a stale context resulting in crashes.
- Fix multi-protocol NPN/ALPN negotiations.
- Provide helpers to make it so we can more easily register new ALPN keys.
- Allow ssl contexts to override/add ALPN keys.
- Have the libmtev http1 implementation advertise HTTP/1.1 over ALPN.
- More robust HTTP testing, including intermediate cert chains.
- HTTP/2 no long performs unsolicited compression, to appease curl.
Allow header inclusion from C++.
Update lua ssl bindings to support extensible context creation.
Update mtev.exec (and this mtev.sh) lua to accept environment that is a key,value table instead of just an array of key=value strings.
luamtev improvements:
- Make luamtev print default to stdout instead of stderr.
- Fix non-tty input when in interactive mode.
- Fix -e flag for running methods other than main.
- Add -h flag for help with a clean exit code.
Fix eventer JSON output to include timers.
Tolerate EPERM in epoll_ctl adding (so adding /dev/null doesn't assert).
New ptrace module that enumerates HTTP requests when using http_observer.

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1590613152 (a153713)

Broker: Fix race in check recycle bin, whereby a check that is being restored could wind up being deleted instead.
Broker: Add filters/show REST endpoint for getting filterset information.
Separate counts of dropped messages in noit_metric_director to distinguish between those that arrived too late from those dropped because the queue was full.
```
dropped|ST[app:noit,reason:too_full,subsystem:metric_director,units:messages]
dropped|ST[app:noit,reason:too_late,subsystem:metric_director,units:messages]
```
Support a limit for queue backlogs.

Web UI/API

Version: 0.1.1590764539 (f56c57a)

Bugfix: Fix RuleSet dependency modal so that it doesn't clip the autocomplete dropdown.
The Metrics Explorer interface is all new! Now it contains a query builder to help users find metrics easier, without needing to understand our search syntax.
Bugfix: API validation error with rule_set parent=null.
The Kubernetes automatic dashboards now have a bread crumb navigation bar at the top to make it easier to add and remove filters.
The /maintenance API endpoint will now be handled by GoAPI for all installations.
Bugfix: API would deny fetch of /broker/ID endpoint to shared account.

web_stream

Version: 0.1.1588616219 (65d35fd)

No changes since 2020-05-18

2020-05-18

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200518/x86_64/

caql_broker

Version: 0.1.1589293899 (7a5e664)

No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1589443427 (7e9056f)

Function Removal Notice

No user-facing changes.

fault_detection

Version (faultd): 0.1.1588711075 (e4fa0c9)

No user-facing changes.

Version (ernie): 0.1.1544639627 (db11a23)

Composite Checks Removal Notice

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.7.17

Added the X-Snowth-Advisory-Limit header to the allowed headers for the IRONdb proxy service.
Fix: Fixes a bug that would prevent check service data checks from being able to identify a check if it had been created since the last cache refresh.
Fix: Corrects a bug affecting filter searching for maintenance windows.
Adds the ability to filter and search maintenance windows based on start and stop time and active status.
Fix: Fixes a problem which could cause failures and HTTP 500 errors when searching for maintenance windows using certain search terms.
Fix: Fixes a problem with the format of the data in the item field in values returned by or sent to the /maintenance endpoints.
Fix: Corrects a problem that could cause the GoAPI process to hang during graceful server shutdown.
All log entries generated within the span of any request will contain a unique trace_id value for that request. This should make it easier to associate all log entries related to an individual request.
Fix: Corrects a formatting issue with the /caql endpoint default data response.
Fix: Corrects a bug that could cause process crashes when handing POST requests to the soon-to-be-deprecated /metric_cluster endpoint.

Hooper

Version: 0.1.1588734490 (4023be3)

Add an exclusion to the log pruning script to skip stratcon jlog directories. These are not disposable logs.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.10.8

Fix NPE when freeing a broken SSL context.
The /cluster PUT endpoint will now synchronously write the configuration to storage if the "x-mtev-conf-synch" header is present and set to "1".
Revert a behavior change in jobqs introduced in 1.10.3.
Further optimize mtev_log filtering.
Logic optimizations and string slices.
Logging optimizations: avoid allocations in some cases, and filter prior to Flatbuffer serialization.
Fix non-string matching in log filtering.
Fix short allocation in filter parsing.
Expose log volume statistics as metrics in mtev/stats for configured outlets.
Fix thread naming in fq module.
Add fanout and fanout_pool config options in fq module.
Assist subroutine name lookups with libunwind (for crash dumps).
Change MTEV_DWARF to default to 0, enable by setting to non zero value.
Add support for environment variable replacement in config values: ENV:fallback:{ENVKEY}

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1589321339 (6478478)

Broker: Some applications that emit statsd metrics omit the trailing line-feed, and the statsd module was unconditionally skipping over the last character, leading to erroneous tag values.

Web UI/API

Version: 0.1.1589813998 (a392058)

Fix: When viewing Rulesets through a check UUID+metric name URL instead of the standard URL, you are now rerouted properly instead of being shown an empty ruleset.
metric_tags (not to be confused with metric StreamTags or check-tags) have been deprecated for a long time. Support for these in the product have now been removed, as have the associated Postgres functions/tables for keeping them in sync.

web_stream

Version: 0.1.1588616219 (65d35fd)

No user-facing changes.

2020-05-04

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200504/x86_64/

caql_broker

Version: 0.1.1587744126 (7a5e664)

Sweep tag-search cache regularly.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1588351051 (b79bc31)

Function Removal Notice

Fix bug where we would attempt to abort LMDB transaction after failed commits, which can cause double frees.
CAQL: Add package filter:*.
Lua: Fix a bug where partially-initialized lua extensions were kept around.

fault_detection

Version (faultd): 0.1.1588034330 (6f9e5ef)

Enhancements to debug logging.

Version (ernie): 0.1.1544639627 (db11a23)

Composite Checks Removal Notice

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.7.7

Fix: Allows a value of null to be used for the contact_group escalations field. This is treated equivalently to omitting the field.
Fix: If an attempt to delete an invalid or inaccessible contact group ID is made, the correct error will now be returned, not a false indication of a successful deletion.
The field name is now a required field for contact_group types. It is now an error to set name to an empty or null value.
Styling updates for internal documentation.

Hooper

Version: 0.1.1588275656 (1c57d1c)

Remove bert-irc-bot service. The IRC notification method has not been supported for some time.
Restart the GoAPI service when circonus.conf is updated.
Remove web_db items from Stratcon's configuration. It does not make use of the Postgres database.
Configure all MQ hosts for caql_broker. Previously only the first MQ host was used, but this can cause issues if Stratcon is configured to round-robin over available MQ nodes.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.10.4

Fix use-after-free race in http_observer module.
Allow null strings in mtev_log KV metadata.
Fix metadata reporting in consul module.
Add metadata in http logging.
Implement filtering logic for log outlets.
Update mtevEL to support new types.
Require -std=c11 (use of _Generic)

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1587744126 (8373e31)

Clustering: Fetch checks and filtersets in batches (default 500).
Fix null-pointer exception in websocket handler.

Web UI/API

Version: 0.1.1588348430 (4982101)

Bugfix: Ruleset Group page would fail intermittently in some installations.
Improvement: API support for numeric CID as rule_set parent attribute.
Improvement: When editing a graph, the graph config will load immediately to allow editing of the graph without needing to wait for the data to load.

web_stream

Version: 0.1.1587405457 (2e34540)

Fix null-pointer dereference in FQ message handling.

2020-04-20

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200420/x86_64/

caql_broker

Version: 0.1.1587054536 (69ce22d)

Add tag-search result caching.
Performance improvements to message handling.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1587050127 (3af5a96)

Function Removal Notice

CAQL: Validate UUIDs passed to metric:* functions.
[raw ingestor] Lower default timeout/connect timeout for sending data to the snowth cluster from 60s/2.5s to 8s/1s. This helps avoid backlogs when an IRONdb node is responding slowly to ingestion.

fault_detection

Version (faultd): 0.1.1586976934 (78b9ee0)

Fix numeric windows for patterns against all checks.

Version (ernie): 0.1.1544639627 (db11a23)

Composite Checks Removal Notice

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.7.4

Improvements to memory utilization for resources shared between API and proxy servers.
Fix: Improve validation error messages, on multiple endpoints, that were confusing because they were being wrapped inside JSON decoding error messages.
Improvements to the clarity of error messages returned by the /data endpoint when required query parameters are missing or out of range.
Include a tag containing authentication information in the GoAPI documentation.
Fix: Metric search pagination no longer returns an incorrect total count.
When annotations are created or modified, a check will be performed to verify that any provided related metrics actually exist.
Validation has been improved for start and stop time fields for requests placed to the /annotation endpoint.
Fix: Correct a problem where a successful annotation DELETE request would return a 404 error code, even though the delete was successful.
Fix: If an account timezone is set to a non-null but empty string, the API will now correctly return a validation error.

Hooper

Version: 0.1.1586979176 (0c21e73)

Expose configuration for NNTBS timeshard retention. This is only valid for deployments using the "nntbs" backing store for the data_storage role.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.10.2

No changes since 2020-04-06

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1586978979 (abda83c)

(Broker) Remove the C-based ssh2 check module. It was replaced with a Lua version in 2018; this is just cleanup and removal of an external library dependency.
(Broker) Drop outdated and unused modules: googleanalytics, keynote, and keynote_pulse.

Web UI/API

Version: 0.1.1587421536 (3df9624)

Bugfix: API was unable to set prefer_reverse_connection=0 in the PUT provision_broker call.
Bugfix: enterprise broker reachability UI notifications were being missed.

web_stream

Version: 0.1.1566849557 (a3e7e11)

No changes since 2019-09-09

2020-04-06

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200406/x86_64/

Note: This is the final release that will be supported for the EL6 and OmniOS platforms.

caql_broker

Version: 0.1.1585256488 (a56f6fd)

Rework statement initialization to avoid double-registrations of checks at startup.
Emit metadata for CAQL find() results, and consistent ordering for find() results.
Allow dynamic numbers of check outputs.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1585928544 (ef8b888)

Function Removal Notice

Fix null-pointer exception crash on absent metric locator during /find.
Improved performance of metric search indices, reducing initial start time and speeding tag searches where the category has wildcards, e.g., and(version-*:v1.*).
Implement a variety of text-based /fetch transforms for counts and rates.

fault_detection

Version (faultd): 0.1.1584040940 (88ae70a)

No changes since 2020-03-23

Version (ernie): 0.1.1544639627 (db11a23)

Composite Checks Removal Notice

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.7.2

add: A new metric search endpoint has been added to accept metric seaches via POST requests. All search parameters in the query string of a GET metric search request can be included in a JSON object in the body of the POST request.
fix: Cleans up the error message returned when POST or PUT requests to the /acknowledgement endpoint contain JSON errors, or data that does not validate.
fix: Improves validation of account country code values and fixes an issue causing a status code 500 error to be returned if an invalid country code is set using the /account endpoint.
fix: Metric search requests which find no results no longer return a 404 error and instead return an empty result set [].
fix: Unauthorized PUT requests to the /account endpoint no longer return the wrong HTTP status code.
fix: Additional validation takes place on contact group adds/updates for pagerduty, opsgenie, victorops, and slack contact groups. These tests prevent certain errors from occurring in Bert or other messaging services.
fix: Empty results are no longer be returned by /data/fetch requests when the request is for the highcharts data format and includes a reduce operation.
fix: Empty tags values will no longer be returned as null in some objects. All objects will now consistently return an empty array [] when there is no data for the tags field.
fix: Search query parameters for all endpoints will no longer return an error if a numeric query parameter is provided but is an empty value.

Hooper

Version: 0.1.1586206581 (fe1b966) (EL7, OmniOS)

Version: 0.1.1585315278 (c63b6a8) (EL6)

(EL7/OmniOS only) snowth.conf changes:
- Add a rollup-suppression tag search filter to the raw database. Any metrics having a stream tag of __rollup:false will not be rolled up, and will disappear from the raw database as time shards are deleted.
- Turn on accept_thread and fanout for the 8112 listener. The former dedicates a thread to handling new connections, and the latter fans out events from accepted connections across threads in the default eventer pool. These improve performance.
- Add a log of RocksDB activity, level "warn" and higher. This will capture statistics on compaction, among others. It rotates on file size and will not occupy more than 100 MB of total disk space.
- Explicitly note the surrogate database attribues cache_init_size and activity_tracking, using the in-code default values.
Remove extraneous blank lines from the stratcon.conf template. When the application updates its config, these lines are not present, so avoid putting them back when Hooper runs.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.10.2

Fix thread-safey issue using XPath within mtev_conf.
Fix mtev.uname on Solaris/illumos.
Fix missed http/1 request count increment.
Include -lssl in mtev_lua/mtev.so.

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1585237221 (84fbfd8)

Fix xpath for global check parameter overrides.
Accommodate double firing of dns check timeout events (crash fix).
Change config lock type from write to read for cluster filter changes.
(CAQL Broker) Expose check tags to Lua.
Delay check/filterset replication until initialization is complete (crash fix).
Delay setting of global cluster identity until check configuration read is complete.

Web UI/API

Version: 0.1.1585872178 (8f1cba3)

New Feature: API provision_broker support for assert_status and cluster_name.

web_stream

Version: 0.1.1566849557 (a3e7e11)

No changes since 2019-09-09

2020-03-23

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200323/x86_64/

caql_broker

Version: 0.1.1584715301 (92fcf48)

No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1584542652 (8dbe49b)

Function Removal Notice

CAQL: Remove 1y limitation on maximum request span. This limit was added to prevent heavy CAQL queries from overloading the system. There are now limits on the total number of samples fetched, and timeouts on key operations.
CAQL: Improve performance of all binary operators, by defaulting to approximate processing.
[raw ingestor] Add a force_jlog field that will force brokers go through jlog, if configured to.

fault_detection

Version (faultd): 0.1.1584040940 (88ae70a)

No user-facing changes.

Version (ernie): 0.1.1544639627 (db11a23)

Composite Checks Removal Notice

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.6.12

fix: Issue causing some cache misses not to be recorded correctly.
fix: Error causing contact group and metric cluster delete requests to always return not found errors has been fixed.
fix: All endpoints that accept JSON data in the request body will now return more descriptive errors with the correct HTTP code if the JSON is invalid.
fix: Improved validation when saving contact group data via the API. The API will now check and return errors if contact group type and contact methods are mismatched.
fix: Parsing error that occured during metric by ID lookup with metric name containing certain special characters has been corrected.
fix: Results from the /acknowledgement endpoint will always include the notes and other fields even when they contain no data. Now, they will be included with a value of null.
fix: Results from the /alert endpoint will always contain the _cleared_on and _acknowledgement fields. Even if they are empty, they will now still be included in the results with a value of null.
upd: Changes the auth service process used for token based authentication to not use caching by default. Auth token caching can be re-activated with an environment variable setting.
fix: multiple instances where 500 errors were being returned have been fixed to return the correct 4XX errors and messaging.
fix: when using token auth, if either the token or app name header is missing, the error code returned will be 401, previously it was incorrectly returning code 403 if only one of the headers was missing.
fix: all endpoints that were returning 404 errors when performing a search operation that finds no results will now return [] instead of the error.
fix: bug causing some tag searches for acknowledgement values to fail with a non-descriptive 500 error.
fix: issue resulting in pending auth token application statuses being cached and not being correctly updated when the associated application is approved.
fix: issue causing account/current not to report errors correctly when retrieving some types of account data.
add: new share_id parameter supported by the /data/fetch and /caql endpoints. This allows these API endpoints to be used to fetch data for shared dashboards, graphs, and worksheets.
add: new "victorops" contact method supported in the "contact_info" values returned by calls to the /user API endpoint. It contains the same VictorOps ID that is shown in the User Profile of the UI.

Hooper

Version: 0.1.1584478659 (062dd79) (EL7, OmniOS)

Version: 0.1.1584537406 (b252290) (EL6)

Facilitate CAQL debugging.
- Populate the Snowth::Override configurations in circonus.conf, permitting a user to set a UI preference forcing all browser requests to a single IRONdb node.
- Configure the Lua request log on all data_storage nodes. This log is written to /snowth/logs/luarequestlog and will capture details of CAQL requests.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.10.1

Don't truncate SSL errors in logs.
Add mtev.get_ipv4 and mtev.uname lua functions.
Add mtev.reverse_details lua function to monitor reverse connections.
Deprecate mtev_conf_{get,release}_section{,s} and add _read and _write variants to allow concurrent configuration access.
Fix crash when HTTP/2 fails to setup upon connection.
Add cross-thread lua support for serialized waitfor/notify.
Add jlog log configs: segment_size and precommit.

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1584631919 (2fad75d)

(Broker) Decouple XML culling from check deactivation. This should solve issues where removing large numbers of checks performs too much work with the master poller lock held.
Adopt new mtev_conf_* section APIs.
(Broker) Support tagged stats in the broker(selfcheck) module.
(Broker) Make thread pool selection hierarchical and unify logic. We already had check-based thread selection; this expands that to allow for check-based, module-based, and module-name-based logic. The ping_icmp, graphite, and prometheus checks are updated to use this shared logic.
Fix crash in socket listener when encoutering an incomplete buffer.
(Stratcon) Add statistics related to broker jlog feed processing.
(Broker) Add locking for safety in mtev_hash manipulation by socket listeners.
(Broker) Fix crashes and multi-thread listener. The listener and handler shared the same describe function which was incorrect as their closure types are different. The opentsdb and graphite modules both registered (conflicting) names for the same callback -- wrap them so this isn't an issue. Lookup a pool by module name and balance new connections across the threads in that pool.

Web UI/API

Version: 0.1.1584651786 (5992246)

New Feature: Adds the /docs endpoint to GoAPI forwarding to allow public access to the GoAPI documentation. Accessible at https://<api_url_host>/docs, e.g., https://api.circonus.com/docs.
Bugfix: fix bug preventing the addition of metrics to graphs with the default numeric type.
New Feature: Dark Theme for the entire user interface. By default, it will follow your OS or browser preference, but you may set it explicitly under User Preferences.

web_stream

Version: 0.1.1566849557 (a3e7e11)

No changes since 2019-09-09

2020-03-09

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200309/x86_64/

caql_broker

Version: 0.1.1583507886 (31f2298)

Add debugging tools: fqaudit and caql-broker.sh library

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1583366271 (4c6a0a8)

Function Removal Notice

raw ingestor: Add jlog error count to stats so operators can see if a jlog has been corrupted.
raw ingestor: Expose space usage statistics.
CAQL: Improve fill() performance.
CAQL: Improve window:sum() performance.
Fix /fetch histogram transforms.

fault_detection

Version (faultd): 0.1.1582567726 (070c631)

No user-facing changes

Version (ernie): 0.1.1544639627 (db11a23)

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.6.8

No changes since 2020-02-24

Hooper

Version: 0.1.1583173745 (ba69445) (EL7, OmniOS)

Version: 0.1.1583173709 (c34b44d) (EL6)

web_frontend: Update CSS/JS bundles in circonus.conf.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.9.12

Expose counter stats for http[12] requests and responses.
Support managed applications (service restarting of arbitrary sidecars)
Make dropping privileges a noop instead of an error when the the process is already running as the requested user/group.

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1583515571 (006e2fd)

Packaging: the broker's jezebel service is removed, and the jezebel process is now directly spawned/managed by noitd. Going forward, operators need only be concerned with the noitd service.
Extend metric/filterset buffer lengths.
Broker: implement graphite_pickle module.
Broker: fix startup speed for large configs.
- Improve check-to-thread distribution.
- Cache the common mtev.conf_get_string calls in modules.
libnoit: add instrumentation to noit_metric_director.

Web UI/API

Version: 0.1.1583528882 (697a19a)

Bugfix: API would not allow provision_broker with account token.
New Feature: Add UI support for the new Graphite (Pickle) module and unified all Graphite modules under a single display.
New Feature: Histogram datapoints on graphs now have a "value type" field which allows them to be plotted as a rate instead of as a heatmap.
New Feature: Enabled metric filtering by stream tag query for checks using allow/deny metric filters.
Bugfix: Removing parent from rule_set via API with "parent":null was not working.

web_stream

Version: 0.1.1566849557 (a3e7e11)

No changes since 2019-09-09

2020-02-24

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200224/x86_64/

caql_broker

Version: 0.1.1582119106 (b37f6c4)

No user-facing changes.

data_storage

Related roles: caql_broker, fault_detection, stratcon

Version: 0.1.1582269016 (3f76f8b)

Function Removal Notice

CAQL: Add coalesce() function.
(Stratcon) Fix potential crash that can occur when two or more nodes in an IRONdb cluster are down. Issue was caused by concurrent writes to a hash table.
(Stratcon) Fix potential infinite recursion bug that could cause crashes if stratcond is unable to write to any node in the IRONdb cluster for an extended period of time.
Implement rate transform on histograms in the /fetch endpoint.
Make existing stddev and average transforms work for histograms in /fetch.
Make existing mean transform work for numerics in /fetch.

fault_detection

Version (faultd): 0.1.1581526619 (4c52760)

Relocate faultd config files to /opt/circonus/etc/fault/ directory.
Expose /config/ via REST

Version (ernie): 0.1.1544639627 (db11a23)

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

No changes since 2020-02-10

GoAPI

Related roles: api, web_frontend

Version: 0.6.8

Note: GoAPI is now packaged and released according to its internal semantic version.
Return CAQL explain metadata from IRONdb when requested.
Fix: remove deprecated metrics_time_to_live db field from /account endpoint.
Fix: remove references to users.language db field from /user endpoint

Hooper

Version: 0.1.1582310744 (5ba1479) (EL7, OmniOS)

Version: 0.1.1582310744 (04c5fae) (EL6)

Add Fault::hosts config token to circonus.conf for web and script usage.
Configure jlog save directory for stratcon raw ingestor. Will not be used in most Inside deployments.
Relocate faultd configuration files to a subdirectory of /opt/circonus/etc and load additional IRONdb client modules to enable tracking of check-level tags.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.9.11

Pass the siginfo context into the stacktrace for self-diagnosis.
Fix short HTTP/1 payload reads.

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1581719449 (ee1a5d8)

Allow metric-less filters.
Fix tag search evaluation with empty tag values.

Web UI/API

Version: 0.1.1582324336 (d6e68d6)

Improvement: Added "Keyboard Shortcuts" link to user menu.
Bugfix: Fixed the layout of advanced search builder modal inside of the utility modal, at certain window sizes.
Bugfix: Fixed customizable dashboards so they respect the 1200px minimum width set on the page.
Bugfix: When using the advanced query builder to populate a Metric Explorer search query, broker names containing commas now work properly.
Bugfix: Creating a check_bundle would lose metric_filters when creating on more than one broker.
Bugfix: Postgres replica check function gets incorrect ownership on initial Inside installs.
Improvement: Added dashboard state widget.
Bugfix: Minimum height set by dashboard API did not match UI.

web_stream

Version: 0.1.1566849557 (a3e7e11)

No changes since 2019-09-09

2020-02-10

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200210/x86_64/

caql_broker

Version: 0.1.1580893408 (2274d22)

No user-facing changes.

data_storage

Related roles: caql_broker

Version: 0.1.1581090623 (947e2c9)

Function Removal Notice

CAQL: Add integrate:while() function
CAQL: Add forecasting:auto() method
CAQL: Add broadcasting support to forecasting:slope()/:regression()

fault_detection

Version (faultd): 0.1.1580504794 (ad15a94)

Surface an existing statistic, collected, from each broker that faultd has seen metrics from. This metric is sourced from the metrics_collected statistic exposed by the broker's "selfcheck" module. It represents the total metric observations recorded by that broker, prior to filtering.

Version (ernie): 0.1.1544639627 (db11a23)

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.13.0

Automatically load all available modules.
FQ plugin modules are now in a subdirectory of /opt/circonus/libexec.

GoAPI

Related roles: api, web_frontend

Version: 0.1.1579636255 (0b6b018)

No changes since 2020-01-27

Hooper

Version: 0.1.1579804588 (178c6b6) (EL7, OmniOS)

Version: 0.1.1579804588 (0d6e5fe) (EL6)

No changes since 2020-01-27

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.9.10

No changes since 2020-01-27

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1580893408 (d1156f8)

(Broker) Fix SNMP value type coercion.
(Broker) Support 0.0.0.0 as a global target for statsd checks.
Simplify MQ routing and make it hookable.

Web UI/API

Version: 0.1.1581103420 (7b53748)

Bugfix: rule_set_group API "unexpected error" with double-submission or duplicate key conflicts.
Bugfix: PUT to rule_set_group API would always return an error.
Bugfix: fix Postgres database scaffolding order to avoid an SQL error on initial Inside setup.
Bugfix: graph datapoint.alpha property was inconsistently numeric or string-quoted in API output.

web_stream

Version: 0.1.1566849557 (a3e7e11)

No changes since 2019-09-09

2020-01-27

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200127/x86_64/

caql_broker

Version: 0.1.1578942442 (1ff7ebb)

No user-facing changes.

data_storage

Related roles: caql_broker

Version: 0.1.1579718901 (3e406ed)

Function Removal Notice

CAQL: Add histogram:ratio_above() / :ratio_below() functions.
CAQL: Fix a bug where fill() would not fill in missing data.
CAQL: Improve window:merge performance.
CAQL: Fix a bug where optimization rules were falsely applied.
CAQL: Fix a bug where long-running queries could trigger watchdog timeouts.
CAQL: Fix an issue where failed proxy attempts could cause find() errors even when N-W+1 nodes were successfully interrogated.

fault_detection

Version (faultd): 0.1.1578944874 (3e3ea63)

No user-facing changes.

Version (ernie): 0.1.1544639627 (db11a23)

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.12.0

No changes since 2019-08-12

GoAPI

Related roles: api, web_frontend

Version: 0.1.1579636255 (0b6b018)

Fix: Results from IRONdb are not guaranteed to be in the order requested. Ensure that request labels match result labels before returning data to original requestor.
Adds support for activity and latest query parameters for requests to metric search and metric lookup by ID. Also adds the corresponding new fields to the result data from /metric endpoint requests. The data returned in the new activity and latest fields uses the same JSON formats used by IRONdb.

Hooper

Version: 0.1.1579804588 (178c6b6) (EL7, OmniOS)

Version: 0.1.1579804588 (0d6e5fe) (EL6)

(API role) Template a previously hard-coded Apache config file that deals with HTTP request headers for direct browser-to-API requests. These requests are not yet extensively used.
Add DB::admin user parameters to the circonus.conf file. This is currently used only by a script that gets run once during initial Inside deployment.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.9.10

Move eventer SSL debug logging to debug/eventer/ssl.
Fix use-after-free in http logging when HTTP/2 sessions are interrupted.
Make the lua subsystem interrupt as it approaches a watchdog timeout.
Fix compressed uploads in rest_get_raw_upload.
mtev_json_object ref counting is now atomic.

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1578942442 (df23193)

(Broker) Fix short reads of compressed httptrap submissions.

Web UI/API

Version: 0.1.1580074212 (c5ecd76)

Bugfix: display_name property was lost in Cloudwatch check creation via API.
Bugfix: Ruleset Groups in the API could not reference a pattern-based ruleset.

web_stream

Version: 0.1.1566849557 (a3e7e11)

No changes since 2019-09-09

2020-01-13

EL7 pinned repo: baseurl=http://updates.circonus.net/centos/7/release-20200113/x86_64/

caql_broker

Version: 0.1.1573917743 (1ff7ebb)

No changes since 2019-12-16

data_storage

Related roles: caql_broker

Version: 0.1.1578416138 (6f32a35)

Function Removal Notice

Support tag query filter for suppressing rollups from raw database.

fault_detection

Version (faultd): 0.1.1578060087 (6829a12)

Bug: text rule notifications lacked most recent value.
Feature: expose triggering value in a rule stream's trigger history.

Version (ernie): 0.1.1544639627 (db11a23)

No changes since 2018-12-31

FQ

Related roles: caql_broker, fault_detection, mq, stratcon, web_stream

Version: 0.12.0

No changes since 2019-08-12

GoAPI

Related roles: api, web_frontend

Version: 0.1.1574368914 (d2bb13a)

No changes since 2019-12-02

Hooper

Version: 0.1.1578599667 (71b7806) (EL7, OmniOS)

Version: 0.1.1578599667 (fd3a80f) (EL6)

Always try to update PostgreSQL contrib module packages.
Additional configuration checks for data_storage "ncopies" values.

libmtev

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 1.9.8

No changes since 2019-12-16

notification

Version: 0.1.1570571834 (00b2aa2)

No changes since 2019-10-21

Reconnoiter

Related roles: broker, caql_broker, data_storage, fault_detection, stratcon, web_stream

Version: 0.1.1573917743 (d2eacec)

No changes since 2019-12-02

Web UI/API

Version: 0.1.1578699304 (7e9e7fb)

Updated module and schema for PostgresSQL-RabbitMQ communication (pg_amqp).

web_stream

Version: 0.1.1566849557 (a3e7e11)

No changes since 2019-09-09

Release Notes 2020

2020-11-16​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2020-11-02​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2020-10-19​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2020-10-05​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2020-09-21​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2020-09-07​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

libmtev​

notification​

Reconnoiter​

Web UI/API​

web_stream​

2020-08-24​

caql_broker​

data_storage​

fault_detection​

FQ​

GoAPI​

Hooper​

2020-11-16

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2020-11-02

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2020-10-19

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2020-10-05

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2020-09-21

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2020-09-07

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev

notification

Reconnoiter

Web UI/API

web_stream

2020-08-24

caql_broker

data_storage

fault_detection

FQ

GoAPI

Hooper

libmtev