This merges the deduplication_put and deduplication_locked Redis queries
into a single pipelined query. Simulating 400 uplinks / seconds through
3 gateways (thus 1200 messages / second), and an artificial TCP latency
of 5ms between ChirpStack <> Redis, this reduces the
storage_redis_conn_get_duration_seconds_sum metric by about 50%.
What we are trying to solve here is that under high (simulated) load, we
exhaust the Redis connection pool. This situation gets worse when there
is a significant latency between ChirpStack <> Redis, because the query
takes longer to complete and is reserved from the pool for a longer
time. The result can be that during the de-duplication process, the key
containing the uplink set has already expired before the
deduplicate_collect function is able to get a Redis connection from the
pool.
This query merge is a quick win, because each uplink can be received by
N gateways, thus this merge saves N Redis queries per uplink.
This data is useful when debugging connection-pool size related issues.
E.g. when more connections are requested than the connection pool allows
then this will result in a longer wait time.
Same as the previous commit, this will returns the PG connection back to
the pool after usage, avoiding the risk that we are holding the conn
longer than needed.
Before the Redis connection would be returned once variable c went out
of scope, in most cases at the return of the function. This would mean
that during the execution of the remaining code within the function,
the Redis connection would be reserved.
With this change, the Redis connection is immediately returned to the
pool after usage.
This refactors the last bit of code that depends on the OpenSSL C
library. Note that the openssl-probe does not depend on OpenSSL, it only
tries to detect the CA certificate directory on the host system.
This code still used channel.blocking_send(..) and sleep from std::time.
As async anonymous functions are not (yet) supported, this moves the
stream handling into a separate async function.
This still depends on unreleased diesel and diesel-async code. As soon
as new diesel and diesel-async code has been released, we can remove
the [patch.crates-io] from Cargo.toml.
This flag helps to migrate from v3 to v4 and adds a compatibility layer
to support ChirpStack Gateway Bridge v3.14+. The plan was to default
this to false in an earlier release, but this change never happened.
This was changed from error to warn, but a debug is better as there is
not really something to warn the user about and with many devices
this might become very noisy.
In case a Relay would send an uplink with adr_ack_req=true, this would
not result in an empty downlink in case there is no downlink to relay
back to the relay end-device.
Initially this was added to make development using Docker Compose easier.
However, in my experience Docker Compose is not great for the reason
that there might be environmental differences between Rust running in
Compose and Rust running outside Compose (if using VSCode with Rust
plugin, then rust-analyzer will perform checks in the background). This
and 'cross' not working properly in Compose was the reason to migrate
to nix, as it provides a reproducible environment for development that
can also be used by VSCode (using the Nix environment selector
extension) and GitHub Actions (for the CI pipeline).
As the chirpstack-docker repository provides its own configuration files,
these files are not included in the Dockerfile and because we are
rewriting these env variables anyway to 'localhost' in the .deb and .rpm
post-installation script, it is better to set these to localhost by
default.
The post-installation rewrite to 'localhost' is causing issues on
upgrade (#295). If we only do the rewrite on initial installation, then
we run in an other issue; the package-manager will prompt that the
config file has changed (from 'localhost' to '$MQTT_BROKER_HOST' for
example) and will ask if you would like to overwrite or not. If the
end-user would accept the config changes without looking at the diff
this would break the installation as most likely, these environment are
not set (which is why we were rewriting these to 'localhost' in the first
place).
