Commit Graph

766 Commits

Author SHA1 Message Date
Orne Brocaar
f283261636 api: Remove Python proto folder.
There is no point in tracking this folder in git, as on each make we
will copy it from the original api/proto folder.
2024-04-04 14:30:03 +01:00
Orne Brocaar
ac9923e120 api: Add duty-cycle stats messages + types. 2024-04-04 14:28:24 +01:00
Orne Brocaar
2bb05fba58 Update dependencies. 2024-04-02 11:40:47 +01:00
Orne Brocaar
161cd9008c Fix multicast class-b ping-slot migration in case value is 0. 2024-04-02 09:25:58 +01:00
Orne Brocaar
c238e8f6da api: Use protoc-gen-go & -go-grpc from nix. 2024-04-01 14:37:41 +01:00
Orne Brocaar
d170c7dd79 api: Remove generated API files from repo + update build.
All these files can be generated using the `make api` command and there
is no real need to commit these into the repo. Only the api/go files
need to be comitted of how the Go import system works.

This also updates the Rust, Go, JS and gRPC-web (JS) code generation and
UI build to use the nix-shell environment instead of using Docker.
2024-04-01 14:27:15 +01:00
SAGAR PATEL
a91f5ff1d3
ui: Fix drawer header z-index issue in UI. (#393)
The z-index of the header was set to 1000 and the drawer header is behind the UI header so single frame download and title not displaying

Setting the z-index to 1002 is enough to fix the Drawer header issue.
2024-04-01 12:20:10 +01:00
dependabot[bot]
cd179dd16f
Bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /ui (#388)
Bumps [webpack-dev-middleware](https://github.com/webpack/webpack-dev-middleware) from 5.3.3 to 5.3.4.
- [Release notes](https://github.com/webpack/webpack-dev-middleware/releases)
- [Changelog](https://github.com/webpack/webpack-dev-middleware/blob/v5.3.4/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack-dev-middleware/compare/v5.3.3...v5.3.4)

---
updated-dependencies:
- dependency-name: webpack-dev-middleware
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-27 14:13:31 +00:00
dependabot[bot]
59271f6966
Bump async-trait from 0.1.78 to 0.1.79 (#389)
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.78 to 0.1.79.
- [Release notes](https://github.com/dtolnay/async-trait/releases)
- [Commits](https://github.com/dtolnay/async-trait/compare/0.1.78...0.1.79)

---
updated-dependencies:
- dependency-name: async-trait
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-27 14:13:18 +00:00
dependabot[bot]
23e0613970
Bump express from 4.18.2 to 4.19.2 in /ui (#392)
Bumps [express](https://github.com/expressjs/express) from 4.18.2 to 4.19.2.
- [Release notes](https://github.com/expressjs/express/releases)
- [Changelog](https://github.com/expressjs/express/blob/master/History.md)
- [Commits](https://github.com/expressjs/express/compare/4.18.2...4.19.2)

---
updated-dependencies:
- dependency-name: express
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-27 14:10:24 +00:00
Orne Brocaar
a66f085782 Use PKCS#8 fallback for all private keys reads.
This fallback was already implemented for reading the private key when
generating MQTT client-certificates, but it was not used when reading a
private key for connecting to the MQTT broker.

While it is recommended to use PKCS#8 private-keys, this will perform an
auto conversion for RSA PKCS#1 and EC P256 SEC1 private-keys to PKCS#8.
2024-03-27 10:25:22 +00:00
Orne Brocaar
2c06edd6ff Add auto conversion for SEC1 EC keys to PKCS#8.
Fixes #386.
2024-03-27 09:50:18 +00:00
Orne Brocaar
286d8b33b2 Generate API code. 2024-03-26 16:47:55 +00:00
Orne Brocaar
c71b856c78 Align multicast class-b ping-slot config.
This aligns the multicast class-b ping-slot configuration with the way
how it is configured in the device-profile. This deprecates the
class_b_ping_slot_period field in favor of the class_b_ping_slot_nb_k
field, which should be a value between 0 - 7 (this is defined and
explained by the LoRaWAN specification).

Closes #255.
2024-03-26 16:45:24 +00:00
Orne Brocaar
e96e828d3f Document that keys need to be in PKCS#8.
Closes #386.
2024-03-26 12:47:20 +00:00
Orne Brocaar
e3752fb19d Expose more verbose JS codec error output.
Closes #391.
2024-03-26 11:06:43 +00:00
Orne Brocaar
f970e94cca Show version in UI (for admin users).
Closes #73 and #158.
2024-03-26 09:01:21 +00:00
Orne Brocaar
54b10cb693 ui: Run code formatter. 2024-03-19 14:14:18 +00:00
Orne Brocaar
8cde912fdf Show error when navigator.clipboard is not available.
There can be multiple reasons for this, e.g. it is not supported by the
browser or the browser does not provide this feature because there is no
secure context (HTTP).

Closes #231.
2024-03-19 14:12:03 +00:00
Orne Brocaar
5a865c924c Fix setting gateway after 0cd5dd8d31. 2024-03-19 14:00:47 +00:00
Orne Brocaar
56336fa685 Update dependencies. 2024-03-19 12:41:18 +00:00
Orne Brocaar
c426e48b8d Update dependencies + fix check / clippy feedback. 2024-03-19 12:15:48 +00:00
Orne Brocaar
9de0354f13 api: Update dependencies. 2024-03-19 10:57:29 +00:00
dependabot[bot]
7e0c18cf00
Bump google.golang.org/protobuf from 1.31.0 to 1.33.0 in /api/go (#379)
Bumps google.golang.org/protobuf from 1.31.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 10:39:55 +00:00
dependabot[bot]
b02f2a21d2
Bump google.golang.org/protobuf in /examples/frame_log/go (#380)
Bumps google.golang.org/protobuf from 1.31.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 10:39:46 +00:00
dependabot[bot]
1505abf215
Bump google.golang.org/protobuf in /examples/event_log/go (#381)
Bumps google.golang.org/protobuf from 1.31.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 10:39:36 +00:00
dependabot[bot]
81d70fba14
Bump google.golang.org/protobuf in /examples/request_log/go (#382)
Bumps google.golang.org/protobuf from 1.31.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 10:39:26 +00:00
dependabot[bot]
9dad578d89
Bump google.golang.org/protobuf in /examples/meta_log/go (#383)
Bumps google.golang.org/protobuf from 1.31.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 10:39:18 +00:00
dependabot[bot]
a254f70fe1
Bump follow-redirects from 1.15.4 to 1.15.6 in /ui (#384)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.4 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.4...v1.15.6)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 10:38:57 +00:00
Orne Brocaar
cef454920e Document missing recvTime codec option.
Closes #385.
2024-03-19 10:38:08 +00:00
Orne Brocaar
a78bc226f6 Update redis crate.
This version solves inconsistent pipeline output types when using
multiplexed async connections.
2024-03-19 10:38:08 +00:00
Orne Brocaar
06bcca147c Update tonic. 2024-03-19 10:38:08 +00:00
Orne Brocaar
26cef0b2ca Update rustls / tls functions. 2024-03-19 10:38:08 +00:00
Simon Arnell
b118ab56be
Fix typo in error string. (#367) 2024-03-14 14:53:38 +00:00
Orne Brocaar
0cd5dd8d31 Update updating gw loc + tls cert.
This refactors the multiple update functions into a single function
accepting a change-set struct. This also adds a sanity check to make
sure we are not updating the gw loc when lat / lon / alt are all set to
0.
2024-03-14 12:20:11 +00:00
Orne Brocaar
8c875c3bf1 Bump version to 4.7.0 2024-03-13 10:44:21 +00:00
Orne Brocaar
c815c6be88 Update Rust + dev env. 2024-03-13 09:44:29 +00:00
Aidar
5f72db234f
Updated docker-compose to docker compose (#377) 2024-03-12 11:27:27 +00:00
Orne Brocaar
34ca1c2d6b Hide change password on oAuth2 login. 2024-03-05 10:06:08 +00:00
Orne Brocaar
519f53940e Bump version to 4.7.0-test.4 2024-02-27 16:48:40 +00:00
Orne Brocaar
f816ca469b Update generated api code. 2024-02-27 16:34:31 +00:00
Orne Brocaar
098f8db4c6 Migrate device-sessions from Redis to PostgreSQL.
This migrates the device-sessions from Redis into PostgreSQL. This fixes
a performance issue in case the same DevAddr is reused many times
(e.g. devices rejoining very often or a NetID with small DevAddr space).

There were two issues:

The Redis key containing the DevAddr -> DevEUIs mapping could contain
DevEUIs that no longer used the DevAddr. This mapping would only expire
from the Redis database after none of the devices would use the DevAddr
for more than the configured device_session_ttl.

The other issue with the previous approach was that on for example a
Type 7 NetID, a single DevAddr could be re-used multiple times. As each
device-session could be stored on a different Redis Cluster instance,
there was no option to retrieve all device-sessions at once. Thus a high
re-usage of a single DevAddr would cause an increase in Redis queries.

Both issues are solved by moving the device-session into PostgreSQL
as the DevAddr is a column of the device record and thus filtering on
this DevAddr would always result in the devices using that DevAddr, as
well all device-sessions for a DevAddr can be retrieved by a single
query.

Note that to migrate the device-sessions, you must run:

chirpstack -c path/to/config migrate-device-sessions-to-postgres

A nice side-effect is that a PostgreSQL backup / restore will also
restore the device connectivity.

Closes #362 and #74.
2024-02-27 16:17:15 +00:00
Orne Brocaar
fc30a3b7e5 Add OIDC CustomClaims struct.
This is needed when custom claims are returned that are not used by
ChirpStack, but might be used during user registration.
2024-02-22 13:38:29 +00:00
Orne Brocaar
d599e7a276 Add js context to get_activation api method. 2024-02-06 13:16:26 +00:00
Orne Brocaar
8cde64c4b7 Save metrics in one pipelined Redis query.
This saves the metrics in one pipelined Redis query, rather than one
query per aggregation level. This saves 2 queries per save.
2024-01-31 12:11:54 +00:00
Orne Brocaar
b65faf7b98 Merge deduplication_put and _locked queries.
This merges the deduplication_put and deduplication_locked Redis queries
into a single pipelined query. Simulating 400 uplinks / seconds through
3 gateways (thus 1200 messages / second), and an artificial TCP latency
of 5ms between ChirpStack <> Redis, this reduces the
storage_redis_conn_get_duration_seconds_sum metric by about 50%.

What we are trying to solve here is that under high (simulated) load, we
exhaust the Redis connection pool. This situation gets worse when there
is a significant latency between ChirpStack <> Redis, because the query
takes longer to complete and is reserved from the pool for a longer
time. The result can be that during the de-duplication process, the key
containing the uplink set has already expired before the
deduplicate_collect function is able to get a Redis connection from the
pool.

This query merge is a quick win, because each uplink can be received by
N gateways, thus this merge saves N Redis queries per uplink.
2024-01-31 11:06:03 +00:00
Orne Brocaar
9de86ffdec Track pg and redis get conn metrics.
This data is useful when debugging connection-pool size related issues.
E.g. when more connections are requested than the connection pool allows
then this will result in a longer wait time.
2024-01-31 09:54:00 +00:00
Orne Brocaar
107fbfa7bb Fix setting dev_addr field on ABP activate. 2024-01-25 11:23:33 +00:00
dependabot[bot]
4ce886b21d
Bump unsafe-libyaml from 0.2.9 to 0.2.10 (#341)
Bumps [unsafe-libyaml](https://github.com/dtolnay/unsafe-libyaml) from 0.2.9 to 0.2.10.
- [Release notes](https://github.com/dtolnay/unsafe-libyaml/releases)
- [Commits](https://github.com/dtolnay/unsafe-libyaml/compare/0.2.9...0.2.10)

---
updated-dependencies:
- dependency-name: unsafe-libyaml
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-12 15:55:03 +00:00
dependabot[bot]
a1c3431719
Bump follow-redirects from 1.15.3 to 1.15.4 in /ui (#353)
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.3 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.3...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-12 15:54:33 +00:00