Commit Graph

682 Commits

Author SHA1 Message Date
Adam Ierymenko
b270d527f4
Basic plumbing for authentication requirement and piping through of URL information. 2021-05-24 22:58:17 -04:00
Adam Ierymenko
d64c5a92c6
Merge pull request #1233 from dosuperuser/improvement/optimizations
Minor C++ optimizations
2020-11-24 19:24:36 -05:00
Adam Ierymenko
24769219b5 Merge branch 'dev' 2020-10-08 18:08:24 -04:00
Grant Limberg
c80843e496
Revert "remove redundant writes when changes come from Central"
This reverts commit f9396f979f.
2020-10-05 13:32:47 -07:00
Grant Limberg
f9396f979f
remove redundant writes when changes come from Central
network & member changes tagged with `"fromCentral": true` will not be rewritten to the db
2020-10-05 11:03:03 -07:00
Adam Ierymenko
bb45f9ca3c Upgrade cpp-httplib 2020-09-30 15:21:58 -04:00
Grant Limberg
ff23d3051f
self hosted controller JSON format fix for DNS 2020-09-23 12:16:23 -07:00
Grant Limberg
3db263284b
not sure how this got reverted 2020-09-10 13:18:25 -07:00
Grant Limberg
43c108f077
missed a rollback 2020-08-26 15:50:36 -07:00
Grant Limberg
33cc59914a
Merge branch 'dev-redis' into dev 2020-08-26 14:52:54 -07:00
Grant Limberg
45b6d11126
Put upserts from controller into single transaction 2020-08-26 14:52:23 -07:00
Adam Ierymenko
47e9fb3ddb Merge branch 'dev' of http://git.int.zerotier.com/zerotier/ZeroTierOne into dev 2020-08-25 08:07:23 -07:00
Grant Limberg
e7891bfcb0
Merge branch 'dev-redis' into dev 2020-08-21 16:29:13 -07:00
Grant Limberg
7f99c4a779
Sleep 10 seconds between writes to DB 2020-08-21 16:22:28 -07:00
Adam Ierymenko
06730c7d1d BSL date bump 2020-08-20 12:51:39 -07:00
Grant Limberg
c0c215c83c
single dns config per network 2020-08-12 13:08:47 -07:00
Grant Limberg
058d888311
More debug logging cleanup 2020-08-12 10:00:09 -07:00
Grant Limberg
6868e98904 Get stuff building on Windows
Been a minute since any of this has been updated
2020-08-05 16:04:02 -07:00
Grant Limberg
d2708daa8e
debug output 2020-08-04 14:33:18 -07:00
Grant Limberg
88a3c685fb
latest 2020-08-04 13:52:57 -07:00
Grant Limberg
2e52a1eebf
forgot a couple queries in postgres. trying to pull dns of member not network in embedded network controller
also some debug logging
2020-08-04 09:45:45 -07:00
Grant Limberg
5b700fa497
println for which notification stream the controller is listening to 2020-07-27 18:37:45 -07:00
Grant Limberg
251b06d812
revert redis for member status 2020-07-23 09:38:50 -07:00
Grant Limberg
387039456d
Pass 1 at adding DNS to controller 2020-07-20 14:34:19 -07:00
Suad Halilovic
ffebcd247f 2020/07/16, Minor optmizations 2020-07-16 18:31:56 +02:00
Grant Limberg
d0aacfddb7
one more spot 2020-06-04 14:12:20 -07:00
Grant Limberg
9b92ad7772
handle case of no networks and/or no members 2020-06-04 14:06:30 -07:00
Grant Limberg
135a547889
No longer update both pgsql & redis 2020-05-28 19:22:07 -07:00
Grant Limberg
06de25a680 bump online notification pause to 100ms
also put all online notification redis commands into a single tx
2020-05-27 22:25:07 -07:00
Grant Limberg
5692402d32 A little more cleanup 2020-05-27 21:02:30 -07:00
Grant Limberg
ad7ae5a372 list of all network-node pairs 2020-05-27 20:43:31 -07:00
Grant Limberg
9794e31a64 Temporary online notification thread hack
Updates both Redis and Postgres
2020-05-27 20:43:31 -07:00
Grant Limberg
2f0f0e4f53 redis init optimization 2020-05-27 20:43:31 -07:00
Grant Limberg
7bde004c7c Replace scan of Redis with iteration of _networks map 2020-05-27 20:41:47 -07:00
Grant Limberg
39da360725
add online controller list in Redis 2020-05-22 14:16:04 -07:00
Grant Limberg
beedee4401
fix typo 2020-05-22 11:07:12 -07:00
Grant Limberg
fb0e8aebdb
keep list of active networks in redis 2020-05-22 10:07:39 -07:00
Grant Limberg
13929aee6f
reduce log chattiness 2020-05-21 09:49:55 -07:00
Grant Limberg
c2409ad6c9
fix connection to redis cluster in online notification thread 2020-05-21 09:49:41 -07:00
Grant Limberg
08cb72bdba
Temp object was being destroyed before connection was used 2020-05-21 09:33:03 -07:00
Grant Limberg
d24c8d858c
include climits for Linux 2020-05-20 16:54:18 -07:00
Grant Limberg
879ef58565
Finalize Redis integration 2020-05-20 16:28:28 -07:00
Grant Limberg
8b8399efbc
Error recovery for network & member stream watchers 2020-05-20 11:42:51 -07:00
Grant Limberg
0f17508cac
error recovery in redis online notification
If a redis cluster member fails over to the slave, we'll get an error from not specifying the key for the insert.  Recover from that instead of crashing the controller
2020-05-20 11:38:04 -07:00
Grant Limberg
7a138f963c
TIL: Creating a redis transaction without sending any commands throws an exception
So let's not do that
2020-05-19 17:47:06 -07:00
Grant Limberg
b637ced1ee
Track member status in Redis 2020-05-19 17:47:02 -07:00
Grant Limberg
8ac42b4600
controller status query fix 2020-05-19 17:40:11 -07:00
Grant Limberg
15c0c1db39
finish the RabbitMQ-ectomy 2020-05-13 09:46:41 -07:00
Grant Limberg
aab96964b6
Put debug output behind ZT_TRACE 2020-05-12 12:48:58 -07:00
Grant Limberg
c6518afa7a
Make sure the streams clean up after themselves 2020-05-12 12:37:05 -07:00
Grant Limberg
563655a1a4
Redis now usable as a message queue 2020-05-12 11:56:19 -07:00
Grant Limberg
d699116795
mac deps 2020-05-11 16:48:14 -07:00
Grant Limberg
154470b570
add original hiredis 2020-05-11 15:03:56 -07:00
Grant Limberg
73b1d57b13
rip out rabbitMQ 2020-05-11 12:29:06 -07:00
Grant Limberg
acb4ef0f12
add hiredis-vip to controller build 2020-05-11 11:48:05 -07:00
Adam Ierymenko
f12c75e68b Merge branch 'master' of github.com:zerotier/ZeroTierOne 2020-03-25 13:40:13 -07:00
Grant Limberg
0d42108254
member ip inserts DO NOTHING on conflict 2020-03-04 09:45:45 -08:00
Grant Limberg
f6026f94a5
More fun with dupes 2020-03-03 23:52:53 -08:00
Grant Limberg
e702942041
Fix duplicate assigned IP check when adding assigned IPs to database 2020-03-03 13:34:24 -08:00
Grant Limberg
12a92fc4a1
Un-log & some production build image fixes 2020-02-24 11:02:05 -08:00
Grant Limberg
baa14a63c9
It's LOG! It's LOG!
It's big, it's heavy, it's wood
2020-02-24 10:20:47 -08:00
Travis LaDuke
e744c95c5b Add Managed Routes example to controller readme. 2019-11-15 10:29:39 -08:00
Grant Limberg
86abd31659
fix log line 2019-11-13 12:46:16 -08:00
Grant Limberg
088dab4f04
Let's not print this to the logs 2019-10-31 15:13:20 -07:00
Grant Limberg
cc79ec6179
use std::string instead of const char* in rmq struct 2019-10-31 15:08:59 -07:00
Adam Ierymenko
52a166a71f
Relicense: GPLv3 -> ZeroTier BSL 1.1 2019-08-23 09:23:39 -07:00
wendelb
fe18579061
Removed duplicate line creationTime 2019-08-17 22:50:13 +02:00
Adam Ierymenko
5341c216a7
Comment ugly query 2019-08-08 19:20:01 -05:00
Adam Ierymenko
96a0c3d764
ugly query is ugly 2019-08-08 19:16:11 -05:00
Adam Ierymenko
783b49e383
more worky less not worky 2019-08-08 18:58:22 -05:00
Adam Ierymenko
d4dc983a01
fix bad column name 2019-08-08 18:31:41 -05:00
Adam Ierymenko
72ccd1020a
Another exception fix 2019-08-08 18:11:06 -05:00
Adam Ierymenko
6a9a577c59
rulesSource needs to be required in network or pgsql has issues 2019-08-08 17:56:27 -05:00
Adam Ierymenko
730305ecc7
One more sync fix. 2019-08-08 17:31:32 -05:00
Adam Ierymenko
f8f2432ece
Shutdown fix 2019-08-08 17:22:20 -05:00
Adam Ierymenko
55087521f6
Add periodic sync check when syncing LF<>another DB 2019-08-08 17:20:50 -05:00
Adam Ierymenko
760ae07d72
Sync bug fix 2019-08-08 16:30:52 -05:00
Adam Ierymenko
28d0070ce2
Fix race in multiple DB mirroring configurations. 2019-08-08 15:29:13 -05:00
Adam Ierymenko
7bb0d9be1b
warning removal 2019-08-08 11:04:11 -05:00
Adam Ierymenko
af7a70bf02
cleanup 2019-08-07 21:11:48 -05:00
Adam Ierymenko
6073e7972f
Merge branch 'dev' of http://git.int.zerotier.com/zerotier/ZeroTierOne into dev 2019-08-07 11:24:04 -05:00
Adam Ierymenko
f7e8d8f1dd
Fix LFDB path issue when using with Postgress in DBMirrorSet config 2019-08-07 11:23:48 -05:00
Grant Limberg
87016ae815 network update to upsert 2019-08-06 15:30:50 -07:00
Adam Ierymenko
3db0197865
Add try/catch in LFDB and add introspection into multicast subscriptions in client network status API 2019-08-06 15:22:57 -05:00
Adam Ierymenko
6174c0c103
Fix deadlock bug. 2019-08-06 13:20:37 -05:00
Adam Ierymenko
8898cb342b
. 2019-08-06 13:12:45 -05:00
Adam Ierymenko
9cc76d1115
Remove debug line 2019-08-06 11:01:08 -05:00
Adam Ierymenko
5289d3327c
Build fixes 2019-08-06 11:00:35 -05:00
Adam Ierymenko
00fb9c475e
More work on DB mirroring. 2019-08-06 10:42:54 -05:00
Adam Ierymenko
3c776675b3
Cruftectomy, work in progress on mirrorring 2019-08-06 08:51:23 -05:00
Adam Ierymenko
37d508ab96
Refactoring in prep for mirroring 2019-08-06 07:51:50 -05:00
Adam Ierymenko
f4f8fef82e
LFDB fix 2019-08-05 15:30:03 -05:00
Adam Ierymenko
a77b4ecddb Add optional function in DB change listener for member online events. 2019-08-04 19:55:52 -07:00
Grant Limberg
21467596bc Merge branch 'remove_network_status_table' into dev 2019-08-04 12:38:12 -07:00
Adam Ierymenko
bb99ef1351 Get rid of online status stuff in FileDB, not necessary at all. 2019-07-31 16:41:12 -07:00
Adam Ierymenko
eb7081d2a3 Fix for detecting changes in LF from other instances of the same controller. 2019-07-26 19:28:09 -07:00
Adam Ierymenko
d9befe3fb3 Debug code removal 2019-07-26 18:51:50 -07:00
Adam Ierymenko
0618452615 LF masking key should be a secret 2019-07-26 17:44:05 -07:00
Adam Ierymenko
f6b080b8a2 Abstract out change listener from controller itself to permit DBs to shadow changes from other DBs. 2019-07-26 17:39:00 -07:00
Adam Ierymenko
c8c33db1d1 . 2019-07-26 16:03:21 -07:00
Adam Ierymenko
ac2688de58 More LFDB work 2019-07-23 16:06:35 -07:00
Adam Ierymenko
5f11daadf2 Build fix 2019-07-23 09:30:40 -07:00
Adam Ierymenko
d4d350a285 Work in progress 2019-07-23 09:29:08 -07:00
Adam Ierymenko
5edd04638d LFDB work in progress 2019-07-22 13:43:06 -07:00
Grant Limberg
861a319014 Merge branch 'dev' of github.com:zerotier/ZeroTierOne into dev 2019-07-16 15:20:05 -07:00
Grant Limberg
7afcc17024 no usleep() on Windows 2019-07-16 13:09:10 -07:00
Grant Limberg
2ab2f68725 Add a central database schema version check to central controller startup 2019-07-16 12:15:38 -07:00
Adam Ierymenko
90872c906f Fix FileDB stuff and latency measurement. 2019-07-15 16:30:34 -05:00
Adam Ierymenko
bdc2ec1f5e Track online status in FileDB 2019-07-15 15:27:53 -05:00
Grant Limberg
692733fffb Don't write to ztc_network_status table anymore in central controller 2019-07-15 13:00:16 -07:00
Adam Ierymenko
700cc92083 Indent fix 2019-06-11 11:01:12 -07:00
Adam Ierymenko
848f822171 . 2019-06-06 10:04:11 -07:00
Grant Limberg
37d321afc8 dont spin as much in the onlineNotificationThread 2019-05-31 15:10:45 -07:00
Adam Ierymenko
cacdb445c7 Bump protocol version to 10 2019-05-14 13:54:27 -07:00
Grant Limberg
6a027c9c0a amqp_consume_message now has a timeout
RabbitMQ::consume() will return an empty string if the call to amqp_consume_message times out
2019-04-18 14:57:06 -07:00
Grant Limberg
6014df2847 fix compile & sql errors 2019-04-04 15:11:01 -07:00
Grant Limberg
55a9e6e05e Queue up inserts in onlineNotificationThread() into a multiple insert statement 2019-04-04 12:40:49 -07:00
Adam Ierymenko
2966eac64f Get rid of ZT_FAST_MEMCPY because this is not really needed. 2019-03-22 15:50:15 -07:00
Grant Limberg
e3cc7c9a80 fix non-central controller builds 2019-03-11 11:16:44 -07:00
Grant Limberg
993d850f69 more logs 2019-03-08 10:50:33 -08:00
Grant Limberg
cda07b20a2 add mutex to channel numbering 2019-03-08 10:29:36 -08:00
Grant Limberg
171cc3e01a reorder a couple of things 2019-03-08 10:20:19 -08:00
Grant Limberg
8141043560 finish up rabbitmq integration 2019-03-06 16:16:49 -08:00
Grant Limberg
fcb4221f97 rabbitMQ implementation 2019-03-05 15:11:50 -08:00
Grant Limberg
1f13374a4f added struct for rabbitmq config 2019-03-04 17:01:16 -08:00
Grant Limberg
801401a6e7 Merge branch 'dev' of http://git.int.zerotier.com/ZeroTier/ZeroTierOne into dev 2019-03-04 16:31:36 -08:00
Grant Limberg
de1fc43cc5 added librabbitmq library for central controller 2019-03-04 16:27:39 -08:00
Grant Limberg
a788486803 Remove /controller/network/<network ID>/active from controller documentation as that path no longer exists.
GitHub Issue #921
2019-02-25 12:13:55 -08:00
Adam Ierymenko
5b1ef2fb03 Take netmask bits from most specific route. 2019-02-01 14:05:29 -08:00
Adam Ierymenko
34c1023ba7 Wrong criteria was wrong. Now with less wrong. 2019-02-01 11:37:30 -08:00
Grant Limberg
7414418e2e insert/update host_port on controller start 2019-01-21 11:29:13 -08:00
Grant Limberg
2294770e56 Merge branch 'dev' of http://git.int.zerotier.com/zerotier/ZeroTierOne into dev 2019-01-21 11:18:25 -08:00
Grant Limberg
d98bdb5643 WIP: Pass listen port down to Postgres 2019-01-21 11:18:20 -08:00
Joseph Henry
0e597191b8 Updated licenses for 2019 2019-01-14 10:25:53 -08:00
Grant Limberg
0da2efa633 add application_name to pgbouncer connectio string 2018-12-10 12:26:20 -08:00
Grant Limberg
56a58f2b11 more logging of exits from the central controller 2018-12-06 13:26:53 -08:00
Grant Limberg
e959908c51 log message 2018-12-06 13:19:36 -08:00
Grant Limberg
6e3144370a exit controller if onlinenotification thread exits while _run == 1 2018-12-06 13:10:59 -08:00
Grant Limberg
1c86f25fab update # of threads for Postgres 2018-12-06 13:08:31 -08:00
Grant Limberg
328d08781a finish pgbouncer integration 2018-12-03 16:12:15 -08:00
Grant Limberg
c8d5215870 add handling for PGBOUNCER_CONNSTR environment variable.
alows Central controllers to connect to PGBouncer on some threads.  LISTEN threads still require a direct connection to the DB
2018-12-03 15:19:15 -08:00
Grant Limberg
b75f212d70 Replace RethinkDB section in controller/README with PostgreSQL 2018-11-30 10:41:54 -08:00
Grant Limberg
ccb856749f back to plain old libpq in dev 2018-11-30 10:40:57 -08:00
Grant Limberg
e3381a4c1f fix member init query 2018-11-13 16:19:55 -08:00
Grant Limberg
882b03436d WIP: libpxx instead of libpq 2018-11-13 16:00:13 -08:00
Grant Limberg
2729db9555 exceptions 2018-11-11 22:55:53 -08:00
Adam Ierymenko
7048c77ebc Merge branch 'dev' of http://git.int.zerotier.com/zerotier/ZeroTierOne into dev 2018-11-11 22:35:26 -08:00
Adam Ierymenko
c9c17eaddd Retire RethinkDB, simple receive path multithreading. 2018-11-11 22:35:15 -08:00
Grant Limberg
4c0e7aa8b0 more error checking 2018-11-11 21:08:51 -08:00
Grant Limberg
761cc5c923 remove another noisy log line 2018-10-24 13:36:07 -07:00
Grant Limberg
d87fcca176 Fully operational pgsql controller 2018-10-24 12:06:17 -07:00
Grant Limberg
1da1409db2 get rid of a noisy log line 2018-09-28 10:58:26 -07:00
Grant Limberg
c1ea1329c1 Merge branch 'dev' into libpq 2018-09-28 10:56:40 -07:00
Grant Limberg
417aa9547b Working controller & docker container for GKE 2018-09-28 10:55:39 -07:00
Adam Ierymenko
7a42416a78
Merge pull request #822 from dschulz/controller-patch1-dschulz
Add functionality to erase members from networks using file backend i…
2018-09-25 11:38:25 -07:00
Grant Limberg
5b222d0a09 cleanup connections after errors 2018-09-25 10:17:15 -07:00
Grant Limberg
0ec6215493 Send member/network change notifications via Postgres 2018-09-06 15:14:16 -07:00
Grant Limberg
de45bdb448 quieting things down a little bit 2018-09-05 17:03:49 -07:00
Grant Limberg
7b608c6ca3 Fix testing for true/false. Dont attempt to update member status on networks that we know dont exist 2018-09-05 16:54:23 -07:00
Grant Limberg
6ff78c9c26 fix crashes 2018-09-05 16:08:06 -07:00
Grant Limberg
d701f4e331 Added some exception hadling. Dont want to live *too* dangerously here 2018-09-05 11:49:07 -07:00
Grant Limberg
92abc92463 Theoretically we have PostgreSQL support now 2018-09-05 11:30:17 -07:00
Grant Limberg
a4be9ceb5a dont insert/update creation time. Let the database handle it with the default value 2018-09-04 16:08:26 -07:00
Grant Limberg
e21b6e1daf commit thread WIP 2018-09-04 16:05:34 -07:00
Grant Limberg
2e3b03fff9 members initialized 2018-09-04 14:51:35 -07:00
Grant Limberg
cd657da431 Online Update/Notify thread 2018-09-04 14:00:02 -07:00
Grant Limberg
17af09a5cc Postgres Integration has begun 2018-08-31 14:58:15 -07:00
Diego Schulz
fc6dba0797
Add functionality to erase members from networks using file backend in controller microservice
Signed-off-by: Diego Schulz <dschulz@gmail.com>
2018-08-02 17:13:55 -04:00
Adam Ierymenko
f94aea8119 Return error 503 if RethinkDB is down when built as RethinkDB-based controller. 2018-07-11 10:42:31 -07:00
Adam Ierymenko
37ae3b2b80 Remote debug printf. 2018-07-03 12:52:35 -07:00
Adam Ierymenko
73e4286fbf Fix two controller bugs: filesystem bug and another possible infinite recursion bug. 2018-07-03 12:51:41 -07:00
Grant Limberg
9463d4abe4 Fix for issue #778
Double quote before member name in `/controller/network/network_id/member` API
2018-06-05 12:55:39 -07:00
Adam Ierymenko
bbdb2aa672 Some work on IPv4 enabled ad-hoc networks. 2018-03-08 23:53:57 -08:00
Adam Ierymenko
574b24c082 docs 2018-03-08 22:41:42 -08:00
Adam Ierymenko
874a187c7f cleanup 2018-03-08 22:39:51 -08:00
Adam Ierymenko
e8fb443313 Forgot one other use in RethinkDb driver. 2018-03-08 22:34:51 -08:00
Adam Ierymenko
57b96af2c4 Fix for FileDB _networkChanged / _memberChanged problem. 2018-03-08 22:33:08 -08:00
Adam Ierymenko
f7019d9e80 Tie up the rest of hub and spoke designated replicator multicast mode. 2018-01-26 22:14:10 -05:00
Adam Ierymenko
6d8e1e8783 More cleanup of old stuff. 2018-01-26 21:34:56 -05:00
Adam Ierymenko
4e689998f9 Sanity checks on array sizes and fix a bug in IPv4 auto-assign. 2018-01-12 10:38:19 -08:00
Adam Ierymenko
cd2a4b709c Reduce maximum batcn size for status updates but increase the number of whole record commit threads. 2018-01-12 09:18:27 -08:00
Adam Ierymenko
f17cc1c6d8 cleanup 2018-01-10 15:03:39 -08:00
Adam Ierymenko
a1992d76f2 Doc updates and other fixes. 2018-01-10 14:31:28 -08:00
Adam Ierymenko
9bab49d2f9 Move DB stuff into Db. 2018-01-09 12:39:25 -08:00
Adam Ierymenko
0608704626 Some micro-optimizations, including a memcpy that is faster than Linux for most distro/compiler versions. 2017-12-15 11:03:20 -08:00
Adam Ierymenko
4704d115c4 Some controller fixes. 2017-12-14 08:53:24 -08:00
Adam Ierymenko
e4a10cc510 Member commit bug fix. 2017-12-08 19:00:38 -05:00
Adam Ierymenko
38cc89b2ca Add time to rethinkdb message output in controller. 2017-12-08 12:39:20 -05:00
Adam Ierymenko
aeb1794be1 Comment out storage of secret ID in RethinkDb. 2017-12-07 16:04:41 -08:00
Adam Ierymenko
dae728124e Fix to network path in conventional filesystem controller DB. 2017-12-07 14:42:33 -08:00
Adam Ierymenko
19bfef36f1 Build fix. 2017-12-07 16:45:43 -05:00
Adam Ierymenko
8d9464c414 docs, and make RethinkDB controller DB driver upsert into the Controller DB and also update the hostname field. 2017-12-07 13:39:25 -08:00
Adam Ierymenko
520b0601db Merge branch 'dev' of http://10.6.6.2/zerotier/ZeroTierOne into dev 2017-12-05 14:51:06 -08:00
Adam Ierymenko
926ecf9640 docs 2017-12-05 14:50:59 -08:00
Adam Ierymenko
c57198decd Add remoteTraceLevel to controller. 2017-12-05 17:49:20 -05:00
Adam Ierymenko
431716e249 Disable trace by default, stderr message improvements in RethinkDB driver. 2017-12-05 17:16:45 -05:00
Adam Ierymenko
694709f392 Add timestamp to network status. 2017-12-04 19:21:56 -05:00
Adam Ierymenko
0fb22df633 Get ephemeral status fields out of the configs. They do not belong there and it just complicates things. 2017-11-09 17:01:16 -05:00
Adam Ierymenko
f7f658605d Move more ephemeral stuff to a tiny MemberLastRequest table instead of the main Member table. 2017-11-08 20:19:46 -05:00
Adam Ierymenko
192e5dbff8 Another fix for uptime logging. 2017-11-08 20:02:10 -05:00