Commit Graph

404 Commits

Author SHA1 Message Date
Adam Ierymenko
5dad73647d Lengthen backup period again 2016-02-04 14:22:54 -08:00
Adam Ierymenko
13b39a0c3e SQLite perf tuning 2016-02-04 14:03:37 -08:00
Adam Ierymenko
90801a94d3 Track client version and tell whether active nodes support circuit test. 2016-02-04 13:38:42 -08:00
Adam Ierymenko
fab6f4450d /active subpath off networks 2016-02-04 12:17:55 -08:00
Adam Ierymenko
2e04dc03f2 Logging to NodeHistory, SQL queries. 2016-02-03 18:10:56 -08:00
Adam Ierymenko
f8eb6b0067 Add NodeHistory table on sqlite controller. 2016-02-03 13:56:35 -08:00
Adam Ierymenko
9cb4bbe2b8 Save test results for circuit tests in memory and then cancel the test and send the results when the test is queried later. This way you can POST a test and then come GET the result at the appointed time. 2016-01-26 12:42:44 -08:00
Ren Jie
21656ba015 Update controller README.md
Sync make parameter with code.
2016-01-12 22:51:08 +08:00
Adam Ierymenko
436c1fac1d Selectively move over changes from "edge" to "dev" excluding netcon. 2015-12-21 16:15:39 -08:00
Adam Ierymenko
523412edfb Abort backup in progress if thread is told to shut down. 2015-11-03 16:03:00 -08:00
Adam Ierymenko
f7a407ffa0 Tweak timings and use lock in backup to make it a bit faster and still permit main thread to work. 2015-11-03 15:56:24 -08:00
Adam Ierymenko
7903f24a8f Create periodic backup copies of controller.db in network controller from the main process itself to facilitate easier and safer backups of controller.db. 2015-11-03 15:52:10 -08:00
Adam Ierymenko
eff1fe3c61 Create files for each hop (more convenient) and fix a packet parse bug. 2015-10-09 16:22:34 -07:00
Adam Ierymenko
7d01fab132 Reorg fields to be in same order as FS scheme. 2015-10-09 15:18:01 -07:00
Adam Ierymenko
aec13b50fd Be a bit more verbose in circuit test reports to more clearly track current and upstream hop in graph traversal history. 2015-10-09 15:05:26 -07:00
Adam Ierymenko
a95fa379cc Circuit tests basically work but need some tweaks, and fix some issues found with valgrind. 2015-10-09 14:51:38 -07:00
Adam Ierymenko
6b5bb0b278 Eliminate format string warnings. 2015-10-09 12:22:13 -07:00
Adam Ierymenko
59da8b2a4b Logging of circuit test results to disk. 2015-10-08 15:44:06 -07:00
Adam Ierymenko
a3876353ca Abiltiy to post a test via the controller web API, and parsing of CIRCUIT_TEST_REPORT messages. 2015-10-08 13:25:38 -07:00
Adam Ierymenko
7394ec6f6a Prep in controller code to run tests. 2015-10-06 15:56:18 -07:00
Adam Ierymenko
a7bd1eaa40 Never assign v4 IPs ending in .255 even within range. 2015-09-28 15:28:30 -07:00
Adam Ierymenko
ddf3d1f949 Controller side support for IPv6 assignment. 2015-09-18 13:35:00 -07:00
Adam Ierymenko
610ab0750c Drop Sqlite-based Log table for now and switch to an in-memory log for recent activity. Log table gets too big on busy nodes. Should probably support push of events to some kind of event system later. 2015-09-15 10:59:23 -07:00
Adam Ierymenko
ef316ced3b Fix JSON. 2015-09-14 11:59:43 -07:00
Adam Ierymenko
cd005341c5 Extra statement to clean up Members -- cascade did not seem to work, possibly due to dual key. 2015-09-11 15:02:26 -07:00
Adam Ierymenko
a35fa7ac93 Add expansion of netconf in _test field. 2015-09-10 15:14:10 -07:00
Adam Ierymenko
bebe3d7cfa Fix deadlock in test mode. 2015-09-10 14:47:04 -07:00
Adam Ierymenko
1f7a41cff8 Fix to allowing identity to be populated if not present. 2015-09-10 14:37:34 -07:00
Adam Ierymenko
4fbcad2468 Allow identity to be populated for newly inserted Member objects to permit transfer from old network controller and testing. 2015-09-08 13:02:42 -07:00
Adam Ierymenko
0d386f1c31 Add a bit of useful testing instrumentation to SqliteNetworkController. 2015-09-08 11:35:55 -07:00
Adam Ierymenko
2aa1b5d9b7 Add clock helper field to both member and network to permit time duration calculation easily. 2015-08-24 12:44:07 -07:00
Adam Ierymenko
9a5be0a092 typo 2015-08-24 11:24:33 -07:00
Adam Ierymenko
4da794b389 Add authorizedMemberCount to controller network config records. 2015-08-19 11:43:56 -07:00
Adam Ierymenko
0a5429cab0 Lookup of member must be a left outer join in case the member is being manually inserted before we see the node. 2015-08-17 21:08:02 +00:00
Adam Ierymenko
fcc5bf1e66 Go ahead and spec out controller DB support for AuthToken -- GitHub issue #211 -- even though full implementation won't make it into 1.0.4. 2015-07-29 15:09:23 -07:00
Adam Ierymenko
d57ea671d7 Add version to log. 2015-07-24 09:59:17 -07:00
Adam Ierymenko
d647a587a1 (1) Fix updating of network revision counter on member change.
(2) Go back to timestamp as certificate revision number. This is simpler
    and more robust than using the network revision number for this and
    forcing network revision fast-forward, which could cause some peers
    to fall off the horizon when you don't want them to.
2015-07-23 17:18:20 -07:00
Adam Ierymenko
b3516c599b Add a rate limiting circuit breaker to the network controller to prevent flooding attacks and race conditions. 2015-07-23 10:10:17 -07:00
Adam Ierymenko
3ba54c7e35 Eliminate some poorly thought out optimizations from the netconf/controller interaction,
and go ahead and bump version to 1.0.4.

For a while in 1.0.3 -dev I was trying to optimize out repeated network controller
requests by using a ratcheting mechanism. If the client received a network config
that was indeed different from the one it had, it would respond by instantlly
requesting it again.

Not sure what I was thinking. It's fundamentally unsafe to respond to a message
with another message of the same type -- it risks a race condition. In this case
that's exactly what could happen.

It just isn't worth the added complexity to avoid a tiny, tiny amount of network
overhead, so I've taken this whole path out.

A few extra bytes every two minutes isn't worth fretting about, but as I recall
the reason for this optimization was to save CPU on the controller. This can be
achieved by just caching responses in memory *there* and serving those same
responses back out if they haven't changed.

I think I developed that 'ratcheting' stuff before I went full time on this. It's
hard to develop stuff like this without hours of sustained focus.
2015-07-23 09:50:10 -07:00
Adam Ierymenko
e2a2993b18 Add a Log table to log queries for debugging and security logging. No JSON API support for querying the log yet, but will probably come via /network/###/member/###/log/... or something. 2015-07-22 14:01:49 -07:00
Kees Bos
53c7f61f98 Fix for output of empty (no members) network 2015-07-05 13:27:27 +02:00
Adam Ierymenko
7c761dea72 Fix to member listing: I wanted an object with member IDs as keys and member revisions as values, not an array. 2015-07-21 14:12:22 -07:00
Adam Ierymenko
3f8a5b8b76 List members in the form of a hash of member ID and member revision so code can quickly detect which members have changed. 2015-07-21 13:38:59 -07:00
Adam Ierymenko
a061aa3d87 Remove "members" from Network record and instead enumerate members via specific query to /network/nwid/member sub-path. More RESTful, scalable, and compatible with how OnePoint code works. 2015-07-21 12:57:01 -07:00
Adam Ierymenko
b343eac10d Fix IP auto-assign bug due to missing subnet routes. 2015-07-21 12:42:43 -07:00
Adam Ierymenko
649a12472b Report controllerInstanceId in all objects so that controller resets can be easily detected by whatever is using the service. 2015-07-21 10:39:29 -07:00
Adam Ierymenko
cac6be87ba Fix bug in rules JSON output. 2015-07-20 16:31:37 -07:00
Adam Ierymenko
38d34a7495 Proper handling of NULL entry for etherType in rules table. 2015-07-20 15:11:53 -07:00
Adam Ierymenko
fb4c3dd8d4 Fix string overwrite bug. 2015-07-20 14:31:33 -07:00
Adam Ierymenko
1ffd67e014 Get rid of false foreign key in Relay. 2015-07-20 14:28:30 -07:00
Adam Ierymenko
bca8886ff8 IP assignment pool range bug fix. 2015-07-17 15:09:28 -07:00
Adam Ierymenko
1f7bb67069 Fix some SQL and make instanceId more robustly random. 2015-07-17 13:09:53 -07:00
Adam Ierymenko
712e2785f2 Fix bad JSON in response. 2015-07-17 12:24:42 -07:00
Adam Ierymenko
5515909c1e Add a concept of an "instanceId" to the controller, which the OnePoint can use to determine whether it is the same running database instance it already knows. 2015-07-17 10:47:21 -07:00
Adam Ierymenko
0db7c94c90 Add memberRevision stuff to JSON output, and update docs. 2015-07-16 17:42:47 -07:00
Adam Ierymenko
99969b186b Add a concept of a member revision counter to networks. This can be used to select all members that have been added or changed since a given point. 2015-07-16 17:34:03 -07:00
Adam Ierymenko
f9f7de0ec7 Networks don't need their ID as a default name. 2015-07-14 15:54:56 -07:00
Adam Ierymenko
d27c14af48 Don't allow zero as a network number. 2015-07-14 12:32:57 -07:00
Adam Ierymenko
30e4a188d0 ipLocalRoutes now exposed via network objects in JSON controller API, and documentation changes. 2015-06-29 15:34:26 -07:00
Adam Ierymenko
5c9411a671 Untested -- modifications to support IP ranges instead of ip/mask for IP assignment pools, also add portId to Rule for future use. 2015-06-29 14:52:09 -07:00
Adam Ierymenko
48a2ad032a (1) Both nodeId and portId in Rule can be NULL, (2) remove on delete cascade since rules should never mysteriously disappear from the rules table. If it let you delete a node with rules, that would be a UI or cleanup function bug. 2015-06-29 10:47:47 -07:00
Adam Ierymenko
f05e62deae DB schema changes: separate portId in rules, ranges in IP assignment pools. (No code changes yet so code is broken.) 2015-06-29 10:40:31 -07:00
Adam Ierymenko
dbee1b38b3 Fix semantics of std::unique() to actually remove duplicates (hidden memory leak?) 2015-06-29 10:21:28 -07:00
Kees Bos
3eca5d9c29 Fix reporting of ipAssignments for ipv4 2015-06-26 07:22:30 +02:00
Adam Ierymenko
57c7992c78 GitHub issue #191 - kill intra-network multicast rate limits (which were not well supported or easily configurable anyway) -- this is really left over from the old collaborative multicast propagation algorithm. New algorithm (in for a while) has been sender-side replication in which sender "pays" all bandwidth, which intrinsically limits multicast. 2015-06-26 12:36:45 -07:00
Kees Bos
50d4f66d73 Fixed member authorization bug and minor cleanup 2015-06-19 21:19:42 +02:00
Kees Bos
16eae132fa Fix for ipv4 assignment 2015-06-18 19:14:52 +02:00
Kees Bos
4affa10ca0 Fix 404 on creation of new network 2015-06-15 10:29:12 +02:00
Kees Bos
2e1d363a86 Removed a superfluous cross join 2015-06-15 03:19:25 +02:00
Kees Bos
1cbdae65fe Fix controller/network/*/member/*
Cross join works other than expected or something changed. The
_sGetMember2 returned too many rows. Replaced it with an explicit
join statement.
2015-06-15 03:19:25 +02:00
Kees Bos
dcbae5f313 Bugfix controller get member info 2015-06-15 03:19:25 +02:00
Adam Ierymenko
96a58becf8 Gateways support in network controller schema and database (not implemented yet in client) toward GitHub issue #178 2015-06-13 11:34:31 +02:00
Adam Ierymenko
8a9715f183 Rename ruleId to ruleNo and optimize some indexes in Sqlite3 schema. 2015-06-13 10:05:34 +02:00
Kees Bos
7a55c6b388 Return 404 on delete if network member is missing (controller) 2015-06-11 12:20:52 +02:00
Kees Bos
c2ce018202 Return 404 on delete if network doesn't exist (controller) 2015-06-11 12:10:25 +02:00
Kees Bos
0d0af07ce9 Get deletion of networks in controller going
Multiple statements in a sqlite3_prepare_v2 is not usable. Only
the first statement will be executed.

Since the schema now uses 'ON DELETE CASCADE', there's only
one statement needed.

If multiple statements are needed, there should be either multiple
sqlite3_prepare_v2 calls be used or the sqlite3_exec function.
2015-06-11 11:49:13 +02:00
Kees Bos
de697a1c45 Change schema to enforce foreing keys
The foreign keys have 'ON DELETE CASCADE' to simplify the removal
of networks etc. (controller code)

Some unique constraints are replaced with a multi column primary
key.

To update an existing database:
 * install updated binaries
 * stop service
 * sqlite3 controller.db .dump | \
    egrep '((^PRAGMA)|(^BEGIN)|(^INSERT)|(^COMMIT))' | \
    grep -v 'schemaVersion' > data.sql
 * mv controller.db controller.db.backup
 * start service
 * stop service
 * sqlite3 controller.db < data.sql
 * start service
2015-06-11 11:35:25 +02:00
Adam Ierymenko
d8ad555b9a Go ahead and add flags and invFlags to the Rule table. 2015-05-25 13:20:10 -07:00
Adam Ierymenko
d41b6eb0c8 docs 2015-05-17 10:14:12 -07:00
Adam Ierymenko
651e67f2e5 Add a feature to generate a new network ID on POST. 2015-05-17 09:36:35 -07:00
Adam Ierymenko
69ceb7e730 Basic controller JSON API seems to be working. 2015-05-16 17:12:29 -07:00
Adam Ierymenko
cf51961d52 . 2015-05-16 16:32:13 -07:00
Adam Ierymenko
c9fd8de007 . 2015-05-16 16:22:38 -07:00
Adam Ierymenko
a187d290f1 Fixes to control plane, API, eliminate problematic inheritance pattern, and start on a NodeJS class for talking to the network controller. 2015-05-16 16:09:28 -07:00
Adam Ierymenko
4be4908914 Fix some prepared statement problems. 2015-05-16 14:34:51 -07:00
Adam Ierymenko
0bb92715f4 DELETE function in network controller JSON API, and a newIdentity convenience request in ControlPlane for scripted testing. 2015-05-16 13:42:53 -07:00
Adam Ierymenko
78769900a9 More network controller cleanup, and some features to permit scripted testing. 2015-05-16 12:50:42 -07:00
Adam Ierymenko
65a9a9a6f2 typo 2015-05-15 15:30:44 -07:00
Adam Ierymenko
e269846f84 Netconf docs, add clock field to status, simplify netconf a bit by eliminating caching for now. We will re-add if it is needed. 2015-05-15 15:20:12 -07:00
Adam Ierymenko
6d2376eb9c Controller API status message. 2015-05-15 09:41:45 -07:00
Adam Ierymenko
f693d4d0c8 Network controller cleanup and an extra sanity check. 2015-05-15 09:32:10 -07:00
Adam Ierymenko
883a216d2a Build fixes. 2015-04-24 12:29:31 -07:00
Adam Ierymenko
5202fbdaf3 CRUD 2015-04-22 18:06:26 -07:00
Adam Ierymenko
103dcb072d CRUD 2015-04-21 19:49:04 -07:00
Adam Ierymenko
79f63ba30a Fix: make sure we do not assign broadcast address as an IP to new members. 2015-04-21 18:37:17 -07:00
Adam Ierymenko
71f006cbeb More CRUD, almost done... 2015-04-21 18:08:33 -07:00
Adam Ierymenko
e4046964f0 Forgot to run schema2c. 2015-04-21 16:50:02 -07:00
Adam Ierymenko
ddebe2d4c7 Network controller CRUD... :P 2015-04-21 16:41:35 -07:00
Adam Ierymenko
ed107c4daf Network preferred relay stuff in netconf controller. 2015-04-20 17:47:12 -07:00
Adam Ierymenko
69076f8a45 Add per-network relay stuff to sqlite table schema. 2015-04-20 15:46:20 -07:00
Adam Ierymenko
740121504f Add a timestamp to netconf cache, fix some SQL queries in NC. 2015-04-17 15:21:53 -07:00
Adam Ierymenko
91ca238163 Compile fixes. 2015-04-15 18:47:38 -07:00
Adam Ierymenko
ea1859541c More cleanup, and fix for the extremely unlikely case of identity collision. 2015-04-15 18:32:25 -07:00
Adam Ierymenko
6369c264e2 Rename netconf to controller and NetworkConfigMaster to NetworkController for consistency. 2015-04-15 15:12:09 -07:00