Merge dev to edge

2025-02-20 17:52:46 +00:00 · 2019-08-06 14:13:07 -05:00 · 2019-08-06 14:13:07 -05:00 · f9900cc6fb
commit f9900cc6fb
parent 7e105343e2 6174c0c103
50 changed files with 4563 additions and 624 deletions
--- a/.github/ISSUE_TEMPLATE/bug_report.md
+++ b/.github/ISSUE_TEMPLATE/bug_report.md
@ -6,7 +6,7 @@ about: Create a report to help us improve
 **Alternative, faster ways to get help**
 If you have just started using ZeroTier, here are some places to get help:
 - my.zerotier.com has a _Community_ tab. It's a live chat with other users and the developers. 
- [ZeroTier Knowledge Base](https://support.zerotier.com/knowledgebase.php?s=MQ__)
+- [ZeroTier Knowledge Base](https://zerotier.atlassian.net/wiki/spaces/SD/overview)
 - www.zerotier.com has a Contact Us button
 - email contact@zerotier.com

--- a/AUTHORS.md
+++ b/AUTHORS.md
@ -65,3 +65,9 @@ ZeroTier includes the following third party code, either in ext/ or incorporated
   * Files: ext/libnatpmp/* ext/miniupnpc/*
   * Home page: http://miniupnp.free.fr/
   * License grant: BSD attribution no-endorsement
+
+ * cpp-httplib by yhirose
+
+   * Files: ext/cpp-httplib/*
+   * Home page: https://github.com/yhirose/cpp-httplib
+   * License grant: MIT
--- a/2
+++ b/2
@ -1,5 +1,5 @@
 ZeroTier One, an endpoint server for the ZeroTier virtual network layer.
-Copyright © 2011–2018 ZeroTier, Inc.
+Copyright © 2011–2019 ZeroTier, Inc.

 ZeroTier One is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
--- a/LICENSE.txt
+++ b/LICENSE.txt
@ -1,5 +1,5 @@
 ZeroTier One - Network Virtualization Everywhere
-Copyright (C) 2011-2017  ZeroTier, Inc.  https://www.zerotier.com/
+Copyright (C) 2011-2019  ZeroTier, Inc.  https://www.zerotier.com/

 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
--- a/OFFICIAL-RELEASE-STEPS.md
+++ b/OFFICIAL-RELEASE-STEPS.md
@ -13,7 +13,6 @@ The version must be incremented in all of the following files:
    /zerotier-one.spec
    /debian/changelog
    /ext/installfiles/mac/ZeroTier One.pkgproj
-    /ext/installfiles/windows/chocolatey/zerotier-one.nuspec
    /ext/installfiles/windows/ZeroTier One.aip
    /windows/WinUI/AboutView.xaml

@ -29,21 +28,6 @@ Mac's easy. Just type:

 You will need [Packages](http://s.sudre.free.fr/Software/Packages/about.html) and our release signing key in the keychain.

-## Linux
-
-See `LinuxBuild` environment on `linux-build` VM and use: `chroots/mount-build.sh`, `chroots/build.sh`, and the scripts in `build/` to make APT and RPM repositories.
-
 ## Windows

 First load the Visual Studio solution and rebuild the UI and ZeroTier One in both x64 and i386 `Release` mode. Then load [Advanced Installer Enterprise](http://www.advancedinstaller.com/), check that the version is correct, and build. The build will fail if any build artifacts are missing, and Windows must have our product singing key (from DigiCert) available to sign the resulting MSI file. The MSI must then be tested on at least a few different CLEAN Windows VMs to ensure that the installer is valid and properly signed.
-
-*After the MSI is published to download.zerotier.com in the proper RELEASE/#.#.#/dist subfolder for its version* the Chocolatey package must be rebuilt and published. Open a command prompt, change to `ext/installfiles/windows/chocolatey`, and type `choco pack`. Then use `choco push` to push it to Chocolatey (API key required).
-
-    choco pack
-    choco push zerotier-one.#.#.#.nupkg -s https://chocolatey.org/
-
-Note that this does not cover rebuilding the drivers or their containing MSI projects, as this is typically not necessary and they are shipped in binary form in the repository for convenience.
-
-## iOS, Android
-
-... no docs here yet since this is done entirely out of band with regular installs.
--- a/README.md
+++ b/README.md
@ -1,62 +1,57 @@
-ZeroTier - A Planetary Ethernet Switch
+ZeroTier - Global Area Networking
 ======

-ZeroTier is a smart programmable Ethernet switch for planet Earth.
+ZeroTier is a smart programmable Ethernet switch for planet Earth. It allows networked devices and applications to be managed as if the entire world is one data center or cloud region.

 It replaces the physical LAN/WAN boundary with a virtual one, allowing devices of any type at any location to be managed as if they all reside in the same cloud region or data center. All traffic is encrypted end-to-end and takes the most direct path available for minimum latency and maximum performance. The goals and design of ZeroTier are inspired by among other things the original [Google BeyondCorp](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43231.pdf) paper and the [Jericho Forum](https://en.wikipedia.org/wiki/Jericho_Forum).

-Visit [ZeroTier's site](https://www.zerotier.com/?pk_campaign=github_ZeroTierOne) for more information and [pre-built binary packages](https://www.zerotier.com/download.shtml?pk_campaign=github_ZeroTierOne). Apps for Android and iOS are available for free in the Google Play and Apple app stores.
+Visit [ZeroTier's site](https://www.zerotier.com/) for more information and [pre-built binary packages](https://www.zerotier.com/download/). Apps for Android and iOS are available for free in the Google Play and Apple app stores.

 ### Getting Started

 Everything in the ZeroTier world is controlled by two types of identifier: 40-bit/10-digit *ZeroTier addresses* and 64-bit/16-digit *network IDs*. A ZeroTier address identifies a node or "device" (laptop, phone, server, VM, app, etc.) while a network ID identifies a virtual Ethernet network that can be joined by devices.

-Another way of thinking about it is that ZeroTier addresses are port numbers on a giant planetary-sized smart switch while network IDs are VLANs to which these ports can be assigned. For more details read about VL1 and VL2 in [the ZeroTier manual](https://www.zerotier.com/manual.shtml).
+Another way of thinking about it is that ZeroTier addresses are port numbers on a giant planetary-sized smart switch while network IDs are VLANs to which these ports can be assigned. For more details read about VL1 and VL2 in [the ZeroTier manual](https://www.zerotier.com/manual/).

 *Network controllers* are ZeroTier nodes that act as access control certificate authorities and configuration managers for virtual networks. The first 40 bits (or 10 digits) of a network ID is the ZeroTier address of its controller. You can create networks with our [hosted controllers](https://my.zerotier.com/) and web UI/API or [host your own](controller/) if you don't mind posting some JSON configuration info or writing a script to do so.

 ### Project Layout

+The base path contains the ZeroTier One service main entry point (`one.cpp`), self test code, makefiles, etc.
+
 - `artwork/`: icons, logos, etc.
 - `attic/`: old stuff and experimental code that we want to keep around for reference.
 - `controller/`: the reference network controller implementation, which is built and included by default on desktop and server build targets.
 - `debian/`: files for building Debian packages on Linux.
 - `doc/`: manual pages and other documentation.
+ - `docker/`: Dockerfile to build as a container for containerized Linux systems and Kubernetes clusters.
 - `ext/`: third party libraries, binaries that we ship for convenience on some platforms (Mac and Windows), and installation support files.
 - `include/`: include files for the ZeroTier core.
 - `java/`: a JNI wrapper used with our Android mobile app. (The whole Android app is not open source but may be made so in the future.)
 - `macui/`: a Macintosh menu-bar app for controlling ZeroTier One, written in Objective C.
 - `node/`: the ZeroTier virtual Ethernet switch core, which is designed to be entirely separate from the rest of the code and able to be built as a stand-alone OS-independent library. Note to developers: do not use C++11 features in here, since we want this to build on old embedded platforms that lack C++11 support. C++11 can be used elsewhere.
 - `osdep/`: code to support and integrate with OSes, including platform-specific stuff only built for certain targets.
+ - `rule-compiler/`: JavaScript rules language compiler for defining network-level rules.
 - `service/`: the ZeroTier One service, which wraps the ZeroTier core and provides VPN-like connectivity to virtual networks for desktops, laptops, servers, VMs, and containers.
- - `tcp-proxy/`: TCP proxy code run by ZeroTier, Inc. to provide TCP fallback (this will die soon!).
 - `windows/`: Visual Studio solution files, Windows service code for ZeroTier One, and the Windows task bar app UI.

-The base path contains the ZeroTier One service main entry point (`one.cpp`), self test code, makefiles, etc.
-
 ### Build and Platform Notes

 To build on Mac and Linux just type `make`. On FreeBSD and OpenBSD `gmake` (GNU make) is required and can be installed from packages or ports. For Windows there is a Visual Studio solution in `windows/'.

 - **Mac**
-   - Xcode command line tools for OSX 10.7 or newer are required.
-   - Tap device driver kext source is in `ext/tap-mac` and a signed pre-built binary can be found in `ext/bin/tap-mac`. You should not need to build it yourself. It's a fork of [tuntaposx](http://tuntaposx.sourceforge.net) with device names changed to `zt#`, support for a larger MTU, and tun functionality removed.
+   - Xcode command line tools for OSX 10.8 or newer are required.
 - **Linux**
-   - The minimum compiler versions required are GCC/G++ 4.9.3 or CLANG/CLANG++ 3.4.2.
+   - The minimum compiler versions required are GCC/G++ 4.9.3 or CLANG/CLANG++ 3.4.2. (Install `clang` on CentOS 7 as G++ is too old.)
   - Linux makefiles automatically detect and prefer clang/clang++ if present as it produces smaller and slightly faster binaries in most cases. You can override by supplying CC and CXX variables on the make command line.
-   - CentOS 7 ships with a version of GCC/G++ that is too old, but a new enough version of CLANG can be found in the *epel* repositories. Type `yum install epel-release` and then `yum install clang` to build there.
 - **Windows**
   - Windows 7 or newer is supported. This *may* work on Vista but isn't officially supported there. It will not work on Windows XP.
-   - We build with Visual Studio 2015. Older versions may not work. Clang or MinGW will also probably work but may require some makefile hacking.
-   - Pre-built signed Windows drivers are included in `ext/bin/tap-windows-ndis6`. The MSI files found there will install them on 32-bit and 64-bit systems. We don't recommend trying to build Windows drivers from scratch unless you know what you're doing. One does not simply "build" a Windows driver.
+   - We build with Visual Studio 2017. Older versions may not work. Clang or MinGW will also probably work but may require some makefile hacking.
 - **FreeBSD**
-   - Tested most recently on FreeBSD-11. Older versions may work but we're not sure.
-   - GCC/G++ 4.9 and gmake are required. These can be installed from packages or ports. Type `gmake` to build.
+   - GNU make is required. Type `gmake` to build.
 - **OpenBSD**
-   - There is a limit of four network memberships on OpenBSD as there are only four tap devices (`/dev/tap0` through `/dev/tap3`). We're not sure if this can be increased.
-   - OpenBSD lacks `getifmaddrs` (or any equivalent method) to get interface multicast memberships. As a result multicast will only work on OpenBSD for ARP and NDP (IP/MAC lookup) and not for other purposes.
-   - Only tested on OpenBSD 6.0. Older versions may not work.
-   - GCC/G++ 4.9 and gmake are required and can be installed using `pkg_add` or from ports. They get installed in `/usr/local/bin` as `egcc` and `eg++` and our makefile is pre-configured to use them on OpenBSD.
+   - There is a limit of four network memberships on OpenBSD as there are only four tap devices (`/dev/tap0` through `/dev/tap3`).
+   - GNU make is required. Type `gmake` to build.

 Typing `make selftest` will build a *zerotier-selftest* binary which unit tests various internals and reports on a few aspects of the build environment. It's a good idea to try this on novel platforms or architectures.

--- a/RELEASE-NOTES.md
+++ b/RELEASE-NOTES.md
@ -1,16 +1,23 @@
 ZeroTier Release Notes
 ======

-# 2019-??-?? -- Version 1.4.0 (currently in prerelease as 1.2.99)
+# 2019-07-29 -- Version 1.4.0

+### Major Changes
+
+ * Mac version no longer requires a kernel extension, instead making use of the [feth interfaces](https://apple.stackexchange.com/questions/337715/fake-ethernet-interfaces-feth-if-fake-anyone-ever-seen-this).
 * Added support for concurrent multipath (multiple paths at once) with traffic weighting by link quality and faster recovery from lost links.
 * Added under-the-hood support for QoS (not yet exposed) that will eventually be configurable via our rules engine.
+
+### Minor Changes and Bug Fixes
+
+ * Experimental controller DB driver for [LF](https://github.com/zerotier/lf) to store network controller data (LFDB.cpp / LFDB.hpp).
 * Modified credential push and direct path push timings and algorithms to somewhat reduce "chattiness" of the protocol when idle. More radical background overhead reductions will have to wait for the 2.x line.
- * Mac version no longer requires a kernel extension, instead making use of the [feth interfaces](https://apple.stackexchange.com/questions/337715/fake-ethernet-interfaces-feth-if-fake-anyone-ever-seen-this).
 * Removed our beta/half-baked integration of Central with the Windows UI. We're going to do a whole new UI of some kind in the future at least for Windows and Mac.
 * Fixed stack overflow issues on Linux versions using musl libc.
 * Fixed some alignment problems reported on ARM and ARM64, but some reports we could not reproduce so please report any issues with exact chip, OS/distro, and ZeroTier version in use.
- * Fixed numerous other small issues and bugs.
+ * Fixed numerous other small issues and bugs such as ARM alignment issues causing crashes on some devices.
+ * Windows now sets the adapter name such that it is consistent in both the Windows UI and command line utilities.

 # 2018-07-27 -- Version 1.2.12

--- a/controller/DB.cpp
+++ b/controller/DB.cpp
@ -104,20 +104,8 @@ void DB::cleanMember(nlohmann::json &member)
 	member.erase("lastRequestMetaData");
 }

-DB::DB(EmbeddedNetworkController *const nc,const Identity &myId,const char *path) :
-	_controller(nc),
-	_myId(myId),
-	_myAddress(myId.address()),
-	_path((path) ? path : "")
-{
-	char tmp[32];
-	_myAddress.toString(tmp);
-	_myAddressStr = tmp;
-}
-
-DB::~DB()
-{
-}
+DB::DB() {}
+DB::~DB() {}

 bool DB::get(const uint64_t networkId,nlohmann::json &network)
 {
@ -202,34 +190,15 @@ bool DB::get(const uint64_t networkId,nlohmann::json &network,std::vector<nlohma
 	return true;
 }

-bool DB::summary(const uint64_t networkId,NetworkSummaryInfo &info)
-{
-	waitForReady();
-	std::shared_ptr<_Network> nw;
-	{
-		std::lock_guard<std::mutex> l(_networks_l);
-		auto nwi = _networks.find(networkId);
-		if (nwi == _networks.end())
-			return false;
-		nw = nwi->second;
-	}
-	{
-		std::lock_guard<std::mutex> l2(nw->lock);
-		_fillSummaryInfo(nw,info);
-	}
-	return true;
-}
-
-void DB::networks(std::vector<uint64_t> &networks)
+void DB::networks(std::set<uint64_t> &networks)
 {
 	waitForReady();
 	std::lock_guard<std::mutex> l(_networks_l);
-	networks.reserve(_networks.size() + 1);
 	for(auto n=_networks.begin();n!=_networks.end();++n)
-		networks.push_back(n->first);
+		networks.insert(n->first);
 }

-void DB::_memberChanged(nlohmann::json &old,nlohmann::json &memberConfig,bool push)
+void DB::_memberChanged(nlohmann::json &old,nlohmann::json &memberConfig,bool notifyListeners)
 {
 	uint64_t memberId = 0;
 	uint64_t networkId = 0;
@ -313,8 +282,12 @@ void DB::_memberChanged(nlohmann::json &old,nlohmann::json &memberConfig,bool pu
 			}
 		}

-		if (push)
-			_controller->onNetworkMemberUpdate(networkId,memberId);
+		if (notifyListeners) {
+			std::lock_guard<std::mutex> ll(_changeListeners_l);
+			for(auto i=_changeListeners.begin();i!=_changeListeners.end();++i) {
+				(*i)->onNetworkMemberUpdate(this,networkId,memberId,memberConfig);
+			}
+		}
 	} else if (memberId) {
 		if (nw) {
 			std::lock_guard<std::mutex> l(nw->lock);
@ -332,20 +305,24 @@ void DB::_memberChanged(nlohmann::json &old,nlohmann::json &memberConfig,bool pu
 		}
 	}

-	if ((push)&&((wasAuth)&&(!isAuth)&&(networkId)&&(memberId)))
-		_controller->onNetworkMemberDeauthorize(networkId,memberId);
+	if ((notifyListeners)&&((wasAuth)&&(!isAuth)&&(networkId)&&(memberId))) {
+		std::lock_guard<std::mutex> ll(_changeListeners_l);
+		for(auto i=_changeListeners.begin();i!=_changeListeners.end();++i) {
+			(*i)->onNetworkMemberDeauthorize(this,networkId,memberId);
+		}
+	}
 }

-void DB::_networkChanged(nlohmann::json &old,nlohmann::json &networkConfig,bool push)
+void DB::_networkChanged(nlohmann::json &old,nlohmann::json &networkConfig,bool notifyListeners)
 {
 	if (networkConfig.is_object()) {
 		const std::string ids = networkConfig["id"];
-		const uint64_t id = Utils::hexStrToU64(ids.c_str());
-		if (id) {
+		const uint64_t networkId = Utils::hexStrToU64(ids.c_str());
+		if (networkId) {
 			std::shared_ptr<_Network> nw;
 			{
 				std::lock_guard<std::mutex> l(_networks_l);
-				std::shared_ptr<_Network> &nw2 = _networks[id];
+				std::shared_ptr<_Network> &nw2 = _networks[networkId];
 				if (!nw2)
 					nw2.reset(new _Network);
 				nw = nw2;
@ -354,15 +331,19 @@ void DB::_networkChanged(nlohmann::json &old,nlohmann::json &networkConfig,bool
 				std::lock_guard<std::mutex> l2(nw->lock);
 				nw->config = networkConfig;
 			}
-			if (push)
-				_controller->onNetworkUpdate(id);
+			if (notifyListeners) {
+				std::lock_guard<std::mutex> ll(_changeListeners_l);
+				for(auto i=_changeListeners.begin();i!=_changeListeners.end();++i) {
+					(*i)->onNetworkUpdate(this,networkId,networkConfig);
+				}
+			}
 		}
 	} else if (old.is_object()) {
 		const std::string ids = old["id"];
-		const uint64_t id = Utils::hexStrToU64(ids.c_str());
-		if (id) {
+		const uint64_t networkId = Utils::hexStrToU64(ids.c_str());
+		if (networkId) {
 			std::lock_guard<std::mutex> l(_networks_l);
-			_networks.erase(id);
+			_networks.erase(networkId);
 		}
 	}
 }
--- a/controller/DB.hpp
+++ b/controller/DB.hpp
@ -27,6 +27,8 @@
 #ifndef ZT_CONTROLLER_DB_HPP
 #define ZT_CONTROLLER_DB_HPP

+//#define ZT_CONTROLLER_USE_LIBPQ
+
 #include "../node/Constants.hpp"
 #include "../node/Identity.hpp"
 #include "../node/InetAddress.hpp"
@ -41,20 +43,29 @@
 #include <vector>
 #include <atomic>
 #include <mutex>
+#include <set>

 #include "../ext/json/json.hpp"

 namespace ZeroTier
 {

-class EmbeddedNetworkController;
-
 /**
 * Base class with common infrastructure for all controller DB implementations
 */
 class DB
 {
 public:
+	class ChangeListener
+	{
+	public:
+		ChangeListener() {}
+		virtual ~ChangeListener() {}
+		virtual void onNetworkUpdate(const void *db,uint64_t networkId,const nlohmann::json &network) {}
+		virtual void onNetworkMemberUpdate(const void *db,uint64_t networkId,uint64_t memberId,const nlohmann::json &member) {}
+		virtual void onNetworkMemberDeauthorize(const void *db,uint64_t networkId,uint64_t memberId) {}
+	};
+
 	struct NetworkSummaryInfo
 	{
 		NetworkSummaryInfo() : authorizedMemberCount(0),totalMemberCount(0),mostRecentDeauthTime(0) {}
@ -65,27 +76,12 @@ public:
 		int64_t mostRecentDeauthTime;
 	};

-	/**
-	 * Ensure that all network fields are present
-	 */
 	static void initNetwork(nlohmann::json &network);
-
-	/**
-	 * Ensure that all member fields are present
-	 */
 	static void initMember(nlohmann::json &member);
-
-	/**
-	 * Remove old and temporary network fields
-	 */
 	static void cleanNetwork(nlohmann::json &network);
-
-	/**
-	 * Remove old and temporary member fields
-	 */
 	static void cleanMember(nlohmann::json &member);

-	DB(EmbeddedNetworkController *const nc,const Identity &myId,const char *path);
+	DB();
 	virtual ~DB();

 	virtual bool waitForReady() = 0;
@ -102,18 +98,21 @@ public:
 	bool get(const uint64_t networkId,nlohmann::json &network,const uint64_t memberId,nlohmann::json &member,NetworkSummaryInfo &info);
 	bool get(const uint64_t networkId,nlohmann::json &network,std::vector<nlohmann::json> &members);

-	bool summary(const uint64_t networkId,NetworkSummaryInfo &info);
+	void networks(std::set<uint64_t> &networks);

-	void networks(std::vector<uint64_t> &networks);
-
-	virtual void save(nlohmann::json *orig,nlohmann::json &record) = 0;
+	virtual bool save(nlohmann::json &record,bool notifyListeners) = 0;

 	virtual void eraseNetwork(const uint64_t networkId) = 0;
-
 	virtual void eraseMember(const uint64_t networkId,const uint64_t memberId) = 0;

 	virtual void nodeIsOnline(const uint64_t networkId,const uint64_t memberId,const InetAddress &physicalAddress) = 0;

+	inline void addListener(DB::ChangeListener *const listener)
+	{
+		std::lock_guard<std::mutex> l(_changeListeners_l);
+		_changeListeners.push_back(listener);
+	}
+
 protected:
 	struct _Network
 	{
@ -127,18 +126,14 @@ protected:
 		std::mutex lock;
 	};

-	void _memberChanged(nlohmann::json &old,nlohmann::json &memberConfig,bool push);
-	void _networkChanged(nlohmann::json &old,nlohmann::json &networkConfig,bool push);
+	void _memberChanged(nlohmann::json &old,nlohmann::json &memberConfig,bool notifyListeners);
+	void _networkChanged(nlohmann::json &old,nlohmann::json &networkConfig,bool notifyListeners);
 	void _fillSummaryInfo(const std::shared_ptr<_Network> &nw,NetworkSummaryInfo &info);

-	EmbeddedNetworkController *const _controller;
-	const Identity _myId;
-	const Address _myAddress;
-	const std::string _path;
-	std::string _myAddressStr;
-
+	std::vector<DB::ChangeListener *> _changeListeners;
 	std::unordered_map< uint64_t,std::shared_ptr<_Network> > _networks;
 	std::unordered_multimap< uint64_t,uint64_t > _networkByMember;
+	mutable std::mutex _changeListeners_l;
 	mutable std::mutex _networks_l;
 };

--- a/controller/DBMirrorSet.cpp
+++ b/controller/DBMirrorSet.cpp
@ -0,0 +1,194 @@
+/*
+ * ZeroTier One - Network Virtualization Everywhere
+ * Copyright (C) 2011-2019  ZeroTier, Inc.  https://www.zerotier.com/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * You can be released from the requirements of the license by purchasing
+ * a commercial license. Buying such a license is mandatory as soon as you
+ * develop commercial closed-source software that incorporates or links
+ * directly against ZeroTier software without disclosing the source code
+ * of your own application.
+ */
+
+#include "DBMirrorSet.hpp"
+
+namespace ZeroTier {
+
+DBMirrorSet::DBMirrorSet(DB::ChangeListener *listener) :
+	_listener(listener)
+{
+}
+
+DBMirrorSet::~DBMirrorSet()
+{
+}
+
+bool DBMirrorSet::hasNetwork(const uint64_t networkId) const
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		if ((*d)->hasNetwork(networkId))
+			return true;
+	}
+	return false;
+}
+
+bool DBMirrorSet::get(const uint64_t networkId,nlohmann::json &network)
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		if ((*d)->get(networkId,network)) {
+			return true;
+		}
+	}
+	return false;
+}
+
+bool DBMirrorSet::get(const uint64_t networkId,nlohmann::json &network,const uint64_t memberId,nlohmann::json &member)
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		if ((*d)->get(networkId,network,memberId,member))
+			return true;
+	}
+	return false;
+}
+
+bool DBMirrorSet::get(const uint64_t networkId,nlohmann::json &network,const uint64_t memberId,nlohmann::json &member,DB::NetworkSummaryInfo &info)
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		if ((*d)->get(networkId,network,memberId,member,info))
+			return true;
+	}
+	return false;
+}
+
+bool DBMirrorSet::get(const uint64_t networkId,nlohmann::json &network,std::vector<nlohmann::json> &members)
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		if ((*d)->get(networkId,network,members))
+			return true;
+	}
+	return false;
+}
+
+void DBMirrorSet::networks(std::set<uint64_t> &networks)
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		(*d)->networks(networks);
+	}
+}
+
+bool DBMirrorSet::waitForReady()
+{
+	bool r = false;
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		r |= (*d)->waitForReady();
+	}
+	return r;
+}
+
+bool DBMirrorSet::isReady()
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		if (!(*d)->isReady())
+			return false;
+	}
+	return true;
+}
+
+bool DBMirrorSet::save(nlohmann::json &record,bool notifyListeners)
+{
+	std::vector< std::shared_ptr<DB> > dbs;
+	{
+		std::lock_guard<std::mutex> l(_dbs_l);
+		dbs = _dbs;
+	}
+	if (notifyListeners) {
+		for(auto d=dbs.begin();d!=dbs.end();++d) {
+			if ((*d)->save(record,notifyListeners))
+				return true;
+		}
+		return false;
+	} else {
+		bool modified = false;
+		for(auto d=dbs.begin();d!=dbs.end();++d) {
+			modified |= (*d)->save(record,notifyListeners);
+		}
+		return modified;
+	}
+}
+
+void DBMirrorSet::eraseNetwork(const uint64_t networkId)
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		(*d)->eraseNetwork(networkId);
+	}
+}
+
+void DBMirrorSet::eraseMember(const uint64_t networkId,const uint64_t memberId)
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		(*d)->eraseMember(networkId,memberId);
+	}
+}
+
+void DBMirrorSet::nodeIsOnline(const uint64_t networkId,const uint64_t memberId,const InetAddress &physicalAddress)
+{
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		(*d)->nodeIsOnline(networkId,memberId,physicalAddress);
+	}
+}
+
+void DBMirrorSet::onNetworkUpdate(const void *db,uint64_t networkId,const nlohmann::json &network)
+{
+	nlohmann::json record(network);
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		if (d->get() != db) {
+			(*d)->save(record,false);
+		}
+	}
+	_listener->onNetworkUpdate(this,networkId,network);
+}
+
+void DBMirrorSet::onNetworkMemberUpdate(const void *db,uint64_t networkId,uint64_t memberId,const nlohmann::json &member)
+{
+	nlohmann::json record(member);
+	std::lock_guard<std::mutex> l(_dbs_l);
+	for(auto d=_dbs.begin();d!=_dbs.end();++d) {
+		if (d->get() != db) {
+			(*d)->save(record,false);
+		}
+	}
+	_listener->onNetworkMemberUpdate(this,networkId,memberId,member);
+}
+
+void DBMirrorSet::onNetworkMemberDeauthorize(const void *db,uint64_t networkId,uint64_t memberId)
+{
+	_listener->onNetworkMemberDeauthorize(this,networkId,memberId);
+}
+
+} // namespace ZeroTier
--- a/controller/DBMirrorSet.hpp
+++ b/controller/DBMirrorSet.hpp
@ -0,0 +1,81 @@
+/*
+ * ZeroTier One - Network Virtualization Everywhere
+ * Copyright (C) 2011-2019  ZeroTier, Inc.  https://www.zerotier.com/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * You can be released from the requirements of the license by purchasing
+ * a commercial license. Buying such a license is mandatory as soon as you
+ * develop commercial closed-source software that incorporates or links
+ * directly against ZeroTier software without disclosing the source code
+ * of your own application.
+ */
+
+#ifndef ZT_DBMIRRORSET_HPP
+#define ZT_DBMIRRORSET_HPP
+
+#include "DB.hpp"
+
+#include <vector>
+#include <memory>
+#include <mutex>
+#include <set>
+
+namespace ZeroTier {
+
+class DBMirrorSet : public DB::ChangeListener
+{
+public:
+	DBMirrorSet(DB::ChangeListener *listener);
+	virtual ~DBMirrorSet();
+
+	bool hasNetwork(const uint64_t networkId) const;
+
+	bool get(const uint64_t networkId,nlohmann::json &network);
+	bool get(const uint64_t networkId,nlohmann::json &network,const uint64_t memberId,nlohmann::json &member);
+	bool get(const uint64_t networkId,nlohmann::json &network,const uint64_t memberId,nlohmann::json &member,DB::NetworkSummaryInfo &info);
+	bool get(const uint64_t networkId,nlohmann::json &network,std::vector<nlohmann::json> &members);
+
+	void networks(std::set<uint64_t> &networks);
+
+	bool waitForReady();
+	bool isReady();
+	bool save(nlohmann::json &record,bool notifyListeners);
+	void eraseNetwork(const uint64_t networkId);
+	void eraseMember(const uint64_t networkId,const uint64_t memberId);
+	void nodeIsOnline(const uint64_t networkId,const uint64_t memberId,const InetAddress &physicalAddress);
+
+	// These are called by various DB instances when changes occur.
+	virtual void onNetworkUpdate(const void *db,uint64_t networkId,const nlohmann::json &network);
+	virtual void onNetworkMemberUpdate(const void *db,uint64_t networkId,uint64_t memberId,const nlohmann::json &member);
+	virtual void onNetworkMemberDeauthorize(const void *db,uint64_t networkId,uint64_t memberId);
+
+	inline void addDB(const std::shared_ptr<DB> &db)
+	{
+		db->addListener(this);
+		std::lock_guard<std::mutex> l(_dbs_l);
+		_dbs.push_back(db);
+	}
+
+private:
+	DB::ChangeListener *const _listener;
+	std::vector< std::shared_ptr< DB > > _dbs;
+	mutable std::mutex _dbs_l;
+};
+
+} // namespace ZeroTier
+
+#endif
--- a/controller/EmbeddedNetworkController.cpp
+++ b/controller/EmbeddedNetworkController.cpp
@ -46,6 +46,11 @@
 #include "../version.h"

 #include "EmbeddedNetworkController.hpp"
+#include "LFDB.hpp"
+#include "FileDB.hpp"
+#ifdef ZT_CONTROLLER_USE_LIBPQ
+#include "PostgreSQL.hpp"
+#endif

 #include "../node/Node.hpp"
 #include "../node/CertificateOfMembership.hpp"
@ -470,6 +475,7 @@ EmbeddedNetworkController::EmbeddedNetworkController(Node *node,const char *dbPa
 	_node(node),
 	_path(dbPath),
 	_sender((NetworkController::Sender *)0),
+	_db(this),
 	_mqc(mqc)
 {
 }
@ -488,13 +494,48 @@ void EmbeddedNetworkController::init(const Identity &signingId,Sender *sender)
 	_signingId = signingId;
 	_sender = sender;
 	_signingIdAddressString = signingId.address().toString(tmp);
+
 #ifdef ZT_CONTROLLER_USE_LIBPQ
-	if ((_path.length() > 9)&&(_path.substr(0,9) == "postgres:"))
-		_db.reset(new PostgreSQL(this,_signingId,_path.substr(9).c_str(), _listenPort, _mqc));
-	else // else use FileDB after endif
+	if ((_path.length() > 9)&&(_path.substr(0,9) == "postgres:")) {
+		_db.addDB(std::shared_ptr<DB>(new PostgreSQL(_signingId,_path.substr(9).c_str(), _listenPort, _mqc)));
+	} else {
 #endif
-		_db.reset(new FileDB(this,_signingId,_path.c_str()));
-	_db->waitForReady();
+		_db.addDB(std::shared_ptr<DB>(new FileDB(_path.c_str())));
+#ifdef ZT_CONTROLLER_USE_LIBPQ
+	}
+#endif
+
+	std::string lfJSON;
+	OSUtils::readFile((_path + ZT_PATH_SEPARATOR_S ".." ZT_PATH_SEPARATOR_S "local.conf").c_str(),lfJSON);
+	if (lfJSON.length() > 0) {
+		nlohmann::json lfConfig(OSUtils::jsonParse(lfJSON));
+		nlohmann::json &settings = lfConfig["settings"];
+		if (settings.is_object()) {
+			nlohmann::json &controllerDb = settings["controllerDb"];
+			if (controllerDb.is_object()) {
+				std::string type = controllerDb["type"];
+				if (type == "lf") {
+					std::string lfOwner = controllerDb["owner"];
+					std::string lfHost = controllerDb["host"];
+					int lfPort = controllerDb["port"];
+					bool storeOnlineState = controllerDb["storeOnlineState"];
+					if ((lfOwner.length())&&(lfHost.length())&&(lfPort > 0)&&(lfPort < 65536)) {
+						std::size_t pubHdrLoc = lfOwner.find("Public: ");
+						if ((pubHdrLoc > 0)&&((pubHdrLoc + 8) < lfOwner.length())) {
+							std::string lfOwnerPublic = lfOwner.substr(pubHdrLoc + 8);
+							std::size_t pubHdrEnd = lfOwnerPublic.find_first_of("\n\r\t ");
+							if (pubHdrEnd != std::string::npos) {
+								lfOwnerPublic = lfOwnerPublic.substr(0,pubHdrEnd);
+								_db.addDB(std::shared_ptr<DB>(new LFDB(_signingId,_path.c_str(),lfOwner.c_str(),lfOwnerPublic.c_str(),lfHost.c_str(),lfPort,storeOnlineState)));
+							}
+						}
+					}
+				}
+			}
+		}
+	}
+
+	_db.waitForReady();
 }

 void EmbeddedNetworkController::request(
@ -525,15 +566,12 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpGET(
 	std::string &responseBody,
 	std::string &responseContentType)
 {
-	if (!_db)
-		return 500;
-
 	if ((path.size() > 0)&&(path[0] == "network")) {

 		if ((path.size() >= 2)&&(path[1].length() == 16)) {
 			const uint64_t nwid = Utils::hexStrToU64(path[1].c_str());
 			json network;
-			if (!_db->get(nwid,network))
+			if (!_db.get(nwid,network))
 				return 404;

 			if (path.size() >= 3) {
@ -545,7 +583,7 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpGET(

 						const uint64_t address = Utils::hexStrToU64(path[3].c_str());
 						json member;
-						if (!_db->get(nwid,network,address,member))
+						if (!_db.get(nwid,network,address,member))
 							return 404;
 						responseBody = OSUtils::jsonDump(member);
 						responseContentType = "application/json";
@ -555,7 +593,7 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpGET(

 						responseBody = "{";
 						std::vector<json> members;
-						if (_db->get(nwid,network,members)) {
+						if (_db.get(nwid,network,members)) {
 							responseBody.reserve((members.size() + 2) * 32);
 							std::string mid;
 							for(auto member=members.begin();member!=members.end();++member) {
@ -584,12 +622,12 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpGET(
 		} else if (path.size() == 1) {
 			// List networks

-			std::vector<uint64_t> networkIds;
-			_db->networks(networkIds);
+			std::set<uint64_t> networkIds;
+			_db.networks(networkIds);
 			char tmp[64];
 			responseBody = "[";
 			responseBody.reserve((networkIds.size() + 1) * 24);
-			for(std::vector<uint64_t>::const_iterator i(networkIds.begin());i!=networkIds.end();++i) {
+			for(std::set<uint64_t>::const_iterator i(networkIds.begin());i!=networkIds.end();++i) {
 				if (responseBody.length() > 1)
 					responseBody.push_back(',');
 				OSUtils::ztsnprintf(tmp,sizeof(tmp),"\"%.16llx\"",(unsigned long long)*i);
@ -606,7 +644,7 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpGET(
 		// Controller status

 		char tmp[4096];
-		const bool dbOk = _db->isReady();
+		const bool dbOk = _db.isReady();
 		OSUtils::ztsnprintf(tmp,sizeof(tmp),"{\n\t\"controller\": true,\n\t\"apiVersion\": %d,\n\t\"clock\": %llu,\n\t\"databaseReady\": %s\n}\n",ZT_NETCONF_CONTROLLER_API_VERSION,(unsigned long long)OSUtils::now(),dbOk ? "true" : "false");
 		responseBody = tmp;
 		responseContentType = "application/json";
@ -625,8 +663,6 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpPOST(
 	std::string &responseBody,
 	std::string &responseContentType)
 {
-	if (!_db)
-		return 500;
 	if (path.empty())
 		return 404;

@ -660,8 +696,7 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpPOST(
 					OSUtils::ztsnprintf(addrs,sizeof(addrs),"%.10llx",(unsigned long long)address);

 					json member,network;
-					_db->get(nwid,network,address,member);
-					json origMember(member); // for detecting changes
+					_db.get(nwid,network,address,member);
 					DB::initMember(member);

 					try {
@ -755,7 +790,7 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpPOST(
 					member["nwid"] = nwids;

 					DB::cleanMember(member);
-					_db->save(&origMember,member);
+					_db.save(member,true);
 					responseBody = OSUtils::jsonDump(member);
 					responseContentType = "application/json";

@ -774,7 +809,7 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpPOST(
 						Utils::getSecureRandom(&nwidPostfix,sizeof(nwidPostfix));
 						uint64_t tryNwid = nwidPrefix | (nwidPostfix & 0xffffffULL);
 						if ((tryNwid & 0xffffffULL) == 0ULL) tryNwid |= 1ULL;
-						if (!_db->hasNetwork(tryNwid)) {
+						if (!_db.hasNetwork(tryNwid)) {
 							nwid = tryNwid;
 							break;
 						}
@ -785,8 +820,7 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpPOST(
 				OSUtils::ztsnprintf(nwids,sizeof(nwids),"%.16llx",(unsigned long long)nwid);

 				json network;
-				_db->get(nwid,network);
-				json origNetwork(network); // for detecting changes
+				_db.get(nwid,network);
 				DB::initNetwork(network);

 				try {
@ -1017,7 +1051,7 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpPOST(
 				network["nwid"] = nwids; // legacy

 				DB::cleanNetwork(network);
-				_db->save(&origNetwork,network);
+				_db.save(network,true);

 				responseBody = OSUtils::jsonDump(network);
 				responseContentType = "application/json";
@ -1039,8 +1073,6 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpDELETE(
 	std::string &responseBody,
 	std::string &responseContentType)
 {
-	if (!_db)
-		return 500;
 	if (path.empty())
 		return 404;

@ -1052,8 +1084,8 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpDELETE(
 					const uint64_t address = Utils::hexStrToU64(path[3].c_str());

 					json network,member;
-					_db->get(nwid,network,address,member);
-					_db->eraseMember(nwid, address);
+					_db.get(nwid,network,address,member);
+					_db.eraseMember(nwid, address);

 					{
 						std::lock_guard<std::mutex> l(_memberStatus_l);
@ -1068,8 +1100,8 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpDELETE(
 				}
 			} else {
 				json network;
-				_db->get(nwid,network);
-				_db->eraseNetwork(nwid);
+				_db.get(nwid,network);
+				_db.eraseNetwork(nwid);

 				{
 					std::lock_guard<std::mutex> l(_memberStatus_l);
@ -1093,7 +1125,57 @@ unsigned int EmbeddedNetworkController::handleControlPlaneHttpDELETE(
 	return 404;
 }

-void EmbeddedNetworkController::onNetworkUpdate(const uint64_t networkId)
+void EmbeddedNetworkController::handleRemoteTrace(const ZT_RemoteTrace &rt)
+{
+	static volatile unsigned long idCounter = 0;
+	char id[128],tmp[128];
+	std::string k,v;
+
+	try {
+		// Convert Dictionary into JSON object
+		json d;
+		char *saveptr = (char *)0;
+		for(char *l=Utils::stok(rt.data,"\n",&saveptr);(l);l=Utils::stok((char *)0,"\n",&saveptr)) {
+			char *eq = strchr(l,'=');
+			if (eq > l) {
+				k.assign(l,(unsigned long)(eq - l));
+				v.clear();
+				++eq;
+				while (*eq) {
+					if (*eq == '\\') {
+						++eq;
+						if (*eq) {
+							switch(*eq) {
+								case 'r': v.push_back('\r'); break;
+								case 'n': v.push_back('\n'); break;
+								case '0': v.push_back((char)0); break;
+								case 'e': v.push_back('='); break;
+								default: v.push_back(*eq); break;
+							}
+							++eq;
+						}
+					} else {
+						v.push_back(*(eq++));
+					}
+				}
+				if ((k.length() > 0)&&(v.length() > 0))
+					d[k] = v;
+			}
+		}
+
+		const int64_t now = OSUtils::now();
+		OSUtils::ztsnprintf(id,sizeof(id),"%.10llx-%.16llx-%.10llx-%.4x",_signingId.address().toInt(),now,rt.origin,(unsigned int)(idCounter++ & 0xffff));
+		d["id"] = id;
+		d["objtype"] = "trace";
+		d["ts"] = now;
+		d["nodeId"] = Utils::hex10(rt.origin,tmp);
+		_db.save(d,true);
+	} catch ( ... ) {
+		// drop invalid trace messages if an error occurs
+	}
+}
+
+void EmbeddedNetworkController::onNetworkUpdate(const void *db,uint64_t networkId,const nlohmann::json &network)
 {
 	// Send an update to all members of the network that are online
 	const int64_t now = OSUtils::now();
@ -1104,7 +1186,7 @@ void EmbeddedNetworkController::onNetworkUpdate(const uint64_t networkId)
 	}
 }

-void EmbeddedNetworkController::onNetworkMemberUpdate(const uint64_t networkId,const uint64_t memberId)
+void EmbeddedNetworkController::onNetworkMemberUpdate(const void *db,uint64_t networkId,uint64_t memberId,const nlohmann::json &member)
 {
 	// Push update to member if online
 	try {
@ -1115,7 +1197,7 @@ void EmbeddedNetworkController::onNetworkMemberUpdate(const uint64_t networkId,c
 	} catch ( ... ) {}
 }

-void EmbeddedNetworkController::onNetworkMemberDeauthorize(const uint64_t networkId,const uint64_t memberId)
+void EmbeddedNetworkController::onNetworkMemberDeauthorize(const void *db,uint64_t networkId,uint64_t memberId)
 {
 	const int64_t now = OSUtils::now();
 	Revocation rev((uint32_t)_node->prng(),networkId,0,now,ZT_REVOCATION_FLAG_FAST_PROPAGATE,Address(memberId),Revocation::CREDENTIAL_TYPE_COM);
@ -1138,10 +1220,7 @@ void EmbeddedNetworkController::_request(
 {
 	char nwids[24];
 	DB::NetworkSummaryInfo ns;
-	json network,member,origMember;
-
-	if (!_db)
-		return;
+	json network,member;

 	if (((!_signingId)||(!_signingId.hasPrivate()))||(_signingId.address().toInt() != (nwid >> 24))||(!_sender))
 		return;
@ -1156,15 +1235,14 @@ void EmbeddedNetworkController::_request(
 		ms.lastRequestTime = now;
 	}

-	_db->nodeIsOnline(nwid,identity.address().toInt(),fromAddr);
+	_db.nodeIsOnline(nwid,identity.address().toInt(),fromAddr);

 	Utils::hex(nwid,nwids);
-	_db->get(nwid,network,identity.address().toInt(),member,ns);
+	_db.get(nwid,network,identity.address().toInt(),member,ns);
 	if ((!network.is_object())||(network.size() == 0)) {
 		_sender->ncSendError(nwid,requestPacketId,identity.address(),NetworkController::NC_ERROR_OBJECT_NOT_FOUND);
 		return;
 	}
-	origMember = member;
 	const bool newMember = ((!member.is_object())||(member.size() == 0));
 	DB::initMember(member);

@ -1265,7 +1343,7 @@ void EmbeddedNetworkController::_request(
 	} else {
 		// If they are not authorized, STOP!
 		DB::cleanMember(member);
-		_db->save(&origMember,member);
+		_db.save(member,true);
 		_sender->ncSendError(nwid,requestPacketId,identity.address(),NetworkController::NC_ERROR_ACCESS_DENIED);
 		return;
 	}
@ -1637,7 +1715,7 @@ void EmbeddedNetworkController::_request(
 	}

 	DB::cleanMember(member);
-	_db->save(&origMember,member);
+	_db.save(member,true);
 	_sender->ncSendConfig(nwid,requestPacketId,identity.address(),*(nc.get()),metaData.getUI(ZT_NETWORKCONFIG_REQUEST_METADATA_KEY_VERSION,0) < 6);
 }

--- a/controller/EmbeddedNetworkController.hpp
+++ b/controller/EmbeddedNetworkController.hpp
@ -51,10 +51,7 @@
 #include "../ext/json/json.hpp"

 #include "DB.hpp"
-#include "FileDB.hpp"
-#ifdef ZT_CONTROLLER_USE_LIBPQ
-#include "PostgreSQL.hpp"
-#endif
+#include "DBMirrorSet.hpp"

 namespace ZeroTier {

@ -62,7 +59,7 @@ class Node;

 struct MQConfig;

-class EmbeddedNetworkController : public NetworkController
+class EmbeddedNetworkController : public NetworkController,public DB::ChangeListener
 {
 public:
 	/**
@ -103,10 +100,11 @@ public:
 		std::string &responseBody,
 		std::string &responseContentType);

-	// Called on update via POST or by JSONDB on external update of network or network member records
-	void onNetworkUpdate(const uint64_t networkId);
-	void onNetworkMemberUpdate(const uint64_t networkId,const uint64_t memberId);
-	void onNetworkMemberDeauthorize(const uint64_t networkId,const uint64_t memberId);
+	void handleRemoteTrace(const ZT_RemoteTrace &rt);
+
+	virtual void onNetworkUpdate(const void *db,uint64_t networkId,const nlohmann::json &network);
+	virtual void onNetworkMemberUpdate(const void *db,uint64_t networkId,uint64_t memberId,const nlohmann::json &member);
+	virtual void onNetworkMemberDeauthorize(const void *db,uint64_t networkId,uint64_t memberId);

 private:
 	void _request(uint64_t nwid,const InetAddress &fromAddr,uint64_t requestPacketId,const Identity &identity,const Dictionary<ZT_NETWORKCONFIG_METADATA_DICT_CAPACITY> &metaData);
@ -156,7 +154,7 @@ private:
 	std::string _signingIdAddressString;
 	NetworkController::Sender *_sender;

-	std::unique_ptr<DB> _db;
+	DBMirrorSet _db;
 	BlockingQueue< _RQEntry * > _queue;

 	std::vector<std::thread> _threads;
--- a/controller/FileDB.cpp
+++ b/controller/FileDB.cpp
@ -29,10 +29,10 @@
 namespace ZeroTier
 {

-FileDB::FileDB(EmbeddedNetworkController *const nc,const Identity &myId,const char *path) :
-	DB(nc,myId,path),
+FileDB::FileDB(const char *path) :
+	DB(),
+	_path(path),
 	_networksPath(_path + ZT_PATH_SEPARATOR_S + "network"),
-	_onlineChanged(false),
 	_running(true)
 {
 	OSUtils::mkdir(_path.c_str());
@ -69,54 +69,6 @@ FileDB::FileDB(EmbeddedNetworkController *const nc,const Identity &myId,const ch
 			} catch ( ... ) {}
 		}
 	}
-
-	_onlineUpdateThread = std::thread([this]() {
-		unsigned int cnt = 0;
-		while (this->_running) {
-			usleep(250);
-			if ((++cnt % 20) == 0) { // 5 seconds
-				std::lock_guard<std::mutex> l(this->_online_l);
-				if (!this->_running) return;
-				if (this->_onlineChanged) {
-					char p[4096],atmp[64];
-					for(auto nw=this->_online.begin();nw!=this->_online.end();++nw) {
-						OSUtils::ztsnprintf(p,sizeof(p),"%s" ZT_PATH_SEPARATOR_S "%.16llx-online.json",_networksPath.c_str(),(unsigned long long)nw->first);
-						FILE *f = fopen(p,"wb");
-						if (f) {
-							fprintf(f,"{");
-							const char *memberPrefix = "";
-							for(auto m=nw->second.begin();m!=nw->second.end();++m) {
-								fprintf(f,"%s\"%.10llx\":{" ZT_EOL_S,memberPrefix,(unsigned long long)m->first);
-								memberPrefix = ",";
-								InetAddress lastAddr;
-								const char *timestampPrefix = " ";
-								int cnt = 0;
-								for(auto ts=m->second.rbegin();ts!=m->second.rend();) {
-									if (cnt < 25) {
-										if (lastAddr != ts->second) {
-											lastAddr = ts->second;
-											fprintf(f,"%s\"%lld\":\"%s\"" ZT_EOL_S,timestampPrefix,(long long)ts->first,ts->second.toString(atmp));
-											timestampPrefix = ",";
-											++cnt;
-											++ts;
-										} else {
-											ts = std::map<int64_t,InetAddress>::reverse_iterator(m->second.erase(std::next(ts).base()));
-										}
-									} else {
-										ts = std::map<int64_t,InetAddress>::reverse_iterator(m->second.erase(std::next(ts).base()));
-									}
-								}
-								fprintf(f,"}");
-							}
-							fprintf(f,"}" ZT_EOL_S);
-							fclose(f);
-						}
-					}
-					this->_onlineChanged = false;
-				}
-			}
-		}
-	});
 }

 FileDB::~FileDB()
@ -132,38 +84,37 @@ FileDB::~FileDB()
 bool FileDB::waitForReady() { return true; }
 bool FileDB::isReady() { return true; }

-void FileDB::save(nlohmann::json *orig,nlohmann::json &record)
+bool FileDB::save(nlohmann::json &record,bool notifyListeners)
 {
 	char p1[4096],p2[4096],pb[4096];
+	bool modified = false;
 	try {
-		if (orig) {
-			if (*orig != record) {
-				record["revision"] = OSUtils::jsonInt(record["revision"],0ULL) + 1;
-			}
-		} else {
-			record["revision"] = 1;
-		}
-
 		const std::string objtype = record["objtype"];
 		if (objtype == "network") {
+
 			const uint64_t nwid = OSUtils::jsonIntHex(record["id"],0ULL);
 			if (nwid) {
 				nlohmann::json old;
 				get(nwid,old);
 				if ((!old.is_object())||(old != record)) {
+					record["revision"] = OSUtils::jsonInt(record["revision"],0ULL) + 1ULL;
 					OSUtils::ztsnprintf(p1,sizeof(p1),"%s" ZT_PATH_SEPARATOR_S "%.16llx.json",_networksPath.c_str(),nwid);
 					if (!OSUtils::writeFile(p1,OSUtils::jsonDump(record,-1)))
 						fprintf(stderr,"WARNING: controller unable to write to path: %s" ZT_EOL_S,p1);
-					_networkChanged(old,record,true);
+					_networkChanged(old,record,notifyListeners);
+					modified = true;
 				}
 			}
+
 		} else if (objtype == "member") {
+
 			const uint64_t id = OSUtils::jsonIntHex(record["id"],0ULL);
 			const uint64_t nwid = OSUtils::jsonIntHex(record["nwid"],0ULL);
 			if ((id)&&(nwid)) {
 				nlohmann::json network,old;
 				get(nwid,network,id,old);
 				if ((!old.is_object())||(old != record)) {
+					record["revision"] = OSUtils::jsonInt(record["revision"],0ULL) + 1ULL;
 					OSUtils::ztsnprintf(pb,sizeof(pb),"%s" ZT_PATH_SEPARATOR_S "%.16llx" ZT_PATH_SEPARATOR_S "member",_networksPath.c_str(),(unsigned long long)nwid);
 					OSUtils::ztsnprintf(p1,sizeof(p1),"%s" ZT_PATH_SEPARATOR_S "%.10llx.json",pb,(unsigned long long)id);
 					if (!OSUtils::writeFile(p1,OSUtils::jsonDump(record,-1))) {
@ -173,11 +124,14 @@ void FileDB::save(nlohmann::json *orig,nlohmann::json &record)
 						if (!OSUtils::writeFile(p1,OSUtils::jsonDump(record,-1)))
 							fprintf(stderr,"WARNING: controller unable to write to path: %s" ZT_EOL_S,p1);
 					}
-					_memberChanged(old,record,true);
+					_memberChanged(old,record,notifyListeners);
+					modified = true;
 				}
 			}
+
 		}
 	} catch ( ... ) {} // drop invalid records missing fields
+	return modified;
 }

 void FileDB::eraseNetwork(const uint64_t networkId)
@ -187,14 +141,11 @@ void FileDB::eraseNetwork(const uint64_t networkId)
 	char p[16384];
 	OSUtils::ztsnprintf(p,sizeof(p),"%s" ZT_PATH_SEPARATOR_S "%.16llx.json",_networksPath.c_str(),networkId);
 	OSUtils::rm(p);
-	OSUtils::ztsnprintf(p,sizeof(p),"%s" ZT_PATH_SEPARATOR_S "%.16llx-online.json",_networksPath.c_str(),networkId);
-	OSUtils::rm(p);
 	OSUtils::ztsnprintf(p,sizeof(p),"%s" ZT_PATH_SEPARATOR_S "%.16llx" ZT_PATH_SEPARATOR_S "member",_networksPath.c_str(),(unsigned long long)networkId);
 	OSUtils::rmDashRf(p);
 	_networkChanged(network,nullJson,true);
 	std::lock_guard<std::mutex> l(this->_online_l);
 	this->_online.erase(networkId);
-	this->_onlineChanged = true;
 }

 void FileDB::eraseMember(const uint64_t networkId,const uint64_t memberId)
@ -208,7 +159,6 @@ void FileDB::eraseMember(const uint64_t networkId,const uint64_t memberId)
 	_memberChanged(member,nullJson,true);
 	std::lock_guard<std::mutex> l(this->_online_l);
 	this->_online[networkId].erase(memberId);
-	this->_onlineChanged = true;
 }

 void FileDB::nodeIsOnline(const uint64_t networkId,const uint64_t memberId,const InetAddress &physicalAddress)
@ -218,7 +168,6 @@ void FileDB::nodeIsOnline(const uint64_t networkId,const uint64_t memberId,const
 	physicalAddress.toString(atmp);
 	std::lock_guard<std::mutex> l(this->_online_l);
 	this->_online[networkId][memberId][OSUtils::now()] = physicalAddress;
-	this->_onlineChanged = true;
 }

 } // namespace ZeroTier
--- a/controller/FileDB.hpp
+++ b/controller/FileDB.hpp
@ -35,22 +35,22 @@ namespace ZeroTier
 class FileDB : public DB
 {
 public:
-	FileDB(EmbeddedNetworkController *const nc,const Identity &myId,const char *path);
+	FileDB(const char *path);
 	virtual ~FileDB();

 	virtual bool waitForReady();
 	virtual bool isReady();
-	virtual void save(nlohmann::json *orig,nlohmann::json &record);
+	virtual bool save(nlohmann::json &record,bool notifyListeners);
 	virtual void eraseNetwork(const uint64_t networkId);
 	virtual void eraseMember(const uint64_t networkId,const uint64_t memberId);
 	virtual void nodeIsOnline(const uint64_t networkId,const uint64_t memberId,const InetAddress &physicalAddress);

 protected:
+	std::string _path;
 	std::string _networksPath;
 	std::thread _onlineUpdateThread;
 	std::map< uint64_t,std::map<uint64_t,std::map<int64_t,InetAddress> > > _online;
 	std::mutex _online_l;
-	bool _onlineChanged;
 	bool _running;
 };

--- a/controller/LFDB.cpp
+++ b/controller/LFDB.cpp
@ -0,0 +1,402 @@
+/*
+ * ZeroTier One - Network Virtualization Everywhere
+ * Copyright (C) 2011-2019  ZeroTier, Inc.  https://www.zerotier.com/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * You can be released from the requirements of the license by purchasing
+ * a commercial license. Buying such a license is mandatory as soon as you
+ * develop commercial closed-source software that incorporates or links
+ * directly against ZeroTier software without disclosing the source code
+ * of your own application.
+ */
+
+#include "LFDB.hpp"
+
+#include <thread>
+#include <chrono>
+#include <iostream>
+#include <sstream>
+
+#include "../osdep/OSUtils.hpp"
+#include "../ext/cpp-httplib/httplib.h"
+
+namespace ZeroTier
+{
+
+LFDB::LFDB(const Identity &myId,const char *path,const char *lfOwnerPrivate,const char *lfOwnerPublic,const char *lfNodeHost,int lfNodePort,bool storeOnlineState) :
+	DB(),
+	_myId(myId),
+	_lfOwnerPrivate((lfOwnerPrivate) ? lfOwnerPrivate : ""),
+	_lfOwnerPublic((lfOwnerPublic) ? lfOwnerPublic : ""),
+	_lfNodeHost((lfNodeHost) ? lfNodeHost : "127.0.0.1"),
+	_lfNodePort(((lfNodePort > 0)&&(lfNodePort < 65536)) ? lfNodePort : 9980),
+	_running(true),
+	_ready(false),
+	_storeOnlineState(storeOnlineState)
+{
+	_syncThread = std::thread([this]() {
+		char controllerAddress[24];
+		const uint64_t controllerAddressInt = _myId.address().toInt();
+		_myId.address().toString(controllerAddress);
+		std::string networksSelectorName("com.zerotier.controller.lfdb:"); networksSelectorName.append(controllerAddress); networksSelectorName.append("/network");
+		std::string membersSelectorName("com.zerotier.controller.lfdb:"); membersSelectorName.append(controllerAddress); membersSelectorName.append("/member");
+
+		// LF record masking key is the first 32 bytes of SHA512(controller private key) in hex,
+		// hiding record values from anything but the controller or someone who has its key.
+		uint8_t sha512pk[64];
+		_myId.sha512PrivateKey(sha512pk);
+		char maskingKey [128];
+		Utils::hex(sha512pk,32,maskingKey);
+
+		httplib::Client htcli(_lfNodeHost.c_str(),_lfNodePort,600);
+		int64_t timeRangeStart = 0;
+		while (_running.load()) {
+			{
+				std::lock_guard<std::mutex> sl(_state_l);
+				for(auto ns=_state.begin();ns!=_state.end();++ns) {
+					if (ns->second.dirty) {
+						nlohmann::json network;
+						if (get(ns->first,network)) {
+							nlohmann::json newrec,selector0;
+							selector0["Name"] = networksSelectorName;
+							selector0["Ordinal"] = ns->first;
+							newrec["Selectors"].push_back(selector0);
+							newrec["Value"] = network.dump();
+							newrec["OwnerPrivate"] = _lfOwnerPrivate;
+							newrec["MaskingKey"] = maskingKey;
+							newrec["PulseIfUnchanged"] = true;
+							auto resp = htcli.Post("/makerecord",newrec.dump(),"application/json");
+							if (resp) {
+								if (resp->status == 200) {
+									ns->second.dirty = false;
+									//printf("SET network %.16llx %s\n",ns->first,resp->body.c_str());
+								} else {
+									fprintf(stderr,"ERROR: LFDB: %d from node (create/update network): %s" ZT_EOL_S,resp->status,resp->body.c_str());
+								}
+							} else {
+								fprintf(stderr,"ERROR: LFDB: node is offline" ZT_EOL_S);
+							}
+						}
+					}
+
+					for(auto ms=ns->second.members.begin();ms!=ns->second.members.end();++ms) {
+						if ((_storeOnlineState)&&(ms->second.lastOnlineDirty)&&(ms->second.lastOnlineAddress)) {
+							nlohmann::json newrec,selector0,selector1,selectors,ip;
+							char tmp[1024],tmp2[128];
+							OSUtils::ztsnprintf(tmp,sizeof(tmp),"com.zerotier.controller.lfdb:%s/network/%.16llx/online",controllerAddress,(unsigned long long)ns->first);
+							ms->second.lastOnlineAddress.toIpString(tmp2);
+							selector0["Name"] = tmp;
+							selector0["Ordinal"] = ms->first;
+							selector1["Name"] = tmp2;
+							selector1["Ordinal"] = 0;
+							selectors.push_back(selector0);
+							selectors.push_back(selector1);
+							newrec["Selectors"] = selectors;
+							const uint8_t *const rawip = (const uint8_t *)ms->second.lastOnlineAddress.rawIpData();
+							switch(ms->second.lastOnlineAddress.ss_family) {
+								case AF_INET:
+									for(int j=0;j<4;++j)
+										ip.push_back((unsigned int)rawip[j]);
+									break;
+								case AF_INET6:
+									for(int j=0;j<16;++j)
+										ip.push_back((unsigned int)rawip[j]);
+									break;
+								default:
+									ip = tmp2; // should never happen since only IP transport is currently supported
+									break;
+							}
+							newrec["Value"] = ip;
+							newrec["OwnerPrivate"] = _lfOwnerPrivate;
+							newrec["MaskingKey"] = maskingKey;
+							newrec["Timestamp"] = ms->second.lastOnlineTime;
+							newrec["PulseIfUnchanged"] = true;
+							auto resp = htcli.Post("/makerecord",newrec.dump(),"application/json");
+							if (resp) {
+								if (resp->status == 200) {
+									ms->second.lastOnlineDirty = false;
+									//printf("SET member online %.16llx %.10llx %s\n",ns->first,ms->first,resp->body.c_str());
+								} else {
+									fprintf(stderr,"ERROR: LFDB: %d from node (create/update member online status): %s" ZT_EOL_S,resp->status,resp->body.c_str());
+								}
+							} else {
+								fprintf(stderr,"ERROR: LFDB: node is offline" ZT_EOL_S);
+							}
+						}
+
+						if (ms->second.dirty) {
+							nlohmann::json network,member;
+							if (get(ns->first,network,ms->first,member)) {
+								nlohmann::json newrec,selector0,selector1,selectors;
+								selector0["Name"] = networksSelectorName;
+								selector0["Ordinal"] = ns->first;
+								selector1["Name"] = membersSelectorName;
+								selector1["Ordinal"] = ms->first;
+								selectors.push_back(selector0);
+								selectors.push_back(selector1);
+								newrec["Selectors"] = selectors;
+								newrec["Value"] = member.dump();
+								newrec["OwnerPrivate"] = _lfOwnerPrivate;
+								newrec["MaskingKey"] = maskingKey;
+								newrec["PulseIfUnchanged"] = true;
+								auto resp = htcli.Post("/makerecord",newrec.dump(),"application/json");
+								if (resp) {
+									if (resp->status == 200) {
+										ms->second.dirty = false;
+										//printf("SET member %.16llx %.10llx %s\n",ns->first,ms->first,resp->body.c_str());
+									} else {
+										fprintf(stderr,"ERROR: LFDB: %d from node (create/update member): %s" ZT_EOL_S,resp->status,resp->body.c_str());
+									}
+								} else {
+									fprintf(stderr,"ERROR: LFDB: node is offline" ZT_EOL_S);
+								}
+							}
+						}
+					}
+				}
+			}
+
+			{
+				std::ostringstream query;
+				query
+					<< '{'
+						<< "\"Ranges\":[{"
+							<< "\"Name\":\"" << networksSelectorName << "\","
+							<< "\"Range\":[0,18446744073709551615]"
+						<< "}],"
+						<< "\"TimeRange\":[" << timeRangeStart << ",9223372036854775807],"
+						<< "\"MaskingKey\":\"" << maskingKey << "\","
+						<< "\"Owners\":[\"" << _lfOwnerPublic << "\"]"
+					<< '}';
+				auto resp = htcli.Post("/query",query.str(),"application/json");
+				if (resp) {
+					if (resp->status == 200) {
+						nlohmann::json results(OSUtils::jsonParse(resp->body));
+						if ((results.is_array())&&(results.size() > 0)) {
+							for(std::size_t ri=0;ri<results.size();++ri) {
+								nlohmann::json &rset = results[ri];
+								if ((rset.is_array())&&(rset.size() > 0)) {
+
+									nlohmann::json &result = rset[0];
+									if (result.is_object()) {
+										nlohmann::json &record = result["Record"];
+										if (record.is_object()) {
+											const std::string recordValue = result["Value"];
+											//printf("GET network %s\n",recordValue.c_str());
+											nlohmann::json network(OSUtils::jsonParse(recordValue));
+											if (network.is_object()) {
+												const std::string idstr = network["id"];
+												const uint64_t id = Utils::hexStrToU64(idstr.c_str());
+												if ((id >> 24) == controllerAddressInt) { // sanity check
+
+													std::lock_guard<std::mutex> sl(_state_l);
+													_NetworkState &ns = _state[id];
+													if (!ns.dirty) {
+														nlohmann::json oldNetwork;
+														if ((timeRangeStart > 0)&&(get(id,oldNetwork))) {
+															const uint64_t revision = network["revision"];
+															const uint64_t prevRevision = oldNetwork["revision"];
+															if (prevRevision < revision) {
+																_networkChanged(oldNetwork,network,timeRangeStart > 0);
+															}
+														} else {
+															nlohmann::json nullJson;
+															_networkChanged(nullJson,network,timeRangeStart > 0);
+														}
+													}
+
+												}
+											}
+										}
+									}
+
+								}
+							}
+						}
+					} else {
+						fprintf(stderr,"ERROR: LFDB: %d from node: %s" ZT_EOL_S,resp->status,resp->body.c_str());
+					}
+				} else {
+					fprintf(stderr,"ERROR: LFDB: node is offline" ZT_EOL_S);
+				}
+			}
+
+			{
+				std::ostringstream query;
+				query
+					<< '{'
+						<< "\"Ranges\":[{"
+							<< "\"Name\":\"" << networksSelectorName << "\","
+							<< "\"Range\":[0,18446744073709551615]"
+						<< "},{"
+							<< "\"Name\":\"" << membersSelectorName << "\","
+							<< "\"Range\":[0,18446744073709551615]"
+						<< "}],"
+						<< "\"TimeRange\":[" << timeRangeStart << ",9223372036854775807],"
+						<< "\"MaskingKey\":\"" << maskingKey << "\","
+						<< "\"Owners\":[\"" << _lfOwnerPublic << "\"]"
+					<< '}';
+				auto resp = htcli.Post("/query",query.str(),"application/json");
+				if (resp) {
+					if (resp->status == 200) {
+						nlohmann::json results(OSUtils::jsonParse(resp->body));
+						if ((results.is_array())&&(results.size() > 0)) {
+							for(std::size_t ri=0;ri<results.size();++ri) {
+								nlohmann::json &rset = results[ri];
+								if ((rset.is_array())&&(rset.size() > 0)) {
+
+									nlohmann::json &result = rset[0];
+									if (result.is_object()) {
+										nlohmann::json &record = result["Record"];
+										if (record.is_object()) {
+											const std::string recordValue = result["Value"];
+											//printf("GET member %s\n",recordValue.c_str());
+											nlohmann::json member(OSUtils::jsonParse(recordValue));
+											if (member.is_object()) {
+												const std::string nwidstr = member["nwid"];
+												const std::string idstr = member["id"];
+												const uint64_t nwid = Utils::hexStrToU64(nwidstr.c_str());
+												const uint64_t id = Utils::hexStrToU64(idstr.c_str());
+												if ((id)&&((nwid >> 24) == controllerAddressInt)) { // sanity check
+
+													std::lock_guard<std::mutex> sl(_state_l);
+													auto ns = _state.find(nwid);
+													if ((ns == _state.end())||(!ns->second.members[id].dirty)) {
+														nlohmann::json network,oldMember;
+														if ((timeRangeStart > 0)&&(get(nwid,network,id,oldMember))) {
+															const uint64_t revision = member["revision"];
+															const uint64_t prevRevision = oldMember["revision"];
+															if (prevRevision < revision)
+																_memberChanged(oldMember,member,timeRangeStart > 0);
+														}
+													} else {
+														nlohmann::json nullJson;
+														_memberChanged(nullJson,member,timeRangeStart > 0);
+													}
+
+												}
+											}
+										}
+									}
+
+								}
+							}
+						}
+					} else {
+						fprintf(stderr,"ERROR: LFDB: %d from node: %s" ZT_EOL_S,resp->status,resp->body.c_str());
+					}
+				} else {
+					fprintf(stderr,"ERROR: LFDB: node is offline" ZT_EOL_S);
+				}
+			}
+
+			timeRangeStart = time(nullptr) - 120; // start next query 2m before now to avoid losing updates
+			_ready.store(true);
+
+			for(int k=0;k<4;++k) { // 2s delay between queries for remotely modified networks or members
+				if (!_running.load())
+					return;
+				std::this_thread::sleep_for(std::chrono::milliseconds(500));
+			}
+		}
+	});
+}
+
+LFDB::~LFDB()
+{
+	_running.store(false);
+	_syncThread.join();
+}
+
+bool LFDB::waitForReady()
+{
+	while (!_ready.load()) {
+		std::this_thread::sleep_for(std::chrono::milliseconds(500));
+	}
+	return true;
+}
+
+bool LFDB::isReady()
+{
+	return (_ready.load());
+}
+
+bool LFDB::save(nlohmann::json &record,bool notifyListeners)
+{
+	bool modified = false;
+	const std::string objtype = record["objtype"];
+	if (objtype == "network") {
+		const uint64_t nwid = OSUtils::jsonIntHex(record["id"],0ULL);
+		if (nwid) {
+			nlohmann::json old;
+			get(nwid,old);
+			if ((!old.is_object())||(old != record)) {
+				record["revision"] = OSUtils::jsonInt(record["revision"],0ULL) + 1ULL;
+				_networkChanged(old,record,notifyListeners);
+				{
+					std::lock_guard<std::mutex> l(_state_l);
+					_state[nwid].dirty = true;
+				}
+				modified = true;
+			}
+		}
+	} else if (objtype == "member") {
+		const uint64_t nwid = OSUtils::jsonIntHex(record["nwid"],0ULL);
+		const uint64_t id = OSUtils::jsonIntHex(record["id"],0ULL);
+		if ((id)&&(nwid)) {
+			nlohmann::json network,old;
+			get(nwid,network,id,old);
+			if ((!old.is_object())||(old != record)) {
+				record["revision"] = OSUtils::jsonInt(record["revision"],0ULL) + 1ULL;
+				_memberChanged(old,record,notifyListeners);
+				{
+					std::lock_guard<std::mutex> l(_state_l);
+					_state[nwid].members[id].dirty = true;
+				}
+				modified = true;
+			}
+		}
+	}
+	return modified;
+}
+
+void LFDB::eraseNetwork(const uint64_t networkId)
+{
+	// TODO
+}
+
+void LFDB::eraseMember(const uint64_t networkId,const uint64_t memberId)
+{
+	// TODO
+}
+
+void LFDB::nodeIsOnline(const uint64_t networkId,const uint64_t memberId,const InetAddress &physicalAddress)
+{
+	std::lock_guard<std::mutex> l(_state_l);
+	auto nw = _state.find(networkId);
+	if (nw != _state.end()) {
+		auto m = nw->second.members.find(memberId);
+		if (m != nw->second.members.end()) {
+			m->second.lastOnlineTime = OSUtils::now();
+			if (physicalAddress)
+				m->second.lastOnlineAddress = physicalAddress;
+			m->second.lastOnlineDirty = true;
+		}
+	}
+}
+
+} // namespace ZeroTier
--- a/controller/LFDB.hpp
+++ b/controller/LFDB.hpp
@ -0,0 +1,102 @@
+/*
+ * ZeroTier One - Network Virtualization Everywhere
+ * Copyright (C) 2011-2019  ZeroTier, Inc.  https://www.zerotier.com/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * You can be released from the requirements of the license by purchasing
+ * a commercial license. Buying such a license is mandatory as soon as you
+ * develop commercial closed-source software that incorporates or links
+ * directly against ZeroTier software without disclosing the source code
+ * of your own application.
+ */
+
+#ifndef ZT_CONTROLLER_LFDB_HPP
+#define ZT_CONTROLLER_LFDB_HPP
+
+#include "DB.hpp"
+
+#include <mutex>
+#include <string>
+#include <unordered_map>
+#include <atomic>
+
+namespace ZeroTier {
+
+/**
+ * DB implementation for controller that stores data in LF
+ */
+class LFDB : public DB
+{
+public:
+	/**
+	 * @param myId This controller's identity
+	 * @param path Base path for ZeroTier node itself
+	 * @param lfOwnerPrivate LF owner private in PEM format
+	 * @param lfOwnerPublic LF owner public in @base62 format
+	 * @param lfNodeHost LF node host
+	 * @param lfNodePort LF node http (not https) port
+	 * @param storeOnlineState If true, store online/offline state and IP info in LF (a lot of data, only for private networks!)
+	 */
+	LFDB(const Identity &myId,const char *path,const char *lfOwnerPrivate,const char *lfOwnerPublic,const char *lfNodeHost,int lfNodePort,bool storeOnlineState);
+	virtual ~LFDB();
+
+	virtual bool waitForReady();
+	virtual bool isReady();
+	virtual bool save(nlohmann::json &record,bool notifyListeners);
+	virtual void eraseNetwork(const uint64_t networkId);
+	virtual void eraseMember(const uint64_t networkId,const uint64_t memberId);
+	virtual void nodeIsOnline(const uint64_t networkId,const uint64_t memberId,const InetAddress &physicalAddress);
+
+protected:
+	const Identity _myId;
+
+	std::string _lfOwnerPrivate,_lfOwnerPublic;
+	std::string _lfNodeHost;
+	int _lfNodePort;
+
+	struct _MemberState
+	{
+		_MemberState() :
+			lastOnlineAddress(),
+			lastOnlineTime(0),
+			dirty(false),
+			lastOnlineDirty(false) {}
+		InetAddress lastOnlineAddress;
+		int64_t lastOnlineTime;
+		bool dirty;
+		bool lastOnlineDirty;
+	};
+	struct _NetworkState
+	{
+		_NetworkState() :
+			members(),
+			dirty(false) {}
+		std::unordered_map<uint64_t,_MemberState> members;
+		bool dirty;
+	};
+	std::unordered_map<uint64_t,_NetworkState> _state;
+	std::mutex _state_l;
+
+	std::atomic_bool _running;
+	std::atomic_bool _ready;
+	std::thread _syncThread;
+	bool _storeOnlineState;
+};
+	
+} // namespace ZeroTier
+
+#endif
--- a/controller/PostgreSQL.cpp
+++ b/controller/PostgreSQL.cpp
@ -24,9 +24,11 @@
 * of your own application.
 */

+#include "PostgreSQL.hpp"
+
 #ifdef ZT_CONTROLLER_USE_LIBPQ

-#include "PostgreSQL.hpp"
+#include "../node/Constants.hpp"
 #include "EmbeddedNetworkController.hpp"
 #include "RabbitMQ.hpp"
 #include "../version.h"
@ -37,8 +39,11 @@
 #include <amqp_tcp_socket.h>

 using json = nlohmann::json;
+
 namespace {

+static const int DB_MINIMUM_VERSION = 5;
+
 static const char *_timestr()
 {
 	time_t t = time(0);
@ -56,6 +61,7 @@ static const char *_timestr()
 	return ts;
 }

+/*
 std::string join(const std::vector<std::string> &elements, const char * const separator)
 {
 	switch(elements.size()) {
@ -70,21 +76,56 @@ std::string join(const std::vector<std::string> &elements, const char * const se
 		return os.str();
 	}
 }
+*/

-}
+} // anonymous namespace

 using namespace ZeroTier;

-PostgreSQL::PostgreSQL(EmbeddedNetworkController *const nc, const Identity &myId, const char *path, int listenPort, MQConfig *mqc)
-    : DB(nc, myId, path)
-    , _ready(0)
+PostgreSQL::PostgreSQL(const Identity &myId, const char *path, int listenPort, MQConfig *mqc)
+	: DB()
+	, _myId(myId)
+	, _myAddress(myId.address())
+	, _ready(0)
 	, _connected(1)
-    , _run(1)
-    , _waitNoticePrinted(false)
+	, _run(1)
+	, _waitNoticePrinted(false)
 	, _listenPort(listenPort)
 	, _mqc(mqc)
 {
-	_connString = std::string(path) + " application_name=controller_" +_myAddressStr;
+	char myAddress[64];
+	_myAddressStr = myId.address().toString(myAddress);
+	_connString = std::string(path) + " application_name=controller_" + _myAddressStr;
+
+	// Database Schema Version Check
+	PGconn *conn = getPgConn();
+	if (PQstatus(conn) != CONNECTION_OK) {
+		fprintf(stderr, "Bad Database Connection: %s", PQerrorMessage(conn));
+		exit(1);
+	}
+
+	PGresult *res = PQexec(conn, "SELECT version FROM ztc_database");
+	if (PQresultStatus(res) != PGRES_TUPLES_OK) {
+		fprintf(stderr, "Error determining database version");
+		exit(1);
+	}
+
+	if (PQntuples(res) != 1) {
+		fprintf(stderr, "Invalid number of db version tuples returned.");
+		exit(1);
+	}
+
+	int dbVersion = std::stoi(PQgetvalue(res, 0, 0));
+
+	if (dbVersion < DB_MINIMUM_VERSION) {
+		fprintf(stderr, "Central database schema version too low.  This controller version requires a minimum schema version of %d. Please upgrade your Central instance", DB_MINIMUM_VERSION);
+		exit(1);
+	}
+
+	PQclear(res);
+	res = NULL;
+	PQfinish(conn);
+	conn = NULL;

 	_readyLock.lock();
 	_heartbeatThread = std::thread(&PostgreSQL::heartbeat, this);
@ -130,12 +171,38 @@ bool PostgreSQL::isReady()
 	return ((_ready == 2)&&(_connected));
 }

-void PostgreSQL::save(nlohmann::json *orig, nlohmann::json &record)
+bool PostgreSQL::save(nlohmann::json &record,bool notifyListeners)
 {
+	bool modified = false;
 	try {
-		if (!record.is_object()) {
-			return;
+		if (!record.is_object())
+			return false;
+		const std::string objtype = record["objtype"];
+		if (objtype == "network") {
+			const uint64_t nwid = OSUtils::jsonIntHex(record["id"],0ULL);
+			if (nwid) {
+				nlohmann::json old;
+				get(nwid,old);
+				if ((!old.is_object())||(old != record)) {
+					record["revision"] = OSUtils::jsonInt(record["revision"],0ULL) + 1ULL;
+					_commitQueue.post(std::pair<nlohmann::json,bool>(record,notifyListeners));
+					modified = true;
+				}
+			}
+		} else if (objtype == "member") {
+			const uint64_t nwid = OSUtils::jsonIntHex(record["nwid"],0ULL);
+			const uint64_t id = OSUtils::jsonIntHex(record["id"],0ULL);
+			if ((id)&&(nwid)) {
+				nlohmann::json network,old;
+				get(nwid,network,id,old);
+				if ((!old.is_object())||(old != record)) {
+					record["revision"] = OSUtils::jsonInt(record["revision"],0ULL) + 1ULL;
+					_commitQueue.post(std::pair<nlohmann::json,bool>(record,notifyListeners));
+					modified = true;
+				}
+			}
 		}
+		/*
 		waitForReady();
 		if (orig) {
 			if (*orig != record) {
@ -146,11 +213,13 @@ void PostgreSQL::save(nlohmann::json *orig, nlohmann::json &record)
 			record["revision"] = 1;
 			_commitQueue.post(new nlohmann::json(record));
 		}
+		*/
 	} catch (std::exception &e) {
 		fprintf(stderr, "Error on PostgreSQL::save: %s\n", e.what());
 	} catch (...) {
 		fprintf(stderr, "Unknown error on PostgreSQL::save\n");
 	}
+	return modified;
 }

 void PostgreSQL::eraseNetwork(const uint64_t networkId)
@ -158,21 +227,23 @@ void PostgreSQL::eraseNetwork(const uint64_t networkId)
 	char tmp2[24];
 	waitForReady();
 	Utils::hex(networkId, tmp2);
-	json *tmp = new json();
-	(*tmp)["id"] = tmp2;
-	(*tmp)["objtype"] = "_delete_network";
+	std::pair<nlohmann::json,bool> tmp;
+	tmp.first["id"] = tmp2;
+	tmp.first["objtype"] = "_delete_network";
+	tmp.second = true;
 	_commitQueue.post(tmp);
 }

 void PostgreSQL::eraseMember(const uint64_t networkId, const uint64_t memberId) 
 {
 	char tmp2[24];
-	json *tmp = new json();
+	std::pair<nlohmann::json,bool> tmp;
 	Utils::hex(networkId, tmp2);
-	(*tmp)["nwid"] = tmp2;
+	tmp.first["nwid"] = tmp2;
 	Utils::hex(memberId, tmp2);
-	(*tmp)["id"] = tmp2;
-	(*tmp)["objtype"] = "_delete_member";
+	tmp.first["id"] = tmp2;
+	tmp.first["objtype"] = "_delete_member";
+	tmp.second = true;
 	_commitQueue.post(tmp);
 }

@ -563,8 +634,8 @@ void PostgreSQL::heartbeat()
 				"public_identity = EXCLUDED.public_identity, v_major = EXCLUDED.v_major, v_minor = EXCLUDED.v_minor, "
 				"v_rev = EXCLUDED.v_rev, v_build = EXCLUDED.v_rev, host_port = EXCLUDED.host_port, "
 				"use_rabbitmq = EXCLUDED.use_rabbitmq",
-				10,       // number of parameters
-				NULL,    // oid field.   ignore
+				10,	   // number of parameters
+				NULL,	// oid field.   ignore
 				values,  // values for substitution
 				NULL, // lengths in bytes of each value
 				NULL,  // binary?
@ -685,7 +756,7 @@ void PostgreSQL::_membersWatcher_RabbitMQ() {
 			fprintf(stderr, "RABBITMQ ERROR member change: %s\n", e.what());
 		} catch(...) {
 			fprintf(stderr, "RABBITMQ ERROR member change: unknown error\n");
-        }
+		}
 	}
 }

@ -802,18 +873,18 @@ void PostgreSQL::commitThread()
 		exit(1);
 	}

-	json *config = nullptr;
-	while(_commitQueue.get(config)&(_run == 1)) {
-		if (!config) {
+	std::pair<nlohmann::json,bool> qitem;
+	while(_commitQueue.get(qitem)&(_run == 1)) {
+		if (!qitem.first.is_object()) {
 			continue;
 		}
 		if (PQstatus(conn) == CONNECTION_BAD) {
 			fprintf(stderr, "ERROR: Connection to database failed: %s\n", PQerrorMessage(conn));
 			PQfinish(conn);
-			delete config;
 			exit(1);
 		}
-		try { 
+		try {
+			nlohmann::json *config = &(qitem.first);
 			const std::string objtype = (*config)["objtype"];
 			if (objtype == "member") {
 				try {
@ -968,12 +1039,12 @@ void PostgreSQL::commitThread()
 						nlohmann::json memOrig;

 						nlohmann::json memNew(*config);
-						
+
 						get(nwidInt, nwOrig, memberidInt, memOrig);
 				
-						_memberChanged(memOrig, memNew, (this->_ready>=2));
+						_memberChanged(memOrig, memNew, qitem.second);
 					} else {
-						fprintf(stderr, "Can't notify of change.  Error parsing nwid or memberid: %lu-%lu\n", nwidInt, memberidInt);
+						fprintf(stderr, "Can't notify of change.  Error parsing nwid or memberid: %llu-%llu\n", (unsigned long long)nwidInt, (unsigned long long)memberidInt);
 					}

 				} catch (std::exception &e) {
@ -1194,16 +1265,14 @@ void PostgreSQL::commitThread()

 						get(nwidInt, nwOrig);

-						_networkChanged(nwOrig, nwNew, true);
+						_networkChanged(nwOrig, nwNew, qitem.second);
 					} else {
-						fprintf(stderr, "Can't notify network changed: %lu\n", nwidInt);
+						fprintf(stderr, "Can't notify network changed: %llu\n", (unsigned long long)nwidInt);
 					}

 				} catch (std::exception &e) {
 					fprintf(stderr, "ERROR: Error updating member: %s\n", e.what());
 				}
-			} else if (objtype == "trace") {
-				fprintf(stderr, "ERROR: Trace not yet implemented");
 			} else if (objtype == "_delete_network") {
 				try {
 					std::string networkId = (*config)["nwid"];
@ -1260,8 +1329,6 @@ void PostgreSQL::commitThread()
 		} catch (std::exception &e) {
 			fprintf(stderr, "ERROR: Error getting objtype: %s\n", e.what());
 		}
-		delete config;
-		config = nullptr;

 		std::this_thread::sleep_for(std::chrono::milliseconds(10));
 	}
@ -1283,9 +1350,9 @@ void PostgreSQL::onlineNotificationThread()
 	}
 	_connected = 1;

-	int64_t	lastUpdatedNetworkStatus = 0;
+	//int64_t	lastUpdatedNetworkStatus = 0;
 	std::unordered_map< std::pair<uint64_t,uint64_t>,int64_t,_PairHasher > lastOnlineCumulative;
-	
+
 	while (_run == 1) {
 		if (PQstatus(conn) != CONNECTION_OK) {
 			fprintf(stderr, "ERROR: Online Notification thread lost connection to Postgres.");
@ -1389,129 +1456,6 @@ void PostgreSQL::onlineNotificationThread()
 			PQclear(res);
 		}

-		const int64_t now = OSUtils::now();
-		if ((now - lastUpdatedNetworkStatus) > 10000) {
-			lastUpdatedNetworkStatus = now;
-
-			std::vector<std::pair<uint64_t, std::shared_ptr<_Network>>> networks;
-			{
-				std::lock_guard<std::mutex> l(_networks_l);
-				for (auto i = _networks.begin(); i != _networks.end(); ++i) {
-					networks.push_back(*i);
-				}
-			}
-
-			std::stringstream networkUpdate;
-			networkUpdate << "INSERT INTO ztc_network_status (network_id, bridge_count, authorized_member_count, online_member_count, total_member_count, last_modified) VALUES ";
-			bool nwFirstRun = true;
-			bool networkAdded = false;
-			for (auto i = networks.begin(); i != networks.end(); ++i) {
-				char tmp[64];
-				Utils::hex(i->first, tmp);
-
-				std::string networkId(tmp);
-
-				std::vector<std::string> &_notUsed = updateMap[networkId];
-				(void)_notUsed;
-
-				uint64_t authMemberCount = 0;
-				uint64_t totalMemberCount = 0;
-				uint64_t onlineMemberCount = 0;
-				uint64_t bridgeCount = 0;
-				uint64_t ts = now;
-				{
-					std::lock_guard<std::mutex> l2(i->second->lock);
-					authMemberCount = i->second->authorizedMembers.size();
-					totalMemberCount = i->second->members.size();
-					bridgeCount = i->second->activeBridgeMembers.size();
-					for (auto m=i->second->members.begin(); m != i->second->members.end(); ++m) {
-						auto lo = lastOnlineCumulative.find(std::pair<uint64_t,uint64_t>(i->first, m->first));
-						if (lo != lastOnlineCumulative.end()) {
-							if ((now - lo->second) <= (ZT_NETWORK_AUTOCONF_DELAY * 2)) {
-								++onlineMemberCount;
-							} else {
-								lastOnlineCumulative.erase(lo);
-							}
-						}
-					}
-				}
-
-				const char *nvals[1] = {
-					networkId.c_str()
-				};
-
-				res = PQexecParams(conn,
-					"SELECT id FROM ztc_network WHERE id = $1",
-					1,
-					NULL,
-					nvals,
-					NULL,
-					NULL,
-					0);
-
-				if (PQresultStatus(res) != PGRES_TUPLES_OK) {
-					fprintf(stderr, "Network lookup failed: %s", PQerrorMessage(conn));
-					PQclear(res);
-					continue;
-				}
-
-				int nrows = PQntuples(res);
-				PQclear(res);
-
-				if (nrows == 1) {
-					std::string bc = std::to_string(bridgeCount);
-					std::string amc = std::to_string(authMemberCount);
-					std::string omc = std::to_string(onlineMemberCount);
-					std::string tmc = std::to_string(totalMemberCount);
-					std::string timestamp = std::to_string(ts);
-
-					if (nwFirstRun) {
-						nwFirstRun = false;
-					} else {
-						networkUpdate << ", ";
-					}
-
-					networkUpdate << "('" << networkId << "', " << bc << ", " << amc << ", " << omc << ", " << tmc << ", "
-							<< "TO_TIMESTAMP(" << timestamp << "::double precision/1000))";
-
-					networkAdded = true;
-
-				} else if (nrows > 1) {
-					fprintf(stderr, "Number of networks > 1?!?!?");
-					continue;
-				} else {
-					continue;
-				}
-			}
-			networkUpdate << " ON CONFLICT (network_id) DO UPDATE SET bridge_count = EXCLUDED.bridge_count, "
-					<< "authorized_member_count = EXCLUDED.authorized_member_count, online_member_count = EXCLUDED.online_member_count, "
-					<< "total_member_count = EXCLUDED.total_member_count, last_modified = EXCLUDED.last_modified";
-			if (networkAdded) {
-				res = PQexec(conn, networkUpdate.str().c_str());
-				if (PQresultStatus(res) != PGRES_COMMAND_OK) {
-					fprintf(stderr, "Error during multiple network upsert: %s", PQresultErrorMessage(res));
-				}
-				PQclear(res);
-			}
-		}
-
-		// for (auto it = updateMap.begin(); it != updateMap.end(); ++it) {
-		// 	std::string networkId = it->first;
-		// 	std::vector<std::string> members = it->second;
-		// 	std::stringstream queryBuilder;
-
-		// 	std::string membersStr = ::join(members, ",");
-
-		// 	queryBuilder << "NOTIFY controller, '" << networkId << ":" << membersStr << "'";
-		// 	std::string query = queryBuilder.str();
-
-		// 	PGresult *res = PQexec(conn,query.c_str());
-		// 	if (PQresultStatus(res) != PGRES_COMMAND_OK) {
-		// 		fprintf(stderr, "ERROR: Error sending NOTIFY: %s\n", PQresultErrorMessage(res));
-		// 	}
-		// 	PQclear(res);
-		// }
-
 		std::this_thread::sleep_for(std::chrono::milliseconds(10));
 	}
 	fprintf(stderr, "%s: Fell out of run loop in onlineNotificationThread\n", _myAddressStr.c_str());
@ -1522,7 +1466,8 @@ void PostgreSQL::onlineNotificationThread()
 	}
 }

-PGconn *PostgreSQL::getPgConn(OverrideMode m) {
+PGconn *PostgreSQL::getPgConn(OverrideMode m)
+{
 	if (m == ALLOW_PGBOUNCER_OVERRIDE) {
 		char *connStr = getenv("PGBOUNCER_CONNSTR");
 		if (connStr != NULL) {
@ -1536,4 +1481,5 @@ PGconn *PostgreSQL::getPgConn(OverrideMode m) {

 	return PQconnectdb(_connString.c_str());
 }
+
 #endif //ZT_CONTROLLER_USE_LIBPQ
--- a/controller/PostgreSQL.hpp
+++ b/controller/PostgreSQL.hpp
@ -23,22 +23,21 @@
 * directly against ZeroTier software without disclosing the source code
 * of your own application.
 */
- 
+
+#include "DB.hpp"
+
 #ifdef ZT_CONTROLLER_USE_LIBPQ

 #ifndef ZT_CONTROLLER_LIBPQ_HPP
 #define ZT_CONTROLLER_LIBPQ_HPP

-#include "DB.hpp"
-
 #define ZT_CENTRAL_CONTROLLER_COMMIT_THREADS 4

 extern "C" {
-    typedef struct pg_conn PGconn;
+typedef struct pg_conn PGconn;
 }

-namespace ZeroTier
-{
+namespace ZeroTier {

 struct MQConfig;

@ -51,69 +50,70 @@ struct MQConfig;
 class PostgreSQL : public DB
 {
 public:
-    PostgreSQL(EmbeddedNetworkController *const nc, const Identity &myId, const char *path, int listenPort, MQConfig *mqc = NULL);
-    virtual ~PostgreSQL();
+	PostgreSQL(const Identity &myId, const char *path, int listenPort, MQConfig *mqc = NULL);
+	virtual ~PostgreSQL();

-    virtual bool waitForReady();
-    virtual bool isReady();
-    virtual void save(nlohmann::json *orig, nlohmann::json &record);
-    virtual void eraseNetwork(const uint64_t networkId);
-    virtual void eraseMember(const uint64_t networkId, const uint64_t memberId);
-    virtual void nodeIsOnline(const uint64_t networkId, const uint64_t memberId, const InetAddress &physicalAddress);
+	virtual bool waitForReady();
+	virtual bool isReady();
+	virtual bool save(nlohmann::json &record,bool notifyListeners);
+	virtual void eraseNetwork(const uint64_t networkId);
+	virtual void eraseMember(const uint64_t networkId, const uint64_t memberId);
+	virtual void nodeIsOnline(const uint64_t networkId, const uint64_t memberId, const InetAddress &physicalAddress);

 protected:
-    struct _PairHasher
+	struct _PairHasher
 	{
 		inline std::size_t operator()(const std::pair<uint64_t,uint64_t> &p) const { return (std::size_t)(p.first ^ p.second); }
 	};

 private:
-    void initializeNetworks(PGconn *conn);
-    void initializeMembers(PGconn *conn);
-    void heartbeat();
-    void membersDbWatcher();
-    void _membersWatcher_Postgres(PGconn *conn);
-    void _membersWatcher_RabbitMQ();
-    void networksDbWatcher();
-    void _networksWatcher_Postgres(PGconn *conn);
-    void _networksWatcher_RabbitMQ();
+	void initializeNetworks(PGconn *conn);
+	void initializeMembers(PGconn *conn);
+	void heartbeat();
+	void membersDbWatcher();
+	void _membersWatcher_Postgres(PGconn *conn);
+	void _membersWatcher_RabbitMQ();
+	void networksDbWatcher();
+	void _networksWatcher_Postgres(PGconn *conn);
+	void _networksWatcher_RabbitMQ();

+	void commitThread();
+	void onlineNotificationThread();

-    void commitThread();
-    void onlineNotificationThread();
+	enum OverrideMode {
+		ALLOW_PGBOUNCER_OVERRIDE = 0,
+		NO_OVERRIDE = 1
+	};

-    enum OverrideMode {
-        ALLOW_PGBOUNCER_OVERRIDE = 0,
-        NO_OVERRIDE = 1
-    };
+	PGconn * getPgConn( OverrideMode m = ALLOW_PGBOUNCER_OVERRIDE );

-    PGconn * getPgConn( OverrideMode m = ALLOW_PGBOUNCER_OVERRIDE );
+	const Identity _myId;
+	const Address _myAddress;
+	std::string _myAddressStr;
+	std::string _connString;

-    std::string _connString;
+	BlockingQueue< std::pair<nlohmann::json,bool> > _commitQueue;

-    BlockingQueue<nlohmann::json *> _commitQueue;
-
-
-    std::thread _heartbeatThread;
-    std::thread _membersDbWatcher;
-    std::thread _networksDbWatcher;
-    std::thread _commitThread[ZT_CENTRAL_CONTROLLER_COMMIT_THREADS];
-    std::thread _onlineNotificationThread;
+	std::thread _heartbeatThread;
+	std::thread _membersDbWatcher;
+	std::thread _networksDbWatcher;
+	std::thread _commitThread[ZT_CENTRAL_CONTROLLER_COMMIT_THREADS];
+	std::thread _onlineNotificationThread;

 	std::unordered_map< std::pair<uint64_t,uint64_t>,std::pair<int64_t,InetAddress>,_PairHasher > _lastOnline;

-    mutable std::mutex _lastOnline_l;
-    mutable std::mutex _readyLock;
-    std::atomic<int> _ready, _connected, _run;
-    mutable volatile bool _waitNoticePrinted;
+	mutable std::mutex _lastOnline_l;
+	mutable std::mutex _readyLock;
+	std::atomic<int> _ready, _connected, _run;
+	mutable volatile bool _waitNoticePrinted;

-    int _listenPort;
+	int _listenPort;

-    MQConfig *_mqc;
+	MQConfig *_mqc;
 };

-}
+} // namespace ZeroTier

 #endif // ZT_CONTROLLER_LIBPQ_HPP

-#endif // ZT_CONTROLLER_USE_LIBPQ
+#endif // ZT_CONTROLLER_USE_LIBPQ
--- a/controller/RabbitMQ.cpp
+++ b/controller/RabbitMQ.cpp
@ -1,3 +1,30 @@
+/*
+ * ZeroTier One - Network Virtualization Everywhere
+ * Copyright (C) 2011-2019  ZeroTier, Inc.  https://www.zerotier.com/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * You can be released from the requirements of the license by purchasing
+ * a commercial license. Buying such a license is mandatory as soon as you
+ * develop commercial closed-source software that incorporates or links
+ * directly against ZeroTier software without disclosing the source code
+ * of your own application.
+ */
+
+
 #include "RabbitMQ.hpp"

 #ifdef ZT_CONTROLLER_USE_LIBPQ
@ -11,95 +38,95 @@ namespace ZeroTier
 {

 RabbitMQ::RabbitMQ(MQConfig *cfg, const char *queueName)
-    : _mqc(cfg)
-    , _qName(queueName)
-    , _socket(NULL)
-    , _status(0)
+	: _mqc(cfg)
+	, _qName(queueName)
+	, _socket(NULL)
+	, _status(0)
 {   
 }

 RabbitMQ::~RabbitMQ()
 {
-    amqp_channel_close(_conn, _channel, AMQP_REPLY_SUCCESS);
-    amqp_connection_close(_conn, AMQP_REPLY_SUCCESS);
-    amqp_destroy_connection(_conn);
+	amqp_channel_close(_conn, _channel, AMQP_REPLY_SUCCESS);
+	amqp_connection_close(_conn, AMQP_REPLY_SUCCESS);
+	amqp_destroy_connection(_conn);
 }

 void RabbitMQ::init()
 {
-    struct timeval tval;
-    memset(&tval, 0, sizeof(struct timeval));
-    tval.tv_sec = 5;
+	struct timeval tval;
+	memset(&tval, 0, sizeof(struct timeval));
+	tval.tv_sec = 5;

-    fprintf(stderr, "Initializing RabbitMQ %s\n", _qName);
-    _conn = amqp_new_connection();
-    _socket = amqp_tcp_socket_new(_conn);
-    if (!_socket) {
-        throw std::runtime_error("Can't create socket for RabbitMQ");
-    }
-    
-    _status = amqp_socket_open_noblock(_socket, _mqc->host, _mqc->port, &tval);
-    if (_status) {
-        throw std::runtime_error("Can't connect to RabbitMQ");
-    }
-    
-    amqp_rpc_reply_t r = amqp_login(_conn, "/", 0, 131072, 0, AMQP_SASL_METHOD_PLAIN,
-        _mqc->username, _mqc->password);
-    if (r.reply_type != AMQP_RESPONSE_NORMAL) {
-        throw std::runtime_error("RabbitMQ Login Error");
-    }
+	fprintf(stderr, "Initializing RabbitMQ %s\n", _qName);
+	_conn = amqp_new_connection();
+	_socket = amqp_tcp_socket_new(_conn);
+	if (!_socket) {
+		throw std::runtime_error("Can't create socket for RabbitMQ");
+	}
+	
+	_status = amqp_socket_open_noblock(_socket, _mqc->host, _mqc->port, &tval);
+	if (_status) {
+		throw std::runtime_error("Can't connect to RabbitMQ");
+	}
+	
+	amqp_rpc_reply_t r = amqp_login(_conn, "/", 0, 131072, 0, AMQP_SASL_METHOD_PLAIN,
+		_mqc->username, _mqc->password);
+	if (r.reply_type != AMQP_RESPONSE_NORMAL) {
+		throw std::runtime_error("RabbitMQ Login Error");
+	}

-    static int chan = 0;
+	static int chan = 0;
 	{
 		Mutex::Lock l(_chan_m);
-    	_channel = ++chan;
+		_channel = ++chan;
+	}
+	amqp_channel_open(_conn, _channel);
+	r = amqp_get_rpc_reply(_conn);
+	if(r.reply_type != AMQP_RESPONSE_NORMAL) {
+		throw std::runtime_error("Error opening communication channel");
+	}
+	
+	_q = amqp_queue_declare(_conn, _channel, amqp_cstring_bytes(_qName), 0, 0, 0, 0, amqp_empty_table);
+	r = amqp_get_rpc_reply(_conn);
+	if (r.reply_type != AMQP_RESPONSE_NORMAL) {
+		throw std::runtime_error("Error declaring queue " + std::string(_qName));
 	}
-    amqp_channel_open(_conn, _channel);
-    r = amqp_get_rpc_reply(_conn);
-    if(r.reply_type != AMQP_RESPONSE_NORMAL) {
-        throw std::runtime_error("Error opening communication channel");
-    }
-    
-    _q = amqp_queue_declare(_conn, _channel, amqp_cstring_bytes(_qName), 0, 0, 0, 0, amqp_empty_table);
-    r = amqp_get_rpc_reply(_conn);
-    if (r.reply_type != AMQP_RESPONSE_NORMAL) {
-        throw std::runtime_error("Error declaring queue " + std::string(_qName));
-    }

-    amqp_basic_consume(_conn, _channel, amqp_cstring_bytes(_qName), amqp_empty_bytes, 0, 1, 0, amqp_empty_table);
-    r = amqp_get_rpc_reply(_conn);
-    if (r.reply_type != AMQP_RESPONSE_NORMAL) {
-        throw std::runtime_error("Error consuming queue " + std::string(_qName));
-    }
-    fprintf(stderr, "RabbitMQ Init OK %s\n", _qName);
+	amqp_basic_consume(_conn, _channel, amqp_cstring_bytes(_qName), amqp_empty_bytes, 0, 1, 0, amqp_empty_table);
+	r = amqp_get_rpc_reply(_conn);
+	if (r.reply_type != AMQP_RESPONSE_NORMAL) {
+		throw std::runtime_error("Error consuming queue " + std::string(_qName));
+	}
+	fprintf(stderr, "RabbitMQ Init OK %s\n", _qName);
 }

 std::string RabbitMQ::consume()
 {
-    amqp_rpc_reply_t res;
-    amqp_envelope_t envelope;
-    amqp_maybe_release_buffers(_conn);
+	amqp_rpc_reply_t res;
+	amqp_envelope_t envelope;
+	amqp_maybe_release_buffers(_conn);

-    struct timeval timeout;
-    timeout.tv_sec = 1;
-    timeout.tv_usec = 0;
+	struct timeval timeout;
+	timeout.tv_sec = 1;
+	timeout.tv_usec = 0;

-    res = amqp_consume_message(_conn, &envelope, &timeout, 0);
-    if (res.reply_type != AMQP_RESPONSE_NORMAL) {
-        if (res.reply_type == AMQP_RESPONSE_LIBRARY_EXCEPTION && res.library_error == AMQP_STATUS_TIMEOUT) {
-            // timeout waiting for message.  Return empty string
-            return "";
-        } else {
-            throw std::runtime_error("Error getting message");
-        }
-    }
+	res = amqp_consume_message(_conn, &envelope, &timeout, 0);
+	if (res.reply_type != AMQP_RESPONSE_NORMAL) {
+		if (res.reply_type == AMQP_RESPONSE_LIBRARY_EXCEPTION && res.library_error == AMQP_STATUS_TIMEOUT) {
+			// timeout waiting for message.  Return empty string
+			return "";
+		} else {
+			throw std::runtime_error("Error getting message");
+		}
+	}

-    std::string msg(
-        (const char*)envelope.message.body.bytes,
-        envelope.message.body.len
-    );
-    amqp_destroy_envelope(&envelope);
-    return msg;
+	std::string msg(
+		(const char*)envelope.message.body.bytes,
+		envelope.message.body.len
+	);
+	amqp_destroy_envelope(&envelope);
+	return msg;
 }

 }
--- a/controller/RabbitMQ.hpp
+++ b/controller/RabbitMQ.hpp
@ -23,16 +23,19 @@
 * directly against ZeroTier software without disclosing the source code
 * of your own application.
 */
+
 #ifndef ZT_CONTROLLER_RABBITMQ_HPP
 #define ZT_CONTROLLER_RABBITMQ_HPP

+#include "DB.hpp"
+
 namespace ZeroTier
 {
 struct MQConfig {
-    const char *host;
-    int port;
-    const char *username;
-    const char *password;
+	const char *host;
+	int port;
+	const char *username;
+	const char *password;
 };
 }

@ -49,26 +52,25 @@ namespace ZeroTier

 class RabbitMQ {
 public:
-    RabbitMQ(MQConfig *cfg, const char *queueName);
-    ~RabbitMQ();
+	RabbitMQ(MQConfig *cfg, const char *queueName);
+	~RabbitMQ();

-    void init();
+	void init();

-    std::string consume();
+	std::string consume();

 private:
-    MQConfig *_mqc;
-    const char *_qName;
+	MQConfig *_mqc;
+	const char *_qName;

-    amqp_socket_t *_socket;
-    amqp_connection_state_t _conn;
-    amqp_queue_declare_ok_t *_q;
-    int _status;
+	amqp_socket_t *_socket;
+	amqp_connection_state_t _conn;
+	amqp_queue_declare_ok_t *_q;
+	int _status;

-    int _channel;
+	int _channel;

 	Mutex _chan_m;
-
 };

 }
--- a/debian/changelog
+++ b/debian/changelog
@ -1,8 +1,14 @@
-zerotier-one (1.2.99) unstable; urgency=medium
+zerotier-one (1.4.2) unstable; urgency=medium

-  * 1.4.0pre release
+  * See https://github.com/zerotier/ZeroTierOne for release notes.

- -- Adam Ierymenko <adam.ierymenko@zerotier.com>  Thu, 27 Jun 2019 01:00:00 -0700
+ -- Adam Ierymenko <adam.ierymenko@zerotier.com>  Thu, 04 Aug 2019 01:00:00 -0700
+
+zerotier-one (1.4.0) unstable; urgency=medium
+
+  * See https://github.com/zerotier/ZeroTierOne for release notes.
+
+ -- Adam Ierymenko <adam.ierymenko@zerotier.com>  Thu, 29 Jul 2019 01:00:00 -0700

 zerotier-one (1.2.12) unstable; urgency=medium

--- a/ext/cpp-httplib/LICENSE
+++ b/ext/cpp-httplib/LICENSE
@ -0,0 +1,22 @@
+The MIT License (MIT)
+
+Copyright (c) 2017 yhirose
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
--- a/ext/cpp-httplib/README.md
+++ b/ext/cpp-httplib/README.md
@ -0,0 +1,259 @@
+cpp-httplib
+===========
+
+[![Build Status](https://travis-ci.org/yhirose/cpp-httplib.svg?branch=master)](https://travis-ci.org/yhirose/cpp-httplib)
+[![Bulid Status](https://ci.appveyor.com/api/projects/status/github/yhirose/cpp-httplib?branch=master&svg=true)](https://ci.appveyor.com/project/yhirose/cpp-httplib)
+
+A C++ header-only cross platform HTTP/HTTPS library.
+
+It's extremely easy to setup. Just include **httplib.h** file in your code!
+
+Inspired by [Sinatra](http://www.sinatrarb.com/) and [express](https://github.com/visionmedia/express).
+
+Server Example
+--------------
+
+```c++
+#include <httplib.h>
+
+int main(void)
+{
+    using namespace httplib;
+
+    Server svr;
+
+    svr.Get("/hi", [](const Request& req, Response& res) {
+        res.set_content("Hello World!", "text/plain");
+    });
+
+    svr.Get(R"(/numbers/(\d+))", [&](const Request& req, Response& res) {
+        auto numbers = req.matches[1];
+        res.set_content(numbers, "text/plain");
+    });
+
+    svr.listen("localhost", 1234);
+}
+```
+
+`Post`, `Put`, `Delete` and `Options` methods are also supported.
+
+### Bind a socket to multiple interfaces and any available port
+
+```cpp
+int port = svr.bind_to_any_port("0.0.0.0");
+svr.listen_after_bind();
+```
+
+### Method Chain
+
+```cpp
+svr.Get("/get", [](const auto& req, auto& res) {
+        res.set_content("get", "text/plain");
+    })
+    .Post("/post", [](const auto& req, auto& res) {
+        res.set_content(req.body(), "text/plain");
+    })
+    .listen("localhost", 1234);
+```
+
+### Static File Server
+
+```cpp
+svr.set_base_dir("./www");
+```
+
+### Logging
+
+```cpp
+svr.set_logger([](const auto& req, const auto& res) {
+    your_logger(req, res);
+});
+```
+
+### Error Handler
+
+```cpp
+svr.set_error_handler([](const auto& req, auto& res) {
+    const char* fmt = "<p>Error Status: <span style='color:red;'>%d</span></p>";
+    char buf[BUFSIZ];
+    snprintf(buf, sizeof(buf), fmt, res.status);
+    res.set_content(buf, "text/html");
+});
+```
+
+### 'multipart/form-data' POST data
+
+```cpp
+svr.Post("/multipart", [&](const auto& req, auto& res) {
+    auto size = req.files.size();
+    auto ret = req.has_file("name1"));
+    const auto& file = req.get_file_value("name1");
+    // file.filename;
+    // file.content_type;
+    auto body = req.body.substr(file.offset, file.length));
+})
+```
+
+Client Example
+--------------
+
+### GET
+
+```c++
+#include <httplib.h>
+#include <iostream>
+
+int main(void)
+{
+    httplib::Client cli("localhost", 1234);
+
+    auto res = cli.Get("/hi");
+    if (res && res->status == 200) {
+        std::cout << res->body << std::endl;
+    }
+}
+```
+
+### GET with Content Receiver
+
+```c++
+  std::string body;
+  auto res = cli.Get("/large-data", [&](const char *data, size_t len) {
+    body.append(data, len);
+  });
+  assert(res->body.empty());
+```
+
+### POST
+
+```c++
+res = cli.Post("/post", "text", "text/plain");
+res = cli.Post("/person", "name=john1&note=coder", "application/x-www-form-urlencoded");
+```
+
+### POST with parameters
+
+```c++
+httplib::Params params;
+params.emplace("name", "john");
+params.emplace("note", "coder");
+
+auto res = cli.Post("/post", params);
+```
+ or
+
+```c++
+httplib::Params params{
+  { "name", "john" },
+  { "note", "coder" }
+};
+
+auto res = cli.Post("/post", params);
+```
+
+### PUT
+
+```c++
+res = cli.Put("/resource/foo", "text", "text/plain");
+```
+
+### DELETE
+
+```c++
+res = cli.Delete("/resource/foo");
+```
+
+### OPTIONS
+
+```c++
+res = cli.Options("*");
+res = cli.Options("/resource/foo");
+```
+
+### Connection Timeout
+
+```c++
+httplib::Client cli("localhost", 8080, 5); // timeouts in 5 seconds
+```
+### With Progress Callback
+
+```cpp
+httplib::Client client(url, port);
+
+// prints: 0 / 000 bytes => 50% complete
+std::shared_ptr<httplib::Response> res =
+    cli.Get("/", [](uint64_t len, uint64_t total) {
+        printf("%lld / %lld bytes => %d%% complete\n",
+            len, total,
+            (int)((len/total)*100));
+        return true; // return 'false' if you want to cancel the request.
+    }
+);
+```
+
+![progress](https://user-images.githubusercontent.com/236374/33138910-495c4ecc-cf86-11e7-8693-2fc6d09615c4.gif)
+
+This feature was contributed by [underscorediscovery](https://github.com/yhirose/cpp-httplib/pull/23).
+
+### Basic Authentication
+
+```cpp
+httplib::Client cli("httplib.org");
+
+auto res = cli.Get("/basic-auth/hello/world", {
+  httplib::make_basic_authentication_header("hello", "world")
+});
+// res->status should be 200
+// res->body should be "{\n  \"authenticated\": true, \n  \"user\": \"hello\"\n}\n".
+```
+
+### Range
+
+```cpp
+httplib::Client cli("httpbin.org");
+
+auto res = cli.Get("/range/32", {
+  httplib::make_range_header(1, 10) // 'Range: bytes=1-10'
+});
+// res->status should be 206.
+// res->body should be "bcdefghijk".
+```
+
+OpenSSL Support
+---------------
+
+SSL support is available with `CPPHTTPLIB_OPENSSL_SUPPORT`. `libssl` and `libcrypto` should be linked.
+
+```c++
+#define CPPHTTPLIB_OPENSSL_SUPPORT
+
+SSLServer svr("./cert.pem", "./key.pem");
+
+SSLClient cli("localhost", 8080);
+cli.set_ca_cert_path("./ca-bundle.crt");
+cli.enable_server_certificate_verification(true);
+```
+
+Zlib Support
+------------
+
+'gzip' compression is available with `CPPHTTPLIB_ZLIB_SUPPORT`.
+
+The server applies gzip compression to the following MIME type contents:
+
+  * all text types
+  * image/svg+xml
+  * application/javascript
+  * application/json
+  * application/xml
+  * application/xhtml+xml
+
+NOTE
+----
+
+g++ 4.8 cannot build this library since `<regex>` in g++4.8 is [broken](https://stackoverflow.com/questions/12530406/is-gcc-4-8-or-earlier-buggy-about-regular-expressions).
+
+License
+-------
+
+MIT license (© 2019 Yuji Hirose)
--- a/ext/cpp-httplib/httplib.h
+++ b/ext/cpp-httplib/httplib.h
--- a/ext/installfiles/linux/zerotier-containerized/Dockerfile
+++ b/ext/installfiles/linux/zerotier-containerized/Dockerfile
@ -1,30 +1,32 @@
-FROM debian:stretch as builder
+## NOTE: to retain configuration; mount a Docker volume, or use a bind-mount, on /var/lib/zerotier-one
+
+FROM debian:buster-slim as builder

 ## Supports x86_64, x86, arm, and arm64

 RUN apt-get update && apt-get install -y curl gnupg
 RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys 0x1657198823e52a61  && \
-    echo "deb http://download.zerotier.com/debian/stretch stretch main" > /etc/apt/sources.list.d/zerotier.list
+    echo "deb http://download.zerotier.com/debian/buster buster main" > /etc/apt/sources.list.d/zerotier.list
 RUN apt-get update && apt-get install -y zerotier-one=1.2.12
+RUN curl https://raw.githubusercontent.com/zerotier/ZeroTierOne/master/ext/installfiles/linux/zerotier-containerized/main.sh > /var/lib/zerotier-one/main.sh

 FROM alpine:latest
-MAINTAINER Adam Ierymenko <adam.ierymenko@zerotier.com>
-
 LABEL version="1.2.12"
 LABEL description="Containerized ZeroTier One for use on CoreOS or other Docker-only Linux hosts."

 # Uncomment to build in container
-#RUN apk add --update alpine-sdk linux-headers
-
+# RUN apk add --update alpine-sdk linux-headers
 RUN apk add --update libgcc libstdc++

+# ZeroTier relies on UDP port 9993
+EXPOSE 9993/udp
+
 RUN mkdir -p /var/lib/zerotier-one
-
-COPY --from=builder /var/lib/zerotier-one/zerotier-cli /usr/sbin/zerotier-cli
-COPY --from=builder /var/lib/zerotier-one/zerotier-idtool /usr/sbin/zerotier-idtool
+COPY --from=builder /usr/sbin/zerotier-cli /usr/sbin/zerotier-cli
+COPY --from=builder /usr/sbin/zerotier-idtool /usr/sbin/zerotier-idtool
 COPY --from=builder /usr/sbin/zerotier-one /usr/sbin/zerotier-one
+COPY --from=builder /var/lib/zerotier-one/main.sh /main.sh

-ADD main.sh /
 RUN chmod 0755 /main.sh
 ENTRYPOINT ["/main.sh"]
-CMD ["zerotier-one"]
+CMD ["zerotier-one"]
--- a/ext/installfiles/mac/ZeroTier
+++ b/ext/installfiles/mac/ZeroTier
@ -664,7 +664,7 @@
 			<key>USE_HFS+_COMPRESSION</key>
 			<false/>
 			<key>VERSION</key>
-			<string>1.2.99</string>
+			<string>1.4.2</string>
 		</dict>
 		<key>PROJECT_COMMENTS</key>
 		<dict>
--- a/ext/installfiles/windows/ZeroTier
+++ b/ext/installfiles/windows/ZeroTier
@ -27,10 +27,10 @@
    <ROW Property="CTRLS" Value="2"/>
    <ROW Property="MSIFASTINSTALL" MultiBuildValue="DefaultBuild:2"/>
    <ROW Property="Manufacturer" Value="ZeroTier, Inc."/>
-    <ROW Property="ProductCode" Value="1033:{FF2B4EDA-7B30-41A3-8A3F-BF81E1085234} " Type="16"/>
+    <ROW Property="ProductCode" Value="1033:{E5E38B77-644B-48BA-A120-3CA86EECA9B1} " Type="16"/>
    <ROW Property="ProductLanguage" Value="1033"/>
    <ROW Property="ProductName" Value="ZeroTier One"/>
-    <ROW Property="ProductVersion" Value="1.2.99" Type="32"/>
+    <ROW Property="ProductVersion" Value="1.4.2" Type="32"/>
    <ROW Property="REBOOT" MultiBuildValue="DefaultBuild:ReallySuppress"/>
    <ROW Property="RUNAPPLICATION" Value="1" Type="4"/>
    <ROW Property="SecureCustomProperties" Value="OLDPRODUCTS;AI_NEWERPRODUCTFOUND;AI_SETUPEXEPATH;SETUPEXEDIR"/>
@ -64,7 +64,7 @@
    <ROW Directory="x86_Dir" Directory_Parent="tapwindows_Dir" DefaultDir="x86"/>
  </COMPONENT>
  <COMPONENT cid="caphyon.advinst.msicomp.MsiCompsComponent">
-    <ROW Component="AI_CustomARPName" ComponentId="{6CAB31CD-CEDA-4B74-A52D-3D35D4A426D8}" Directory_="APPDIR" Attributes="4" KeyPath="DisplayName" Options="1"/>
+    <ROW Component="AI_CustomARPName" ComponentId="{A660DE2F-81EF-45AB-AC9B-BAD4AB0E806D}" Directory_="APPDIR" Attributes="4" KeyPath="DisplayName" Options="1"/>
    <ROW Component="AI_DisableModify" ComponentId="{020DCABD-5D56-49B9-AF48-F07F0B55E590}" Directory_="APPDIR" Attributes="4" KeyPath="NoModify" Options="1"/>
    <ROW Component="AI_ExePath" ComponentId="{8E02B36C-7A19-429B-A93E-77A9261AC918}" Directory_="APPDIR" Attributes="4" KeyPath="AI_ExePath"/>
    <ROW Component="Hardcodet.Wpf.TaskbarNotification.dll" ComponentId="{BEA825AF-2555-44AF-BE40-47FFC16DCBA6}" Directory_="APPDIR" Attributes="0" KeyPath="Hardcodet.Wpf.TaskbarNotification.dll"/>
@ -454,10 +454,10 @@
    <ROW XmlAttribute="xsischemaLocation" XmlElement="swidsoftware_identification_tag" Name="xsi:schemaLocation" Flags="14" Order="3" Value="http://standards.iso.org/iso/19770/-2/2008/schema.xsd software_identification_tag.xsd"/>
  </COMPONENT>
  <COMPONENT cid="caphyon.advinst.msicomp.XmlElementComponent">
-    <ROW XmlElement="swidbuild" ParentElement="swidnumeric" Name="swid:build" Condition="1" Order="2" Flags="14" Text="99"/>
+    <ROW XmlElement="swidbuild" ParentElement="swidnumeric" Name="swid:build" Condition="1" Order="2" Flags="14" Text="2"/>
    <ROW XmlElement="swidentitlement_required_indicator" ParentElement="swidsoftware_identification_tag" Name="swid:entitlement_required_indicator" Condition="1" Order="0" Flags="14" Text="false"/>
    <ROW XmlElement="swidmajor" ParentElement="swidnumeric" Name="swid:major" Condition="1" Order="0" Flags="14" Text="1"/>
-    <ROW XmlElement="swidminor" ParentElement="swidnumeric" Name="swid:minor" Condition="1" Order="1" Flags="14" Text="2"/>
+    <ROW XmlElement="swidminor" ParentElement="swidnumeric" Name="swid:minor" Condition="1" Order="1" Flags="14" Text="4"/>
    <ROW XmlElement="swidname" ParentElement="swidproduct_version" Name="swid:name" Condition="1" Order="0" Flags="14" Text="[ProductVersion]"/>
    <ROW XmlElement="swidname_1" ParentElement="swidsoftware_creator" Name="swid:name" Condition="1" Order="0" Flags="14" Text="ZeroTier, Inc."/>
    <ROW XmlElement="swidname_2" ParentElement="swidsoftware_licensor" Name="swid:name" Condition="1" Order="0" Flags="14" Text="ZeroTier, Inc."/>
--- a/make-linux.mk
+++ b/make-linux.mk
@ -95,6 +95,10 @@ ifeq ($(ZT_SYNOLOGY), 1)
 	override DEFS+=-D__SYNOLOGY__
 endif

+ifeq ($(ZT_DISABLE_COMPRESSION), 1)
+	override DEFS+=-DZT_DISABLE_COMPRESSION
+endif
+
 ifeq ($(ZT_TRACE),1)
 	override DEFS+=-DZT_TRACE
 endif
@ -174,6 +178,11 @@ ifeq ($(CC_MACH),armv6)
 	override DEFS+=-DZT_NO_TYPE_PUNNING
 	ZT_USE_ARM32_NEON_ASM_CRYPTO=1
 endif
+ifeq ($(CC_MACH),armv6l)
+	ZT_ARCHITECTURE=3
+	override DEFS+=-DZT_NO_TYPE_PUNNING
+	ZT_USE_ARM32_NEON_ASM_CRYPTO=1
+endif
 ifeq ($(CC_MACH),armv6zk)
 	ZT_ARCHITECTURE=3
 	override DEFS+=-DZT_NO_TYPE_PUNNING
@ -278,20 +287,22 @@ ifeq ($(ZT_USE_ARM32_NEON_ASM_CRYPTO),1)
 	override CORE_OBJS+=ext/arm32-neon-salsa2012-asm/salsa2012.o
 endif

+.PHONY: all
 all:	one

-one:	$(CORE_OBJS) $(ONE_OBJS) one.o
+.PHONY: one
+one: zerotier-one zerotier-idtool zerotier-cli
+
+zerotier-one:	$(CORE_OBJS) $(ONE_OBJS) one.o
 	$(CXX) $(CXXFLAGS) $(LDFLAGS) -o zerotier-one $(CORE_OBJS) $(ONE_OBJS) one.o $(LDLIBS)
 	$(STRIP) zerotier-one
+
+zerotier-idtool: zerotier-one
 	ln -sf zerotier-one zerotier-idtool
+
+zerotier-cli: zerotier-one
 	ln -sf zerotier-one zerotier-cli

-zerotier-one: one
-
-zerotier-idtool: one
-
-zerotier-cli: one
-
 libzerotiercore.a:	FORCE
 	make CFLAGS="-O3 -fstack-protector -fPIC" CXXFLAGS="-O3 -std=c++11 -fstack-protector -fPIC" $(CORE_OBJS)
 	ar rcs libzerotiercore.a $(CORE_OBJS)
--- a/node/Constants.hpp
+++ b/node/Constants.hpp
@ -72,14 +72,6 @@
 #include <machine/endian.h>
 #endif

-// Defined this macro to disable "type punning" on a number of targets that
-// have issues with unaligned memory access.
-#if defined(__arm__) || defined(__ARMEL__) || (defined(__APPLE__) && ( (defined(TARGET_OS_IPHONE) && (TARGET_OS_IPHONE != 0)) || (defined(TARGET_OS_WATCH) && (TARGET_OS_WATCH != 0)) || (defined(TARGET_IPHONE_SIMULATOR) && (TARGET_IPHONE_SIMULATOR != 0)) ) )
-#ifndef ZT_NO_TYPE_PUNNING
-#define ZT_NO_TYPE_PUNNING
-#endif
-#endif
-
 #if defined(__FreeBSD__) || defined(__OpenBSD__) || defined(__NetBSD__)
 #ifndef __UNIX_LIKE__
 #define __UNIX_LIKE__
@ -107,13 +99,23 @@
 #pragma warning(disable : 4101)
 #undef __UNIX_LIKE__
 #undef __BSD__
-#define ZT_PATH_SEPARATOR '\\'
-#define ZT_PATH_SEPARATOR_S "\\"
-#define ZT_EOL_S "\r\n"
 #include <WinSock2.h>
 #include <Windows.h>
 #endif

+#ifdef __NetBSD__
+#ifndef RTF_MULTICAST
+#define RTF_MULTICAST   0x20000000
+#endif
+#endif
+
+// Define ZT_NO_TYPE_PUNNING to disable reckless casts on anything other than x86/x64.
+#if (!(defined(__amd64__) || defined(__amd64) || defined(__x86_64__) || defined(__x86_64) || defined(_M_AMD64) || defined(_M_X64) || defined(i386) || defined(__i386) || defined(__i386__) || defined(__i486__) || defined(__i586__) || defined(__i686__) || defined(_M_IX86) || defined(__X86__) || defined(_X86_) || defined(__I86__) || defined(__INTEL__) || defined(__386)))
+#ifndef ZT_NO_TYPE_PUNNING
+#define ZT_NO_TYPE_PUNNING
+#endif
+#endif
+
 // Assume little endian if not defined
 #if (defined(__APPLE__) || defined(__WINDOWS__)) && (!defined(__BYTE_ORDER))
 #undef __BYTE_ORDER
@ -124,7 +126,11 @@
 #define __BYTE_ORDER 1234
 #endif

-#ifdef __UNIX_LIKE__
+#ifdef __WINDOWS__
+#define ZT_PATH_SEPARATOR '\\'
+#define ZT_PATH_SEPARATOR_S "\\"
+#define ZT_EOL_S "\r\n"
+#else
 #define ZT_PATH_SEPARATOR '/'
 #define ZT_PATH_SEPARATOR_S "/"
 #define ZT_EOL_S "\n"
@ -134,10 +140,6 @@
 #include <endian.h>
 #endif

-#ifdef __NetBSD__
-#define RTF_MULTICAST   0x20000000
-#endif
-
 #if (defined(__GNUC__) && (__GNUC__ >= 3)) || (defined(__INTEL_COMPILER) && (__INTEL_COMPILER >= 800)) || defined(__clang__)
 #ifndef likely
 #define likely(x) __builtin_expect((x),1)
@ -632,16 +634,7 @@
 */
 #define ZT_THREAD_MIN_STACK_SIZE 1048576

-/* Ethernet frame types that might be relevant to us */
-#define ZT_ETHERTYPE_IPV4 0x0800
-#define ZT_ETHERTYPE_ARP 0x0806
-#define ZT_ETHERTYPE_RARP 0x8035
-#define ZT_ETHERTYPE_ATALK 0x809b
-#define ZT_ETHERTYPE_AARP 0x80f3
-#define ZT_ETHERTYPE_IPX_A 0x8137
-#define ZT_ETHERTYPE_IPX_B 0x8138
-#define ZT_ETHERTYPE_IPV6 0x86dd
-
+// Exceptions thrown in core ZT code
 #define ZT_EXCEPTION_OUT_OF_BOUNDS 100
 #define ZT_EXCEPTION_OUT_OF_MEMORY 101
 #define ZT_EXCEPTION_PRIVATE_KEY_REQUIRED 102
--- a/node/InetAddress.cpp
+++ b/node/InetAddress.cpp
@ -57,7 +57,6 @@ InetAddress::IpScope InetAddress::ipScope() const
 				case 0x1c: return IP_SCOPE_PSEUDOPRIVATE;                          // 28.0.0.0/8 (US DSI-North)
 				case 0x1d: return IP_SCOPE_PSEUDOPRIVATE;                          // 29.0.0.0/8 (US DISA)
 				case 0x1e: return IP_SCOPE_PSEUDOPRIVATE;                          // 30.0.0.0/8 (US DISA)
-				case 0x2c: return IP_SCOPE_PSEUDOPRIVATE;                          // 44.0.0.0/8 (Amateur Radio)
 				case 0x33: return IP_SCOPE_PSEUDOPRIVATE;                          // 51.0.0.0/8 (UK Department of Social Security)
 				case 0x37: return IP_SCOPE_PSEUDOPRIVATE;                          // 55.0.0.0/8 (US DoD)
 				case 0x38: return IP_SCOPE_PSEUDOPRIVATE;                          // 56.0.0.0/8 (US Postal Service)
--- a/node/Membership.cpp
+++ b/node/Membership.cpp
@ -48,9 +48,12 @@ Membership::Membership() :
 {
 }

-void Membership::pushCredentials(const RuntimeEnvironment *RR,void *tPtr,const int64_t now,const Address &peerAddress,const NetworkConfig &nconf,int localCapabilityIndex)
+void Membership::pushCredentials(const RuntimeEnvironment *RR,void *tPtr,const int64_t now,const Address &peerAddress,const NetworkConfig &nconf)
 {
-	const Capability *sendCap = (localCapabilityIndex >= 0) ? &(nconf.capabilities[localCapabilityIndex]) : (const Capability *)0;
+	const Capability *sendCaps[ZT_MAX_NETWORK_CAPABILITIES];
+	unsigned int sendCapCount = 0;
+	for(unsigned int c=0;c<nconf.capabilityCount;++c)
+		sendCaps[sendCapCount++] = &(nconf.capabilities[c]);

 	const Tag *sendTags[ZT_MAX_NETWORK_TAGS];
 	unsigned int sendTagCount = 0;
@ -62,10 +65,11 @@ void Membership::pushCredentials(const RuntimeEnvironment *RR,void *tPtr,const i
 	for(unsigned int c=0;c<nconf.certificateOfOwnershipCount;++c)
 		sendCoos[sendCooCount++] = &(nconf.certificatesOfOwnership[c]);

+	unsigned int capPtr = 0;
 	unsigned int tagPtr = 0;
 	unsigned int cooPtr = 0;
 	bool sendCom = (bool)(nconf.com);
-	while ((tagPtr < sendTagCount)||(cooPtr < sendCooCount)||(sendCom)||(sendCap)) {
+	while ((capPtr < sendCapCount)||(tagPtr < sendTagCount)||(cooPtr < sendCooCount)||(sendCom)) {
 		Packet outp(peerAddress,RR->identity.address(),Packet::VERB_NETWORK_CREDENTIALS);

 		if (sendCom) {
@ -74,11 +78,14 @@ void Membership::pushCredentials(const RuntimeEnvironment *RR,void *tPtr,const i
 		}
 		outp.append((uint8_t)0x00);

-		if (sendCap) {
-			outp.append((uint16_t)1);
-			sendCap->serialize(outp);
-			sendCap = (const Capability *)0;
-		} else outp.append((uint16_t)0);
+		const unsigned int capCountAt = outp.size();
+		outp.addSize(2);
+		unsigned int thisPacketCapCount = 0;
+		while ((capPtr < sendCapCount)&&((outp.size() + sizeof(Capability) + 16) < ZT_PROTO_MAX_PACKET_LENGTH)) {
+			sendCaps[capPtr++]->serialize(outp);
+			++thisPacketCapCount;
+		}
+		outp.setAt(capCountAt,(uint16_t)thisPacketCapCount);

 		const unsigned int tagCountAt = outp.size();
 		outp.addSize(2);
--- a/node/Membership.hpp
+++ b/node/Membership.hpp
@ -74,9 +74,8 @@ public:
 	 * @param now Current time
 	 * @param peerAddress Address of member peer (the one that this Membership describes)
 	 * @param nconf My network config
-	 * @param localCapabilityIndex Index of local capability to include (in nconf.capabilities[]) or -1 if none
 	 */
-	void pushCredentials(const RuntimeEnvironment *RR,void *tPtr,const int64_t now,const Address &peerAddress,const NetworkConfig &nconf,int localCapabilityIndex);
+	void pushCredentials(const RuntimeEnvironment *RR,void *tPtr,const int64_t now,const Address &peerAddress,const NetworkConfig &nconf);

 	/**
 	 * @return True if we haven't pushed credentials in a long time (to cause proactive credential push)
@ -137,7 +136,7 @@ public:
 			if (_isCredentialTimestampValid(nconf,*v)&&(v->owns(r)))
 				return true;
 		}
-		return false;
+		return _isV6NDPEmulated(nconf,r);
 	}

 	/**
@ -192,6 +191,15 @@ public:
 	static uint64_t credentialKey(const Credential::Type &t,const uint32_t i) { return (((uint64_t)t << 32) | (uint64_t)i); }

 private:
+	inline bool _isV6NDPEmulated(const NetworkConfig &nconf,const MAC &m) const { return false; }
+	inline bool _isV6NDPEmulated(const NetworkConfig &nconf,const InetAddress &ip) const
+	{
+		if ((ip.isV6())&&(nconf.ndpEmulation())&&((InetAddress::makeIpv66plane(nconf.networkId,nconf.issuedTo.toInt()).ipsEqual(ip))||(InetAddress::makeIpv6rfc4193(nconf.networkId,nconf.issuedTo.toInt()).ipsEqual(ip)))) {
+			return true;
+		}
+		return false;
+	}
+
 	template<typename C>
 	inline bool _isCredentialTimestampValid(const NetworkConfig &nconf,const C &remoteCredential) const
 	{
--- a/node/Multicaster.hpp
+++ b/node/Multicaster.hpp
@ -168,6 +168,7 @@ private:
 		MulticastGroup mg;

 		inline bool operator==(const Key &k) const { return ((nwid == k.nwid)&&(mg == k.mg)); }
+		inline bool operator!=(const Key &k) const { return ((nwid != k.nwid)||(mg != k.mg)); }
 		inline unsigned long hashCode() const { return (mg.hashCode() ^ (unsigned long)(nwid ^ (nwid >> 32))); }
 	};

@ -176,6 +177,9 @@ private:
 		MulticastGroupMember() {}
 		MulticastGroupMember(const Address &a,uint64_t ts) : address(a),timestamp(ts) {}

+		inline bool operator<(const MulticastGroupMember &a) const { return (address < a.address); }
+		inline bool operator==(const MulticastGroupMember &a) const { return (address == a.address); }
+		inline bool operator!=(const MulticastGroupMember &a) const { return (address != a.address); }
 		inline bool operator<(const Address &a) const { return (address < a); }
 		inline bool operator==(const Address &a) const { return (address == a); }
 		inline bool operator!=(const Address &a) const { return (address != a); }
--- a/node/Network.hpp
+++ b/node/Network.hpp
@ -365,7 +365,7 @@ public:
 	inline void pushCredentialsNow(void *tPtr,const Address &to,const int64_t now)
 	{
 		Mutex::Lock _l(_lock);
-		_membership(to).pushCredentials(RR,tPtr,now,to,_config,-1);
+		_membership(to).pushCredentials(RR,tPtr,now,to,_config);
 	}

 	/**
@ -380,7 +380,7 @@ public:
 		Mutex::Lock _l(_lock);
 		Membership &m = _membership(to);
 		if (m.shouldPushCredentials(now))
-			m.pushCredentials(RR,tPtr,now,to,_config,-1);
+			m.pushCredentials(RR,tPtr,now,to,_config);
 	}

 	/**
--- a/node/NetworkConfig.hpp
+++ b/node/NetworkConfig.hpp
@ -271,10 +271,14 @@ public:
 	 */
 	inline bool disableCompression() const
 	{
-#ifndef ZT_SDK
+#ifndef ZT_DISABLE_COMPRESSION
 		return ((this->flags & ZT_NETWORKCONFIG_FLAG_DISABLE_COMPRESSION) != 0);
 #else
-		return false; // Compression is disabled for SDK builds since it doesn't play nice with lwIP
+		/* Compression is disabled for libzt builds since it causes non-obvious chaotic
+		interference with lwIP's TCP congestion algorithm. Compression is also disabled
+		for some NAS builds due to the usage of low-performance processors in certain
+		older and budget models. */
+		return false;
 #endif
 	}

--- a/node/Switch.hpp
+++ b/node/Switch.hpp
@ -44,6 +44,16 @@
 #include "IncomingPacket.hpp"
 #include "Hashtable.hpp"

+/* Ethernet frame types that might be relevant to us */
+#define ZT_ETHERTYPE_IPV4 0x0800
+#define ZT_ETHERTYPE_ARP 0x0806
+#define ZT_ETHERTYPE_RARP 0x8035
+#define ZT_ETHERTYPE_ATALK 0x809b
+#define ZT_ETHERTYPE_AARP 0x80f3
+#define ZT_ETHERTYPE_IPX_A 0x8137
+#define ZT_ETHERTYPE_IPX_B 0x8138
+#define ZT_ETHERTYPE_IPV6 0x86dd
+
 namespace ZeroTier {

 class RuntimeEnvironment;
--- a/objects.mk
+++ b/objects.mk
@ -29,8 +29,10 @@ CORE_OBJS=\

 ONE_OBJS=\
 	controller/EmbeddedNetworkController.o \
+	controller/DBMirrorSet.o \
 	controller/DB.o \
 	controller/FileDB.o \
+	controller/LFDB.o \
 	controller/PostgreSQL.o \
 	controller/RabbitMQ.o \
 	osdep/ManagedRoute.o \
--- a/osdep/LinuxNetLink.cpp
+++ b/osdep/LinuxNetLink.cpp
@ -124,9 +124,9 @@ int LinuxNetLink::_doRecv(int fd)
 			if(nlp->nlmsg_type == NLMSG_ERROR && (nlp->nlmsg_flags & NLM_F_ACK) != NLM_F_ACK) {
 				struct nlmsgerr *err = (struct nlmsgerr*)NLMSG_DATA(nlp);
 				if (err->error != 0) {
-//#ifdef ZT_TRACE
-					fprintf(stderr, "rtnetlink error: %s\n", strerror(-(err->error)));
-//#endif
+#ifdef ZT_TRACE
+					//fprintf(stderr, "rtnetlink error: %s\n", strerror(-(err->error)));
+#endif
 				}
 				p = buf;
 				nll = 0;
--- a/osdep/MacEthernetTap.cpp
+++ b/osdep/MacEthernetTap.cpp
@ -165,6 +165,7 @@ MacEthernetTap::MacEthernetTap(
 			break;
 		}
 	}
+	_dev = devstr;

 	if (::pipe(_shutdownSignalPipe))
 		throw std::runtime_error("pipe creation failed");
@ -285,7 +286,7 @@ std::vector<InetAddress> MacEthernetTap::ips() const
 	if (!getifaddrs(&ifa)) {
 		struct ifaddrs *p = ifa;
 		while (p) {
-			if ((!strcmp(p->ifa_name,_dev.c_str()))&&(p->ifa_addr)&&(p->ifa_netmask)&&(p->ifa_addr->sa_family == p->ifa_netmask->sa_family)) {
+			if ((p->ifa_name)&&(!strcmp(p->ifa_name,_dev.c_str()))&&(p->ifa_addr)) {
 				switch(p->ifa_addr->sa_family) {
 					case AF_INET: {
 						struct sockaddr_in *sin = (struct sockaddr_in *)p->ifa_addr;
@ -361,7 +362,7 @@ void MacEthernetTap::scanMulticastGroups(std::vector<MulticastGroup> &added,std:
 		newGroups.push_back(MulticastGroup::deriveMulticastGroupForAddressResolution(*ip));

 	std::sort(newGroups.begin(),newGroups.end());
-	std::unique(newGroups.begin(),newGroups.end());
+	newGroups.erase(std::unique(newGroups.begin(),newGroups.end()),newGroups.end());

 	for(std::vector<MulticastGroup>::iterator m(newGroups.begin());m!=newGroups.end();++m) {
 		if (!std::binary_search(_multicastGroups.begin(),_multicastGroups.end(),*m))
--- a/osdep/OSUtils.cpp
+++ b/osdep/OSUtils.cpp
@ -399,10 +399,6 @@ std::string OSUtils::platformDefaultHomePath()
    return homeDir;
 #endif

-#ifdef __SYNOLOGY__
-	return std::string("/var/packages/zerotier/target/var");
-#endif
-
    // Check for user-defined environment variable before using defaults
 #ifdef __WINDOWS__
 	DWORD bufferSize = 65535;
--- a/osdep/WindowsEthernetTap.cpp
+++ b/osdep/WindowsEthernetTap.cpp
@ -58,6 +58,8 @@

 #include "..\windows\TapDriver6\tap-windows.h"

+#include <netcon.h>
+
 // Create a fake unused default route to force detection of network type on networks without gateways
 #define ZT_WINDOWS_CREATE_FAKE_DEFAULT_ROUTE

@ -824,6 +826,61 @@ void WindowsEthernetTap::setFriendlyName(const char *dn)
 		RegSetKeyValueA(ifp,"Connection","Name",REG_SZ,(LPCVOID)dn,(DWORD)(strlen(dn)+1));
 		RegCloseKey(ifp);
 	}
+
+	HRESULT hr = CoInitialize(nullptr);
+	if (hr != S_OK) return;
+	CoInitializeSecurity(NULL, -1, NULL, NULL,
+		RPC_C_AUTHN_LEVEL_PKT,
+		RPC_C_IMP_LEVEL_IMPERSONATE,
+		NULL, EOAC_NONE, NULL);
+	if (hr != S_OK) return;
+
+	INetSharingManager *nsm;
+	hr = CoCreateInstance(__uuidof(NetSharingManager), NULL, CLSCTX_ALL, __uuidof(INetSharingManager), (void**)&nsm);
+	if (hr != S_OK)	return;
+
+	bool found = false;
+	INetSharingEveryConnectionCollection *nsecc = nullptr;
+	hr = nsm->get_EnumEveryConnection(&nsecc);
+	if (!nsecc) {
+		fprintf(stderr, "Failed to get NSM connections");
+		return;
+	}
+
+	IEnumVARIANT *ev = nullptr;
+	IUnknown *unk = nullptr;
+	hr = nsecc->get__NewEnum(&unk);
+	if (unk) {
+		hr = unk->QueryInterface(__uuidof(IEnumVARIANT), (void**)&ev);
+		unk->Release();
+	}
+	if (ev) {
+		VARIANT v;
+		VariantInit(&v);
+
+		while ((S_OK == ev->Next(1, &v, NULL)) && found == FALSE) {
+			if (V_VT(&v) == VT_UNKNOWN) {
+				INetConnection *nc = nullptr;
+				V_UNKNOWN(&v)->QueryInterface(__uuidof(INetConnection), (void**)&nc);
+				if (nc) {
+					NETCON_PROPERTIES *ncp = nullptr;
+					nc->GetProperties(&ncp);
+
+					GUID curId = ncp->guidId;
+					if (curId == _deviceGuid) {
+						wchar_t wtext[255];
+						mbstowcs(wtext, dn, strlen(dn)+1);
+						nc->Rename(wtext);
+						found = true;
+					}
+					nc->Release();
+				}
+			}
+			VariantClear(&v);
+		}
+		ev->Release();
+	}
+	nsecc->Release();
 }

 void WindowsEthernetTap::scanMulticastGroups(std::vector<MulticastGroup> &added,std::vector<MulticastGroup> &removed)
--- a/service/OneService.cpp
+++ b/service/OneService.cpp
@ -961,15 +961,17 @@ public:
 		Mutex::Lock _l2(_localConfig_m);
 		std::string lcbuf;
 		if (OSUtils::readFile((_homePath + ZT_PATH_SEPARATOR_S "local.conf").c_str(),lcbuf)) {
-			try {
-				_localConfig = OSUtils::jsonParse(lcbuf);
-				if (!_localConfig.is_object()) {
-					fprintf(stderr,"ERROR: unable to parse local.conf (root element is not a JSON object)" ZT_EOL_S);
+			if (lcbuf.length() > 0) {
+				try {
+					_localConfig = OSUtils::jsonParse(lcbuf);
+					if (!_localConfig.is_object()) {
+						fprintf(stderr,"ERROR: unable to parse local.conf (root element is not a JSON object)" ZT_EOL_S);
+						exit(1);
+					}
+				} catch ( ... ) {
+					fprintf(stderr,"ERROR: unable to parse local.conf (invalid JSON)" ZT_EOL_S);
 					exit(1);
 				}
-			} catch ( ... ) {
-				fprintf(stderr,"ERROR: unable to parse local.conf (invalid JSON)" ZT_EOL_S);
-				exit(1);
 			}
 		}

--- a/service/README.md
+++ b/service/README.md
@ -5,7 +5,7 @@ This is the actual implementation of ZeroTier One, a service providing connectiv

 ### Local Configuration File

-A file called `local.conf` in the ZeroTier home folder contains configuration options that apply to the local node. It can be used to set up trusted paths, blacklist physical paths, set up physical path hints for certain nodes, and define trusted upstream devices (federated roots). In a large deployment it can be deployed using a tool like Puppet, Chef, SaltStack, etc. to set a uniform configuration across systems. It's a JSON format file that can also be edited and rewritten by ZeroTier One itself, so ensure that proper JSON formatting is used.
+A file called `local.conf` in the ZeroTier home folder contains configuration options that apply to the local node. (It does not exist unless you create it). It can be used to set up trusted paths, blacklist physical paths, set up physical path hints for certain nodes, and define trusted upstream devices (federated roots). In a large deployment it can be deployed using a tool like Puppet, Chef, SaltStack, etc. to set a uniform configuration across systems. It's a JSON format file that can also be edited and rewritten by ZeroTier One itself, so ensure that proper JSON formatting is used. 

 Settings available in `local.conf` (this is not valid JSON, and JSON does not allow comments):

--- a/version.h
+++ b/version.h
@ -40,7 +40,7 @@
 /**
 * Revision
 */
-#define ZEROTIER_ONE_VERSION_REVISION 0
+#define ZEROTIER_ONE_VERSION_REVISION 2

 /**
 * Build version
--- a/windows-clean.bat
+++ b/windows-clean.bat
@ -0,0 +1,9 @@
+DEL "ZeroTier One.msi"
+DEL zt1_update*.exe
+RMDIR /Q /S windows\Build
+RMDIR /Q /S windows\copyutil\bin
+RMDIR /Q /S windows\copyutil\obj
+RMDIR /Q /S windows\WinUI\bin
+RMDIR /Q /S windows\WinUI\obj
+RMDIR /Q /S windows\ZeroTierOne\Release
+RMDIR /Q /S windows\ZeroTierOne\x64
--- a/windows/WinUI/AboutView.xaml
+++ b/windows/WinUI/AboutView.xaml
@ -19,7 +19,7 @@
                    <Run Text="ZeroTier One"/>
                </Paragraph>
                <Paragraph TextAlignment="Center">
-                    <Run FontSize="14" Text="Version 1.2.99 (1.4.0pre)"/>
+                    <Run FontSize="14" Text="Version 1.4.2"/>
                    <LineBreak/>
                    <Run FontSize="14" Text="(c) 2011-2019 ZeroTier, Inc."/>
                    <LineBreak/>
--- a/windows/ZeroTierOne/ZeroTierOne.vcxproj
+++ b/windows/ZeroTierOne/ZeroTierOne.vcxproj
@ -30,6 +30,7 @@
    <ClCompile Include="..\..\controller\DB.cpp" />
    <ClCompile Include="..\..\controller\EmbeddedNetworkController.cpp" />
    <ClCompile Include="..\..\controller\FileDB.cpp" />
+    <ClCompile Include="..\..\controller\LFDB.cpp" />
    <ClCompile Include="..\..\controller\PostgreSQL.cpp" />
    <ClCompile Include="..\..\controller\RabbitMQ.cpp" />
    <ClCompile Include="..\..\ext\http-parser\http_parser.c" />
@ -116,8 +117,10 @@
    <ClInclude Include="..\..\controller\DB.hpp" />
    <ClInclude Include="..\..\controller\EmbeddedNetworkController.hpp" />
    <ClInclude Include="..\..\controller\FileDB.hpp" />
+    <ClInclude Include="..\..\controller\LFDB.hpp" />
    <ClInclude Include="..\..\controller\PostgreSQL.hpp" />
    <ClInclude Include="..\..\controller\RabbitMQ.hpp" />
+    <ClInclude Include="..\..\ext\cpp-httplib\httplib.h" />
    <ClInclude Include="..\..\ext\http-parser\http_parser.h" />
    <ClInclude Include="..\..\ext\json\json.hpp" />
    <ClInclude Include="..\..\ext\libnatpmp\getgateway.h" />
--- a/windows/ZeroTierOne/ZeroTierOne.vcxproj.filters
+++ b/windows/ZeroTierOne/ZeroTierOne.vcxproj.filters
@ -82,6 +82,9 @@
    <Filter Include="Header Files\controller">
      <UniqueIdentifier>{7dc22e9c-f869-41e7-b43d-f07f5b94f6fb}</UniqueIdentifier>
    </Filter>
+    <Filter Include="Header Files\ext\cpp-httplib">
+      <UniqueIdentifier>{4dfde4c7-2950-40ee-92f2-05e0916d36c5}</UniqueIdentifier>
+    </Filter>
  </ItemGroup>
  <ItemGroup>
    <ClCompile Include="..\..\service\OneService.cpp">
@ -264,6 +267,9 @@
    <ClCompile Include="..\..\controller\RabbitMQ.cpp">
      <Filter>Source Files\controller</Filter>
    </ClCompile>
+    <ClCompile Include="..\..\controller\LFDB.cpp">
+      <Filter>Source Files\controller</Filter>
+    </ClCompile>
  </ItemGroup>
  <ItemGroup>
    <ClInclude Include="resource.h">
@ -506,6 +512,12 @@
    <ClInclude Include="..\..\controller\RabbitMQ.hpp">
      <Filter>Header Files\controller</Filter>
    </ClInclude>
+    <ClInclude Include="..\..\controller\LFDB.hpp">
+      <Filter>Header Files\controller</Filter>
+    </ClInclude>
+    <ClInclude Include="..\..\ext\cpp-httplib\httplib.h">
+      <Filter>Header Files\ext\cpp-httplib</Filter>
+    </ClInclude>
  </ItemGroup>
  <ItemGroup>
    <ResourceCompile Include="ZeroTierOne.rc">
--- a/zerotier-one.spec
+++ b/zerotier-one.spec
@ -1,5 +1,5 @@
 Name:           zerotier-one
-Version:        1.2.99
+Version:        1.4.2
 Release:        1%{?dist}
 Summary:        ZeroTier One network virtualization service

@ -145,6 +145,12 @@ esac
 %endif

 %changelog
+* Mon Aug 04 2019 Adam Ierymenko <adam.ierymenko@zerotier.com> - 1.4.2-0.1
+- see https://github.com/zerotier/ZeroTierOne for release notes
+
+* Mon Jul 29 2019 Adam Ierymenko <adam.ierymenko@zerotier.com> - 1.4.0-0.1
+- see https://github.com/zerotier/ZeroTierOne for release notes
+
 * Tue May 08 2018 Adam Ierymenko <adam.ierymenko@zerotier.com> - 1.2.10-0.1
 - see https://github.com/zerotier/ZeroTierOne for release notes