Merge branch 'dev' into libpq

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,46 @@
+---
+name: Bug report
+about: Create a report to help us improve
+
+---
+**Alternative, faster ways to get help**
+If you have just started using ZeroTier, here are some places to get help:
+- my.zerotier.com has a _Community_ tab. It's a live chat with other users and the developers. 
+- [ZeroTier Knowledge Base](https://support.zerotier.com/knowledgebase.php?s=MQ__)
+- www.zerotier.com has a Contact Us button
+- email contact@zerotier.com
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Create a Network '...'
+2. Install zerotier-one '....'
+3. '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots or console output to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. Mac, Linux, Windows, BSD]
+ - OS/Distribution Version
+ - ZeroTier Version [e.g. 1.2.4]
+ - Hardware [e.g. raspberry pi 3]
+ 
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Version [e.g. 1.2.4]
+
+**Additional context**
+Add any other context about the problem here.
+- ZeroTier Network Configuration
+- Router Config
+- Firewall Config (try turning the firewall off)
+- General Network Environment: [ e.g Home, University Campus, Corporate LAN ]
+

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.gitignore

@@ -4,6 +4,7 @@
 /zerotier-cli
 /zerotier-selftest
 /zerotier
+/nltest
 
 # OS-created garbage files from various platforms
 .DS_Store

RELEASE-NOTES.md

@@ -43,7 +43,7 @@ ZeroTier Release Notes
     * Fixed two very rare multithreading issues that were only observed on certain systems
  * Platform-Specific Changes
     * MacOS
-        * Installer now loads the kernel extension right away so that High Sierra users will see the prompt to authorize it. This is done in the "Security & Privacy" preference pane and must be done driectly on the console (not via remote desktop). On High Sierra and newer kexts must be authorized at the console via security settings system preferences pane.
+        * Installer now loads the kernel extension right away so that High Sierra users will see the prompt to authorize it. This is done in the "Security & Privacy" preference pane and must be done directly on the console (not via remote desktop). On High Sierra and newer kexts must be authorized at the console via security settings system preferences pane.
     * Windows
         * The Windows installer should now install the driver without requiring a special prompt in most cases. This should make it easier for our packages to be accepted into and updated in the Chocolatey repository and should make it easier to perform remote installs across groups of machines using IT management and provisioning tools.
         * The Windows official packages are now signed with an EV certificate (with hardware key).
@@ -85,7 +85,7 @@ The largest new feature in 1.2.0, and the product of many months of work, is our
 
 Rules allow you to filter packets on your network and vector traffic to security observers. Security observation can be performed in-band using REDIRECT or out of band using TEE.
 
-Tags and capabilites provide advanced methods for implementing fine grained permission structures and micro-segmentation schemes without bloating the size and complexity of your rules table.
+Tags and capabilities provide advanced methods for implementing fine grained permission structures and micro-segmentation schemes without bloating the size and complexity of your rules table.
 
 See the [rules engine announcement blog post](https://www.zerotier.com/blog/?p=927) for an in-depth discussion of theory and implementation. The [manual](https://www.zerotier.com/manual.shtml) contains detailed information on rule, tag, and capability use, and the `rule-compiler/` subfolder of the ZeroTier source tree contains a JavaScript function to compile rules in our human-readable rule definition language into rules suitable for import into a network controller. (ZeroTier Central uses this same script to compile rules on [my.zerotier.com](https://my.zerotier.com/).)
 
@@ -168,7 +168,7 @@ A special kind of public network called an ad-hoc network may be accessed by joi
     | Start of port range (hex)
     Reserved ZeroTier address prefix indicating a controller-less network
 
-Ad-hoc networks are public (no access control) networks that have no network controller. Instead their configuration and other credentials are generated locally. Ad-hoc networks permit only IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using 6plane format NDP-emulated IPv6 addresses. In addition an ad-hoc network ID encodes an IP port range. UDP packets and TCP SYN (connection open) packets are only allowed to desintation ports within the encoded range.
+Ad-hoc networks are public (no access control) networks that have no network controller. Instead their configuration and other credentials are generated locally. Ad-hoc networks permit only IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using 6plane format NDP-emulated IPv6 addresses. In addition an ad-hoc network ID encodes an IP port range. UDP packets and TCP SYN (connection open) packets are only allowed to destination ports within the encoded range.
 
 For example `ff00160016000000` is an ad-hoc network allowing only SSH, while `ff0000ffff000000` is an ad-hoc network allowing any UDP or TCP port.
 
@@ -183,7 +183,7 @@ If you have data in an old SQLite3 controller we've included a NodeJS script in
 ## Major Bug Fixes in 1.2.0
 
  * **The Windows HyperV 100% CPU bug is FINALLY DEAD**: This long-running problem turns out to have been an issue with Windows itself, but one we were triggering by placing invalid data into the Windows registry. Microsoft is aware of the issue but we've also fixed the triggering problem on our side. ZeroTier should now co-exist quite well with HyperV and should now be able to be bridged with a HyperV virtual switch.
- * **Segmenation faults on musl-libc based Linux systems**: Alpine Linux and some embedded Linux systems that use musl libc (a minimal libc) experienced segmentation faults. These were due to a smaller default stack size. A work-around that sets the stack size for new threads has been added.
+ * **Segmentation faults on musl-libc based Linux systems**: Alpine Linux and some embedded Linux systems that use musl libc (a minimal libc) experienced segmentation faults. These were due to a smaller default stack size. A work-around that sets the stack size for new threads has been added.
  * **Windows firewall blocks local JSON API**: On some Windows systems the firewall likes to block 127.0.0.1:9993 for mysterious reasons. This is now fixed in the installer via the addition of another firewall exemption rule.
  * **UI crash on embedded Windows due to missing fonts**: The MSI installer now ships fonts and will install them if they are not present, so this should be fixed.
 

ext/arm32-neon-salsa2012-asm/salsa2012.h

@@ -5,8 +5,10 @@
 #include <sys/auxv.h>
 #include <asm/hwcap.h>
 #define zt_arm_has_neon() ((getauxval(AT_HWCAP) & HWCAP_NEON) != 0)
-#else
+#elif defined(__ARM_NEON__) || defined(__ARM_NEON)
 #define zt_arm_has_neon() (true)
+#else
+#define zt_arm_has_neon() (false)
 #endif
 
 #ifdef __cplusplus

ext/curl-7.58.0/Win32/include/curl/curlver.h

@@ -0,0 +1,77 @@
+#ifndef __CURL_CURLVER_H
+#define __CURL_CURLVER_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+/* This header file contains nothing but libcurl version info, generated by
+   a script at release-time. This was made its own header file in 7.11.2 */
+
+/* This is the global package copyright */
+#define LIBCURL_COPYRIGHT "1996 - 2017 Daniel Stenberg, <daniel@haxx.se>."
+
+/* This is the version number of the libcurl package from which this header
+   file origins: */
+#define LIBCURL_VERSION "7.58.0"
+
+/* The numeric version number is also available "in parts" by using these
+   defines: */
+#define LIBCURL_VERSION_MAJOR 7
+#define LIBCURL_VERSION_MINOR 58
+#define LIBCURL_VERSION_PATCH 0
+
+/* This is the numeric version of the libcurl version number, meant for easier
+   parsing and comparions by programs. The LIBCURL_VERSION_NUM define will
+   always follow this syntax:
+
+         0xXXYYZZ
+
+   Where XX, YY and ZZ are the main version, release and patch numbers in
+   hexadecimal (using 8 bits each). All three numbers are always represented
+   using two digits.  1.2 would appear as "0x010200" while version 9.11.7
+   appears as "0x090b07".
+
+   This 6-digit (24 bits) hexadecimal number does not show pre-release number,
+   and it is always a greater number in a more recent release. It makes
+   comparisons with greater than and less than work.
+
+   Note: This define is the full hex number and _does not_ use the
+   CURL_VERSION_BITS() macro since curl's own configure script greps for it
+   and needs it to contain the full number.
+*/
+#define LIBCURL_VERSION_NUM 0x073a00
+
+/*
+ * This is the date and time when the full source package was created. The
+ * timestamp is not stored in git, as the timestamp is properly set in the
+ * tarballs by the maketgz script.
+ *
+ * The format of the date follows this template:
+ *
+ * "2007-11-23"
+ */
+#define LIBCURL_TIMESTAMP "2018-01-24"
+
+#define CURL_VERSION_BITS(x,y,z) ((x)<<16|(y)<<8|z)
+#define CURL_AT_LEAST_VERSION(x,y,z) \
+  (LIBCURL_VERSION_NUM >= CURL_VERSION_BITS(x, y, z))
+
+#endif /* __CURL_CURLVER_H */

ext/curl-7.58.0/Win32/include/curl/easy.h

@@ -0,0 +1,102 @@
+#ifndef __CURL_EASY_H
+#define __CURL_EASY_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2016, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+#ifdef  __cplusplus
+extern "C" {
+#endif
+
+CURL_EXTERN CURL *curl_easy_init(void);
+CURL_EXTERN CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);
+CURL_EXTERN CURLcode curl_easy_perform(CURL *curl);
+CURL_EXTERN void curl_easy_cleanup(CURL *curl);
+
+/*
+ * NAME curl_easy_getinfo()
+ *
+ * DESCRIPTION
+ *
+ * Request internal information from the curl session with this function.  The
+ * third argument MUST be a pointer to a long, a pointer to a char * or a
+ * pointer to a double (as the documentation describes elsewhere).  The data
+ * pointed to will be filled in accordingly and can be relied upon only if the
+ * function returns CURLE_OK.  This function is intended to get used *AFTER* a
+ * performed transfer, all results from this function are undefined until the
+ * transfer is completed.
+ */
+CURL_EXTERN CURLcode curl_easy_getinfo(CURL *curl, CURLINFO info, ...);
+
+
+/*
+ * NAME curl_easy_duphandle()
+ *
+ * DESCRIPTION
+ *
+ * Creates a new curl session handle with the same options set for the handle
+ * passed in. Duplicating a handle could only be a matter of cloning data and
+ * options, internal state info and things like persistent connections cannot
+ * be transferred. It is useful in multithreaded applications when you can run
+ * curl_easy_duphandle() for each new thread to avoid a series of identical
+ * curl_easy_setopt() invokes in every thread.
+ */
+CURL_EXTERN CURL *curl_easy_duphandle(CURL *curl);
+
+/*
+ * NAME curl_easy_reset()
+ *
+ * DESCRIPTION
+ *
+ * Re-initializes a CURL handle to the default values. This puts back the
+ * handle to the same state as it was in when it was just created.
+ *
+ * It does keep: live connections, the Session ID cache, the DNS cache and the
+ * cookies.
+ */
+CURL_EXTERN void curl_easy_reset(CURL *curl);
+
+/*
+ * NAME curl_easy_recv()
+ *
+ * DESCRIPTION
+ *
+ * Receives data from the connected socket. Use after successful
+ * curl_easy_perform() with CURLOPT_CONNECT_ONLY option.
+ */
+CURL_EXTERN CURLcode curl_easy_recv(CURL *curl, void *buffer, size_t buflen,
+                                    size_t *n);
+
+/*
+ * NAME curl_easy_send()
+ *
+ * DESCRIPTION
+ *
+ * Sends data over the connected socket. Use after successful
+ * curl_easy_perform() with CURLOPT_CONNECT_ONLY option.
+ */
+CURL_EXTERN CURLcode curl_easy_send(CURL *curl, const void *buffer,
+                                    size_t buflen, size_t *n);
+
+#ifdef  __cplusplus
+}
+#endif
+
+#endif

ext/curl-7.58.0/Win32/include/curl/mprintf.h

@@ -0,0 +1,50 @@
+#ifndef __CURL_MPRINTF_H
+#define __CURL_MPRINTF_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2016, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+#include <stdarg.h>
+#include <stdio.h> /* needed for FILE */
+#include "curl.h"  /* for CURL_EXTERN */
+
+#ifdef  __cplusplus
+extern "C" {
+#endif
+
+CURL_EXTERN int curl_mprintf(const char *format, ...);
+CURL_EXTERN int curl_mfprintf(FILE *fd, const char *format, ...);
+CURL_EXTERN int curl_msprintf(char *buffer, const char *format, ...);
+CURL_EXTERN int curl_msnprintf(char *buffer, size_t maxlength,
+                               const char *format, ...);
+CURL_EXTERN int curl_mvprintf(const char *format, va_list args);
+CURL_EXTERN int curl_mvfprintf(FILE *fd, const char *format, va_list args);
+CURL_EXTERN int curl_mvsprintf(char *buffer, const char *format, va_list args);
+CURL_EXTERN int curl_mvsnprintf(char *buffer, size_t maxlength,
+                                const char *format, va_list args);
+CURL_EXTERN char *curl_maprintf(const char *format, ...);
+CURL_EXTERN char *curl_mvaprintf(const char *format, va_list args);
+
+#ifdef  __cplusplus
+}
+#endif
+
+#endif /* __CURL_MPRINTF_H */

ext/curl-7.58.0/Win32/include/curl/multi.h

@@ -0,0 +1,439 @@
+#ifndef __CURL_MULTI_H
+#define __CURL_MULTI_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+/*
+  This is an "external" header file. Don't give away any internals here!
+
+  GOALS
+
+  o Enable a "pull" interface. The application that uses libcurl decides where
+    and when to ask libcurl to get/send data.
+
+  o Enable multiple simultaneous transfers in the same thread without making it
+    complicated for the application.
+
+  o Enable the application to select() on its own file descriptors and curl's
+    file descriptors simultaneous easily.
+
+*/
+
+/*
+ * This header file should not really need to include "curl.h" since curl.h
+ * itself includes this file and we expect user applications to do #include
+ * <curl/curl.h> without the need for especially including multi.h.
+ *
+ * For some reason we added this include here at one point, and rather than to
+ * break existing (wrongly written) libcurl applications, we leave it as-is
+ * but with this warning attached.
+ */
+#include "curl.h"
+
+#ifdef  __cplusplus
+extern "C" {
+#endif
+
+#if defined(BUILDING_LIBCURL) || defined(CURL_STRICTER)
+typedef struct Curl_multi CURLM;
+#else
+typedef void CURLM;
+#endif
+
+typedef enum {
+  CURLM_CALL_MULTI_PERFORM = -1, /* please call curl_multi_perform() or
+                                    curl_multi_socket*() soon */
+  CURLM_OK,
+  CURLM_BAD_HANDLE,      /* the passed-in handle is not a valid CURLM handle */
+  CURLM_BAD_EASY_HANDLE, /* an easy handle was not good/valid */
+  CURLM_OUT_OF_MEMORY,   /* if you ever get this, you're in deep sh*t */
+  CURLM_INTERNAL_ERROR,  /* this is a libcurl bug */
+  CURLM_BAD_SOCKET,      /* the passed in socket argument did not match */
+  CURLM_UNKNOWN_OPTION,  /* curl_multi_setopt() with unsupported option */
+  CURLM_ADDED_ALREADY,   /* an easy handle already added to a multi handle was
+                            attempted to get added - again */
+  CURLM_LAST
+} CURLMcode;
+
+/* just to make code nicer when using curl_multi_socket() you can now check
+   for CURLM_CALL_MULTI_SOCKET too in the same style it works for
+   curl_multi_perform() and CURLM_CALL_MULTI_PERFORM */
+#define CURLM_CALL_MULTI_SOCKET CURLM_CALL_MULTI_PERFORM
+
+/* bitmask bits for CURLMOPT_PIPELINING */
+#define CURLPIPE_NOTHING   0L
+#define CURLPIPE_HTTP1     1L
+#define CURLPIPE_MULTIPLEX 2L
+
+typedef enum {
+  CURLMSG_NONE, /* first, not used */
+  CURLMSG_DONE, /* This easy handle has completed. 'result' contains
+                   the CURLcode of the transfer */
+  CURLMSG_LAST /* last, not used */
+} CURLMSG;
+
+struct CURLMsg {
+  CURLMSG msg;       /* what this message means */
+  CURL *easy_handle; /* the handle it concerns */
+  union {
+    void *whatever;    /* message-specific data */
+    CURLcode result;   /* return code for transfer */
+  } data;
+};
+typedef struct CURLMsg CURLMsg;
+
+/* Based on poll(2) structure and values.
+ * We don't use pollfd and POLL* constants explicitly
+ * to cover platforms without poll(). */
+#define CURL_WAIT_POLLIN    0x0001
+#define CURL_WAIT_POLLPRI   0x0002
+#define CURL_WAIT_POLLOUT   0x0004
+
+struct curl_waitfd {
+  curl_socket_t fd;
+  short events;
+  short revents; /* not supported yet */
+};
+
+/*
+ * Name:    curl_multi_init()
+ *
+ * Desc:    inititalize multi-style curl usage
+ *
+ * Returns: a new CURLM handle to use in all 'curl_multi' functions.
+ */
+CURL_EXTERN CURLM *curl_multi_init(void);
+
+/*
+ * Name:    curl_multi_add_handle()
+ *
+ * Desc:    add a standard curl handle to the multi stack
+ *
+ * Returns: CURLMcode type, general multi error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_add_handle(CURLM *multi_handle,
+                                            CURL *curl_handle);
+
+ /*
+  * Name:    curl_multi_remove_handle()
+  *
+  * Desc:    removes a curl handle from the multi stack again
+  *
+  * Returns: CURLMcode type, general multi error code.
+  */
+CURL_EXTERN CURLMcode curl_multi_remove_handle(CURLM *multi_handle,
+                                               CURL *curl_handle);
+
+ /*
+  * Name:    curl_multi_fdset()
+  *
+  * Desc:    Ask curl for its fd_set sets. The app can use these to select() or
+  *          poll() on. We want curl_multi_perform() called as soon as one of
+  *          them are ready.
+  *
+  * Returns: CURLMcode type, general multi error code.
+  */
+CURL_EXTERN CURLMcode curl_multi_fdset(CURLM *multi_handle,
+                                       fd_set *read_fd_set,
+                                       fd_set *write_fd_set,
+                                       fd_set *exc_fd_set,
+                                       int *max_fd);
+
+/*
+ * Name:     curl_multi_wait()
+ *
+ * Desc:     Poll on all fds within a CURLM set as well as any
+ *           additional fds passed to the function.
+ *
+ * Returns:  CURLMcode type, general multi error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_wait(CURLM *multi_handle,
+                                      struct curl_waitfd extra_fds[],
+                                      unsigned int extra_nfds,
+                                      int timeout_ms,
+                                      int *ret);
+
+ /*
+  * Name:    curl_multi_perform()
+  *
+  * Desc:    When the app thinks there's data available for curl it calls this
+  *          function to read/write whatever there is right now. This returns
+  *          as soon as the reads and writes are done. This function does not
+  *          require that there actually is data available for reading or that
+  *          data can be written, it can be called just in case. It returns
+  *          the number of handles that still transfer data in the second
+  *          argument's integer-pointer.
+  *
+  * Returns: CURLMcode type, general multi error code. *NOTE* that this only
+  *          returns errors etc regarding the whole multi stack. There might
+  *          still have occurred problems on invidual transfers even when this
+  *          returns OK.
+  */
+CURL_EXTERN CURLMcode curl_multi_perform(CURLM *multi_handle,
+                                         int *running_handles);
+
+ /*
+  * Name:    curl_multi_cleanup()
+  *
+  * Desc:    Cleans up and removes a whole multi stack. It does not free or
+  *          touch any individual easy handles in any way. We need to define
+  *          in what state those handles will be if this function is called
+  *          in the middle of a transfer.
+  *
+  * Returns: CURLMcode type, general multi error code.
+  */
+CURL_EXTERN CURLMcode curl_multi_cleanup(CURLM *multi_handle);
+
+/*
+ * Name:    curl_multi_info_read()
+ *
+ * Desc:    Ask the multi handle if there's any messages/informationals from
+ *          the individual transfers. Messages include informationals such as
+ *          error code from the transfer or just the fact that a transfer is
+ *          completed. More details on these should be written down as well.
+ *
+ *          Repeated calls to this function will return a new struct each
+ *          time, until a special "end of msgs" struct is returned as a signal
+ *          that there is no more to get at this point.
+ *
+ *          The data the returned pointer points to will not survive calling
+ *          curl_multi_cleanup().
+ *
+ *          The 'CURLMsg' struct is meant to be very simple and only contain
+ *          very basic information. If more involved information is wanted,
+ *          we will provide the particular "transfer handle" in that struct
+ *          and that should/could/would be used in subsequent
+ *          curl_easy_getinfo() calls (or similar). The point being that we
+ *          must never expose complex structs to applications, as then we'll
+ *          undoubtably get backwards compatibility problems in the future.
+ *
+ * Returns: A pointer to a filled-in struct, or NULL if it failed or ran out
+ *          of structs. It also writes the number of messages left in the
+ *          queue (after this read) in the integer the second argument points
+ *          to.
+ */
+CURL_EXTERN CURLMsg *curl_multi_info_read(CURLM *multi_handle,
+                                          int *msgs_in_queue);
+
+/*
+ * Name:    curl_multi_strerror()
+ *
+ * Desc:    The curl_multi_strerror function may be used to turn a CURLMcode
+ *          value into the equivalent human readable error string.  This is
+ *          useful for printing meaningful error messages.
+ *
+ * Returns: A pointer to a zero-terminated error message.
+ */
+CURL_EXTERN const char *curl_multi_strerror(CURLMcode);
+
+/*
+ * Name:    curl_multi_socket() and
+ *          curl_multi_socket_all()
+ *
+ * Desc:    An alternative version of curl_multi_perform() that allows the
+ *          application to pass in one of the file descriptors that have been
+ *          detected to have "action" on them and let libcurl perform.
+ *          See man page for details.
+ */
+#define CURL_POLL_NONE   0
+#define CURL_POLL_IN     1
+#define CURL_POLL_OUT    2
+#define CURL_POLL_INOUT  3
+#define CURL_POLL_REMOVE 4
+
+#define CURL_SOCKET_TIMEOUT CURL_SOCKET_BAD
+
+#define CURL_CSELECT_IN   0x01
+#define CURL_CSELECT_OUT  0x02
+#define CURL_CSELECT_ERR  0x04
+
+typedef int (*curl_socket_callback)(CURL *easy,      /* easy handle */
+                                    curl_socket_t s, /* socket */
+                                    int what,        /* see above */
+                                    void *userp,     /* private callback
+                                                        pointer */
+                                    void *socketp);  /* private socket
+                                                        pointer */
+/*
+ * Name:    curl_multi_timer_callback
+ *
+ * Desc:    Called by libcurl whenever the library detects a change in the
+ *          maximum number of milliseconds the app is allowed to wait before
+ *          curl_multi_socket() or curl_multi_perform() must be called
+ *          (to allow libcurl's timed events to take place).
+ *
+ * Returns: The callback should return zero.
+ */
+typedef int (*curl_multi_timer_callback)(CURLM *multi,    /* multi handle */
+                                         long timeout_ms, /* see above */
+                                         void *userp);    /* private callback
+                                                             pointer */
+
+CURL_EXTERN CURLMcode curl_multi_socket(CURLM *multi_handle, curl_socket_t s,
+                                        int *running_handles);
+
+CURL_EXTERN CURLMcode curl_multi_socket_action(CURLM *multi_handle,
+                                               curl_socket_t s,
+                                               int ev_bitmask,
+                                               int *running_handles);
+
+CURL_EXTERN CURLMcode curl_multi_socket_all(CURLM *multi_handle,
+                                            int *running_handles);
+
+#ifndef CURL_ALLOW_OLD_MULTI_SOCKET
+/* This macro below was added in 7.16.3 to push users who recompile to use
+   the new curl_multi_socket_action() instead of the old curl_multi_socket()
+*/
+#define curl_multi_socket(x,y,z) curl_multi_socket_action(x,y,0,z)
+#endif
+
+/*
+ * Name:    curl_multi_timeout()
+ *
+ * Desc:    Returns the maximum number of milliseconds the app is allowed to
+ *          wait before curl_multi_socket() or curl_multi_perform() must be
+ *          called (to allow libcurl's timed events to take place).
+ *
+ * Returns: CURLM error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_timeout(CURLM *multi_handle,
+                                         long *milliseconds);
+
+#undef CINIT /* re-using the same name as in curl.h */
+
+#ifdef CURL_ISOCPP
+#define CINIT(name,type,num) CURLMOPT_ ## name = CURLOPTTYPE_ ## type + num
+#else
+/* The macro "##" is ISO C, we assume pre-ISO C doesn't support it. */
+#define LONG          CURLOPTTYPE_LONG
+#define OBJECTPOINT   CURLOPTTYPE_OBJECTPOINT
+#define FUNCTIONPOINT CURLOPTTYPE_FUNCTIONPOINT
+#define OFF_T         CURLOPTTYPE_OFF_T
+#define CINIT(name,type,number) CURLMOPT_/**/name = type + number
+#endif
+
+typedef enum {
+  /* This is the socket callback function pointer */
+  CINIT(SOCKETFUNCTION, FUNCTIONPOINT, 1),
+
+  /* This is the argument passed to the socket callback */
+  CINIT(SOCKETDATA, OBJECTPOINT, 2),
+
+    /* set to 1 to enable pipelining for this multi handle */
+  CINIT(PIPELINING, LONG, 3),
+
+   /* This is the timer callback function pointer */
+  CINIT(TIMERFUNCTION, FUNCTIONPOINT, 4),
+
+  /* This is the argument passed to the timer callback */
+  CINIT(TIMERDATA, OBJECTPOINT, 5),
+
+  /* maximum number of entries in the connection cache */
+  CINIT(MAXCONNECTS, LONG, 6),
+
+  /* maximum number of (pipelining) connections to one host */
+  CINIT(MAX_HOST_CONNECTIONS, LONG, 7),
+
+  /* maximum number of requests in a pipeline */
+  CINIT(MAX_PIPELINE_LENGTH, LONG, 8),
+
+  /* a connection with a content-length longer than this
+     will not be considered for pipelining */
+  CINIT(CONTENT_LENGTH_PENALTY_SIZE, OFF_T, 9),
+
+  /* a connection with a chunk length longer than this
+     will not be considered for pipelining */
+  CINIT(CHUNK_LENGTH_PENALTY_SIZE, OFF_T, 10),
+
+  /* a list of site names(+port) that are blacklisted from
+     pipelining */
+  CINIT(PIPELINING_SITE_BL, OBJECTPOINT, 11),
+
+  /* a list of server types that are blacklisted from
+     pipelining */
+  CINIT(PIPELINING_SERVER_BL, OBJECTPOINT, 12),
+
+  /* maximum number of open connections in total */
+  CINIT(MAX_TOTAL_CONNECTIONS, LONG, 13),
+
+   /* This is the server push callback function pointer */
+  CINIT(PUSHFUNCTION, FUNCTIONPOINT, 14),
+
+  /* This is the argument passed to the server push callback */
+  CINIT(PUSHDATA, OBJECTPOINT, 15),
+
+  CURLMOPT_LASTENTRY /* the last unused */
+} CURLMoption;
+
+
+/*
+ * Name:    curl_multi_setopt()
+ *
+ * Desc:    Sets options for the multi handle.
+ *
+ * Returns: CURLM error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_setopt(CURLM *multi_handle,
+                                        CURLMoption option, ...);
+
+
+/*
+ * Name:    curl_multi_assign()
+ *
+ * Desc:    This function sets an association in the multi handle between the
+ *          given socket and a private pointer of the application. This is
+ *          (only) useful for curl_multi_socket uses.
+ *
+ * Returns: CURLM error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_assign(CURLM *multi_handle,
+                                        curl_socket_t sockfd, void *sockp);
+
+
+/*
+ * Name: curl_push_callback
+ *
+ * Desc: This callback gets called when a new stream is being pushed by the
+ *       server. It approves or denies the new stream.
+ *
+ * Returns: CURL_PUSH_OK or CURL_PUSH_DENY.
+ */
+#define CURL_PUSH_OK   0
+#define CURL_PUSH_DENY 1
+
+struct curl_pushheaders;  /* forward declaration only */
+
+CURL_EXTERN char *curl_pushheader_bynum(struct curl_pushheaders *h,
+                                        size_t num);
+CURL_EXTERN char *curl_pushheader_byname(struct curl_pushheaders *h,
+                                         const char *name);
+
+typedef int (*curl_push_callback)(CURL *parent,
+                                  CURL *easy,
+                                  size_t num_headers,
+                                  struct curl_pushheaders *headers,
+                                  void *userp);
+
+#ifdef __cplusplus
+} /* end of extern "C" */
+#endif
+
+#endif

ext/curl-7.58.0/Win32/include/curl/stdcheaders.h

@@ -0,0 +1,33 @@
+#ifndef __STDC_HEADERS_H
+#define __STDC_HEADERS_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2016, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+#include <sys/types.h>
+
+size_t fread(void *, size_t, size_t, FILE *);
+size_t fwrite(const void *, size_t, size_t, FILE *);
+
+int strcasecmp(const char *, const char *);
+int strncasecmp(const char *, const char *, size_t);
+
+#endif /* __STDC_HEADERS_H */

ext/curl-7.58.0/Win32/include/curl/system.h

@@ -0,0 +1,473 @@
+#ifndef __CURL_SYSTEM_H
+#define __CURL_SYSTEM_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+/*
+ * Try to keep one section per platform, compiler and architecture, otherwise,
+ * if an existing section is reused for a different one and later on the
+ * original is adjusted, probably the piggybacking one can be adversely
+ * changed.
+ *
+ * In order to differentiate between platforms/compilers/architectures use
+ * only compiler built in predefined preprocessor symbols.
+ *
+ * curl_off_t
+ * ----------
+ *
+ * For any given platform/compiler curl_off_t must be typedef'ed to a 64-bit
+ * wide signed integral data type. The width of this data type must remain
+ * constant and independent of any possible large file support settings.
+ *
+ * As an exception to the above, curl_off_t shall be typedef'ed to a 32-bit
+ * wide signed integral data type if there is no 64-bit type.
+ *
+ * As a general rule, curl_off_t shall not be mapped to off_t. This rule shall
+ * only be violated if off_t is the only 64-bit data type available and the
+ * size of off_t is independent of large file support settings. Keep your
+ * build on the safe side avoiding an off_t gating.  If you have a 64-bit
+ * off_t then take for sure that another 64-bit data type exists, dig deeper
+ * and you will find it.
+ *
+ */
+
+#if defined(__DJGPP__) || defined(__GO32__)
+#  if defined(__DJGPP__) && (__DJGPP__ > 1)
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__SALFORDC__)
+#  define CURL_TYPEOF_CURL_OFF_T     long
+#  define CURL_FORMAT_CURL_OFF_T     "ld"
+#  define CURL_FORMAT_CURL_OFF_TU    "lu"
+#  define CURL_SUFFIX_CURL_OFF_T     L
+#  define CURL_SUFFIX_CURL_OFF_TU    UL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__BORLANDC__)
+#  if (__BORLANDC__ < 0x520)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "I64d"
+#    define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#    define CURL_SUFFIX_CURL_OFF_T     i64
+#    define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__TURBOC__)
+#  define CURL_TYPEOF_CURL_OFF_T     long
+#  define CURL_FORMAT_CURL_OFF_T     "ld"
+#  define CURL_FORMAT_CURL_OFF_TU    "lu"
+#  define CURL_SUFFIX_CURL_OFF_T     L
+#  define CURL_SUFFIX_CURL_OFF_TU    UL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__WATCOMC__)
+#  if defined(__386__)
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "I64d"
+#    define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#    define CURL_SUFFIX_CURL_OFF_T     i64
+#    define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__POCC__)
+#  if (__POCC__ < 280)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  elif defined(_MSC_VER)
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "I64d"
+#    define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#    define CURL_SUFFIX_CURL_OFF_T     i64
+#    define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__LCC__)
+#  define CURL_TYPEOF_CURL_OFF_T     long
+#  define CURL_FORMAT_CURL_OFF_T     "ld"
+#  define CURL_FORMAT_CURL_OFF_TU    "lu"
+#  define CURL_SUFFIX_CURL_OFF_T     L
+#  define CURL_SUFFIX_CURL_OFF_TU    UL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__SYMBIAN32__)
+#  if defined(__EABI__)  /* Treat all ARM compilers equally */
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  elif defined(__CW32__)
+#    pragma longlong on
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  elif defined(__VC32__)
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T unsigned int
+
+#elif defined(__MWERKS__)
+#  define CURL_TYPEOF_CURL_OFF_T     long long
+#  define CURL_FORMAT_CURL_OFF_T     "lld"
+#  define CURL_FORMAT_CURL_OFF_TU    "llu"
+#  define CURL_SUFFIX_CURL_OFF_T     LL
+#  define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(_WIN32_WCE)
+#  define CURL_TYPEOF_CURL_OFF_T     __int64
+#  define CURL_FORMAT_CURL_OFF_T     "I64d"
+#  define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#  define CURL_SUFFIX_CURL_OFF_T     i64
+#  define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__MINGW32__)
+#  define CURL_TYPEOF_CURL_OFF_T     long long
+#  define CURL_FORMAT_CURL_OFF_T     "I64d"
+#  define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#  define CURL_SUFFIX_CURL_OFF_T     LL
+#  define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#  define CURL_PULL_SYS_TYPES_H      1
+#  define CURL_PULL_WS2TCPIP_H       1
+
+#elif defined(__VMS)
+#  if defined(__VAX)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T unsigned int
+
+#elif defined(__OS400__)
+#  if defined(__ILEC400__)
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#    define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#    define CURL_PULL_SYS_TYPES_H      1
+#    define CURL_PULL_SYS_SOCKET_H     1
+#  endif
+
+#elif defined(__MVS__)
+#  if defined(__IBMC__) || defined(__IBMCPP__)
+#    if defined(_ILP32)
+#    elif defined(_LP64)
+#    endif
+#    if defined(_LONG_LONG)
+#      define CURL_TYPEOF_CURL_OFF_T     long long
+#      define CURL_FORMAT_CURL_OFF_T     "lld"
+#      define CURL_FORMAT_CURL_OFF_TU    "llu"
+#      define CURL_SUFFIX_CURL_OFF_T     LL
+#      define CURL_SUFFIX_CURL_OFF_TU    ULL
+#    elif defined(_LP64)
+#      define CURL_TYPEOF_CURL_OFF_T     long
+#      define CURL_FORMAT_CURL_OFF_T     "ld"
+#      define CURL_FORMAT_CURL_OFF_TU    "lu"
+#      define CURL_SUFFIX_CURL_OFF_T     L
+#      define CURL_SUFFIX_CURL_OFF_TU    UL
+#    else
+#      define CURL_TYPEOF_CURL_OFF_T     long
+#      define CURL_FORMAT_CURL_OFF_T     "ld"
+#      define CURL_FORMAT_CURL_OFF_TU    "lu"
+#      define CURL_SUFFIX_CURL_OFF_T     L
+#      define CURL_SUFFIX_CURL_OFF_TU    UL
+#    endif
+#    define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#    define CURL_PULL_SYS_TYPES_H      1
+#    define CURL_PULL_SYS_SOCKET_H     1
+#  endif
+
+#elif defined(__370__)
+#  if defined(__IBMC__) || defined(__IBMCPP__)
+#    if defined(_ILP32)
+#    elif defined(_LP64)
+#    endif
+#    if defined(_LONG_LONG)
+#      define CURL_TYPEOF_CURL_OFF_T     long long
+#      define CURL_FORMAT_CURL_OFF_T     "lld"
+#      define CURL_FORMAT_CURL_OFF_TU    "llu"
+#      define CURL_SUFFIX_CURL_OFF_T     LL
+#      define CURL_SUFFIX_CURL_OFF_TU    ULL
+#    elif defined(_LP64)
+#      define CURL_TYPEOF_CURL_OFF_T     long
+#      define CURL_FORMAT_CURL_OFF_T     "ld"
+#      define CURL_FORMAT_CURL_OFF_TU    "lu"
+#      define CURL_SUFFIX_CURL_OFF_T     L
+#      define CURL_SUFFIX_CURL_OFF_TU    UL
+#    else
+#      define CURL_TYPEOF_CURL_OFF_T     long
+#      define CURL_FORMAT_CURL_OFF_T     "ld"
+#      define CURL_FORMAT_CURL_OFF_TU    "lu"
+#      define CURL_SUFFIX_CURL_OFF_T     L
+#      define CURL_SUFFIX_CURL_OFF_TU    UL
+#    endif
+#    define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#    define CURL_PULL_SYS_TYPES_H      1
+#    define CURL_PULL_SYS_SOCKET_H     1
+#  endif
+
+#elif defined(TPF)
+#  define CURL_TYPEOF_CURL_OFF_T     long
+#  define CURL_FORMAT_CURL_OFF_T     "ld"
+#  define CURL_FORMAT_CURL_OFF_TU    "lu"
+#  define CURL_SUFFIX_CURL_OFF_T     L
+#  define CURL_SUFFIX_CURL_OFF_TU    UL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__TINYC__) /* also known as tcc */
+
+#  define CURL_TYPEOF_CURL_OFF_T     long long
+#  define CURL_FORMAT_CURL_OFF_T     "lld"
+#  define CURL_FORMAT_CURL_OFF_TU    "llu"
+#  define CURL_SUFFIX_CURL_OFF_T     LL
+#  define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#  define CURL_PULL_SYS_TYPES_H      1
+#  define CURL_PULL_SYS_SOCKET_H     1
+
+#elif defined(__SUNPRO_C) /* Oracle Solaris Studio */
+#  if !defined(__LP64) && (defined(__ILP32) ||                          \
+                           defined(__i386) || defined(__sparcv8))
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  elif defined(__LP64) || \
+        defined(__amd64) || defined(__sparcv9)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#  define CURL_PULL_SYS_TYPES_H      1
+#  define CURL_PULL_SYS_SOCKET_H     1
+
+/* ===================================== */
+/*    KEEP MSVC THE PENULTIMATE ENTRY    */
+/* ===================================== */
+
+#elif defined(_MSC_VER)
+#  if (_MSC_VER >= 900) && (_INTEGRAL_MAX_BITS >= 64)
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "I64d"
+#    define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#    define CURL_SUFFIX_CURL_OFF_T     i64
+#    define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+/* ===================================== */
+/*    KEEP GENERIC GCC THE LAST ENTRY    */
+/* ===================================== */
+
+#elif defined(__GNUC__)
+#  if !defined(__LP64__) &&                                             \
+  (defined(__ILP32__) || defined(__i386__) || defined(__hppa__) ||      \
+   defined(__ppc__) || defined(__powerpc__) || defined(__arm__) ||      \
+   defined(__sparc__) || defined(__mips__) || defined(__sh__) ||        \
+   defined(__XTENSA__) ||                                               \
+   (defined(__SIZEOF_LONG__) && __SIZEOF_LONG__ == 4)  ||               \
+   (defined(__LONG_MAX__) && __LONG_MAX__ == 2147483647L))
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  elif defined(__LP64__) || \
+        defined(__x86_64__) || defined(__ppc64__) || defined(__sparc64__) || \
+        (defined(__SIZEOF_LONG__) && __SIZEOF_LONG__ == 8) || \
+        (defined(__LONG_MAX__) && __LONG_MAX__ == 9223372036854775807L)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#  define CURL_PULL_SYS_TYPES_H      1
+#  define CURL_PULL_SYS_SOCKET_H     1
+
+#else
+/* generic "safe guess" on old 32 bit style */
+# define CURL_TYPEOF_CURL_OFF_T     long
+# define CURL_FORMAT_CURL_OFF_T     "ld"
+# define CURL_FORMAT_CURL_OFF_TU    "lu"
+# define CURL_SUFFIX_CURL_OFF_T     L
+# define CURL_SUFFIX_CURL_OFF_TU    UL
+# define CURL_TYPEOF_CURL_SOCKLEN_T int
+#endif
+
+#ifdef _AIX
+/* AIX needs <sys/poll.h> */
+#define CURL_PULL_SYS_POLL_H
+#endif
+
+
+/* CURL_PULL_WS2TCPIP_H is defined above when inclusion of header file  */
+/* ws2tcpip.h is required here to properly make type definitions below. */
+#ifdef CURL_PULL_WS2TCPIP_H
+#  include <winsock2.h>
+#  include <windows.h>
+#  include <ws2tcpip.h>
+#endif
+
+/* CURL_PULL_SYS_TYPES_H is defined above when inclusion of header file  */
+/* sys/types.h is required here to properly make type definitions below. */
+#ifdef CURL_PULL_SYS_TYPES_H
+#  include <sys/types.h>
+#endif
+
+/* CURL_PULL_SYS_SOCKET_H is defined above when inclusion of header file  */
+/* sys/socket.h is required here to properly make type definitions below. */
+#ifdef CURL_PULL_SYS_SOCKET_H
+#  include <sys/socket.h>
+#endif
+
+/* CURL_PULL_SYS_POLL_H is defined above when inclusion of header file    */
+/* sys/poll.h is required here to properly make type definitions below.   */
+#ifdef CURL_PULL_SYS_POLL_H
+#  include <sys/poll.h>
+#endif
+
+/* Data type definition of curl_socklen_t. */
+#ifdef CURL_TYPEOF_CURL_SOCKLEN_T
+  typedef CURL_TYPEOF_CURL_SOCKLEN_T curl_socklen_t;
+#endif
+
+/* Data type definition of curl_off_t. */
+
+#ifdef CURL_TYPEOF_CURL_OFF_T
+  typedef CURL_TYPEOF_CURL_OFF_T curl_off_t;
+#endif
+
+/*
+ * CURL_ISOCPP and CURL_OFF_T_C definitions are done here in order to allow
+ * these to be visible and exported by the external libcurl interface API,
+ * while also making them visible to the library internals, simply including
+ * curl_setup.h, without actually needing to include curl.h internally.
+ * If some day this section would grow big enough, all this should be moved
+ * to its own header file.
+ */
+
+/*
+ * Figure out if we can use the ## preprocessor operator, which is supported
+ * by ISO/ANSI C and C++. Some compilers support it without setting __STDC__
+ * or  __cplusplus so we need to carefully check for them too.
+ */
+
+#if defined(__STDC__) || defined(_MSC_VER) || defined(__cplusplus) || \
+  defined(__HP_aCC) || defined(__BORLANDC__) || defined(__LCC__) || \
+  defined(__POCC__) || defined(__SALFORDC__) || defined(__HIGHC__) || \
+  defined(__ILEC400__)
+  /* This compiler is believed to have an ISO compatible preprocessor */
+#define CURL_ISOCPP
+#else
+  /* This compiler is believed NOT to have an ISO compatible preprocessor */
+#undef CURL_ISOCPP
+#endif
+
+/*
+ * Macros for minimum-width signed and unsigned curl_off_t integer constants.
+ */
+
+#if defined(__BORLANDC__) && (__BORLANDC__ == 0x0551)
+#  define __CURL_OFF_T_C_HLPR2(x) x
+#  define __CURL_OFF_T_C_HLPR1(x) __CURL_OFF_T_C_HLPR2(x)
+#  define CURL_OFF_T_C(Val)  __CURL_OFF_T_C_HLPR1(Val) ## \
+                             __CURL_OFF_T_C_HLPR1(CURL_SUFFIX_CURL_OFF_T)
+#  define CURL_OFF_TU_C(Val) __CURL_OFF_T_C_HLPR1(Val) ## \
+                             __CURL_OFF_T_C_HLPR1(CURL_SUFFIX_CURL_OFF_TU)
+#else
+#  ifdef CURL_ISOCPP
+#    define __CURL_OFF_T_C_HLPR2(Val,Suffix) Val ## Suffix
+#  else
+#    define __CURL_OFF_T_C_HLPR2(Val,Suffix) Val/**/Suffix
+#  endif
+#  define __CURL_OFF_T_C_HLPR1(Val,Suffix) __CURL_OFF_T_C_HLPR2(Val,Suffix)
+#  define CURL_OFF_T_C(Val)  __CURL_OFF_T_C_HLPR1(Val,CURL_SUFFIX_CURL_OFF_T)
+#  define CURL_OFF_TU_C(Val) __CURL_OFF_T_C_HLPR1(Val,CURL_SUFFIX_CURL_OFF_TU)
+#endif
+
+#endif /* __CURL_SYSTEM_H */

ext/curl-7.58.0/Win32/include/curl/typecheck-gcc.h

@@ -0,0 +1,683 @@
+#ifndef __CURL_TYPECHECK_GCC_H
+#define __CURL_TYPECHECK_GCC_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+/* wraps curl_easy_setopt() with typechecking */
+
+/* To add a new kind of warning, add an
+ *   if(_curl_is_sometype_option(_curl_opt))
+ *     if(!_curl_is_sometype(value))
+ *       _curl_easy_setopt_err_sometype();
+ * block and define _curl_is_sometype_option, _curl_is_sometype and
+ * _curl_easy_setopt_err_sometype below
+ *
+ * NOTE: We use two nested 'if' statements here instead of the && operator, in
+ *       order to work around gcc bug #32061.  It affects only gcc 4.3.x/4.4.x
+ *       when compiling with -Wlogical-op.
+ *
+ * To add an option that uses the same type as an existing option, you'll just
+ * need to extend the appropriate _curl_*_option macro
+ */
+#define curl_easy_setopt(handle, option, value)                               \
+__extension__ ({                                                              \
+  __typeof__(option) _curl_opt = option;                                     \
+  if(__builtin_constant_p(_curl_opt)) {                                       \
+    if(_curl_is_long_option(_curl_opt))                                       \
+      if(!_curl_is_long(value))                                               \
+        _curl_easy_setopt_err_long();                                         \
+    if(_curl_is_off_t_option(_curl_opt))                                      \
+      if(!_curl_is_off_t(value))                                              \
+        _curl_easy_setopt_err_curl_off_t();                                   \
+    if(_curl_is_string_option(_curl_opt))                                     \
+      if(!_curl_is_string(value))                                             \
+        _curl_easy_setopt_err_string();                                       \
+    if(_curl_is_write_cb_option(_curl_opt))                                   \
+      if(!_curl_is_write_cb(value))                                           \
+        _curl_easy_setopt_err_write_callback();                               \
+    if((_curl_opt) == CURLOPT_READFUNCTION)                                   \
+      if(!_curl_is_read_cb(value))                                            \
+        _curl_easy_setopt_err_read_cb();                                      \
+    if((_curl_opt) == CURLOPT_IOCTLFUNCTION)                                  \
+      if(!_curl_is_ioctl_cb(value))                                           \
+        _curl_easy_setopt_err_ioctl_cb();                                     \
+    if((_curl_opt) == CURLOPT_SOCKOPTFUNCTION)                                \
+      if(!_curl_is_sockopt_cb(value))                                         \
+        _curl_easy_setopt_err_sockopt_cb();                                   \
+    if((_curl_opt) == CURLOPT_OPENSOCKETFUNCTION)                             \
+      if(!_curl_is_opensocket_cb(value))                                      \
+        _curl_easy_setopt_err_opensocket_cb();                                \
+    if((_curl_opt) == CURLOPT_PROGRESSFUNCTION)                               \
+      if(!_curl_is_progress_cb(value))                                        \
+        _curl_easy_setopt_err_progress_cb();                                  \
+    if((_curl_opt) == CURLOPT_DEBUGFUNCTION)                                  \
+      if(!_curl_is_debug_cb(value))                                           \
+        _curl_easy_setopt_err_debug_cb();                                     \
+    if((_curl_opt) == CURLOPT_SSL_CTX_FUNCTION)                               \
+      if(!_curl_is_ssl_ctx_cb(value))                                         \
+        _curl_easy_setopt_err_ssl_ctx_cb();                                   \
+    if(_curl_is_conv_cb_option(_curl_opt))                                    \
+      if(!_curl_is_conv_cb(value))                                            \
+        _curl_easy_setopt_err_conv_cb();                                      \
+    if((_curl_opt) == CURLOPT_SEEKFUNCTION)                                   \
+      if(!_curl_is_seek_cb(value))                                            \
+        _curl_easy_setopt_err_seek_cb();                                      \
+    if(_curl_is_cb_data_option(_curl_opt))                                    \
+      if(!_curl_is_cb_data(value))                                            \
+        _curl_easy_setopt_err_cb_data();                                      \
+    if((_curl_opt) == CURLOPT_ERRORBUFFER)                                    \
+      if(!_curl_is_error_buffer(value))                                       \
+        _curl_easy_setopt_err_error_buffer();                                 \
+    if((_curl_opt) == CURLOPT_STDERR)                                         \
+      if(!_curl_is_FILE(value))                                               \
+        _curl_easy_setopt_err_FILE();                                         \
+    if(_curl_is_postfields_option(_curl_opt))                                 \
+      if(!_curl_is_postfields(value))                                         \
+        _curl_easy_setopt_err_postfields();                                   \
+    if((_curl_opt) == CURLOPT_HTTPPOST)                                       \
+      if(!_curl_is_arr((value), struct curl_httppost))                        \
+        _curl_easy_setopt_err_curl_httpost();                                 \
+    if((_curl_opt) == CURLOPT_MIMEPOST)                                       \
+      if(!_curl_is_ptr((value), curl_mime))                                   \
+        _curl_easy_setopt_err_curl_mimepost();                                \
+    if(_curl_is_slist_option(_curl_opt))                                      \
+      if(!_curl_is_arr((value), struct curl_slist))                           \
+        _curl_easy_setopt_err_curl_slist();                                   \
+    if((_curl_opt) == CURLOPT_SHARE)                                          \
+      if(!_curl_is_ptr((value), CURLSH))                                      \
+        _curl_easy_setopt_err_CURLSH();                                       \
+  }                                                                           \
+  curl_easy_setopt(handle, _curl_opt, value);                                 \
+})
+
+/* wraps curl_easy_getinfo() with typechecking */
+/* FIXME: don't allow const pointers */
+#define curl_easy_getinfo(handle, info, arg)                                  \
+__extension__ ({                                                              \
+  __typeof__(info) _curl_info = info;                                         \
+  if(__builtin_constant_p(_curl_info)) {                                      \
+    if(_curl_is_string_info(_curl_info))                                      \
+      if(!_curl_is_arr((arg), char *))                                        \
+        _curl_easy_getinfo_err_string();                                      \
+    if(_curl_is_long_info(_curl_info))                                        \
+      if(!_curl_is_arr((arg), long))                                          \
+        _curl_easy_getinfo_err_long();                                        \
+    if(_curl_is_double_info(_curl_info))                                      \
+      if(!_curl_is_arr((arg), double))                                        \
+        _curl_easy_getinfo_err_double();                                      \
+    if(_curl_is_slist_info(_curl_info))                                       \
+      if(!_curl_is_arr((arg), struct curl_slist *))                           \
+        _curl_easy_getinfo_err_curl_slist();                                  \
+    if(_curl_is_tlssessioninfo_info(_curl_info))                              \
+      if(!_curl_is_arr((arg), struct curl_tlssessioninfo *))                  \
+        _curl_easy_getinfo_err_curl_tlssesssioninfo();                        \
+    if(_curl_is_certinfo_info(_curl_info))                                    \
+      if(!_curl_is_arr((arg), struct curl_certinfo *))                        \
+        _curl_easy_getinfo_err_curl_certinfo();                               \
+    if(_curl_is_socket_info(_curl_info))                                      \
+      if(!_curl_is_arr((arg), curl_socket_t))                                 \
+        _curl_easy_getinfo_err_curl_socket();                                 \
+    if(_curl_is_off_t_info(_curl_info))                                       \
+      if(!_curl_is_arr((arg), curl_off_t))                                    \
+        _curl_easy_getinfo_err_curl_off_t();                                  \
+  }                                                                           \
+  curl_easy_getinfo(handle, _curl_info, arg);                                 \
+})
+
+/* TODO: typechecking for curl_share_setopt() and curl_multi_setopt(),
+ * for now just make sure that the functions are called with three
+ * arguments
+ */
+#define curl_share_setopt(share,opt,param) curl_share_setopt(share,opt,param)
+#define curl_multi_setopt(handle,opt,param) curl_multi_setopt(handle,opt,param)
+
+
+/* the actual warnings, triggered by calling the _curl_easy_setopt_err*
+ * functions */
+
+/* To define a new warning, use _CURL_WARNING(identifier, "message") */
+#define _CURL_WARNING(id, message)                                            \
+  static void __attribute__((__warning__(message)))                           \
+  __attribute__((__unused__)) __attribute__((__noinline__))                   \
+  id(void) { __asm__(""); }
+
+_CURL_WARNING(_curl_easy_setopt_err_long,
+  "curl_easy_setopt expects a long argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_curl_off_t,
+  "curl_easy_setopt expects a curl_off_t argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_string,
+              "curl_easy_setopt expects a "
+              "string ('char *' or char[]) argument for this option"
+  )
+_CURL_WARNING(_curl_easy_setopt_err_write_callback,
+  "curl_easy_setopt expects a curl_write_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_read_cb,
+  "curl_easy_setopt expects a curl_read_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_ioctl_cb,
+  "curl_easy_setopt expects a curl_ioctl_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_sockopt_cb,
+  "curl_easy_setopt expects a curl_sockopt_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_opensocket_cb,
+              "curl_easy_setopt expects a "
+              "curl_opensocket_callback argument for this option"
+  )
+_CURL_WARNING(_curl_easy_setopt_err_progress_cb,
+  "curl_easy_setopt expects a curl_progress_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_debug_cb,
+  "curl_easy_setopt expects a curl_debug_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_ssl_ctx_cb,
+  "curl_easy_setopt expects a curl_ssl_ctx_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_conv_cb,
+  "curl_easy_setopt expects a curl_conv_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_seek_cb,
+  "curl_easy_setopt expects a curl_seek_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_cb_data,
+              "curl_easy_setopt expects a "
+              "private data pointer as argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_error_buffer,
+              "curl_easy_setopt expects a "
+              "char buffer of CURL_ERROR_SIZE as argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_FILE,
+  "curl_easy_setopt expects a 'FILE *' argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_postfields,
+  "curl_easy_setopt expects a 'void *' or 'char *' argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_curl_httpost,
+              "curl_easy_setopt expects a 'struct curl_httppost *' "
+              "argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_curl_mimepost,
+              "curl_easy_setopt expects a 'curl_mime *' "
+              "argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_curl_slist,
+  "curl_easy_setopt expects a 'struct curl_slist *' argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_CURLSH,
+  "curl_easy_setopt expects a CURLSH* argument for this option")
+
+_CURL_WARNING(_curl_easy_getinfo_err_string,
+  "curl_easy_getinfo expects a pointer to 'char *' for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_long,
+  "curl_easy_getinfo expects a pointer to long for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_double,
+  "curl_easy_getinfo expects a pointer to double for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_slist,
+  "curl_easy_getinfo expects a pointer to 'struct curl_slist *' for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_tlssesssioninfo,
+              "curl_easy_getinfo expects a pointer to "
+              "'struct curl_tlssessioninfo *' for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_certinfo,
+              "curl_easy_getinfo expects a pointer to "
+              "'struct curl_certinfo *' for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_socket,
+  "curl_easy_getinfo expects a pointer to curl_socket_t for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_off_t,
+  "curl_easy_getinfo expects a pointer to curl_off_t for this info")
+
+/* groups of curl_easy_setops options that take the same type of argument */
+
+/* To add a new option to one of the groups, just add
+ *   (option) == CURLOPT_SOMETHING
+ * to the or-expression. If the option takes a long or curl_off_t, you don't
+ * have to do anything
+ */
+
+/* evaluates to true if option takes a long argument */
+#define _curl_is_long_option(option)                                          \
+  (0 < (option) && (option) < CURLOPTTYPE_OBJECTPOINT)
+
+#define _curl_is_off_t_option(option)                                         \
+  ((option) > CURLOPTTYPE_OFF_T)
+
+/* evaluates to true if option takes a char* argument */
+#define _curl_is_string_option(option)                                        \
+  ((option) == CURLOPT_ABSTRACT_UNIX_SOCKET ||                                \
+   (option) == CURLOPT_ACCEPT_ENCODING ||                                     \
+   (option) == CURLOPT_CAINFO ||                                              \
+   (option) == CURLOPT_CAPATH ||                                              \
+   (option) == CURLOPT_COOKIE ||                                              \
+   (option) == CURLOPT_COOKIEFILE ||                                          \
+   (option) == CURLOPT_COOKIEJAR ||                                           \
+   (option) == CURLOPT_COOKIELIST ||                                          \
+   (option) == CURLOPT_CRLFILE ||                                             \
+   (option) == CURLOPT_CUSTOMREQUEST ||                                       \
+   (option) == CURLOPT_DEFAULT_PROTOCOL ||                                    \
+   (option) == CURLOPT_DNS_INTERFACE ||                                       \
+   (option) == CURLOPT_DNS_LOCAL_IP4 ||                                       \
+   (option) == CURLOPT_DNS_LOCAL_IP6 ||                                       \
+   (option) == CURLOPT_DNS_SERVERS ||                                         \
+   (option) == CURLOPT_EGDSOCKET ||                                           \
+   (option) == CURLOPT_FTPPORT ||                                             \
+   (option) == CURLOPT_FTP_ACCOUNT ||                                         \
+   (option) == CURLOPT_FTP_ALTERNATIVE_TO_USER ||                             \
+   (option) == CURLOPT_INTERFACE ||                                           \
+   (option) == CURLOPT_ISSUERCERT ||                                          \
+   (option) == CURLOPT_KEYPASSWD ||                                           \
+   (option) == CURLOPT_KRBLEVEL ||                                            \
+   (option) == CURLOPT_LOGIN_OPTIONS ||                                       \
+   (option) == CURLOPT_MAIL_AUTH ||                                           \
+   (option) == CURLOPT_MAIL_FROM ||                                           \
+   (option) == CURLOPT_NETRC_FILE ||                                          \
+   (option) == CURLOPT_NOPROXY ||                                             \
+   (option) == CURLOPT_PASSWORD ||                                            \
+   (option) == CURLOPT_PINNEDPUBLICKEY ||                                     \
+   (option) == CURLOPT_PRE_PROXY ||                                           \
+   (option) == CURLOPT_PROXY ||                                               \
+   (option) == CURLOPT_PROXYPASSWORD ||                                       \
+   (option) == CURLOPT_PROXYUSERNAME ||                                       \
+   (option) == CURLOPT_PROXYUSERPWD ||                                        \
+   (option) == CURLOPT_PROXY_CAINFO ||                                        \
+   (option) == CURLOPT_PROXY_CAPATH ||                                        \
+   (option) == CURLOPT_PROXY_CRLFILE ||                                       \
+   (option) == CURLOPT_PROXY_KEYPASSWD ||                                     \
+   (option) == CURLOPT_PROXY_PINNEDPUBLICKEY ||                               \
+   (option) == CURLOPT_PROXY_SERVICE_NAME ||                                  \
+   (option) == CURLOPT_PROXY_SSLCERT ||                                       \
+   (option) == CURLOPT_PROXY_SSLCERTTYPE ||                                   \
+   (option) == CURLOPT_PROXY_SSLKEY ||                                        \
+   (option) == CURLOPT_PROXY_SSLKEYTYPE ||                                    \
+   (option) == CURLOPT_PROXY_SSL_CIPHER_LIST ||                               \
+   (option) == CURLOPT_PROXY_TLSAUTH_PASSWORD ||                              \
+   (option) == CURLOPT_PROXY_TLSAUTH_USERNAME ||                              \
+   (option) == CURLOPT_PROXY_TLSAUTH_TYPE ||                                  \
+   (option) == CURLOPT_RANDOM_FILE ||                                         \
+   (option) == CURLOPT_RANGE ||                                               \
+   (option) == CURLOPT_REFERER ||                                             \
+   (option) == CURLOPT_RTSP_SESSION_ID ||                                     \
+   (option) == CURLOPT_RTSP_STREAM_URI ||                                     \
+   (option) == CURLOPT_RTSP_TRANSPORT ||                                      \
+   (option) == CURLOPT_SERVICE_NAME ||                                        \
+   (option) == CURLOPT_SOCKS5_GSSAPI_SERVICE ||                               \
+   (option) == CURLOPT_SSH_HOST_PUBLIC_KEY_MD5 ||                             \
+   (option) == CURLOPT_SSH_KNOWNHOSTS ||                                      \
+   (option) == CURLOPT_SSH_PRIVATE_KEYFILE ||                                 \
+   (option) == CURLOPT_SSH_PUBLIC_KEYFILE ||                                  \
+   (option) == CURLOPT_SSLCERT ||                                             \
+   (option) == CURLOPT_SSLCERTTYPE ||                                         \
+   (option) == CURLOPT_SSLENGINE ||                                           \
+   (option) == CURLOPT_SSLKEY ||                                              \
+   (option) == CURLOPT_SSLKEYTYPE ||                                          \
+   (option) == CURLOPT_SSL_CIPHER_LIST ||                                     \
+   (option) == CURLOPT_TLSAUTH_PASSWORD ||                                    \
+   (option) == CURLOPT_TLSAUTH_TYPE ||                                        \
+   (option) == CURLOPT_TLSAUTH_USERNAME ||                                    \
+   (option) == CURLOPT_UNIX_SOCKET_PATH ||                                    \
+   (option) == CURLOPT_URL ||                                                 \
+   (option) == CURLOPT_USERAGENT ||                                           \
+   (option) == CURLOPT_USERNAME ||                                            \
+   (option) == CURLOPT_USERPWD ||                                             \
+   (option) == CURLOPT_XOAUTH2_BEARER ||                                      \
+   0)
+
+/* evaluates to true if option takes a curl_write_callback argument */
+#define _curl_is_write_cb_option(option)                                      \
+  ((option) == CURLOPT_HEADERFUNCTION ||                                      \
+   (option) == CURLOPT_WRITEFUNCTION)
+
+/* evaluates to true if option takes a curl_conv_callback argument */
+#define _curl_is_conv_cb_option(option)                                       \
+  ((option) == CURLOPT_CONV_TO_NETWORK_FUNCTION ||                            \
+   (option) == CURLOPT_CONV_FROM_NETWORK_FUNCTION ||                          \
+   (option) == CURLOPT_CONV_FROM_UTF8_FUNCTION)
+
+/* evaluates to true if option takes a data argument to pass to a callback */
+#define _curl_is_cb_data_option(option)                                       \
+  ((option) == CURLOPT_CHUNK_DATA ||                                          \
+   (option) == CURLOPT_CLOSESOCKETDATA ||                                     \
+   (option) == CURLOPT_DEBUGDATA ||                                           \
+   (option) == CURLOPT_FNMATCH_DATA ||                                        \
+   (option) == CURLOPT_HEADERDATA ||                                          \
+   (option) == CURLOPT_INTERLEAVEDATA ||                                      \
+   (option) == CURLOPT_IOCTLDATA ||                                           \
+   (option) == CURLOPT_OPENSOCKETDATA ||                                      \
+   (option) == CURLOPT_PRIVATE ||                                             \
+   (option) == CURLOPT_PROGRESSDATA ||                                        \
+   (option) == CURLOPT_READDATA ||                                            \
+   (option) == CURLOPT_SEEKDATA ||                                            \
+   (option) == CURLOPT_SOCKOPTDATA ||                                         \
+   (option) == CURLOPT_SSH_KEYDATA ||                                         \
+   (option) == CURLOPT_SSL_CTX_DATA ||                                        \
+   (option) == CURLOPT_WRITEDATA ||                                           \
+   0)
+
+/* evaluates to true if option takes a POST data argument (void* or char*) */
+#define _curl_is_postfields_option(option)                                    \
+  ((option) == CURLOPT_POSTFIELDS ||                                          \
+   (option) == CURLOPT_COPYPOSTFIELDS ||                                      \
+   0)
+
+/* evaluates to true if option takes a struct curl_slist * argument */
+#define _curl_is_slist_option(option)                                         \
+  ((option) == CURLOPT_HTTP200ALIASES ||                                      \
+   (option) == CURLOPT_HTTPHEADER ||                                          \
+   (option) == CURLOPT_MAIL_RCPT ||                                           \
+   (option) == CURLOPT_POSTQUOTE ||                                           \
+   (option) == CURLOPT_PREQUOTE ||                                            \
+   (option) == CURLOPT_PROXYHEADER ||                                         \
+   (option) == CURLOPT_QUOTE ||                                               \
+   (option) == CURLOPT_RESOLVE ||                                             \
+   (option) == CURLOPT_TELNETOPTIONS ||                                       \
+   0)
+
+/* groups of curl_easy_getinfo infos that take the same type of argument */
+
+/* evaluates to true if info expects a pointer to char * argument */
+#define _curl_is_string_info(info)                                            \
+  (CURLINFO_STRING < (info) && (info) < CURLINFO_LONG)
+
+/* evaluates to true if info expects a pointer to long argument */
+#define _curl_is_long_info(info)                                              \
+  (CURLINFO_LONG < (info) && (info) < CURLINFO_DOUBLE)
+
+/* evaluates to true if info expects a pointer to double argument */
+#define _curl_is_double_info(info)                                            \
+  (CURLINFO_DOUBLE < (info) && (info) < CURLINFO_SLIST)
+
+/* true if info expects a pointer to struct curl_slist * argument */
+#define _curl_is_slist_info(info)                                       \
+  (((info) == CURLINFO_SSL_ENGINES) || ((info) == CURLINFO_COOKIELIST))
+
+/* true if info expects a pointer to struct curl_tlssessioninfo * argument */
+#define _curl_is_tlssessioninfo_info(info)                              \
+  (((info) == CURLINFO_TLS_SSL_PTR) || ((info) == CURLINFO_TLS_SESSION))
+
+/* true if info expects a pointer to struct curl_certinfo * argument */
+#define _curl_is_certinfo_info(info) ((info) == CURLINFO_CERTINFO)
+
+/* true if info expects a pointer to struct curl_socket_t argument */
+#define _curl_is_socket_info(info)                                            \
+  (CURLINFO_SOCKET < (info) && (info) < CURLINFO_OFF_T)
+
+/* true if info expects a pointer to curl_off_t argument */
+#define _curl_is_off_t_info(info)                                             \
+  (CURLINFO_OFF_T < (info))
+
+
+/* typecheck helpers -- check whether given expression has requested type*/
+
+/* For pointers, you can use the _curl_is_ptr/_curl_is_arr macros,
+ * otherwise define a new macro. Search for __builtin_types_compatible_p
+ * in the GCC manual.
+ * NOTE: these macros MUST NOT EVALUATE their arguments! The argument is
+ * the actual expression passed to the curl_easy_setopt macro. This
+ * means that you can only apply the sizeof and __typeof__ operators, no
+ * == or whatsoever.
+ */
+
+/* XXX: should evaluate to true iff expr is a pointer */
+#define _curl_is_any_ptr(expr)                                                \
+  (sizeof(expr) == sizeof(void *))
+
+/* evaluates to true if expr is NULL */
+/* XXX: must not evaluate expr, so this check is not accurate */
+#define _curl_is_NULL(expr)                                                   \
+  (__builtin_types_compatible_p(__typeof__(expr), __typeof__(NULL)))
+
+/* evaluates to true if expr is type*, const type* or NULL */
+#define _curl_is_ptr(expr, type)                                              \
+  (_curl_is_NULL(expr) ||                                                     \
+   __builtin_types_compatible_p(__typeof__(expr), type *) ||                  \
+   __builtin_types_compatible_p(__typeof__(expr), const type *))
+
+/* evaluates to true if expr is one of type[], type*, NULL or const type* */
+#define _curl_is_arr(expr, type)                                              \
+  (_curl_is_ptr((expr), type) ||                                              \
+   __builtin_types_compatible_p(__typeof__(expr), type []))
+
+/* evaluates to true if expr is a string */
+#define _curl_is_string(expr)                                                 \
+  (_curl_is_arr((expr), char) ||                                              \
+   _curl_is_arr((expr), signed char) ||                                       \
+   _curl_is_arr((expr), unsigned char))
+
+/* evaluates to true if expr is a long (no matter the signedness)
+ * XXX: for now, int is also accepted (and therefore short and char, which
+ * are promoted to int when passed to a variadic function) */
+#define _curl_is_long(expr)                                                   \
+  (__builtin_types_compatible_p(__typeof__(expr), long) ||                    \
+   __builtin_types_compatible_p(__typeof__(expr), signed long) ||             \
+   __builtin_types_compatible_p(__typeof__(expr), unsigned long) ||           \
+   __builtin_types_compatible_p(__typeof__(expr), int) ||                     \
+   __builtin_types_compatible_p(__typeof__(expr), signed int) ||              \
+   __builtin_types_compatible_p(__typeof__(expr), unsigned int) ||            \
+   __builtin_types_compatible_p(__typeof__(expr), short) ||                   \
+   __builtin_types_compatible_p(__typeof__(expr), signed short) ||            \
+   __builtin_types_compatible_p(__typeof__(expr), unsigned short) ||          \
+   __builtin_types_compatible_p(__typeof__(expr), char) ||                    \
+   __builtin_types_compatible_p(__typeof__(expr), signed char) ||             \
+   __builtin_types_compatible_p(__typeof__(expr), unsigned char))
+
+/* evaluates to true if expr is of type curl_off_t */
+#define _curl_is_off_t(expr)                                                  \
+  (__builtin_types_compatible_p(__typeof__(expr), curl_off_t))
+
+/* evaluates to true if expr is abuffer suitable for CURLOPT_ERRORBUFFER */
+/* XXX: also check size of an char[] array? */
+#define _curl_is_error_buffer(expr)                                           \
+  (_curl_is_NULL(expr) ||                                                     \
+   __builtin_types_compatible_p(__typeof__(expr), char *) ||                  \
+   __builtin_types_compatible_p(__typeof__(expr), char[]))
+
+/* evaluates to true if expr is of type (const) void* or (const) FILE* */
+#if 0
+#define _curl_is_cb_data(expr)                                                \
+  (_curl_is_ptr((expr), void) ||                                              \
+   _curl_is_ptr((expr), FILE))
+#else /* be less strict */
+#define _curl_is_cb_data(expr)                                                \
+  _curl_is_any_ptr(expr)
+#endif
+
+/* evaluates to true if expr is of type FILE* */
+#define _curl_is_FILE(expr)                                             \
+  (_curl_is_NULL(expr) ||                                              \
+   (__builtin_types_compatible_p(__typeof__(expr), FILE *)))
+
+/* evaluates to true if expr can be passed as POST data (void* or char*) */
+#define _curl_is_postfields(expr)                                             \
+  (_curl_is_ptr((expr), void) ||                                              \
+   _curl_is_arr((expr), char))
+
+/* FIXME: the whole callback checking is messy...
+ * The idea is to tolerate char vs. void and const vs. not const
+ * pointers in arguments at least
+ */
+/* helper: __builtin_types_compatible_p distinguishes between functions and
+ * function pointers, hide it */
+#define _curl_callback_compatible(func, type)                                 \
+  (__builtin_types_compatible_p(__typeof__(func), type) ||                    \
+   __builtin_types_compatible_p(__typeof__(func) *, type))
+
+/* evaluates to true if expr is of type curl_read_callback or "similar" */
+#define _curl_is_read_cb(expr)                                          \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), __typeof__(fread) *) ||                  \
+   _curl_callback_compatible((expr), curl_read_callback) ||                   \
+   _curl_callback_compatible((expr), _curl_read_callback1) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback2) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback3) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback4) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback5) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback6))
+typedef size_t (*_curl_read_callback1)(char *, size_t, size_t, void *);
+typedef size_t (*_curl_read_callback2)(char *, size_t, size_t, const void *);
+typedef size_t (*_curl_read_callback3)(char *, size_t, size_t, FILE *);
+typedef size_t (*_curl_read_callback4)(void *, size_t, size_t, void *);
+typedef size_t (*_curl_read_callback5)(void *, size_t, size_t, const void *);
+typedef size_t (*_curl_read_callback6)(void *, size_t, size_t, FILE *);
+
+/* evaluates to true if expr is of type curl_write_callback or "similar" */
+#define _curl_is_write_cb(expr)                                               \
+  (_curl_is_read_cb(expr) ||                                            \
+   _curl_callback_compatible((expr), __typeof__(fwrite) *) ||                 \
+   _curl_callback_compatible((expr), curl_write_callback) ||                  \
+   _curl_callback_compatible((expr), _curl_write_callback1) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback2) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback3) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback4) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback5) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback6))
+typedef size_t (*_curl_write_callback1)(const char *, size_t, size_t, void *);
+typedef size_t (*_curl_write_callback2)(const char *, size_t, size_t,
+                                       const void *);
+typedef size_t (*_curl_write_callback3)(const char *, size_t, size_t, FILE *);
+typedef size_t (*_curl_write_callback4)(const void *, size_t, size_t, void *);
+typedef size_t (*_curl_write_callback5)(const void *, size_t, size_t,
+                                       const void *);
+typedef size_t (*_curl_write_callback6)(const void *, size_t, size_t, FILE *);
+
+/* evaluates to true if expr is of type curl_ioctl_callback or "similar" */
+#define _curl_is_ioctl_cb(expr)                                         \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_ioctl_callback) ||                  \
+   _curl_callback_compatible((expr), _curl_ioctl_callback1) ||                \
+   _curl_callback_compatible((expr), _curl_ioctl_callback2) ||                \
+   _curl_callback_compatible((expr), _curl_ioctl_callback3) ||                \
+   _curl_callback_compatible((expr), _curl_ioctl_callback4))
+typedef curlioerr (*_curl_ioctl_callback1)(CURL *, int, void *);
+typedef curlioerr (*_curl_ioctl_callback2)(CURL *, int, const void *);
+typedef curlioerr (*_curl_ioctl_callback3)(CURL *, curliocmd, void *);
+typedef curlioerr (*_curl_ioctl_callback4)(CURL *, curliocmd, const void *);
+
+/* evaluates to true if expr is of type curl_sockopt_callback or "similar" */
+#define _curl_is_sockopt_cb(expr)                                       \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_sockopt_callback) ||                \
+   _curl_callback_compatible((expr), _curl_sockopt_callback1) ||              \
+   _curl_callback_compatible((expr), _curl_sockopt_callback2))
+typedef int (*_curl_sockopt_callback1)(void *, curl_socket_t, curlsocktype);
+typedef int (*_curl_sockopt_callback2)(const void *, curl_socket_t,
+                                      curlsocktype);
+
+/* evaluates to true if expr is of type curl_opensocket_callback or
+   "similar" */
+#define _curl_is_opensocket_cb(expr)                                    \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_opensocket_callback) ||             \
+   _curl_callback_compatible((expr), _curl_opensocket_callback1) ||           \
+   _curl_callback_compatible((expr), _curl_opensocket_callback2) ||           \
+   _curl_callback_compatible((expr), _curl_opensocket_callback3) ||           \
+   _curl_callback_compatible((expr), _curl_opensocket_callback4))
+typedef curl_socket_t (*_curl_opensocket_callback1)
+  (void *, curlsocktype, struct curl_sockaddr *);
+typedef curl_socket_t (*_curl_opensocket_callback2)
+  (void *, curlsocktype, const struct curl_sockaddr *);
+typedef curl_socket_t (*_curl_opensocket_callback3)
+  (const void *, curlsocktype, struct curl_sockaddr *);
+typedef curl_socket_t (*_curl_opensocket_callback4)
+  (const void *, curlsocktype, const struct curl_sockaddr *);
+
+/* evaluates to true if expr is of type curl_progress_callback or "similar" */
+#define _curl_is_progress_cb(expr)                                      \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_progress_callback) ||               \
+   _curl_callback_compatible((expr), _curl_progress_callback1) ||             \
+   _curl_callback_compatible((expr), _curl_progress_callback2))
+typedef int (*_curl_progress_callback1)(void *,
+    double, double, double, double);
+typedef int (*_curl_progress_callback2)(const void *,
+    double, double, double, double);
+
+/* evaluates to true if expr is of type curl_debug_callback or "similar" */
+#define _curl_is_debug_cb(expr)                                         \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_debug_callback) ||                  \
+   _curl_callback_compatible((expr), _curl_debug_callback1) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback2) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback3) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback4) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback5) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback6) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback7) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback8))
+typedef int (*_curl_debug_callback1) (CURL *,
+    curl_infotype, char *, size_t, void *);
+typedef int (*_curl_debug_callback2) (CURL *,
+    curl_infotype, char *, size_t, const void *);
+typedef int (*_curl_debug_callback3) (CURL *,
+    curl_infotype, const char *, size_t, void *);
+typedef int (*_curl_debug_callback4) (CURL *,
+    curl_infotype, const char *, size_t, const void *);
+typedef int (*_curl_debug_callback5) (CURL *,
+    curl_infotype, unsigned char *, size_t, void *);
+typedef int (*_curl_debug_callback6) (CURL *,
+    curl_infotype, unsigned char *, size_t, const void *);
+typedef int (*_curl_debug_callback7) (CURL *,
+    curl_infotype, const unsigned char *, size_t, void *);
+typedef int (*_curl_debug_callback8) (CURL *,
+    curl_infotype, const unsigned char *, size_t, const void *);
+
+/* evaluates to true if expr is of type curl_ssl_ctx_callback or "similar" */
+/* this is getting even messier... */
+#define _curl_is_ssl_ctx_cb(expr)                                       \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_ssl_ctx_callback) ||                \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback1) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback2) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback3) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback4) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback5) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback6) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback7) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback8))
+typedef CURLcode (*_curl_ssl_ctx_callback1)(CURL *, void *, void *);
+typedef CURLcode (*_curl_ssl_ctx_callback2)(CURL *, void *, const void *);
+typedef CURLcode (*_curl_ssl_ctx_callback3)(CURL *, const void *, void *);
+typedef CURLcode (*_curl_ssl_ctx_callback4)(CURL *, const void *,
+                                            const void *);
+#ifdef HEADER_SSL_H
+/* hack: if we included OpenSSL's ssl.h, we know about SSL_CTX
+ * this will of course break if we're included before OpenSSL headers...
+ */
+typedef CURLcode (*_curl_ssl_ctx_callback5)(CURL *, SSL_CTX, void *);
+typedef CURLcode (*_curl_ssl_ctx_callback6)(CURL *, SSL_CTX, const void *);
+typedef CURLcode (*_curl_ssl_ctx_callback7)(CURL *, const SSL_CTX, void *);
+typedef CURLcode (*_curl_ssl_ctx_callback8)(CURL *, const SSL_CTX,
+                                           const void *);
+#else
+typedef _curl_ssl_ctx_callback1 _curl_ssl_ctx_callback5;
+typedef _curl_ssl_ctx_callback1 _curl_ssl_ctx_callback6;
+typedef _curl_ssl_ctx_callback1 _curl_ssl_ctx_callback7;
+typedef _curl_ssl_ctx_callback1 _curl_ssl_ctx_callback8;
+#endif
+
+/* evaluates to true if expr is of type curl_conv_callback or "similar" */
+#define _curl_is_conv_cb(expr)                                          \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_conv_callback) ||                   \
+   _curl_callback_compatible((expr), _curl_conv_callback1) ||                 \
+   _curl_callback_compatible((expr), _curl_conv_callback2) ||                 \
+   _curl_callback_compatible((expr), _curl_conv_callback3) ||                 \
+   _curl_callback_compatible((expr), _curl_conv_callback4))
+typedef CURLcode (*_curl_conv_callback1)(char *, size_t length);
+typedef CURLcode (*_curl_conv_callback2)(const char *, size_t length);
+typedef CURLcode (*_curl_conv_callback3)(void *, size_t length);
+typedef CURLcode (*_curl_conv_callback4)(const void *, size_t length);
+
+/* evaluates to true if expr is of type curl_seek_callback or "similar" */
+#define _curl_is_seek_cb(expr)                                          \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_seek_callback) ||                   \
+   _curl_callback_compatible((expr), _curl_seek_callback1) ||                 \
+   _curl_callback_compatible((expr), _curl_seek_callback2))
+typedef CURLcode (*_curl_seek_callback1)(void *, curl_off_t, int);
+typedef CURLcode (*_curl_seek_callback2)(const void *, curl_off_t, int);
+
+
+#endif /* __CURL_TYPECHECK_GCC_H */

ext/curl-7.58.0/x64/include/curl/curlver.h

@@ -0,0 +1,77 @@
+#ifndef __CURL_CURLVER_H
+#define __CURL_CURLVER_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+/* This header file contains nothing but libcurl version info, generated by
+   a script at release-time. This was made its own header file in 7.11.2 */
+
+/* This is the global package copyright */
+#define LIBCURL_COPYRIGHT "1996 - 2017 Daniel Stenberg, <daniel@haxx.se>."
+
+/* This is the version number of the libcurl package from which this header
+   file origins: */
+#define LIBCURL_VERSION "7.58.0"
+
+/* The numeric version number is also available "in parts" by using these
+   defines: */
+#define LIBCURL_VERSION_MAJOR 7
+#define LIBCURL_VERSION_MINOR 58
+#define LIBCURL_VERSION_PATCH 0
+
+/* This is the numeric version of the libcurl version number, meant for easier
+   parsing and comparions by programs. The LIBCURL_VERSION_NUM define will
+   always follow this syntax:
+
+         0xXXYYZZ
+
+   Where XX, YY and ZZ are the main version, release and patch numbers in
+   hexadecimal (using 8 bits each). All three numbers are always represented
+   using two digits.  1.2 would appear as "0x010200" while version 9.11.7
+   appears as "0x090b07".
+
+   This 6-digit (24 bits) hexadecimal number does not show pre-release number,
+   and it is always a greater number in a more recent release. It makes
+   comparisons with greater than and less than work.
+
+   Note: This define is the full hex number and _does not_ use the
+   CURL_VERSION_BITS() macro since curl's own configure script greps for it
+   and needs it to contain the full number.
+*/
+#define LIBCURL_VERSION_NUM 0x073a00
+
+/*
+ * This is the date and time when the full source package was created. The
+ * timestamp is not stored in git, as the timestamp is properly set in the
+ * tarballs by the maketgz script.
+ *
+ * The format of the date follows this template:
+ *
+ * "2007-11-23"
+ */
+#define LIBCURL_TIMESTAMP "2018-01-24"
+
+#define CURL_VERSION_BITS(x,y,z) ((x)<<16|(y)<<8|z)
+#define CURL_AT_LEAST_VERSION(x,y,z) \
+  (LIBCURL_VERSION_NUM >= CURL_VERSION_BITS(x, y, z))
+
+#endif /* __CURL_CURLVER_H */

ext/curl-7.58.0/x64/include/curl/easy.h

@@ -0,0 +1,102 @@
+#ifndef __CURL_EASY_H
+#define __CURL_EASY_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2016, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+#ifdef  __cplusplus
+extern "C" {
+#endif
+
+CURL_EXTERN CURL *curl_easy_init(void);
+CURL_EXTERN CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);
+CURL_EXTERN CURLcode curl_easy_perform(CURL *curl);
+CURL_EXTERN void curl_easy_cleanup(CURL *curl);
+
+/*
+ * NAME curl_easy_getinfo()
+ *
+ * DESCRIPTION
+ *
+ * Request internal information from the curl session with this function.  The
+ * third argument MUST be a pointer to a long, a pointer to a char * or a
+ * pointer to a double (as the documentation describes elsewhere).  The data
+ * pointed to will be filled in accordingly and can be relied upon only if the
+ * function returns CURLE_OK.  This function is intended to get used *AFTER* a
+ * performed transfer, all results from this function are undefined until the
+ * transfer is completed.
+ */
+CURL_EXTERN CURLcode curl_easy_getinfo(CURL *curl, CURLINFO info, ...);
+
+
+/*
+ * NAME curl_easy_duphandle()
+ *
+ * DESCRIPTION
+ *
+ * Creates a new curl session handle with the same options set for the handle
+ * passed in. Duplicating a handle could only be a matter of cloning data and
+ * options, internal state info and things like persistent connections cannot
+ * be transferred. It is useful in multithreaded applications when you can run
+ * curl_easy_duphandle() for each new thread to avoid a series of identical
+ * curl_easy_setopt() invokes in every thread.
+ */
+CURL_EXTERN CURL *curl_easy_duphandle(CURL *curl);
+
+/*
+ * NAME curl_easy_reset()
+ *
+ * DESCRIPTION
+ *
+ * Re-initializes a CURL handle to the default values. This puts back the
+ * handle to the same state as it was in when it was just created.
+ *
+ * It does keep: live connections, the Session ID cache, the DNS cache and the
+ * cookies.
+ */
+CURL_EXTERN void curl_easy_reset(CURL *curl);
+
+/*
+ * NAME curl_easy_recv()
+ *
+ * DESCRIPTION
+ *
+ * Receives data from the connected socket. Use after successful
+ * curl_easy_perform() with CURLOPT_CONNECT_ONLY option.
+ */
+CURL_EXTERN CURLcode curl_easy_recv(CURL *curl, void *buffer, size_t buflen,
+                                    size_t *n);
+
+/*
+ * NAME curl_easy_send()
+ *
+ * DESCRIPTION
+ *
+ * Sends data over the connected socket. Use after successful
+ * curl_easy_perform() with CURLOPT_CONNECT_ONLY option.
+ */
+CURL_EXTERN CURLcode curl_easy_send(CURL *curl, const void *buffer,
+                                    size_t buflen, size_t *n);
+
+#ifdef  __cplusplus
+}
+#endif
+
+#endif

ext/curl-7.58.0/x64/include/curl/mprintf.h

@@ -0,0 +1,50 @@
+#ifndef __CURL_MPRINTF_H
+#define __CURL_MPRINTF_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2016, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+#include <stdarg.h>
+#include <stdio.h> /* needed for FILE */
+#include "curl.h"  /* for CURL_EXTERN */
+
+#ifdef  __cplusplus
+extern "C" {
+#endif
+
+CURL_EXTERN int curl_mprintf(const char *format, ...);
+CURL_EXTERN int curl_mfprintf(FILE *fd, const char *format, ...);
+CURL_EXTERN int curl_msprintf(char *buffer, const char *format, ...);
+CURL_EXTERN int curl_msnprintf(char *buffer, size_t maxlength,
+                               const char *format, ...);
+CURL_EXTERN int curl_mvprintf(const char *format, va_list args);
+CURL_EXTERN int curl_mvfprintf(FILE *fd, const char *format, va_list args);
+CURL_EXTERN int curl_mvsprintf(char *buffer, const char *format, va_list args);
+CURL_EXTERN int curl_mvsnprintf(char *buffer, size_t maxlength,
+                                const char *format, va_list args);
+CURL_EXTERN char *curl_maprintf(const char *format, ...);
+CURL_EXTERN char *curl_mvaprintf(const char *format, va_list args);
+
+#ifdef  __cplusplus
+}
+#endif
+
+#endif /* __CURL_MPRINTF_H */

ext/curl-7.58.0/x64/include/curl/multi.h

@@ -0,0 +1,439 @@
+#ifndef __CURL_MULTI_H
+#define __CURL_MULTI_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+/*
+  This is an "external" header file. Don't give away any internals here!
+
+  GOALS
+
+  o Enable a "pull" interface. The application that uses libcurl decides where
+    and when to ask libcurl to get/send data.
+
+  o Enable multiple simultaneous transfers in the same thread without making it
+    complicated for the application.
+
+  o Enable the application to select() on its own file descriptors and curl's
+    file descriptors simultaneous easily.
+
+*/
+
+/*
+ * This header file should not really need to include "curl.h" since curl.h
+ * itself includes this file and we expect user applications to do #include
+ * <curl/curl.h> without the need for especially including multi.h.
+ *
+ * For some reason we added this include here at one point, and rather than to
+ * break existing (wrongly written) libcurl applications, we leave it as-is
+ * but with this warning attached.
+ */
+#include "curl.h"
+
+#ifdef  __cplusplus
+extern "C" {
+#endif
+
+#if defined(BUILDING_LIBCURL) || defined(CURL_STRICTER)
+typedef struct Curl_multi CURLM;
+#else
+typedef void CURLM;
+#endif
+
+typedef enum {
+  CURLM_CALL_MULTI_PERFORM = -1, /* please call curl_multi_perform() or
+                                    curl_multi_socket*() soon */
+  CURLM_OK,
+  CURLM_BAD_HANDLE,      /* the passed-in handle is not a valid CURLM handle */
+  CURLM_BAD_EASY_HANDLE, /* an easy handle was not good/valid */
+  CURLM_OUT_OF_MEMORY,   /* if you ever get this, you're in deep sh*t */
+  CURLM_INTERNAL_ERROR,  /* this is a libcurl bug */
+  CURLM_BAD_SOCKET,      /* the passed in socket argument did not match */
+  CURLM_UNKNOWN_OPTION,  /* curl_multi_setopt() with unsupported option */
+  CURLM_ADDED_ALREADY,   /* an easy handle already added to a multi handle was
+                            attempted to get added - again */
+  CURLM_LAST
+} CURLMcode;
+
+/* just to make code nicer when using curl_multi_socket() you can now check
+   for CURLM_CALL_MULTI_SOCKET too in the same style it works for
+   curl_multi_perform() and CURLM_CALL_MULTI_PERFORM */
+#define CURLM_CALL_MULTI_SOCKET CURLM_CALL_MULTI_PERFORM
+
+/* bitmask bits for CURLMOPT_PIPELINING */
+#define CURLPIPE_NOTHING   0L
+#define CURLPIPE_HTTP1     1L
+#define CURLPIPE_MULTIPLEX 2L
+
+typedef enum {
+  CURLMSG_NONE, /* first, not used */
+  CURLMSG_DONE, /* This easy handle has completed. 'result' contains
+                   the CURLcode of the transfer */
+  CURLMSG_LAST /* last, not used */
+} CURLMSG;
+
+struct CURLMsg {
+  CURLMSG msg;       /* what this message means */
+  CURL *easy_handle; /* the handle it concerns */
+  union {
+    void *whatever;    /* message-specific data */
+    CURLcode result;   /* return code for transfer */
+  } data;
+};
+typedef struct CURLMsg CURLMsg;
+
+/* Based on poll(2) structure and values.
+ * We don't use pollfd and POLL* constants explicitly
+ * to cover platforms without poll(). */
+#define CURL_WAIT_POLLIN    0x0001
+#define CURL_WAIT_POLLPRI   0x0002
+#define CURL_WAIT_POLLOUT   0x0004
+
+struct curl_waitfd {
+  curl_socket_t fd;
+  short events;
+  short revents; /* not supported yet */
+};
+
+/*
+ * Name:    curl_multi_init()
+ *
+ * Desc:    inititalize multi-style curl usage
+ *
+ * Returns: a new CURLM handle to use in all 'curl_multi' functions.
+ */
+CURL_EXTERN CURLM *curl_multi_init(void);
+
+/*
+ * Name:    curl_multi_add_handle()
+ *
+ * Desc:    add a standard curl handle to the multi stack
+ *
+ * Returns: CURLMcode type, general multi error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_add_handle(CURLM *multi_handle,
+                                            CURL *curl_handle);
+
+ /*
+  * Name:    curl_multi_remove_handle()
+  *
+  * Desc:    removes a curl handle from the multi stack again
+  *
+  * Returns: CURLMcode type, general multi error code.
+  */
+CURL_EXTERN CURLMcode curl_multi_remove_handle(CURLM *multi_handle,
+                                               CURL *curl_handle);
+
+ /*
+  * Name:    curl_multi_fdset()
+  *
+  * Desc:    Ask curl for its fd_set sets. The app can use these to select() or
+  *          poll() on. We want curl_multi_perform() called as soon as one of
+  *          them are ready.
+  *
+  * Returns: CURLMcode type, general multi error code.
+  */
+CURL_EXTERN CURLMcode curl_multi_fdset(CURLM *multi_handle,
+                                       fd_set *read_fd_set,
+                                       fd_set *write_fd_set,
+                                       fd_set *exc_fd_set,
+                                       int *max_fd);
+
+/*
+ * Name:     curl_multi_wait()
+ *
+ * Desc:     Poll on all fds within a CURLM set as well as any
+ *           additional fds passed to the function.
+ *
+ * Returns:  CURLMcode type, general multi error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_wait(CURLM *multi_handle,
+                                      struct curl_waitfd extra_fds[],
+                                      unsigned int extra_nfds,
+                                      int timeout_ms,
+                                      int *ret);
+
+ /*
+  * Name:    curl_multi_perform()
+  *
+  * Desc:    When the app thinks there's data available for curl it calls this
+  *          function to read/write whatever there is right now. This returns
+  *          as soon as the reads and writes are done. This function does not
+  *          require that there actually is data available for reading or that
+  *          data can be written, it can be called just in case. It returns
+  *          the number of handles that still transfer data in the second
+  *          argument's integer-pointer.
+  *
+  * Returns: CURLMcode type, general multi error code. *NOTE* that this only
+  *          returns errors etc regarding the whole multi stack. There might
+  *          still have occurred problems on invidual transfers even when this
+  *          returns OK.
+  */
+CURL_EXTERN CURLMcode curl_multi_perform(CURLM *multi_handle,
+                                         int *running_handles);
+
+ /*
+  * Name:    curl_multi_cleanup()
+  *
+  * Desc:    Cleans up and removes a whole multi stack. It does not free or
+  *          touch any individual easy handles in any way. We need to define
+  *          in what state those handles will be if this function is called
+  *          in the middle of a transfer.
+  *
+  * Returns: CURLMcode type, general multi error code.
+  */
+CURL_EXTERN CURLMcode curl_multi_cleanup(CURLM *multi_handle);
+
+/*
+ * Name:    curl_multi_info_read()
+ *
+ * Desc:    Ask the multi handle if there's any messages/informationals from
+ *          the individual transfers. Messages include informationals such as
+ *          error code from the transfer or just the fact that a transfer is
+ *          completed. More details on these should be written down as well.
+ *
+ *          Repeated calls to this function will return a new struct each
+ *          time, until a special "end of msgs" struct is returned as a signal
+ *          that there is no more to get at this point.
+ *
+ *          The data the returned pointer points to will not survive calling
+ *          curl_multi_cleanup().
+ *
+ *          The 'CURLMsg' struct is meant to be very simple and only contain
+ *          very basic information. If more involved information is wanted,
+ *          we will provide the particular "transfer handle" in that struct
+ *          and that should/could/would be used in subsequent
+ *          curl_easy_getinfo() calls (or similar). The point being that we
+ *          must never expose complex structs to applications, as then we'll
+ *          undoubtably get backwards compatibility problems in the future.
+ *
+ * Returns: A pointer to a filled-in struct, or NULL if it failed or ran out
+ *          of structs. It also writes the number of messages left in the
+ *          queue (after this read) in the integer the second argument points
+ *          to.
+ */
+CURL_EXTERN CURLMsg *curl_multi_info_read(CURLM *multi_handle,
+                                          int *msgs_in_queue);
+
+/*
+ * Name:    curl_multi_strerror()
+ *
+ * Desc:    The curl_multi_strerror function may be used to turn a CURLMcode
+ *          value into the equivalent human readable error string.  This is
+ *          useful for printing meaningful error messages.
+ *
+ * Returns: A pointer to a zero-terminated error message.
+ */
+CURL_EXTERN const char *curl_multi_strerror(CURLMcode);
+
+/*
+ * Name:    curl_multi_socket() and
+ *          curl_multi_socket_all()
+ *
+ * Desc:    An alternative version of curl_multi_perform() that allows the
+ *          application to pass in one of the file descriptors that have been
+ *          detected to have "action" on them and let libcurl perform.
+ *          See man page for details.
+ */
+#define CURL_POLL_NONE   0
+#define CURL_POLL_IN     1
+#define CURL_POLL_OUT    2
+#define CURL_POLL_INOUT  3
+#define CURL_POLL_REMOVE 4
+
+#define CURL_SOCKET_TIMEOUT CURL_SOCKET_BAD
+
+#define CURL_CSELECT_IN   0x01
+#define CURL_CSELECT_OUT  0x02
+#define CURL_CSELECT_ERR  0x04
+
+typedef int (*curl_socket_callback)(CURL *easy,      /* easy handle */
+                                    curl_socket_t s, /* socket */
+                                    int what,        /* see above */
+                                    void *userp,     /* private callback
+                                                        pointer */
+                                    void *socketp);  /* private socket
+                                                        pointer */
+/*
+ * Name:    curl_multi_timer_callback
+ *
+ * Desc:    Called by libcurl whenever the library detects a change in the
+ *          maximum number of milliseconds the app is allowed to wait before
+ *          curl_multi_socket() or curl_multi_perform() must be called
+ *          (to allow libcurl's timed events to take place).
+ *
+ * Returns: The callback should return zero.
+ */
+typedef int (*curl_multi_timer_callback)(CURLM *multi,    /* multi handle */
+                                         long timeout_ms, /* see above */
+                                         void *userp);    /* private callback
+                                                             pointer */
+
+CURL_EXTERN CURLMcode curl_multi_socket(CURLM *multi_handle, curl_socket_t s,
+                                        int *running_handles);
+
+CURL_EXTERN CURLMcode curl_multi_socket_action(CURLM *multi_handle,
+                                               curl_socket_t s,
+                                               int ev_bitmask,
+                                               int *running_handles);
+
+CURL_EXTERN CURLMcode curl_multi_socket_all(CURLM *multi_handle,
+                                            int *running_handles);
+
+#ifndef CURL_ALLOW_OLD_MULTI_SOCKET
+/* This macro below was added in 7.16.3 to push users who recompile to use
+   the new curl_multi_socket_action() instead of the old curl_multi_socket()
+*/
+#define curl_multi_socket(x,y,z) curl_multi_socket_action(x,y,0,z)
+#endif
+
+/*
+ * Name:    curl_multi_timeout()
+ *
+ * Desc:    Returns the maximum number of milliseconds the app is allowed to
+ *          wait before curl_multi_socket() or curl_multi_perform() must be
+ *          called (to allow libcurl's timed events to take place).
+ *
+ * Returns: CURLM error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_timeout(CURLM *multi_handle,
+                                         long *milliseconds);
+
+#undef CINIT /* re-using the same name as in curl.h */
+
+#ifdef CURL_ISOCPP
+#define CINIT(name,type,num) CURLMOPT_ ## name = CURLOPTTYPE_ ## type + num
+#else
+/* The macro "##" is ISO C, we assume pre-ISO C doesn't support it. */
+#define LONG          CURLOPTTYPE_LONG
+#define OBJECTPOINT   CURLOPTTYPE_OBJECTPOINT
+#define FUNCTIONPOINT CURLOPTTYPE_FUNCTIONPOINT
+#define OFF_T         CURLOPTTYPE_OFF_T
+#define CINIT(name,type,number) CURLMOPT_/**/name = type + number
+#endif
+
+typedef enum {
+  /* This is the socket callback function pointer */
+  CINIT(SOCKETFUNCTION, FUNCTIONPOINT, 1),
+
+  /* This is the argument passed to the socket callback */
+  CINIT(SOCKETDATA, OBJECTPOINT, 2),
+
+    /* set to 1 to enable pipelining for this multi handle */
+  CINIT(PIPELINING, LONG, 3),
+
+   /* This is the timer callback function pointer */
+  CINIT(TIMERFUNCTION, FUNCTIONPOINT, 4),
+
+  /* This is the argument passed to the timer callback */
+  CINIT(TIMERDATA, OBJECTPOINT, 5),
+
+  /* maximum number of entries in the connection cache */
+  CINIT(MAXCONNECTS, LONG, 6),
+
+  /* maximum number of (pipelining) connections to one host */
+  CINIT(MAX_HOST_CONNECTIONS, LONG, 7),
+
+  /* maximum number of requests in a pipeline */
+  CINIT(MAX_PIPELINE_LENGTH, LONG, 8),
+
+  /* a connection with a content-length longer than this
+     will not be considered for pipelining */
+  CINIT(CONTENT_LENGTH_PENALTY_SIZE, OFF_T, 9),
+
+  /* a connection with a chunk length longer than this
+     will not be considered for pipelining */
+  CINIT(CHUNK_LENGTH_PENALTY_SIZE, OFF_T, 10),
+
+  /* a list of site names(+port) that are blacklisted from
+     pipelining */
+  CINIT(PIPELINING_SITE_BL, OBJECTPOINT, 11),
+
+  /* a list of server types that are blacklisted from
+     pipelining */
+  CINIT(PIPELINING_SERVER_BL, OBJECTPOINT, 12),
+
+  /* maximum number of open connections in total */
+  CINIT(MAX_TOTAL_CONNECTIONS, LONG, 13),
+
+   /* This is the server push callback function pointer */
+  CINIT(PUSHFUNCTION, FUNCTIONPOINT, 14),
+
+  /* This is the argument passed to the server push callback */
+  CINIT(PUSHDATA, OBJECTPOINT, 15),
+
+  CURLMOPT_LASTENTRY /* the last unused */
+} CURLMoption;
+
+
+/*
+ * Name:    curl_multi_setopt()
+ *
+ * Desc:    Sets options for the multi handle.
+ *
+ * Returns: CURLM error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_setopt(CURLM *multi_handle,
+                                        CURLMoption option, ...);
+
+
+/*
+ * Name:    curl_multi_assign()
+ *
+ * Desc:    This function sets an association in the multi handle between the
+ *          given socket and a private pointer of the application. This is
+ *          (only) useful for curl_multi_socket uses.
+ *
+ * Returns: CURLM error code.
+ */
+CURL_EXTERN CURLMcode curl_multi_assign(CURLM *multi_handle,
+                                        curl_socket_t sockfd, void *sockp);
+
+
+/*
+ * Name: curl_push_callback
+ *
+ * Desc: This callback gets called when a new stream is being pushed by the
+ *       server. It approves or denies the new stream.
+ *
+ * Returns: CURL_PUSH_OK or CURL_PUSH_DENY.
+ */
+#define CURL_PUSH_OK   0
+#define CURL_PUSH_DENY 1
+
+struct curl_pushheaders;  /* forward declaration only */
+
+CURL_EXTERN char *curl_pushheader_bynum(struct curl_pushheaders *h,
+                                        size_t num);
+CURL_EXTERN char *curl_pushheader_byname(struct curl_pushheaders *h,
+                                         const char *name);
+
+typedef int (*curl_push_callback)(CURL *parent,
+                                  CURL *easy,
+                                  size_t num_headers,
+                                  struct curl_pushheaders *headers,
+                                  void *userp);
+
+#ifdef __cplusplus
+} /* end of extern "C" */
+#endif
+
+#endif

ext/curl-7.58.0/x64/include/curl/stdcheaders.h

@@ -0,0 +1,33 @@
+#ifndef __STDC_HEADERS_H
+#define __STDC_HEADERS_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2016, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+#include <sys/types.h>
+
+size_t fread(void *, size_t, size_t, FILE *);
+size_t fwrite(const void *, size_t, size_t, FILE *);
+
+int strcasecmp(const char *, const char *);
+int strncasecmp(const char *, const char *, size_t);
+
+#endif /* __STDC_HEADERS_H */

ext/curl-7.58.0/x64/include/curl/system.h

@@ -0,0 +1,473 @@
+#ifndef __CURL_SYSTEM_H
+#define __CURL_SYSTEM_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+/*
+ * Try to keep one section per platform, compiler and architecture, otherwise,
+ * if an existing section is reused for a different one and later on the
+ * original is adjusted, probably the piggybacking one can be adversely
+ * changed.
+ *
+ * In order to differentiate between platforms/compilers/architectures use
+ * only compiler built in predefined preprocessor symbols.
+ *
+ * curl_off_t
+ * ----------
+ *
+ * For any given platform/compiler curl_off_t must be typedef'ed to a 64-bit
+ * wide signed integral data type. The width of this data type must remain
+ * constant and independent of any possible large file support settings.
+ *
+ * As an exception to the above, curl_off_t shall be typedef'ed to a 32-bit
+ * wide signed integral data type if there is no 64-bit type.
+ *
+ * As a general rule, curl_off_t shall not be mapped to off_t. This rule shall
+ * only be violated if off_t is the only 64-bit data type available and the
+ * size of off_t is independent of large file support settings. Keep your
+ * build on the safe side avoiding an off_t gating.  If you have a 64-bit
+ * off_t then take for sure that another 64-bit data type exists, dig deeper
+ * and you will find it.
+ *
+ */
+
+#if defined(__DJGPP__) || defined(__GO32__)
+#  if defined(__DJGPP__) && (__DJGPP__ > 1)
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__SALFORDC__)
+#  define CURL_TYPEOF_CURL_OFF_T     long
+#  define CURL_FORMAT_CURL_OFF_T     "ld"
+#  define CURL_FORMAT_CURL_OFF_TU    "lu"
+#  define CURL_SUFFIX_CURL_OFF_T     L
+#  define CURL_SUFFIX_CURL_OFF_TU    UL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__BORLANDC__)
+#  if (__BORLANDC__ < 0x520)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "I64d"
+#    define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#    define CURL_SUFFIX_CURL_OFF_T     i64
+#    define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__TURBOC__)
+#  define CURL_TYPEOF_CURL_OFF_T     long
+#  define CURL_FORMAT_CURL_OFF_T     "ld"
+#  define CURL_FORMAT_CURL_OFF_TU    "lu"
+#  define CURL_SUFFIX_CURL_OFF_T     L
+#  define CURL_SUFFIX_CURL_OFF_TU    UL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__WATCOMC__)
+#  if defined(__386__)
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "I64d"
+#    define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#    define CURL_SUFFIX_CURL_OFF_T     i64
+#    define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__POCC__)
+#  if (__POCC__ < 280)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  elif defined(_MSC_VER)
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "I64d"
+#    define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#    define CURL_SUFFIX_CURL_OFF_T     i64
+#    define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__LCC__)
+#  define CURL_TYPEOF_CURL_OFF_T     long
+#  define CURL_FORMAT_CURL_OFF_T     "ld"
+#  define CURL_FORMAT_CURL_OFF_TU    "lu"
+#  define CURL_SUFFIX_CURL_OFF_T     L
+#  define CURL_SUFFIX_CURL_OFF_TU    UL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__SYMBIAN32__)
+#  if defined(__EABI__)  /* Treat all ARM compilers equally */
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  elif defined(__CW32__)
+#    pragma longlong on
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  elif defined(__VC32__)
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T unsigned int
+
+#elif defined(__MWERKS__)
+#  define CURL_TYPEOF_CURL_OFF_T     long long
+#  define CURL_FORMAT_CURL_OFF_T     "lld"
+#  define CURL_FORMAT_CURL_OFF_TU    "llu"
+#  define CURL_SUFFIX_CURL_OFF_T     LL
+#  define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(_WIN32_WCE)
+#  define CURL_TYPEOF_CURL_OFF_T     __int64
+#  define CURL_FORMAT_CURL_OFF_T     "I64d"
+#  define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#  define CURL_SUFFIX_CURL_OFF_T     i64
+#  define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__MINGW32__)
+#  define CURL_TYPEOF_CURL_OFF_T     long long
+#  define CURL_FORMAT_CURL_OFF_T     "I64d"
+#  define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#  define CURL_SUFFIX_CURL_OFF_T     LL
+#  define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#  define CURL_PULL_SYS_TYPES_H      1
+#  define CURL_PULL_WS2TCPIP_H       1
+
+#elif defined(__VMS)
+#  if defined(__VAX)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T unsigned int
+
+#elif defined(__OS400__)
+#  if defined(__ILEC400__)
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#    define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#    define CURL_PULL_SYS_TYPES_H      1
+#    define CURL_PULL_SYS_SOCKET_H     1
+#  endif
+
+#elif defined(__MVS__)
+#  if defined(__IBMC__) || defined(__IBMCPP__)
+#    if defined(_ILP32)
+#    elif defined(_LP64)
+#    endif
+#    if defined(_LONG_LONG)
+#      define CURL_TYPEOF_CURL_OFF_T     long long
+#      define CURL_FORMAT_CURL_OFF_T     "lld"
+#      define CURL_FORMAT_CURL_OFF_TU    "llu"
+#      define CURL_SUFFIX_CURL_OFF_T     LL
+#      define CURL_SUFFIX_CURL_OFF_TU    ULL
+#    elif defined(_LP64)
+#      define CURL_TYPEOF_CURL_OFF_T     long
+#      define CURL_FORMAT_CURL_OFF_T     "ld"
+#      define CURL_FORMAT_CURL_OFF_TU    "lu"
+#      define CURL_SUFFIX_CURL_OFF_T     L
+#      define CURL_SUFFIX_CURL_OFF_TU    UL
+#    else
+#      define CURL_TYPEOF_CURL_OFF_T     long
+#      define CURL_FORMAT_CURL_OFF_T     "ld"
+#      define CURL_FORMAT_CURL_OFF_TU    "lu"
+#      define CURL_SUFFIX_CURL_OFF_T     L
+#      define CURL_SUFFIX_CURL_OFF_TU    UL
+#    endif
+#    define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#    define CURL_PULL_SYS_TYPES_H      1
+#    define CURL_PULL_SYS_SOCKET_H     1
+#  endif
+
+#elif defined(__370__)
+#  if defined(__IBMC__) || defined(__IBMCPP__)
+#    if defined(_ILP32)
+#    elif defined(_LP64)
+#    endif
+#    if defined(_LONG_LONG)
+#      define CURL_TYPEOF_CURL_OFF_T     long long
+#      define CURL_FORMAT_CURL_OFF_T     "lld"
+#      define CURL_FORMAT_CURL_OFF_TU    "llu"
+#      define CURL_SUFFIX_CURL_OFF_T     LL
+#      define CURL_SUFFIX_CURL_OFF_TU    ULL
+#    elif defined(_LP64)
+#      define CURL_TYPEOF_CURL_OFF_T     long
+#      define CURL_FORMAT_CURL_OFF_T     "ld"
+#      define CURL_FORMAT_CURL_OFF_TU    "lu"
+#      define CURL_SUFFIX_CURL_OFF_T     L
+#      define CURL_SUFFIX_CURL_OFF_TU    UL
+#    else
+#      define CURL_TYPEOF_CURL_OFF_T     long
+#      define CURL_FORMAT_CURL_OFF_T     "ld"
+#      define CURL_FORMAT_CURL_OFF_TU    "lu"
+#      define CURL_SUFFIX_CURL_OFF_T     L
+#      define CURL_SUFFIX_CURL_OFF_TU    UL
+#    endif
+#    define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#    define CURL_PULL_SYS_TYPES_H      1
+#    define CURL_PULL_SYS_SOCKET_H     1
+#  endif
+
+#elif defined(TPF)
+#  define CURL_TYPEOF_CURL_OFF_T     long
+#  define CURL_FORMAT_CURL_OFF_T     "ld"
+#  define CURL_FORMAT_CURL_OFF_TU    "lu"
+#  define CURL_SUFFIX_CURL_OFF_T     L
+#  define CURL_SUFFIX_CURL_OFF_TU    UL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+#elif defined(__TINYC__) /* also known as tcc */
+
+#  define CURL_TYPEOF_CURL_OFF_T     long long
+#  define CURL_FORMAT_CURL_OFF_T     "lld"
+#  define CURL_FORMAT_CURL_OFF_TU    "llu"
+#  define CURL_SUFFIX_CURL_OFF_T     LL
+#  define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#  define CURL_PULL_SYS_TYPES_H      1
+#  define CURL_PULL_SYS_SOCKET_H     1
+
+#elif defined(__SUNPRO_C) /* Oracle Solaris Studio */
+#  if !defined(__LP64) && (defined(__ILP32) ||                          \
+                           defined(__i386) || defined(__sparcv8))
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  elif defined(__LP64) || \
+        defined(__amd64) || defined(__sparcv9)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#  define CURL_PULL_SYS_TYPES_H      1
+#  define CURL_PULL_SYS_SOCKET_H     1
+
+/* ===================================== */
+/*    KEEP MSVC THE PENULTIMATE ENTRY    */
+/* ===================================== */
+
+#elif defined(_MSC_VER)
+#  if (_MSC_VER >= 900) && (_INTEGRAL_MAX_BITS >= 64)
+#    define CURL_TYPEOF_CURL_OFF_T     __int64
+#    define CURL_FORMAT_CURL_OFF_T     "I64d"
+#    define CURL_FORMAT_CURL_OFF_TU    "I64u"
+#    define CURL_SUFFIX_CURL_OFF_T     i64
+#    define CURL_SUFFIX_CURL_OFF_TU    ui64
+#  else
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T int
+
+/* ===================================== */
+/*    KEEP GENERIC GCC THE LAST ENTRY    */
+/* ===================================== */
+
+#elif defined(__GNUC__)
+#  if !defined(__LP64__) &&                                             \
+  (defined(__ILP32__) || defined(__i386__) || defined(__hppa__) ||      \
+   defined(__ppc__) || defined(__powerpc__) || defined(__arm__) ||      \
+   defined(__sparc__) || defined(__mips__) || defined(__sh__) ||        \
+   defined(__XTENSA__) ||                                               \
+   (defined(__SIZEOF_LONG__) && __SIZEOF_LONG__ == 4)  ||               \
+   (defined(__LONG_MAX__) && __LONG_MAX__ == 2147483647L))
+#    define CURL_TYPEOF_CURL_OFF_T     long long
+#    define CURL_FORMAT_CURL_OFF_T     "lld"
+#    define CURL_FORMAT_CURL_OFF_TU    "llu"
+#    define CURL_SUFFIX_CURL_OFF_T     LL
+#    define CURL_SUFFIX_CURL_OFF_TU    ULL
+#  elif defined(__LP64__) || \
+        defined(__x86_64__) || defined(__ppc64__) || defined(__sparc64__) || \
+        (defined(__SIZEOF_LONG__) && __SIZEOF_LONG__ == 8) || \
+        (defined(__LONG_MAX__) && __LONG_MAX__ == 9223372036854775807L)
+#    define CURL_TYPEOF_CURL_OFF_T     long
+#    define CURL_FORMAT_CURL_OFF_T     "ld"
+#    define CURL_FORMAT_CURL_OFF_TU    "lu"
+#    define CURL_SUFFIX_CURL_OFF_T     L
+#    define CURL_SUFFIX_CURL_OFF_TU    UL
+#  endif
+#  define CURL_TYPEOF_CURL_SOCKLEN_T socklen_t
+#  define CURL_PULL_SYS_TYPES_H      1
+#  define CURL_PULL_SYS_SOCKET_H     1
+
+#else
+/* generic "safe guess" on old 32 bit style */
+# define CURL_TYPEOF_CURL_OFF_T     long
+# define CURL_FORMAT_CURL_OFF_T     "ld"
+# define CURL_FORMAT_CURL_OFF_TU    "lu"
+# define CURL_SUFFIX_CURL_OFF_T     L
+# define CURL_SUFFIX_CURL_OFF_TU    UL
+# define CURL_TYPEOF_CURL_SOCKLEN_T int
+#endif
+
+#ifdef _AIX
+/* AIX needs <sys/poll.h> */
+#define CURL_PULL_SYS_POLL_H
+#endif
+
+
+/* CURL_PULL_WS2TCPIP_H is defined above when inclusion of header file  */
+/* ws2tcpip.h is required here to properly make type definitions below. */
+#ifdef CURL_PULL_WS2TCPIP_H
+#  include <winsock2.h>
+#  include <windows.h>
+#  include <ws2tcpip.h>
+#endif
+
+/* CURL_PULL_SYS_TYPES_H is defined above when inclusion of header file  */
+/* sys/types.h is required here to properly make type definitions below. */
+#ifdef CURL_PULL_SYS_TYPES_H
+#  include <sys/types.h>
+#endif
+
+/* CURL_PULL_SYS_SOCKET_H is defined above when inclusion of header file  */
+/* sys/socket.h is required here to properly make type definitions below. */
+#ifdef CURL_PULL_SYS_SOCKET_H
+#  include <sys/socket.h>
+#endif
+
+/* CURL_PULL_SYS_POLL_H is defined above when inclusion of header file    */
+/* sys/poll.h is required here to properly make type definitions below.   */
+#ifdef CURL_PULL_SYS_POLL_H
+#  include <sys/poll.h>
+#endif
+
+/* Data type definition of curl_socklen_t. */
+#ifdef CURL_TYPEOF_CURL_SOCKLEN_T
+  typedef CURL_TYPEOF_CURL_SOCKLEN_T curl_socklen_t;
+#endif
+
+/* Data type definition of curl_off_t. */
+
+#ifdef CURL_TYPEOF_CURL_OFF_T
+  typedef CURL_TYPEOF_CURL_OFF_T curl_off_t;
+#endif
+
+/*
+ * CURL_ISOCPP and CURL_OFF_T_C definitions are done here in order to allow
+ * these to be visible and exported by the external libcurl interface API,
+ * while also making them visible to the library internals, simply including
+ * curl_setup.h, without actually needing to include curl.h internally.
+ * If some day this section would grow big enough, all this should be moved
+ * to its own header file.
+ */
+
+/*
+ * Figure out if we can use the ## preprocessor operator, which is supported
+ * by ISO/ANSI C and C++. Some compilers support it without setting __STDC__
+ * or  __cplusplus so we need to carefully check for them too.
+ */
+
+#if defined(__STDC__) || defined(_MSC_VER) || defined(__cplusplus) || \
+  defined(__HP_aCC) || defined(__BORLANDC__) || defined(__LCC__) || \
+  defined(__POCC__) || defined(__SALFORDC__) || defined(__HIGHC__) || \
+  defined(__ILEC400__)
+  /* This compiler is believed to have an ISO compatible preprocessor */
+#define CURL_ISOCPP
+#else
+  /* This compiler is believed NOT to have an ISO compatible preprocessor */
+#undef CURL_ISOCPP
+#endif
+
+/*
+ * Macros for minimum-width signed and unsigned curl_off_t integer constants.
+ */
+
+#if defined(__BORLANDC__) && (__BORLANDC__ == 0x0551)
+#  define __CURL_OFF_T_C_HLPR2(x) x
+#  define __CURL_OFF_T_C_HLPR1(x) __CURL_OFF_T_C_HLPR2(x)
+#  define CURL_OFF_T_C(Val)  __CURL_OFF_T_C_HLPR1(Val) ## \
+                             __CURL_OFF_T_C_HLPR1(CURL_SUFFIX_CURL_OFF_T)
+#  define CURL_OFF_TU_C(Val) __CURL_OFF_T_C_HLPR1(Val) ## \
+                             __CURL_OFF_T_C_HLPR1(CURL_SUFFIX_CURL_OFF_TU)
+#else
+#  ifdef CURL_ISOCPP
+#    define __CURL_OFF_T_C_HLPR2(Val,Suffix) Val ## Suffix
+#  else
+#    define __CURL_OFF_T_C_HLPR2(Val,Suffix) Val/**/Suffix
+#  endif
+#  define __CURL_OFF_T_C_HLPR1(Val,Suffix) __CURL_OFF_T_C_HLPR2(Val,Suffix)
+#  define CURL_OFF_T_C(Val)  __CURL_OFF_T_C_HLPR1(Val,CURL_SUFFIX_CURL_OFF_T)
+#  define CURL_OFF_TU_C(Val) __CURL_OFF_T_C_HLPR1(Val,CURL_SUFFIX_CURL_OFF_TU)
+#endif
+
+#endif /* __CURL_SYSTEM_H */

ext/curl-7.58.0/x64/include/curl/typecheck-gcc.h

@@ -0,0 +1,683 @@
+#ifndef __CURL_TYPECHECK_GCC_H
+#define __CURL_TYPECHECK_GCC_H
+/***************************************************************************
+ *                                  _   _ ____  _
+ *  Project                     ___| | | |  _ \| |
+ *                             / __| | | | |_) | |
+ *                            | (__| |_| |  _ <| |___
+ *                             \___|\___/|_| \_\_____|
+ *
+ * Copyright (C) 1998 - 2017, Daniel Stenberg, <daniel@haxx.se>, et al.
+ *
+ * This software is licensed as described in the file COPYING, which
+ * you should have received as part of this distribution. The terms
+ * are also available at https://curl.haxx.se/docs/copyright.html.
+ *
+ * You may opt to use, copy, modify, merge, publish, distribute and/or sell
+ * copies of the Software, and permit persons to whom the Software is
+ * furnished to do so, under the terms of the COPYING file.
+ *
+ * This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY
+ * KIND, either express or implied.
+ *
+ ***************************************************************************/
+
+/* wraps curl_easy_setopt() with typechecking */
+
+/* To add a new kind of warning, add an
+ *   if(_curl_is_sometype_option(_curl_opt))
+ *     if(!_curl_is_sometype(value))
+ *       _curl_easy_setopt_err_sometype();
+ * block and define _curl_is_sometype_option, _curl_is_sometype and
+ * _curl_easy_setopt_err_sometype below
+ *
+ * NOTE: We use two nested 'if' statements here instead of the && operator, in
+ *       order to work around gcc bug #32061.  It affects only gcc 4.3.x/4.4.x
+ *       when compiling with -Wlogical-op.
+ *
+ * To add an option that uses the same type as an existing option, you'll just
+ * need to extend the appropriate _curl_*_option macro
+ */
+#define curl_easy_setopt(handle, option, value)                               \
+__extension__ ({                                                              \
+  __typeof__(option) _curl_opt = option;                                     \
+  if(__builtin_constant_p(_curl_opt)) {                                       \
+    if(_curl_is_long_option(_curl_opt))                                       \
+      if(!_curl_is_long(value))                                               \
+        _curl_easy_setopt_err_long();                                         \
+    if(_curl_is_off_t_option(_curl_opt))                                      \
+      if(!_curl_is_off_t(value))                                              \
+        _curl_easy_setopt_err_curl_off_t();                                   \
+    if(_curl_is_string_option(_curl_opt))                                     \
+      if(!_curl_is_string(value))                                             \
+        _curl_easy_setopt_err_string();                                       \
+    if(_curl_is_write_cb_option(_curl_opt))                                   \
+      if(!_curl_is_write_cb(value))                                           \
+        _curl_easy_setopt_err_write_callback();                               \
+    if((_curl_opt) == CURLOPT_READFUNCTION)                                   \
+      if(!_curl_is_read_cb(value))                                            \
+        _curl_easy_setopt_err_read_cb();                                      \
+    if((_curl_opt) == CURLOPT_IOCTLFUNCTION)                                  \
+      if(!_curl_is_ioctl_cb(value))                                           \
+        _curl_easy_setopt_err_ioctl_cb();                                     \
+    if((_curl_opt) == CURLOPT_SOCKOPTFUNCTION)                                \
+      if(!_curl_is_sockopt_cb(value))                                         \
+        _curl_easy_setopt_err_sockopt_cb();                                   \
+    if((_curl_opt) == CURLOPT_OPENSOCKETFUNCTION)                             \
+      if(!_curl_is_opensocket_cb(value))                                      \
+        _curl_easy_setopt_err_opensocket_cb();                                \
+    if((_curl_opt) == CURLOPT_PROGRESSFUNCTION)                               \
+      if(!_curl_is_progress_cb(value))                                        \
+        _curl_easy_setopt_err_progress_cb();                                  \
+    if((_curl_opt) == CURLOPT_DEBUGFUNCTION)                                  \
+      if(!_curl_is_debug_cb(value))                                           \
+        _curl_easy_setopt_err_debug_cb();                                     \
+    if((_curl_opt) == CURLOPT_SSL_CTX_FUNCTION)                               \
+      if(!_curl_is_ssl_ctx_cb(value))                                         \
+        _curl_easy_setopt_err_ssl_ctx_cb();                                   \
+    if(_curl_is_conv_cb_option(_curl_opt))                                    \
+      if(!_curl_is_conv_cb(value))                                            \
+        _curl_easy_setopt_err_conv_cb();                                      \
+    if((_curl_opt) == CURLOPT_SEEKFUNCTION)                                   \
+      if(!_curl_is_seek_cb(value))                                            \
+        _curl_easy_setopt_err_seek_cb();                                      \
+    if(_curl_is_cb_data_option(_curl_opt))                                    \
+      if(!_curl_is_cb_data(value))                                            \
+        _curl_easy_setopt_err_cb_data();                                      \
+    if((_curl_opt) == CURLOPT_ERRORBUFFER)                                    \
+      if(!_curl_is_error_buffer(value))                                       \
+        _curl_easy_setopt_err_error_buffer();                                 \
+    if((_curl_opt) == CURLOPT_STDERR)                                         \
+      if(!_curl_is_FILE(value))                                               \
+        _curl_easy_setopt_err_FILE();                                         \
+    if(_curl_is_postfields_option(_curl_opt))                                 \
+      if(!_curl_is_postfields(value))                                         \
+        _curl_easy_setopt_err_postfields();                                   \
+    if((_curl_opt) == CURLOPT_HTTPPOST)                                       \
+      if(!_curl_is_arr((value), struct curl_httppost))                        \
+        _curl_easy_setopt_err_curl_httpost();                                 \
+    if((_curl_opt) == CURLOPT_MIMEPOST)                                       \
+      if(!_curl_is_ptr((value), curl_mime))                                   \
+        _curl_easy_setopt_err_curl_mimepost();                                \
+    if(_curl_is_slist_option(_curl_opt))                                      \
+      if(!_curl_is_arr((value), struct curl_slist))                           \
+        _curl_easy_setopt_err_curl_slist();                                   \
+    if((_curl_opt) == CURLOPT_SHARE)                                          \
+      if(!_curl_is_ptr((value), CURLSH))                                      \
+        _curl_easy_setopt_err_CURLSH();                                       \
+  }                                                                           \
+  curl_easy_setopt(handle, _curl_opt, value);                                 \
+})
+
+/* wraps curl_easy_getinfo() with typechecking */
+/* FIXME: don't allow const pointers */
+#define curl_easy_getinfo(handle, info, arg)                                  \
+__extension__ ({                                                              \
+  __typeof__(info) _curl_info = info;                                         \
+  if(__builtin_constant_p(_curl_info)) {                                      \
+    if(_curl_is_string_info(_curl_info))                                      \
+      if(!_curl_is_arr((arg), char *))                                        \
+        _curl_easy_getinfo_err_string();                                      \
+    if(_curl_is_long_info(_curl_info))                                        \
+      if(!_curl_is_arr((arg), long))                                          \
+        _curl_easy_getinfo_err_long();                                        \
+    if(_curl_is_double_info(_curl_info))                                      \
+      if(!_curl_is_arr((arg), double))                                        \
+        _curl_easy_getinfo_err_double();                                      \
+    if(_curl_is_slist_info(_curl_info))                                       \
+      if(!_curl_is_arr((arg), struct curl_slist *))                           \
+        _curl_easy_getinfo_err_curl_slist();                                  \
+    if(_curl_is_tlssessioninfo_info(_curl_info))                              \
+      if(!_curl_is_arr((arg), struct curl_tlssessioninfo *))                  \
+        _curl_easy_getinfo_err_curl_tlssesssioninfo();                        \
+    if(_curl_is_certinfo_info(_curl_info))                                    \
+      if(!_curl_is_arr((arg), struct curl_certinfo *))                        \
+        _curl_easy_getinfo_err_curl_certinfo();                               \
+    if(_curl_is_socket_info(_curl_info))                                      \
+      if(!_curl_is_arr((arg), curl_socket_t))                                 \
+        _curl_easy_getinfo_err_curl_socket();                                 \
+    if(_curl_is_off_t_info(_curl_info))                                       \
+      if(!_curl_is_arr((arg), curl_off_t))                                    \
+        _curl_easy_getinfo_err_curl_off_t();                                  \
+  }                                                                           \
+  curl_easy_getinfo(handle, _curl_info, arg);                                 \
+})
+
+/* TODO: typechecking for curl_share_setopt() and curl_multi_setopt(),
+ * for now just make sure that the functions are called with three
+ * arguments
+ */
+#define curl_share_setopt(share,opt,param) curl_share_setopt(share,opt,param)
+#define curl_multi_setopt(handle,opt,param) curl_multi_setopt(handle,opt,param)
+
+
+/* the actual warnings, triggered by calling the _curl_easy_setopt_err*
+ * functions */
+
+/* To define a new warning, use _CURL_WARNING(identifier, "message") */
+#define _CURL_WARNING(id, message)                                            \
+  static void __attribute__((__warning__(message)))                           \
+  __attribute__((__unused__)) __attribute__((__noinline__))                   \
+  id(void) { __asm__(""); }
+
+_CURL_WARNING(_curl_easy_setopt_err_long,
+  "curl_easy_setopt expects a long argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_curl_off_t,
+  "curl_easy_setopt expects a curl_off_t argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_string,
+              "curl_easy_setopt expects a "
+              "string ('char *' or char[]) argument for this option"
+  )
+_CURL_WARNING(_curl_easy_setopt_err_write_callback,
+  "curl_easy_setopt expects a curl_write_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_read_cb,
+  "curl_easy_setopt expects a curl_read_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_ioctl_cb,
+  "curl_easy_setopt expects a curl_ioctl_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_sockopt_cb,
+  "curl_easy_setopt expects a curl_sockopt_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_opensocket_cb,
+              "curl_easy_setopt expects a "
+              "curl_opensocket_callback argument for this option"
+  )
+_CURL_WARNING(_curl_easy_setopt_err_progress_cb,
+  "curl_easy_setopt expects a curl_progress_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_debug_cb,
+  "curl_easy_setopt expects a curl_debug_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_ssl_ctx_cb,
+  "curl_easy_setopt expects a curl_ssl_ctx_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_conv_cb,
+  "curl_easy_setopt expects a curl_conv_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_seek_cb,
+  "curl_easy_setopt expects a curl_seek_callback argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_cb_data,
+              "curl_easy_setopt expects a "
+              "private data pointer as argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_error_buffer,
+              "curl_easy_setopt expects a "
+              "char buffer of CURL_ERROR_SIZE as argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_FILE,
+  "curl_easy_setopt expects a 'FILE *' argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_postfields,
+  "curl_easy_setopt expects a 'void *' or 'char *' argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_curl_httpost,
+              "curl_easy_setopt expects a 'struct curl_httppost *' "
+              "argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_curl_mimepost,
+              "curl_easy_setopt expects a 'curl_mime *' "
+              "argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_curl_slist,
+  "curl_easy_setopt expects a 'struct curl_slist *' argument for this option")
+_CURL_WARNING(_curl_easy_setopt_err_CURLSH,
+  "curl_easy_setopt expects a CURLSH* argument for this option")
+
+_CURL_WARNING(_curl_easy_getinfo_err_string,
+  "curl_easy_getinfo expects a pointer to 'char *' for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_long,
+  "curl_easy_getinfo expects a pointer to long for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_double,
+  "curl_easy_getinfo expects a pointer to double for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_slist,
+  "curl_easy_getinfo expects a pointer to 'struct curl_slist *' for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_tlssesssioninfo,
+              "curl_easy_getinfo expects a pointer to "
+              "'struct curl_tlssessioninfo *' for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_certinfo,
+              "curl_easy_getinfo expects a pointer to "
+              "'struct curl_certinfo *' for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_socket,
+  "curl_easy_getinfo expects a pointer to curl_socket_t for this info")
+_CURL_WARNING(_curl_easy_getinfo_err_curl_off_t,
+  "curl_easy_getinfo expects a pointer to curl_off_t for this info")
+
+/* groups of curl_easy_setops options that take the same type of argument */
+
+/* To add a new option to one of the groups, just add
+ *   (option) == CURLOPT_SOMETHING
+ * to the or-expression. If the option takes a long or curl_off_t, you don't
+ * have to do anything
+ */
+
+/* evaluates to true if option takes a long argument */
+#define _curl_is_long_option(option)                                          \
+  (0 < (option) && (option) < CURLOPTTYPE_OBJECTPOINT)
+
+#define _curl_is_off_t_option(option)                                         \
+  ((option) > CURLOPTTYPE_OFF_T)
+
+/* evaluates to true if option takes a char* argument */
+#define _curl_is_string_option(option)                                        \
+  ((option) == CURLOPT_ABSTRACT_UNIX_SOCKET ||                                \
+   (option) == CURLOPT_ACCEPT_ENCODING ||                                     \
+   (option) == CURLOPT_CAINFO ||                                              \
+   (option) == CURLOPT_CAPATH ||                                              \
+   (option) == CURLOPT_COOKIE ||                                              \
+   (option) == CURLOPT_COOKIEFILE ||                                          \
+   (option) == CURLOPT_COOKIEJAR ||                                           \
+   (option) == CURLOPT_COOKIELIST ||                                          \
+   (option) == CURLOPT_CRLFILE ||                                             \
+   (option) == CURLOPT_CUSTOMREQUEST ||                                       \
+   (option) == CURLOPT_DEFAULT_PROTOCOL ||                                    \
+   (option) == CURLOPT_DNS_INTERFACE ||                                       \
+   (option) == CURLOPT_DNS_LOCAL_IP4 ||                                       \
+   (option) == CURLOPT_DNS_LOCAL_IP6 ||                                       \
+   (option) == CURLOPT_DNS_SERVERS ||                                         \
+   (option) == CURLOPT_EGDSOCKET ||                                           \
+   (option) == CURLOPT_FTPPORT ||                                             \
+   (option) == CURLOPT_FTP_ACCOUNT ||                                         \
+   (option) == CURLOPT_FTP_ALTERNATIVE_TO_USER ||                             \
+   (option) == CURLOPT_INTERFACE ||                                           \
+   (option) == CURLOPT_ISSUERCERT ||                                          \
+   (option) == CURLOPT_KEYPASSWD ||                                           \
+   (option) == CURLOPT_KRBLEVEL ||                                            \
+   (option) == CURLOPT_LOGIN_OPTIONS ||                                       \
+   (option) == CURLOPT_MAIL_AUTH ||                                           \
+   (option) == CURLOPT_MAIL_FROM ||                                           \
+   (option) == CURLOPT_NETRC_FILE ||                                          \
+   (option) == CURLOPT_NOPROXY ||                                             \
+   (option) == CURLOPT_PASSWORD ||                                            \
+   (option) == CURLOPT_PINNEDPUBLICKEY ||                                     \
+   (option) == CURLOPT_PRE_PROXY ||                                           \
+   (option) == CURLOPT_PROXY ||                                               \
+   (option) == CURLOPT_PROXYPASSWORD ||                                       \
+   (option) == CURLOPT_PROXYUSERNAME ||                                       \
+   (option) == CURLOPT_PROXYUSERPWD ||                                        \
+   (option) == CURLOPT_PROXY_CAINFO ||                                        \
+   (option) == CURLOPT_PROXY_CAPATH ||                                        \
+   (option) == CURLOPT_PROXY_CRLFILE ||                                       \
+   (option) == CURLOPT_PROXY_KEYPASSWD ||                                     \
+   (option) == CURLOPT_PROXY_PINNEDPUBLICKEY ||                               \
+   (option) == CURLOPT_PROXY_SERVICE_NAME ||                                  \
+   (option) == CURLOPT_PROXY_SSLCERT ||                                       \
+   (option) == CURLOPT_PROXY_SSLCERTTYPE ||                                   \
+   (option) == CURLOPT_PROXY_SSLKEY ||                                        \
+   (option) == CURLOPT_PROXY_SSLKEYTYPE ||                                    \
+   (option) == CURLOPT_PROXY_SSL_CIPHER_LIST ||                               \
+   (option) == CURLOPT_PROXY_TLSAUTH_PASSWORD ||                              \
+   (option) == CURLOPT_PROXY_TLSAUTH_USERNAME ||                              \
+   (option) == CURLOPT_PROXY_TLSAUTH_TYPE ||                                  \
+   (option) == CURLOPT_RANDOM_FILE ||                                         \
+   (option) == CURLOPT_RANGE ||                                               \
+   (option) == CURLOPT_REFERER ||                                             \
+   (option) == CURLOPT_RTSP_SESSION_ID ||                                     \
+   (option) == CURLOPT_RTSP_STREAM_URI ||                                     \
+   (option) == CURLOPT_RTSP_TRANSPORT ||                                      \
+   (option) == CURLOPT_SERVICE_NAME ||                                        \
+   (option) == CURLOPT_SOCKS5_GSSAPI_SERVICE ||                               \
+   (option) == CURLOPT_SSH_HOST_PUBLIC_KEY_MD5 ||                             \
+   (option) == CURLOPT_SSH_KNOWNHOSTS ||                                      \
+   (option) == CURLOPT_SSH_PRIVATE_KEYFILE ||                                 \
+   (option) == CURLOPT_SSH_PUBLIC_KEYFILE ||                                  \
+   (option) == CURLOPT_SSLCERT ||                                             \
+   (option) == CURLOPT_SSLCERTTYPE ||                                         \
+   (option) == CURLOPT_SSLENGINE ||                                           \
+   (option) == CURLOPT_SSLKEY ||                                              \
+   (option) == CURLOPT_SSLKEYTYPE ||                                          \
+   (option) == CURLOPT_SSL_CIPHER_LIST ||                                     \
+   (option) == CURLOPT_TLSAUTH_PASSWORD ||                                    \
+   (option) == CURLOPT_TLSAUTH_TYPE ||                                        \
+   (option) == CURLOPT_TLSAUTH_USERNAME ||                                    \
+   (option) == CURLOPT_UNIX_SOCKET_PATH ||                                    \
+   (option) == CURLOPT_URL ||                                                 \
+   (option) == CURLOPT_USERAGENT ||                                           \
+   (option) == CURLOPT_USERNAME ||                                            \
+   (option) == CURLOPT_USERPWD ||                                             \
+   (option) == CURLOPT_XOAUTH2_BEARER ||                                      \
+   0)
+
+/* evaluates to true if option takes a curl_write_callback argument */
+#define _curl_is_write_cb_option(option)                                      \
+  ((option) == CURLOPT_HEADERFUNCTION ||                                      \
+   (option) == CURLOPT_WRITEFUNCTION)
+
+/* evaluates to true if option takes a curl_conv_callback argument */
+#define _curl_is_conv_cb_option(option)                                       \
+  ((option) == CURLOPT_CONV_TO_NETWORK_FUNCTION ||                            \
+   (option) == CURLOPT_CONV_FROM_NETWORK_FUNCTION ||                          \
+   (option) == CURLOPT_CONV_FROM_UTF8_FUNCTION)
+
+/* evaluates to true if option takes a data argument to pass to a callback */
+#define _curl_is_cb_data_option(option)                                       \
+  ((option) == CURLOPT_CHUNK_DATA ||                                          \
+   (option) == CURLOPT_CLOSESOCKETDATA ||                                     \
+   (option) == CURLOPT_DEBUGDATA ||                                           \
+   (option) == CURLOPT_FNMATCH_DATA ||                                        \
+   (option) == CURLOPT_HEADERDATA ||                                          \
+   (option) == CURLOPT_INTERLEAVEDATA ||                                      \
+   (option) == CURLOPT_IOCTLDATA ||                                           \
+   (option) == CURLOPT_OPENSOCKETDATA ||                                      \
+   (option) == CURLOPT_PRIVATE ||                                             \
+   (option) == CURLOPT_PROGRESSDATA ||                                        \
+   (option) == CURLOPT_READDATA ||                                            \
+   (option) == CURLOPT_SEEKDATA ||                                            \
+   (option) == CURLOPT_SOCKOPTDATA ||                                         \
+   (option) == CURLOPT_SSH_KEYDATA ||                                         \
+   (option) == CURLOPT_SSL_CTX_DATA ||                                        \
+   (option) == CURLOPT_WRITEDATA ||                                           \
+   0)
+
+/* evaluates to true if option takes a POST data argument (void* or char*) */
+#define _curl_is_postfields_option(option)                                    \
+  ((option) == CURLOPT_POSTFIELDS ||                                          \
+   (option) == CURLOPT_COPYPOSTFIELDS ||                                      \
+   0)
+
+/* evaluates to true if option takes a struct curl_slist * argument */
+#define _curl_is_slist_option(option)                                         \
+  ((option) == CURLOPT_HTTP200ALIASES ||                                      \
+   (option) == CURLOPT_HTTPHEADER ||                                          \
+   (option) == CURLOPT_MAIL_RCPT ||                                           \
+   (option) == CURLOPT_POSTQUOTE ||                                           \
+   (option) == CURLOPT_PREQUOTE ||                                            \
+   (option) == CURLOPT_PROXYHEADER ||                                         \
+   (option) == CURLOPT_QUOTE ||                                               \
+   (option) == CURLOPT_RESOLVE ||                                             \
+   (option) == CURLOPT_TELNETOPTIONS ||                                       \
+   0)
+
+/* groups of curl_easy_getinfo infos that take the same type of argument */
+
+/* evaluates to true if info expects a pointer to char * argument */
+#define _curl_is_string_info(info)                                            \
+  (CURLINFO_STRING < (info) && (info) < CURLINFO_LONG)
+
+/* evaluates to true if info expects a pointer to long argument */
+#define _curl_is_long_info(info)                                              \
+  (CURLINFO_LONG < (info) && (info) < CURLINFO_DOUBLE)
+
+/* evaluates to true if info expects a pointer to double argument */
+#define _curl_is_double_info(info)                                            \
+  (CURLINFO_DOUBLE < (info) && (info) < CURLINFO_SLIST)
+
+/* true if info expects a pointer to struct curl_slist * argument */
+#define _curl_is_slist_info(info)                                       \
+  (((info) == CURLINFO_SSL_ENGINES) || ((info) == CURLINFO_COOKIELIST))
+
+/* true if info expects a pointer to struct curl_tlssessioninfo * argument */
+#define _curl_is_tlssessioninfo_info(info)                              \
+  (((info) == CURLINFO_TLS_SSL_PTR) || ((info) == CURLINFO_TLS_SESSION))
+
+/* true if info expects a pointer to struct curl_certinfo * argument */
+#define _curl_is_certinfo_info(info) ((info) == CURLINFO_CERTINFO)
+
+/* true if info expects a pointer to struct curl_socket_t argument */
+#define _curl_is_socket_info(info)                                            \
+  (CURLINFO_SOCKET < (info) && (info) < CURLINFO_OFF_T)
+
+/* true if info expects a pointer to curl_off_t argument */
+#define _curl_is_off_t_info(info)                                             \
+  (CURLINFO_OFF_T < (info))
+
+
+/* typecheck helpers -- check whether given expression has requested type*/
+
+/* For pointers, you can use the _curl_is_ptr/_curl_is_arr macros,
+ * otherwise define a new macro. Search for __builtin_types_compatible_p
+ * in the GCC manual.
+ * NOTE: these macros MUST NOT EVALUATE their arguments! The argument is
+ * the actual expression passed to the curl_easy_setopt macro. This
+ * means that you can only apply the sizeof and __typeof__ operators, no
+ * == or whatsoever.
+ */
+
+/* XXX: should evaluate to true iff expr is a pointer */
+#define _curl_is_any_ptr(expr)                                                \
+  (sizeof(expr) == sizeof(void *))
+
+/* evaluates to true if expr is NULL */
+/* XXX: must not evaluate expr, so this check is not accurate */
+#define _curl_is_NULL(expr)                                                   \
+  (__builtin_types_compatible_p(__typeof__(expr), __typeof__(NULL)))
+
+/* evaluates to true if expr is type*, const type* or NULL */
+#define _curl_is_ptr(expr, type)                                              \
+  (_curl_is_NULL(expr) ||                                                     \
+   __builtin_types_compatible_p(__typeof__(expr), type *) ||                  \
+   __builtin_types_compatible_p(__typeof__(expr), const type *))
+
+/* evaluates to true if expr is one of type[], type*, NULL or const type* */
+#define _curl_is_arr(expr, type)                                              \
+  (_curl_is_ptr((expr), type) ||                                              \
+   __builtin_types_compatible_p(__typeof__(expr), type []))
+
+/* evaluates to true if expr is a string */
+#define _curl_is_string(expr)                                                 \
+  (_curl_is_arr((expr), char) ||                                              \
+   _curl_is_arr((expr), signed char) ||                                       \
+   _curl_is_arr((expr), unsigned char))
+
+/* evaluates to true if expr is a long (no matter the signedness)
+ * XXX: for now, int is also accepted (and therefore short and char, which
+ * are promoted to int when passed to a variadic function) */
+#define _curl_is_long(expr)                                                   \
+  (__builtin_types_compatible_p(__typeof__(expr), long) ||                    \
+   __builtin_types_compatible_p(__typeof__(expr), signed long) ||             \
+   __builtin_types_compatible_p(__typeof__(expr), unsigned long) ||           \
+   __builtin_types_compatible_p(__typeof__(expr), int) ||                     \
+   __builtin_types_compatible_p(__typeof__(expr), signed int) ||              \
+   __builtin_types_compatible_p(__typeof__(expr), unsigned int) ||            \
+   __builtin_types_compatible_p(__typeof__(expr), short) ||                   \
+   __builtin_types_compatible_p(__typeof__(expr), signed short) ||            \
+   __builtin_types_compatible_p(__typeof__(expr), unsigned short) ||          \
+   __builtin_types_compatible_p(__typeof__(expr), char) ||                    \
+   __builtin_types_compatible_p(__typeof__(expr), signed char) ||             \
+   __builtin_types_compatible_p(__typeof__(expr), unsigned char))
+
+/* evaluates to true if expr is of type curl_off_t */
+#define _curl_is_off_t(expr)                                                  \
+  (__builtin_types_compatible_p(__typeof__(expr), curl_off_t))
+
+/* evaluates to true if expr is abuffer suitable for CURLOPT_ERRORBUFFER */
+/* XXX: also check size of an char[] array? */
+#define _curl_is_error_buffer(expr)                                           \
+  (_curl_is_NULL(expr) ||                                                     \
+   __builtin_types_compatible_p(__typeof__(expr), char *) ||                  \
+   __builtin_types_compatible_p(__typeof__(expr), char[]))
+
+/* evaluates to true if expr is of type (const) void* or (const) FILE* */
+#if 0
+#define _curl_is_cb_data(expr)                                                \
+  (_curl_is_ptr((expr), void) ||                                              \
+   _curl_is_ptr((expr), FILE))
+#else /* be less strict */
+#define _curl_is_cb_data(expr)                                                \
+  _curl_is_any_ptr(expr)
+#endif
+
+/* evaluates to true if expr is of type FILE* */
+#define _curl_is_FILE(expr)                                             \
+  (_curl_is_NULL(expr) ||                                              \
+   (__builtin_types_compatible_p(__typeof__(expr), FILE *)))
+
+/* evaluates to true if expr can be passed as POST data (void* or char*) */
+#define _curl_is_postfields(expr)                                             \
+  (_curl_is_ptr((expr), void) ||                                              \
+   _curl_is_arr((expr), char))
+
+/* FIXME: the whole callback checking is messy...
+ * The idea is to tolerate char vs. void and const vs. not const
+ * pointers in arguments at least
+ */
+/* helper: __builtin_types_compatible_p distinguishes between functions and
+ * function pointers, hide it */
+#define _curl_callback_compatible(func, type)                                 \
+  (__builtin_types_compatible_p(__typeof__(func), type) ||                    \
+   __builtin_types_compatible_p(__typeof__(func) *, type))
+
+/* evaluates to true if expr is of type curl_read_callback or "similar" */
+#define _curl_is_read_cb(expr)                                          \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), __typeof__(fread) *) ||                  \
+   _curl_callback_compatible((expr), curl_read_callback) ||                   \
+   _curl_callback_compatible((expr), _curl_read_callback1) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback2) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback3) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback4) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback5) ||                 \
+   _curl_callback_compatible((expr), _curl_read_callback6))
+typedef size_t (*_curl_read_callback1)(char *, size_t, size_t, void *);
+typedef size_t (*_curl_read_callback2)(char *, size_t, size_t, const void *);
+typedef size_t (*_curl_read_callback3)(char *, size_t, size_t, FILE *);
+typedef size_t (*_curl_read_callback4)(void *, size_t, size_t, void *);
+typedef size_t (*_curl_read_callback5)(void *, size_t, size_t, const void *);
+typedef size_t (*_curl_read_callback6)(void *, size_t, size_t, FILE *);
+
+/* evaluates to true if expr is of type curl_write_callback or "similar" */
+#define _curl_is_write_cb(expr)                                               \
+  (_curl_is_read_cb(expr) ||                                            \
+   _curl_callback_compatible((expr), __typeof__(fwrite) *) ||                 \
+   _curl_callback_compatible((expr), curl_write_callback) ||                  \
+   _curl_callback_compatible((expr), _curl_write_callback1) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback2) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback3) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback4) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback5) ||                \
+   _curl_callback_compatible((expr), _curl_write_callback6))
+typedef size_t (*_curl_write_callback1)(const char *, size_t, size_t, void *);
+typedef size_t (*_curl_write_callback2)(const char *, size_t, size_t,
+                                       const void *);
+typedef size_t (*_curl_write_callback3)(const char *, size_t, size_t, FILE *);
+typedef size_t (*_curl_write_callback4)(const void *, size_t, size_t, void *);
+typedef size_t (*_curl_write_callback5)(const void *, size_t, size_t,
+                                       const void *);
+typedef size_t (*_curl_write_callback6)(const void *, size_t, size_t, FILE *);
+
+/* evaluates to true if expr is of type curl_ioctl_callback or "similar" */
+#define _curl_is_ioctl_cb(expr)                                         \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_ioctl_callback) ||                  \
+   _curl_callback_compatible((expr), _curl_ioctl_callback1) ||                \
+   _curl_callback_compatible((expr), _curl_ioctl_callback2) ||                \
+   _curl_callback_compatible((expr), _curl_ioctl_callback3) ||                \
+   _curl_callback_compatible((expr), _curl_ioctl_callback4))
+typedef curlioerr (*_curl_ioctl_callback1)(CURL *, int, void *);
+typedef curlioerr (*_curl_ioctl_callback2)(CURL *, int, const void *);
+typedef curlioerr (*_curl_ioctl_callback3)(CURL *, curliocmd, void *);
+typedef curlioerr (*_curl_ioctl_callback4)(CURL *, curliocmd, const void *);
+
+/* evaluates to true if expr is of type curl_sockopt_callback or "similar" */
+#define _curl_is_sockopt_cb(expr)                                       \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_sockopt_callback) ||                \
+   _curl_callback_compatible((expr), _curl_sockopt_callback1) ||              \
+   _curl_callback_compatible((expr), _curl_sockopt_callback2))
+typedef int (*_curl_sockopt_callback1)(void *, curl_socket_t, curlsocktype);
+typedef int (*_curl_sockopt_callback2)(const void *, curl_socket_t,
+                                      curlsocktype);
+
+/* evaluates to true if expr is of type curl_opensocket_callback or
+   "similar" */
+#define _curl_is_opensocket_cb(expr)                                    \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_opensocket_callback) ||             \
+   _curl_callback_compatible((expr), _curl_opensocket_callback1) ||           \
+   _curl_callback_compatible((expr), _curl_opensocket_callback2) ||           \
+   _curl_callback_compatible((expr), _curl_opensocket_callback3) ||           \
+   _curl_callback_compatible((expr), _curl_opensocket_callback4))
+typedef curl_socket_t (*_curl_opensocket_callback1)
+  (void *, curlsocktype, struct curl_sockaddr *);
+typedef curl_socket_t (*_curl_opensocket_callback2)
+  (void *, curlsocktype, const struct curl_sockaddr *);
+typedef curl_socket_t (*_curl_opensocket_callback3)
+  (const void *, curlsocktype, struct curl_sockaddr *);
+typedef curl_socket_t (*_curl_opensocket_callback4)
+  (const void *, curlsocktype, const struct curl_sockaddr *);
+
+/* evaluates to true if expr is of type curl_progress_callback or "similar" */
+#define _curl_is_progress_cb(expr)                                      \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_progress_callback) ||               \
+   _curl_callback_compatible((expr), _curl_progress_callback1) ||             \
+   _curl_callback_compatible((expr), _curl_progress_callback2))
+typedef int (*_curl_progress_callback1)(void *,
+    double, double, double, double);
+typedef int (*_curl_progress_callback2)(const void *,
+    double, double, double, double);
+
+/* evaluates to true if expr is of type curl_debug_callback or "similar" */
+#define _curl_is_debug_cb(expr)                                         \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_debug_callback) ||                  \
+   _curl_callback_compatible((expr), _curl_debug_callback1) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback2) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback3) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback4) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback5) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback6) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback7) ||                \
+   _curl_callback_compatible((expr), _curl_debug_callback8))
+typedef int (*_curl_debug_callback1) (CURL *,
+    curl_infotype, char *, size_t, void *);
+typedef int (*_curl_debug_callback2) (CURL *,
+    curl_infotype, char *, size_t, const void *);
+typedef int (*_curl_debug_callback3) (CURL *,
+    curl_infotype, const char *, size_t, void *);
+typedef int (*_curl_debug_callback4) (CURL *,
+    curl_infotype, const char *, size_t, const void *);
+typedef int (*_curl_debug_callback5) (CURL *,
+    curl_infotype, unsigned char *, size_t, void *);
+typedef int (*_curl_debug_callback6) (CURL *,
+    curl_infotype, unsigned char *, size_t, const void *);
+typedef int (*_curl_debug_callback7) (CURL *,
+    curl_infotype, const unsigned char *, size_t, void *);
+typedef int (*_curl_debug_callback8) (CURL *,
+    curl_infotype, const unsigned char *, size_t, const void *);
+
+/* evaluates to true if expr is of type curl_ssl_ctx_callback or "similar" */
+/* this is getting even messier... */
+#define _curl_is_ssl_ctx_cb(expr)                                       \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_ssl_ctx_callback) ||                \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback1) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback2) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback3) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback4) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback5) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback6) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback7) ||              \
+   _curl_callback_compatible((expr), _curl_ssl_ctx_callback8))
+typedef CURLcode (*_curl_ssl_ctx_callback1)(CURL *, void *, void *);
+typedef CURLcode (*_curl_ssl_ctx_callback2)(CURL *, void *, const void *);
+typedef CURLcode (*_curl_ssl_ctx_callback3)(CURL *, const void *, void *);
+typedef CURLcode (*_curl_ssl_ctx_callback4)(CURL *, const void *,
+                                            const void *);
+#ifdef HEADER_SSL_H
+/* hack: if we included OpenSSL's ssl.h, we know about SSL_CTX
+ * this will of course break if we're included before OpenSSL headers...
+ */
+typedef CURLcode (*_curl_ssl_ctx_callback5)(CURL *, SSL_CTX, void *);
+typedef CURLcode (*_curl_ssl_ctx_callback6)(CURL *, SSL_CTX, const void *);
+typedef CURLcode (*_curl_ssl_ctx_callback7)(CURL *, const SSL_CTX, void *);
+typedef CURLcode (*_curl_ssl_ctx_callback8)(CURL *, const SSL_CTX,
+                                           const void *);
+#else
+typedef _curl_ssl_ctx_callback1 _curl_ssl_ctx_callback5;
+typedef _curl_ssl_ctx_callback1 _curl_ssl_ctx_callback6;
+typedef _curl_ssl_ctx_callback1 _curl_ssl_ctx_callback7;
+typedef _curl_ssl_ctx_callback1 _curl_ssl_ctx_callback8;
+#endif
+
+/* evaluates to true if expr is of type curl_conv_callback or "similar" */
+#define _curl_is_conv_cb(expr)                                          \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_conv_callback) ||                   \
+   _curl_callback_compatible((expr), _curl_conv_callback1) ||                 \
+   _curl_callback_compatible((expr), _curl_conv_callback2) ||                 \
+   _curl_callback_compatible((expr), _curl_conv_callback3) ||                 \
+   _curl_callback_compatible((expr), _curl_conv_callback4))
+typedef CURLcode (*_curl_conv_callback1)(char *, size_t length);
+typedef CURLcode (*_curl_conv_callback2)(const char *, size_t length);
+typedef CURLcode (*_curl_conv_callback3)(void *, size_t length);
+typedef CURLcode (*_curl_conv_callback4)(const void *, size_t length);
+
+/* evaluates to true if expr is of type curl_seek_callback or "similar" */
+#define _curl_is_seek_cb(expr)                                          \
+  (_curl_is_NULL(expr) ||                                                     \
+   _curl_callback_compatible((expr), curl_seek_callback) ||                   \
+   _curl_callback_compatible((expr), _curl_seek_callback1) ||                 \
+   _curl_callback_compatible((expr), _curl_seek_callback2))
+typedef CURLcode (*_curl_seek_callback1)(void *, curl_off_t, int);
+typedef CURLcode (*_curl_seek_callback2)(const void *, curl_off_t, int);
+
+
+#endif /* __CURL_TYPECHECK_GCC_H */

ext/installfiles/linux/zerotier-containerized/Dockerfile

@@ -1,15 +1,21 @@
-FROM alpine:latest
+FROM alpine:latest as builder
 MAINTAINER Adam Ierymenko <adam.ierymenko@zerotier.com>
 
-LABEL version="1.2.4"
-LABEL description="Containerized ZeroTier One for use on CoreOS or other Docker-only Linux hosts."
+ARG VERSION=1.2.12
+LABEL version=$VERSION
+LABEL description="Containerized ZeroTier One"
 
-# Uncomment to build in container
-#RUN apk add --update alpine-sdk linux-headers
+RUN apk add --update alpine-sdk linux-headers
+RUN git clone https://github.com/zerotier/ZeroTierOne.git
+WORKDIR ZeroTierOne
+RUN git checkout $VERSION
+RUN make
+
+FROM alpine:latest as final
 
 RUN apk add --update libgcc libstdc++
 
-ADD zerotier-one /
+COPY --from=builder /ZeroTierOne/zerotier-one /
 RUN chmod 0755 /zerotier-one
 RUN ln -sf /zerotier-one /zerotier-cli
 RUN mkdir -p /var/lib/zerotier-one

ext/json/README.md

@@ -5,11 +5,11 @@
 [![Coverage Status](https://img.shields.io/coveralls/nlohmann/json.svg)](https://coveralls.io/r/nlohmann/json)
 [![Coverity Scan Build Status](https://scan.coverity.com/projects/5550/badge.svg)](https://scan.coverity.com/projects/nlohmann-json)
 [![Codacy Badge](https://api.codacy.com/project/badge/Grade/f3732b3327e34358a0e9d1fe9f661f08)](https://www.codacy.com/app/nlohmann/json?utm_source=github.com&utm_medium=referral&utm_content=nlohmann/json&utm_campaign=Badge_Grade)
-[![Try online](https://img.shields.io/badge/try-online-blue.svg)](https://wandbox.org/permlink/Op57X0V7fTf2tdwl)
+[![Try online](https://img.shields.io/badge/try-online-blue.svg)](https://wandbox.org/permlink/TarF5pPn9NtHQjhf)
 [![Documentation](https://img.shields.io/badge/docs-doxygen-blue.svg)](http://nlohmann.github.io/json)
 [![GitHub license](https://img.shields.io/badge/license-MIT-blue.svg)](https://raw.githubusercontent.com/nlohmann/json/master/LICENSE.MIT)
-[![Github Releases](https://img.shields.io/github/release/nlohmann/json.svg)](https://github.com/nlohmann/json/releases)
-[![Github Issues](https://img.shields.io/github/issues/nlohmann/json.svg)](http://github.com/nlohmann/json/issues)
+[![GitHub Releases](https://img.shields.io/github/release/nlohmann/json.svg)](https://github.com/nlohmann/json/releases)
+[![GitHub Issues](https://img.shields.io/github/issues/nlohmann/json.svg)](http://github.com/nlohmann/json/issues)
 [![Average time to resolve an issue](http://isitmaintained.com/badge/resolution/nlohmann/json.svg)](http://isitmaintained.com/project/nlohmann/json "Average time to resolve an issue")
 [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/289/badge)](https://bestpractices.coreinfrastructure.org/projects/289)
 
@@ -21,9 +21,10 @@
   - [STL-like access](#stl-like-access)
   - [Conversion from STL containers](#conversion-from-stl-containers)
   - [JSON Pointer and JSON Patch](#json-pointer-and-json-patch)
+  - [JSON Merge Patch](#json-merge-patch)
   - [Implicit conversions](#implicit-conversions)
   - [Conversions to/from arbitrary types](#arbitrary-types-conversions)
-  - [Binary formats (CBOR and MessagePack)](#binary-formats-cbor-and-messagepack)
+  - [Binary formats (CBOR, MessagePack, and UBJSON)](#binary-formats-cbor-messagepack-and-ubjson)
 - [Supported compilers](#supported-compilers)
 - [License](#license)
 - [Contact](#contact)
@@ -39,9 +40,9 @@ There are myriads of [JSON](http://json.org) libraries out there, and each may e
 
 - **Intuitive syntax**. In languages such as Python, JSON feels like a first class data type. We used all the operator magic of modern C++ to achieve the same feeling in your code. Check out the [examples below](#examples) and you'll know what I mean.
 
-- **Trivial integration**. Our whole code consists of a single header file [`json.hpp`](https://github.com/nlohmann/json/blob/develop/src/json.hpp). That's it. No library, no subproject, no dependencies, no complex build system. The class is written in vanilla C++11. All in all, everything should require no adjustment of your compiler flags or project settings.
+- **Trivial integration**. Our whole code consists of a single header file [`json.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp). That's it. No library, no subproject, no dependencies, no complex build system. The class is written in vanilla C++11. All in all, everything should require no adjustment of your compiler flags or project settings.
 
-- **Serious testing**. Our class is heavily [unit-tested](https://github.com/nlohmann/json/blob/master/test/src/unit.cpp) and covers [100%](https://coveralls.io/r/nlohmann/json) of the code, including all exceptional behavior. Furthermore, we checked with [Valgrind](http://valgrind.org) that there are no memory leaks. To maintain high quality, the project is following the [Core Infrastructure Initiative (CII) best practices](https://bestpractices.coreinfrastructure.org/projects/289).
+- **Serious testing**. Our class is heavily [unit-tested](https://github.com/nlohmann/json/tree/develop/test/src) and covers [100%](https://coveralls.io/r/nlohmann/json) of the code, including all exceptional behavior. Furthermore, we checked with [Valgrind](http://valgrind.org) and the [Clang Sanitizers](https://clang.llvm.org/docs/index.html) that there are no memory leaks. [Google OSS-Fuzz](https://github.com/google/oss-fuzz/tree/master/projects/json) additionally runs fuzz tests agains all parsers 24/7, effectively executing billions of tests so far. To maintain high quality, the project is following the [Core Infrastructure Initiative (CII) best practices](https://bestpractices.coreinfrastructure.org/projects/289).
 
 Other aspects were not so important to us:
 
@@ -54,31 +55,42 @@ See the [contribution guidelines](https://github.com/nlohmann/json/blob/master/.
 
 ## Integration
 
-The single required source, file `json.hpp` is in the `src` directory or [released here](https://github.com/nlohmann/json/releases). All you need to do is add
+[`json.hpp`](https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp) is the single required file in `single_include/nlohmann` or [released here](https://github.com/nlohmann/json/releases). You need to add
 
 ```cpp
-#include "json.hpp"
+#include <nlohmann/json.hpp>
 
 // for convenience
 using json = nlohmann::json;
 ```
 
-to the files you want to use JSON objects. That's it. Do not forget to set the necessary switches to enable C++11 (e.g., `-std=c++11` for GCC and Clang).
+to the files you want to process JSON and set the necessary switches to enable C++11 (e.g., `-std=c++11` for GCC and Clang).
+
+You can further use file [`include/nlohmann/json_fwd.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/json_fwd.hpp) for forward-declarations. The installation of json_fwd.hpp (as part of cmake's install step), can be achieved by setting `-DJSON_MultipleHeaders=ON`.
+
+### Package Managers
 
 :beer: If you are using OS X and [Homebrew](http://brew.sh), just type `brew tap nlohmann/json` and `brew install nlohmann_json` and you're set. If you want the bleeding edge rather than the latest release, use `brew install nlohmann_json --HEAD`.
 
-If you are using the [Meson Build System](http://mesonbuild.com), then you can wrap this repo as a subproject.
+If you are using the [Meson Build System](http://mesonbuild.com), then you can wrap this repository as a subproject.
 
 If you are using [Conan](https://www.conan.io/) to manage your dependencies, merely add `jsonformoderncpp/x.y.z@vthiery/stable` to your `conanfile.py`'s requires, where `x.y.z` is the release version you want to use. Please file issues [here](https://github.com/vthiery/conan-jsonformoderncpp/issues) if you experience problems with the packages.
 
+If you are using [Spack](https://www.spack.io/) to manage your dependencies, you can use the `nlohmann_json` package. Please see the [spack project](https://github.com/spack/spack) for any issues regarding the packaging.
+
 If you are using [hunter](https://github.com/ruslo/hunter/) on your project for external dependencies, then you can use the [nlohmann_json package](https://docs.hunter.sh/en/latest/packages/pkg/nlohmann_json.html). Please see the hunter project for any issues regarding the packaging.
 
+If you are using [Buckaroo](https://buckaroo.pm), you can install this library's module with `buckaroo install nlohmann/json`. Please file issues [here](https://github.com/LoopPerfect/buckaroo-recipes/issues/new?title=nlohmann/nlohmann/json).
+
 If you are using [vcpkg](https://github.com/Microsoft/vcpkg/) on your project for external dependencies, then you can use the [nlohmann-json package](https://github.com/Microsoft/vcpkg/tree/master/ports/nlohmann-json). Please see the vcpkg project for any issues regarding the packaging.
 
+If you are using [cget](http://cget.readthedocs.io/en/latest/), you can install the latest development version with `cget install nlohmann/json`. A specific version can be installed with `cget install nlohmann/json@v3.1.0`. Also, the multiple header version can be installed by adding the `-DJSON_MultipleHeaders=ON` flag (i.e., `cget install nlohmann/json -DJSON_MultipleHeaders=ON`).
+
+If you are using [CocoaPods](https://cocoapods.org), you can use the library by adding pod `"nlohmann_json", '~>3.1.2'` to your podfile (see [an example](https://bitbucket.org/benman/nlohmann_json-cocoapod/src/master/)). Please file issues [here](https://bitbucket.org/benman/nlohmann_json-cocoapod/issues?status=new&status=open).
 
 ## Examples
 
-Beside the examples below, you may want to check the [documentation](https://nlohmann.github.io/json/) where each function contains a separate code example (e.g., check out [`emplace()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a602f275f0359ab181221384989810604.html#a602f275f0359ab181221384989810604)). All [example files](https://github.com/nlohmann/json/tree/develop/doc/examples) can be compiled and executed on their own (e.g., file [emplace.cpp](https://github.com/nlohmann/json/blob/develop/doc/examples/emplace.cpp)).
+Beside the examples below, you may want to check the [documentation](https://nlohmann.github.io/json/) where each function contains a separate code example (e.g., check out [`emplace()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a5338e282d1d02bed389d852dd670d98d.html#a5338e282d1d02bed389d852dd670d98d)). All [example files](https://github.com/nlohmann/json/tree/develop/doc/examples) can be compiled and executed on their own (e.g., file [emplace.cpp](https://github.com/nlohmann/json/blob/develop/doc/examples/emplace.cpp)).
 
 ### JSON as first-class data type
 
@@ -103,7 +115,7 @@ Assume you want to create the JSON object
 }
 ```
 
-With the JSON class, you could write:
+With this library, you could write:
 
 ```cpp
 // create an empty structure (null)
@@ -147,7 +159,7 @@ json j2 = {
 };
 ```
 
-Note that in all these cases, you never need to "tell" the compiler which JSON value you want to use. If you want to be explicit or express some edge cases, the functions `json::array` and `json::object` will help:
+Note that in all these cases, you never need to "tell" the compiler which JSON value type you want to use. If you want to be explicit or express some edge cases, the functions [`json::array`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_aa80485befaffcadaa39965494e0b4d2e.html#aa80485befaffcadaa39965494e0b4d2e) and [`json::object`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_aa13f7c0615867542ce80337cbcf13ada.html#aa13f7c0615867542ce80337cbcf13ada) will help:
 
 ```cpp
 // a way to express the empty array []
@@ -161,12 +173,11 @@ json empty_object_explicit = json::object();
 json array_not_object = json::array({ {"currency", "USD"}, {"value", 42.99} });
 ```
 
-
 ### Serialization / Deserialization
 
 #### To/from strings
 
-You can create an object (deserialization) by appending `_json` to a string literal:
+You can create a JSON value (deserialization) by appending `_json` to a string literal:
 
 ```cpp
 // create object from string literal
@@ -183,14 +194,14 @@ auto j2 = R"(
 
 Note that without appending the `_json` suffix, the passed string literal is not parsed, but just used as JSON string value. That is, `json j = "{ \"happy\": true, \"pi\": 3.141 }"` would just store the string `"{ "happy": true, "pi": 3.141 }"` rather than parsing the actual object.
 
-The above example can also be expressed explicitly using `json::parse()`:
+The above example can also be expressed explicitly using [`json::parse()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_aa9676414f2e36383c4b181fe856aa3c0.html#aa9676414f2e36383c4b181fe856aa3c0):
 
 ```cpp
 // parse explicitly
 auto j3 = json::parse("{ \"happy\": true, \"pi\": 3.141 }");
 ```
 
-You can also get a string representation (serialize):
+You can also get a string representation of a JSON value (serialize):
 
 ```cpp
 // explicit conversion to string
@@ -225,8 +236,9 @@ std::cout << cpp_string << " == " << cpp_string2 << " == " << j_string.get<std::
 std::cout << j_string << " == " << serialized_string << std::endl;
 ```
 
-`.dump()` always returns the serialized value, and `.get<std::string>()` returns the originally stored string value.
+[`.dump()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a5adea76fedba9898d404fef8598aa663.html#a5adea76fedba9898d404fef8598aa663) always returns the serialized value, and [`.get<std::string>()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a16f9445f7629f634221a42b967cdcd43.html#a16f9445f7629f634221a42b967cdcd43) returns the originally stored string value.
 
+Note the library only supports UTF-8. When you store strings with different encodings in the library, calling [`dump()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a5adea76fedba9898d404fef8598aa663.html#a5adea76fedba9898d404fef8598aa663) may throw an exception.
 
 #### To/from streams (e.g. files, string streams)
 
@@ -261,7 +273,7 @@ Please note that setting the exception bit for `failbit` is inappropriate for th
 
 #### Read from iterator range
 
-You can also read JSON from an iterator range; that is, from any container accessible by iterators whose content is stored as contiguous byte sequence, for instance a `std::vector<std::uint8_t>`:
+You can also parse JSON from an iterator range; that is, from any container accessible by iterators whose content is stored as contiguous byte sequence, for instance a `std::vector<std::uint8_t>`:
 
 ```cpp
 std::vector<std::uint8_t> v = {'t', 'r', 'u', 'e'};
@@ -275,10 +287,53 @@ std::vector<std::uint8_t> v = {'t', 'r', 'u', 'e'};
 json j = json::parse(v);
 ```
 
+#### SAX interface
+
+The library uses a SAX-like interface with the following functions:
+
+```cpp
+// called when null is parsed
+bool null();
+
+// called when a boolean is parsed; value is passed
+bool boolean(bool val);
+
+// called when a signed or unsigned integer number is parsed; value is passed
+bool number_integer(number_integer_t val);
+bool number_unsigned(number_unsigned_t val);
+
+// called when a floating-point number is parsed; value and original string is passed
+bool number_float(number_float_t val, const string_t& s);
+
+// called when a string is parsed; value is passed and can be safely moved away
+bool string(string_t& val);
+
+// called when an object or array begins or ends, resp. The number of elements is passed (or -1 if not known)
+bool start_object(std::size_t elements);
+bool end_object();
+bool start_array(std::size_t elements);
+bool end_array();
+// called when an object key is parsed; value is passed and can be safely moved away
+bool key(string_t& val);
+
+// called when a parse error occurs; byte position, the last token, and an exception is passed
+bool parse_error(std::size_t position, const std::string& last_token, const detail::exception& ex);
+```
+
+The return value of each function determines whether parsing should proceed.
+
+To implement your own SAX handler, proceed as follows:
+
+1. Implement the SAX interface in a class. You can use class `nlohmann::json_sax<json>` as base class, but you can also use any class where the functions described above are implemented and public.
+2. Create an object of your SAX interface class, e.g. `my_sax`.
+3. Call `bool json::sax_parse(input, &my_sax)`; where the first parameter can be any input like a string or an input stream and the second parameter is a pointer to your SAX interface.
+
+Note the `sax_parse` function only returns a `bool` indicating the result of the last executed SAX event. It does not return a  `json` value - it is up to you to decide what to do with the SAX events. Furthermore, no exceptions are thrown in case of a parse error - it is up to you what to do with the exception object passed to your `parse_error` implementation. Internally, the SAX interface is used for the DOM parser (class `json_sax_dom_parser`) as well as the acceptor (`json_sax_acceptor`), see file [`json_sax.hpp`](https://github.com/nlohmann/json/blob/develop/include/nlohmann/detail/input/json_sax.hpp).
+
 
 ### STL-like access
 
-We designed the JSON class to behave just like an STL container. In fact, it satisfies the [**ReversibleContainer**](http://en.cppreference.com/w/cpp/concept/ReversibleContainer) requirement.
+We designed the JSON class to behave just like an STL container. In fact, it satisfies the [**ReversibleContainer**](https://en.cppreference.com/w/cpp/named_req/ReversibleContainer) requirement.
 
 ```cpp
 // create an array using push_back
@@ -352,7 +407,7 @@ o.erase("foo");
 
 ### Conversion from STL containers
 
-Any sequence container (`std::array`, `std::vector`, `std::deque`, `std::forward_list`, `std::list`) whose values can be used to construct JSON types (e.g., integers, floating point numbers, Booleans, string types, or again STL containers described in this section) can be used to create a JSON array. The same holds for similar associative containers (`std::set`, `std::multiset`, `std::unordered_set`, `std::unordered_multiset`), but in these cases the order of the elements of the array depends on how the elements are ordered in the respective STL container.
+Any sequence container (`std::array`, `std::vector`, `std::deque`, `std::forward_list`, `std::list`) whose values can be used to construct JSON values (e.g., integers, floating point numbers, Booleans, string types, or again STL containers described in this section) can be used to create a JSON array. The same holds for similar associative containers (`std::set`, `std::multiset`, `std::unordered_set`, `std::unordered_multiset`), but in these cases the order of the elements of the array depends on how the elements are ordered in the respective STL container.
 
 ```cpp
 std::vector<int> c_vector {1, 2, 3, 4};
@@ -392,7 +447,7 @@ json j_umset(c_umset); // both entries for "one" are used
 // maybe ["one", "two", "one", "four"]
 ```
 
-Likewise, any associative key-value containers (`std::map`, `std::multimap`, `std::unordered_map`, `std::unordered_multimap`) whose keys can construct an `std::string` and whose values can be used to construct JSON types (see examples above) can be used to create a JSON object. Note that in case of multimaps only one key is used in the JSON object and the value depends on the internal order of the STL container.
+Likewise, any associative key-value containers (`std::map`, `std::multimap`, `std::unordered_map`, `std::unordered_multimap`) whose keys can construct an `std::string` and whose values can be used to construct JSON values (see examples above) can be used to create a JSON object. Note that in case of multimaps only one key is used in the JSON object and the value depends on the internal order of the STL container.
 
 ```cpp
 std::map<std::string, int> c_map { {"one", 1}, {"two", 2}, {"three", 3} };
@@ -450,6 +505,37 @@ json::diff(j_result, j_original);
 // ]
 ```
 
+### JSON Merge Patch
+
+The library supports **JSON Merge Patch** ([RFC 7386](https://tools.ietf.org/html/rfc7386)) as a patch format. Instead of using JSON Pointer (see above) to specify values to be manipulated, it describes the changes using a syntax that closely mimics the document being modified.
+
+```cpp
+// a JSON value
+json j_document = R"({
+  "a": "b",
+  "c": {
+    "d": "e",
+    "f": "g"
+  }
+})"_json;
+
+// a patch
+json j_patch = R"({
+  "a":"z",
+  "c": {
+    "f": null
+  }
+})"_json;
+
+// apply the patch
+j_original.merge_patch(j_patch);
+// {
+//  "a": "z",
+//  "c": {
+//    "d": "e"
+//  }
+// }
+```
 
 ### Implicit conversions
 
@@ -484,9 +570,17 @@ int vi = jn.get<int>();
 // etc.
 ```
 
+Note that `char` types are not automatically converted to JSON strings, but to integer numbers. A conversion to a string must be specified explicitly:
+
+```cpp
+char ch = 'A';                       // ASCII value 65
+json j_default = ch;                 // stores integer number 65
+json j_string = std::string(1, ch);  // stores string "A"
+```
+
 ### Arbitrary types conversions
 
-Every type can be serialized in JSON, not just STL-containers and scalar types. Usually, you would do something along those lines:
+Every type can be serialized in JSON, not just STL containers and scalar types. Usually, you would do something along those lines:
 
 ```cpp
 namespace ns {
@@ -561,7 +655,8 @@ Likewise, when calling `get<your_type>()`, the `from_json` method will be called
 Some important things:
 
 * Those methods **MUST** be in your type's namespace (which can be the global namespace), or the library will not be able to locate them (in this example, they are in namespace `ns`, where `person` is defined).
-* When using `get<your_type>()`, `your_type` **MUST** be [DefaultConstructible](http://en.cppreference.com/w/cpp/concept/DefaultConstructible). (There is a way to bypass this requirement described later.)
+* Those methods **MUST** be available (e.g., properly headers must be included) everywhere you use the implicit conversions. Look at [issue 1108](https://github.com/nlohmann/json/issues/1108) for errors that may occur otherwise.
+* When using `get<your_type>()`, `your_type` **MUST** be [DefaultConstructible](https://en.cppreference.com/w/cpp/named_req/DefaultConstructible). (There is a way to bypass this requirement described later.)
 * In function `from_json`, use function [`at()`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a93403e803947b86f4da2d1fb3345cf2c.html#a93403e803947b86f4da2d1fb3345cf2c) to access the object values rather than `operator[]`. In case a key does not exist, `at` throws an exception that you can handle, whereas `operator[]` exhibits undefined behavior.
 * In case your type contains several `operator=` definitions, code like `your_variable = your_json;` [may not compile](https://github.com/nlohmann/json/issues/667). You need to write `your_variable = your_json.get<decltype your_variable>();` instead.
 * You do not need to add serializers or deserializers for STL types like `std::vector`: the library already implements these.
@@ -573,7 +668,7 @@ Some important things:
 This requires a bit more advanced technique. But first, let's see how this conversion mechanism works:
 
 The library uses **JSON Serializers** to convert types to json.
-The default serializer for `nlohmann::json` is `nlohmann::adl_serializer` (ADL means [Argument-Dependent Lookup](http://en.cppreference.com/w/cpp/language/adl)).
+The default serializer for `nlohmann::json` is `nlohmann::adl_serializer` (ADL means [Argument-Dependent Lookup](https://en.cppreference.com/w/cpp/language/adl)).
 
 It is implemented like this (simplified):
 
@@ -590,7 +685,7 @@ struct adl_serializer {
 };
 ```
 
-This serializer works fine when you have control over the type's namespace. However, what about `boost::optional`, or `std::filesystem::path` (C++17)? Hijacking the `boost` namespace is pretty bad, and it's illegal to add something other than template specializations to `std`...
+This serializer works fine when you have control over the type's namespace. However, what about `boost::optional` or `std::filesystem::path` (C++17)? Hijacking the `boost` namespace is pretty bad, and it's illegal to add something other than template specializations to `std`...
 
 To solve this, you need to add a specialization of `adl_serializer` to the `nlohmann` namespace, here's an example:
 
@@ -622,7 +717,7 @@ namespace nlohmann {
 
 #### How can I use `get()` for non-default constructible/non-copyable types?
 
-There is a way, if your type is [MoveConstructible](http://en.cppreference.com/w/cpp/concept/MoveConstructible). You will need to specialize the `adl_serializer` as well, but with a special `from_json` overload:
+There is a way, if your type is [MoveConstructible](https://en.cppreference.com/w/cpp/named_req/MoveConstructible). You will need to specialize the `adl_serializer` as well, but with a special `from_json` overload:
 
 ```cpp
 struct move_only_type {
@@ -659,9 +754,9 @@ Yes. You might want to take a look at [`unit-udt.cpp`](https://github.com/nlohma
 
 If you write your own serializer, you'll need to do a few things:
 
-* use a different `basic_json` alias than `nlohmann::json` (the last template parameter of `basic_json` is the `JSONSerializer`)
-* use your `basic_json` alias (or a template parameter) in all your `to_json`/`from_json` methods
-* use `nlohmann::to_json` and `nlohmann::from_json` when you need ADL
+- use a different `basic_json` alias than `nlohmann::json` (the last template parameter of `basic_json` is the `JSONSerializer`)
+- use your `basic_json` alias (or a template parameter) in all your `to_json`/`from_json` methods
+- use `nlohmann::to_json` and `nlohmann::from_json` when you need ADL
 
 Here is an example, without simplifications, that only accepts types with a size <= 32, and uses ADL.
 
@@ -709,9 +804,9 @@ struct bad_serializer
 };
 ```
 
-### Binary formats (CBOR and MessagePack)
+### Binary formats (CBOR, MessagePack, and UBJSON)
 
-Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports [CBOR](http://cbor.io) (Concise Binary Object Representation) and [MessagePack](http://msgpack.org) to efficiently encode JSON values to byte vectors and to decode such vectors.
+Though JSON is a ubiquitous data format, it is not a very compact format suitable for data exchange, for instance over a network. Hence, the library supports [CBOR](http://cbor.io) (Concise Binary Object Representation), [MessagePack](http://msgpack.org), and [UBJSON](http://ubjson.org) (Universal Binary JSON Specification) to efficiently encode JSON values to byte vectors and to decode such vectors.
 
 ```cpp
 // create a JSON value
@@ -720,7 +815,7 @@ json j = R"({"compact": true, "schema": 0})"_json;
 // serialize to CBOR
 std::vector<std::uint8_t> v_cbor = json::to_cbor(j);
 
-// 0xa2, 0x67, 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x63, 0x74, 0xf5, 0x66, 0x73, 0x63, 0x68, 0x65, 0x6d, 0x61, 0x00
+// 0xA2, 0x67, 0x63, 0x6F, 0x6D, 0x70, 0x61, 0x63, 0x74, 0xF5, 0x66, 0x73, 0x63, 0x68, 0x65, 0x6D, 0x61, 0x00
 
 // roundtrip
 json j_from_cbor = json::from_cbor(v_cbor);
@@ -728,19 +823,27 @@ json j_from_cbor = json::from_cbor(v_cbor);
 // serialize to MessagePack
 std::vector<std::uint8_t> v_msgpack = json::to_msgpack(j);
 
-// 0x82, 0xa7, 0x63, 0x6f, 0x6d, 0x70, 0x61, 0x63, 0x74, 0xc3, 0xa6, 0x73, 0x63, 0x68, 0x65, 0x6d, 0x61, 0x00
+// 0x82, 0xA7, 0x63, 0x6F, 0x6D, 0x70, 0x61, 0x63, 0x74, 0xC3, 0xA6, 0x73, 0x63, 0x68, 0x65, 0x6D, 0x61, 0x00
 
 // roundtrip
 json j_from_msgpack = json::from_msgpack(v_msgpack);
+
+// serialize to UBJSON
+std::vector<std::uint8_t> v_ubjson = json::to_ubjson(j);
+
+// 0x7B, 0x69, 0x07, 0x63, 0x6F, 0x6D, 0x70, 0x61, 0x63, 0x74, 0x54, 0x69, 0x06, 0x73, 0x63, 0x68, 0x65, 0x6D, 0x61, 0x69, 0x00, 0x7D
+
+// roundtrip
+json j_from_ubjson = json::from_ubjson(v_ubjson);
 ```
 
 
 ## Supported compilers
 
-Though it's 2017 already, the support for C++11 is still a bit sparse. Currently, the following compilers are known to work:
+Though it's 2018 already, the support for C++11 is still a bit sparse. Currently, the following compilers are known to work:
 
-- GCC 4.9 - 7.2 (and possibly later)
-- Clang 3.4 - 5.0 (and possibly later)
+- GCC 4.9 - 8.2 (and possibly later)
+- Clang 3.4 - 6.1 (and possibly later)
 - Intel C++ Compiler 17.0.2 (and possibly later)
 - Microsoft Visual C++ 2015 / Build Tools 14.0.25123.0 (and possibly later)
 - Microsoft Visual C++ 2017 / Build Tools 15.5.180.51428 (and possibly later)
@@ -762,32 +865,38 @@ Please note:
 
 - For GCC running on MinGW or Android SDK, the error `'to_string' is not a member of 'std'` (or similarly, for `strtod`) may occur. Note this is not an issue with the code,  but rather with the compiler itself. On Android, see above to build with a newer environment.  For MinGW, please refer to [this site](http://tehsausage.com/mingw-to-string) and [this discussion](https://github.com/nlohmann/json/issues/136) for information on how to fix this bug. For Android NDK using `APP_STL := gnustl_static`, please refer to [this discussion](https://github.com/nlohmann/json/issues/219).
 
+- Unsupported versions of GCC and Clang are rejected by `#error` directives. This can be switched off by defining `JSON_SKIP_UNSUPPORTED_COMPILER_CHECK`. Note that you can expect no support in this case.
+
 The following compilers are currently used in continuous integration at [Travis](https://travis-ci.org/nlohmann/json) and [AppVeyor](https://ci.appveyor.com/project/nlohmann/json):
 
 | Compiler        | Operating System             | Version String |
 |-----------------|------------------------------|----------------|
-| GCC 4.9.4       | Ubuntu 14.04.5 LTS           | g++-4.9 (Ubuntu 4.9.4-2ubuntu1~14.04.1) 4.9.4 |
-| GCC 5.4.1       | Ubuntu 14.04.5 LTS           | g++-5 (Ubuntu 5.4.1-2ubuntu1~14.04) 5.4.1 20160904 |
-| GCC 6.3.0       | Ubuntu 14.04.5 LTS           | g++-6 (Ubuntu/Linaro 6.3.0-18ubuntu2~14.04) 6.3.0 20170519 |
-| GCC 7.1.0       | Ubuntu 14.04.5 LTS           | g++-7 (Ubuntu 7.1.0-5ubuntu2~14.04) 7.1.0
-| Clang 3.5.0     | Ubuntu 14.04.5 LTS           | clang version 3.5.0-4ubuntu2~trusty2 (tags/RELEASE_350/final) |
-| Clang 3.6.2     | Ubuntu 14.04.5 LTS           | clang version 3.6.2-svn240577-1~exp1 (branches/release_36) |
-| Clang 3.7.1     | Ubuntu 14.04.5 LTS           | clang version 3.7.1-svn253571-1~exp1 (branches/release_37) |
-| Clang 3.8.0     | Ubuntu 14.04.5 LTS           | clang version 3.8.0-2ubuntu3~trusty5 (tags/RELEASE_380/final) |
-| Clang 3.9.1     | Ubuntu 14.04.5 LTS           | clang version 3.9.1-4ubuntu3~14.04.2 (tags/RELEASE_391/rc2) |
-| Clang 4.0.1     | Ubuntu 14.04.5 LTS           | clang version 4.0.1-svn305264-1~exp1 (branches/release_40) |
-| Clang 5.0.0     | Ubuntu 14.04.5 LTS           | clang version 5.0.0-svn310902-1~exp1 (branches/release_50) |
-| Clang Xcode 6.4 | Darwin Kernel Version 14.3.0 (OSX 10.10.3) | Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn) |
-| Clang Xcode 7.3 | Darwin Kernel Version 15.0.0 (OSX 10.10.5) | Apple LLVM version 7.3.0 (clang-703.0.29) |
-| Clang Xcode 8.0 | Darwin Kernel Version 15.6.0 | Apple LLVM version 8.0.0 (clang-800.0.38) |
-| Clang Xcode 8.1 | Darwin Kernel Version 16.1.0 (macOS 10.12.1) | Apple LLVM version 8.0.0 (clang-800.0.42.1) |
-| Clang Xcode 8.2 | Darwin Kernel Version 16.1.0 (macOS 10.12.1) | Apple LLVM version 8.0.0 (clang-800.0.42.1) |
-| Clang Xcode 8.3 | Darwin Kernel Version 16.5.0 (macOS 10.12.4) | Apple LLVM version 8.1.0 (clang-802.0.38) |
-| Clang Xcode 9.0 | Darwin Kernel Version 16.7.0 (macOS 10.12.6) | Apple LLVM version 9.0.0 (clang-900.0.37) |
-| Clang Xcode 9.1 | Darwin Kernel Version 16.7.0 (macOS 10.12.6) | Apple LLVM version 9.0.0 (clang-900.0.38) |
-| Clang Xcode 9.2 | Darwin Kernel Version 16.7.0 (macOS 10.12.6) | Apple LLVM version 8.1.0 (clang-900.0.39.2) |
+| GCC 4.9.4       | Ubuntu 14.04.1 LTS           | g++-4.9 (Ubuntu 4.9.4-2ubuntu1~14.04.1) 4.9.4 |
+| GCC 5.5.0       | Ubuntu 14.04.1 LTS           | g++-5 (Ubuntu 5.5.0-12ubuntu1~14.04) 5.5.0 20171010 |
+| GCC 6.4.0       | Ubuntu 14.04.1 LTS           | g++-6 (Ubuntu 6.4.0-17ubuntu1~14.04) 6.4.0 20180424 |
+| GCC 7.3.0       | Ubuntu 14.04.1 LTS           | g++-7 (Ubuntu 7.3.0-21ubuntu1~14.04) 7.3.0 |
+| GCC 7.3.0       | Windows Server 2012 R2 (x64) | g++ (x86_64-posix-seh-rev0, Built by MinGW-W64 project) 7.3.0 |
+| GCC 8.1.0       | Ubuntu 14.04.1 LTS           | g++-8 (Ubuntu 8.1.0-5ubuntu1~14.04) 8.1.0 |
+| Clang 3.5.0     | Ubuntu 14.04.1 LTS           | clang version 3.5.0-4ubuntu2~trusty2 (tags/RELEASE_350/final) (based on LLVM 3.5.0) |
+| Clang 3.6.2     | Ubuntu 14.04.1 LTS           | clang version 3.6.2-svn240577-1~exp1 (branches/release_36) (based on LLVM 3.6.2) |
+| Clang 3.7.1     | Ubuntu 14.04.1 LTS           | clang version 3.7.1-svn253571-1~exp1 (branches/release_37) (based on LLVM 3.7.1) |
+| Clang 3.8.0     | Ubuntu 14.04.1 LTS           | clang version 3.8.0-2ubuntu3~trusty5 (tags/RELEASE_380/final) |
+| Clang 3.9.1     | Ubuntu 14.04.1 LTS           | clang version 3.9.1-4ubuntu3~14.04.3 (tags/RELEASE_391/rc2) |
+| Clang 4.0.1     | Ubuntu 14.04.1 LTS           | clang version 4.0.1-svn305264-1~exp1 (branches/release_40) |
+| Clang 5.0.2     | Ubuntu 14.04.1 LTS           | clang version 5.0.2-svn328729-1~exp1~20180509123505.100 (branches/release_50) |
+| Clang 6.0.1     | Ubuntu 14.04.1 LTS           | clang version 6.0.1-svn334776-1~exp1~20180726133705.85 (branches/release_60) |
+| Clang Xcode 6.4 | OSX 10.10.5 | Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn) |
+| Clang Xcode 7.3 | OSX 10.11.6 | Apple LLVM version 7.3.0 (clang-703.0.31) |
+| Clang Xcode 8.0 | OSX 10.11.6 | Apple LLVM version 8.0.0 (clang-800.0.38) |
+| Clang Xcode 8.1 | OSX 10.12.6 | Apple LLVM version 8.0.0 (clang-800.0.42.1) |
+| Clang Xcode 8.2 | OSX 10.12.6 | Apple LLVM version 8.0.0 (clang-800.0.42.1) |
+| Clang Xcode 8.3 | OSX 10.11.6 | Apple LLVM version 8.1.0 (clang-802.0.38) |
+| Clang Xcode 9.0 | OSX 10.12.6 | Apple LLVM version 9.0.0 (clang-900.0.37) |
+| Clang Xcode 9.1 | OSX 10.12.6 | Apple LLVM version 9.0.0 (clang-900.0.38) |
+| Clang Xcode 9.2 | OSX 10.13.3 | Apple LLVM version 9.1.0 (clang-902.0.39.1) |
+| Clang Xcode 9.3 | OSX 10.13.3 | Apple LLVM version 9.1.0 (clang-902.0.39.2) |
 | Visual Studio 14 2015 | Windows Server 2012 R2 (x64) | Microsoft (R) Build Engine version 14.0.25420.1, MSVC 19.0.24215.1 |
-| Visual Studio 2017 | Windows Server 2016 | Microsoft (R) Build Engine version 15.5.180.51428, MSVC 19.12.25830.2 |
+| Visual Studio 2017 | Windows Server 2016 | Microsoft (R) Build Engine version 15.7.180.61344, MSVC 19.14.26433.0 |
 
 ## License
 
@@ -795,7 +904,7 @@ The following compilers are currently used in continuous integration at [Travis]
 
 The class is licensed under the [MIT License](http://opensource.org/licenses/MIT):
 
-Copyright &copy; 2013-2017 [Niels Lohmann](http://nlohmann.me)
+Copyright &copy; 2013-2018 [Niels Lohmann](http://nlohmann.me)
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 
@@ -807,12 +916,17 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I
 
 The class contains the UTF-8 Decoder from Bjoern Hoehrmann which is licensed under the [MIT License](http://opensource.org/licenses/MIT) (see above). Copyright &copy; 2008-2009 [Björn Hoehrmann](http://bjoern.hoehrmann.de/) <bjoern@hoehrmann.de>
 
+The class contains a slightly modified version of the Grisu2 algorithm from Florian Loitsch which is licensed under the [MIT License](http://opensource.org/licenses/MIT) (see above). Copyright &copy; 2009 [Florian Loitsch](http://florian.loitsch.com/)
+
 ## Contact
 
-If you have questions regarding the library, I would like to invite you to [open an issue at Github](https://github.com/nlohmann/json/issues/new). Please describe your request, problem, or question as detailed as possible, and also mention the version of the library you are using as well as the version of your compiler and operating system. Opening an issue at Github allows other users and contributors to this library to collaborate. For instance, I have little experience with MSVC, and most issues in this regard have been solved by a growing community. If you have a look at the [closed issues](https://github.com/nlohmann/json/issues?q=is%3Aissue+is%3Aclosed), you will see that we react quite timely in most cases.
+If you have questions regarding the library, I would like to invite you to [open an issue at GitHub](https://github.com/nlohmann/json/issues/new). Please describe your request, problem, or question as detailed as possible, and also mention the version of the library you are using as well as the version of your compiler and operating system. Opening an issue at GitHub allows other users and contributors to this library to collaborate. For instance, I have little experience with MSVC, and most issues in this regard have been solved by a growing community. If you have a look at the [closed issues](https://github.com/nlohmann/json/issues?q=is%3Aissue+is%3Aclosed), you will see that we react quite timely in most cases.
 
 Only if your request would contain confidential information, please [send me an email](mailto:mail@nlohmann.me). For encrypted messages, please use [this key](https://keybase.io/nlohmann/pgp_keys.asc).
 
+## Security
+
+[Commits by Niels Lohmann](https://github.com/nlohmann/json/commits) and [releases](https://github.com/nlohmann/json/releases) are signed with this [PGP Key](https://keybase.io/nlohmann/pgp_keys.asc?fingerprint=797167ae41c0a6d9232e48457f3cea63ae251b69).
 
 ## Thanks
 
@@ -847,7 +961,7 @@ I deeply appreciate the help of the following people.
 - [Corbin Hughes](https://github.com/nibroc) fixed some typos in the contribution guidelines.
 - [twelsby](https://github.com/twelsby) fixed the array subscript operator, an issue that failed the MSVC build, and floating-point parsing/dumping. He further added support for unsigned integer numbers and implemented better roundtrip support for parsed numbers.
 - [Volker Diels-Grabsch](https://github.com/vog) fixed a link in the README file.
-- [msm-](https://github.com/msm-) added support for american fuzzy lop. 
+- [msm-](https://github.com/msm-) added support for American Fuzzy Lop.
 - [Annihil](https://github.com/Annihil) fixed an example in the README file.
 - [Themercee](https://github.com/Themercee) noted a wrong URL in the README file.
 - [Lv Zheng](https://github.com/lv-zheng) fixed a namespace issue with `int64_t` and `uint64_t`.
@@ -860,14 +974,14 @@ I deeply appreciate the help of the following people.
 - [duncanwerner](https://github.com/duncanwerner) found a really embarrassing performance regression in the 2.0.0 release.
 - [Damien](https://github.com/dtoma) fixed one of the last conversion warnings.
 - [Thomas Braun](https://github.com/t-b) fixed a warning in a test case.
-- [Théo DELRIEU](https://github.com/theodelrieu) patiently and constructively oversaw the long way toward [iterator-range parsing](https://github.com/nlohmann/json/issues/290). He also implemented the magic behind the serialization/deserialization of user-defined types.
+- [Théo DELRIEU](https://github.com/theodelrieu) patiently and constructively oversaw the long way toward [iterator-range parsing](https://github.com/nlohmann/json/issues/290). He also implemented the magic behind the serialization/deserialization of user-defined types and split the single header file into smaller chunks.
 - [Stefan](https://github.com/5tefan) fixed a minor issue in the documentation.
 - [Vasil Dimov](https://github.com/vasild) fixed the documentation regarding conversions from `std::multiset`.
 - [ChristophJud](https://github.com/ChristophJud) overworked the CMake files to ease project inclusion.
 - [Vladimir Petrigo](https://github.com/vpetrigo) made a SFINAE hack more readable and added Visual Studio 17 to the build matrix.
 - [Denis Andrejew](https://github.com/seeekr) fixed a grammar issue in the README file.
 - [Pierre-Antoine Lacaze](https://github.com/palacaze) found a subtle bug in the `dump()` function.
-- [TurpentineDistillery](https://github.com/TurpentineDistillery) pointed to [`std::locale::classic()`](http://en.cppreference.com/w/cpp/locale/locale/classic) to avoid too much locale joggling, found some nice performance improvements in the parser, improved the benchmarking code, and realized locale-independent number parsing and printing.
+- [TurpentineDistillery](https://github.com/TurpentineDistillery) pointed to [`std::locale::classic()`](https://en.cppreference.com/w/cpp/locale/locale/classic) to avoid too much locale joggling, found some nice performance improvements in the parser, improved the benchmarking code, and realized locale-independent number parsing and printing.
 - [cgzones](https://github.com/cgzones) had an idea how to fix the Coverity scan.
 - [Jared Grubb](https://github.com/jaredgrubb) silenced a nasty documentation warning.
 - [Yixin Zhang](https://github.com/qwename) fixed an integer overflow check.
@@ -893,7 +1007,7 @@ I deeply appreciate the help of the following people.
 - [Vincent Thiery](https://github.com/vthiery) maintains a package for the Conan package manager.
 - [Steffen](https://github.com/koemeet) fixed a potential issue with MSVC and `std::min`.
 - [Mike Tzou](https://github.com/Chocobo1) fixed some typos.
-- [amrcode](https://github.com/amrcode) noted a missleading documentation about comparison of floats.
+- [amrcode](https://github.com/amrcode) noted a misleading documentation about comparison of floats.
 - [Oleg Endo](https://github.com/olegendo) reduced the memory consumption by replacing `<iostream>` with `<iosfwd>`.
 - [dan-42](https://github.com/dan-42) cleaned up the CMake files to simplify including/reusing of the library.
 - [Nikita Ofitserov](https://github.com/himikof) allowed for moving values from initializer lists.
@@ -910,25 +1024,50 @@ I deeply appreciate the help of the following people.
 - [Nate Vargas](https://github.com/eld00d) added a Doxygen tag file.
 - [pvleuven](https://github.com/pvleuven) helped fixing a warning in ICC.
 - [Pavel](https://github.com/crea7or) helped fixing some warnings in MSVC.
-- [Jamie Seward](https://github.com/jseward) avoided unneccessary string copies in `find()` and `count()`.
+- [Jamie Seward](https://github.com/jseward) avoided unnecessary string copies in `find()` and `count()`.
 - [Mitja](https://github.com/Itja) fixed some typos.
 - [Jorrit Wronski](https://github.com/jowr) updated the Hunter package links.
 - [Matthias Möller](https://github.com/TinyTinni) added a `.natvis` for the MSVC debug view.
 - [bogemic](https://github.com/bogemic) fixed some C++17 deprecation warnings.
 - [Eren Okka](https://github.com/erengy) fixed some MSVC warnings.
-
+- [abolz](https://github.com/abolz) integrated the Grisu2 algorithm for proper floating-point formatting, allowing more roundtrip checks to succeed.
+- [Vadim Evard](https://github.com/Pipeliner) fixed a Markdown issue in the README.
+- [zerodefect](https://github.com/zerodefect) fixed a compiler warning.
+- [Kert](https://github.com/kaidokert) allowed to template the string type in the serialization and added the possibility to override the exceptional behavior.
+- [mark-99](https://github.com/mark-99) helped fixing an ICC error.
+- [Patrik Huber](https://github.com/patrikhuber) fixed links in the README file.
+- [johnfb](https://github.com/johnfb) found a bug in the implementation of CBOR's indefinite length strings.
+- [Paul Fultz II](https://github.com/pfultz2) added a note on the cget package manager.
+- [Wilson Lin](https://github.com/wla80) made the integration section of the README more concise.
+- [RalfBielig](https://github.com/ralfbielig) detected and fixed a memory leak in the parser callback.
+- [agrianius](https://github.com/agrianius) allowed to dump JSON to an alternative string type.
+- [Kevin Tonon](https://github.com/ktonon) overworked the C++11 compiler checks in CMake.
+- [Axel Huebl](https://github.com/ax3l) simplified a CMake check and added support for the [Spack package manager](https://spack.io).
+- [Carlos O'Ryan](https://github.com/coryan) fixed a typo.
+- [James Upjohn](https://github.com/jammehcow) fixed a version number in the compilers section.
+- [Chuck Atkins](https://github.com/chuckatkins) adjusted the CMake files to the CMake packaging guidelines
+- [Jan Schöppach](https://github.com/dns13) fixed a typo.
+- [martin-mfg](https://github.com/martin-mfg) fixed a typo.
+- [Matthias Möller](https://github.com/TinyTinni) removed the dependency from `std::stringstream`.
+- [agrianius](https://github.com/agrianius) added code to use alternative string implementations.
+- [Daniel599](https://github.com/Daniel599) allowed to use more algorithms with the `items()` function.
+- [Julius Rakow](https://github.com/jrakow) fixed the Meson include directory and fixed the links to [cppreference.com](cppreference.com).
+- [Sonu Lohani](https://github.com/sonulohani) fixed the compilation with MSVC 2015 in debug mode.
+- [grembo](https://github.com/grembo) fixed the test suite and re-enabled several test cases.
+- [Hyeon Kim](https://github.com/simnalamburt) introduced the macro `JSON_INTERNAL_CATCH` to control the exception handling inside the library.
+- [thyu](https://github.com/thyu) fixed a compiler warning.
 
 Thanks a lot for helping out! Please [let me know](mailto:mail@nlohmann.me) if I forgot someone.
 
 
 ## Used third-party tools
 
-The library itself contains of a single header file licensed under the MIT license. However, it is built, tested, documented, and whatnot using a lot of third-party tools and services. Thanks a lot!
+The library itself consists of a single header file licensed under the MIT license. However, it is built, tested, documented, and whatnot using a lot of third-party tools and services. Thanks a lot!
 
+- [**amalgamate.py - Amalgamate C source and header files**](https://github.com/edlund/amalgamate) to create a single header file
 - [**American fuzzy lop**](http://lcamtuf.coredump.cx/afl/) for fuzz testing
 - [**AppVeyor**](https://www.appveyor.com) for [continuous integration](https://ci.appveyor.com/project/nlohmann/json) on Windows
 - [**Artistic Style**](http://astyle.sourceforge.net) for automatic source code identation
-- [**benchpress**](https://github.com/sbs-ableton/benchpress) to benchmark the code
 - [**Catch**](https://github.com/philsquared/Catch) for the unit tests
 - [**Clang**](http://clang.llvm.org) for compilation with code sanitizers
 - [**Cmake**](https://cmake.org) for build automation
@@ -936,17 +1075,17 @@ The library itself contains of a single header file licensed under the MIT licen
 - [**Coveralls**](https://coveralls.io) to measure [code coverage](https://coveralls.io/github/nlohmann/json)
 - [**Coverity Scan**](https://scan.coverity.com) for [static analysis](https://scan.coverity.com/projects/nlohmann-json)
 - [**cppcheck**](http://cppcheck.sourceforge.net) for static analysis
-- [**cxxopts**](https://github.com/jarro2783/cxxopts) to let benchpress parse command-line parameters
 - [**Doxygen**](http://www.stack.nl/~dimitri/doxygen/) to generate [documentation](https://nlohmann.github.io/json/)
 - [**git-update-ghpages**](https://github.com/rstacruz/git-update-ghpages) to upload the documentation to gh-pages
-- [**Github Changelog Generator**](https://github.com/skywinder/github-changelog-generator) to generate the [ChangeLog](https://github.com/nlohmann/json/blob/develop/ChangeLog.md)
+- [**GitHub Changelog Generator**](https://github.com/skywinder/github-changelog-generator) to generate the [ChangeLog](https://github.com/nlohmann/json/blob/develop/ChangeLog.md)
+- [**Google Benchmark**](https://github.com/google/benchmark) to implement the benchmarks
 - [**libFuzzer**](http://llvm.org/docs/LibFuzzer.html) to implement fuzz testing for OSS-Fuzz
-- [**OSS-Fuzz**](https://github.com/google/oss-fuzz) for continuous fuzz testing of the library
+- [**OSS-Fuzz**](https://github.com/google/oss-fuzz) for continuous fuzz testing of the library ([project repository](https://github.com/google/oss-fuzz/tree/master/projects/json))
 - [**Probot**](https://probot.github.io) for automating maintainer tasks such as closing stale issues, requesting missing information, or detecting toxic comments.
 - [**send_to_wandbox**](https://github.com/nlohmann/json/blob/develop/doc/scripts/send_to_wandbox.py) to send code examples to [Wandbox](http://melpon.org/wandbox)
 - [**Travis**](https://travis-ci.org) for [continuous integration](https://travis-ci.org/nlohmann/json) on Linux and macOS
 - [**Valgrind**](http://valgrind.org) to check for correct memory management
-- [**Wandbox**](http://melpon.org/wandbox) for [online examples](https://wandbox.org/permlink/Op57X0V7fTf2tdwl)
+- [**Wandbox**](http://melpon.org/wandbox) for [online examples](https://wandbox.org/permlink/TarF5pPn9NtHQjhf)
 
 
 ## Projects using JSON for Modern C++
@@ -956,7 +1095,7 @@ The library is currently used in Apple macOS Sierra and iOS 10. I am not sure wh
 
 ## Notes
 
-- The code contains numerous debug **assertions** which can be switched off by defining the preprocessor macro `NDEBUG`, see the [documentation of `assert`](http://en.cppreference.com/w/cpp/error/assert). In particular, note [`operator[]`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a2e26bd0b0168abb61f67ad5bcd5b9fa1.html#a2e26bd0b0168abb61f67ad5bcd5b9fa1) implements **unchecked access** for const objects: If the given key is not present, the behavior is undefined (think of a dereferenced null pointer) and yields an [assertion failure](https://github.com/nlohmann/json/issues/289) if assertions are switched on. If you are not sure whether an element in an object exists, use checked access with the [`at()` function](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a674de1ee73e6bf4843fc5dc1351fb726.html#a674de1ee73e6bf4843fc5dc1351fb726).
+- The code contains numerous debug **assertions** which can be switched off by defining the preprocessor macro `NDEBUG`, see the [documentation of `assert`](https://en.cppreference.com/w/cpp/error/assert). In particular, note [`operator[]`](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a2e26bd0b0168abb61f67ad5bcd5b9fa1.html#a2e26bd0b0168abb61f67ad5bcd5b9fa1) implements **unchecked access** for const objects: If the given key is not present, the behavior is undefined (think of a dereferenced null pointer) and yields an [assertion failure](https://github.com/nlohmann/json/issues/289) if assertions are switched on. If you are not sure whether an element in an object exists, use checked access with the [`at()` function](https://nlohmann.github.io/json/classnlohmann_1_1basic__json_a674de1ee73e6bf4843fc5dc1351fb726.html#a674de1ee73e6bf4843fc5dc1351fb726).
 - As the exact type of a number is not defined in the [JSON specification](http://rfc7159.net/rfc7159), this library tries to choose the best fitting C++ number type automatically. As a result, the type `double` may be used to store numbers which may yield [**floating-point exceptions**](https://github.com/nlohmann/json/issues/181) in certain rare situations if floating-point exceptions have been unmasked in the calling code. These exceptions are not caused by the library and need to be fixed in the calling code, such as by re-masking the exceptions prior to calling library functions.
 - The library supports **Unicode input** as follows:
   - Only **UTF-8** encoded input is supported which is the default encoding for JSON according to [RFC 7159](http://rfc7159.net/rfc7159#rfc.section.8.1).

include/ZeroTierDebug.h

@@ -76,6 +76,9 @@
 #ifdef __APPLE__
   #define ZT_THREAD_ID (long)0 // (long)gettid()
 #endif
+#ifdef __FreeBSD__
+  #define ZT_THREAD_ID (long)0 // (long)gettid()
+#endif
 #ifdef _WIN32
   #define ZT_THREAD_ID (long)0 // 
 #endif
@@ -86,7 +89,7 @@
 		#include <android/log.h>
 		#define ZT_LOG_TAG "ZTSDK"
 #endif
-#if defined(ZT_TRACE)
+#if defined(ZT_DEBUG_TRACE)
 	#if ZT_MSG_INFO == true
 		#if defined(__ANDROID__)
 			#define DEBUG_INFO(fmt, args...) ((void)__android_log_print(ANDROID_LOG_VERBOSE, ZT_LOG_TAG, \
@@ -96,7 +99,7 @@
 			#define DEBUG_INFO(fmt, ...) fprintf(stderr, ZT_GRN "INFO [%ld]: %17s:%5d:%25s: " fmt "\n" \
 					ZT_RESET, ZT_THREAD_ID, ZT_FILENAME, __LINE__, __FUNCTION__, __VA_ARGS__)
 		#endif
-		#if defined(__linux__) or defined(__APPLE__)
+		#if defined(__linux__) or defined(__APPLE__) or defined(__FreeBSD__)
 			#define DEBUG_INFO(fmt, args ...) fprintf(stderr, ZT_GRN "INFO [%ld]: %17s:%5d:%25s: " fmt "\n" \
 					ZT_RESET, ZT_THREAD_ID, ZT_FILENAME, __LINE__, __FUNCTION__, ##args)
 		#endif

include/ZeroTierOne.h

@@ -411,7 +411,7 @@ enum ZT_ResultCode
 	ZT_RESULT_ERROR_UNSUPPORTED_OPERATION = 1001,
 
 	/**
-	 * The requestion operation was given a bad parameter or was called in an invalid state
+	 * The requested operation was given a bad parameter or was called in an invalid state
 	 */
 	ZT_RESULT_ERROR_BAD_PARAMETER = 1002
 };
@@ -422,6 +422,35 @@ enum ZT_ResultCode
  */
 #define ZT_ResultCode_isFatal(x) ((((int)(x)) >= 100)&&(((int)(x)) < 1000))
 
+/**
+ * The multipath algorithm in use by this node.
+ */
+enum ZT_MultipathMode
+{
+	/**
+	 * No active multipath.
+	 *
+	 * Traffic is merely sent over the strongest path. That being
+	 * said, this mode will automatically failover in the event that a link goes down.
+	 */
+	ZT_MULTIPATH_NONE = 0,
+
+	/**
+	 * Traffic is randomly distributed among all active paths.
+	 *
+	 * Will cease sending traffic over links that appear to be stale.
+	 */
+	ZT_MULTIPATH_RANDOM = 1,
+
+	/**
+	 * Traffic is allocated across all active paths in proportion to their strength and
+	 * reliability.
+	 *
+	 * Will cease sending traffic over links that appear to be stale.
+	 */
+	ZT_MULTIPATH_PROPORTIONALLY_BALANCED = 2,
+};
+
 /**
  * Status codes sent to status update callback when things happen
  */
@@ -715,6 +744,11 @@ enum ZT_VirtualNetworkRuleType
 	 */
 	ZT_NETWORK_RULE_ACTION_BREAK = 5,
 
+	/**
+	 * Place a matching frame in the specified QoS bucket
+	 */
+	ZT_NETWORK_RULE_ACTION_PRIORITY = 6,
+
 	/**
 	 * Maximum ID for an ACTION, anything higher is a MATCH
 	 */
@@ -905,6 +939,11 @@ typedef struct
 			uint32_t flags;
 			uint16_t length;
 		} fwd;
+
+		/**
+		 * Quality of Service (QoS) bucket we want a frame to be placed in
+		 */
+		uint8_t qosBucket;
 	} v;
 } ZT_VirtualNetworkRule;
 
@@ -1185,6 +1224,56 @@ typedef struct
 	 */
 	uint64_t trustedPathId;
 
+	/**
+	 * One-way latency
+	 */
+	float latency;
+
+	/**
+	 * How much latency varies over time
+	 */
+	float packetDelayVariance;
+
+	/**
+	 * How much observed throughput varies over time
+	 */
+	float throughputDisturbCoeff;
+
+	/**
+	 * Packet Error Ratio (PER)
+	 */
+	float packetErrorRatio;
+
+	/**
+	 * Packet Loss Ratio (PLR)
+	 */
+	float packetLossRatio;
+
+	/**
+	 * Stability of the path
+	 */
+	float stability;
+
+	/**
+	 * Current throughput (moving average)
+	 */
+	uint64_t throughput;
+
+	/**
+	 * Maximum observed throughput for this path
+	 */
+	uint64_t maxThroughput;
+
+	/**
+	 * Percentage of traffic allocated to this path
+	 */
+	float allocation;
+
+	/**
+	 * Name of physical interface (for monitoring)
+	 */
+	char *ifname;
+
 	/**
 	 * Is path expired?
 	 */
@@ -1236,6 +1325,11 @@ typedef struct
 	 */
 	unsigned int pathCount;
 
+	/**
+	 * Whether this peer was ever reachable via an aggregate link
+	 */
+	bool hadAggregateLink;
+
 	/**
 	 * Known network paths to peer
 	 */
@@ -1462,7 +1556,7 @@ typedef int (*ZT_WirePacketSendFunction)(
 /**
  * Function to check whether a path should be used for ZeroTier traffic
  *
- * Paramters:
+ * Parameters:
  *  (1) Node
  *  (2) User pointer
  *  (3) ZeroTier address or 0 for none/any
@@ -1495,7 +1589,7 @@ typedef int (*ZT_PathCheckFunction)(
  *  (1) Node
  *  (2) User pointer
  *  (3) ZeroTier address (least significant 40 bits)
- *  (4) Desried address family or -1 for any
+ *  (4) Desired address family or -1 for any
  *  (5) Buffer to fill with result
  *
  * If provided this function will be occasionally called to get physical
@@ -1660,7 +1754,7 @@ ZT_SDK_API enum ZT_ResultCode ZT_Node_processBackgroundTasks(ZT_Node *node,void
  * Join a network
  *
  * This may generate calls to the port config callback before it returns,
- * or these may be deffered if a netconf is not available yet.
+ * or these may be differed if a netconf is not available yet.
  *
  * If we are already a member of the network, nothing is done and OK is
  * returned.

make-bsd.mk

@@ -123,6 +123,7 @@ endif
 ifeq ($(ZT_USE_ARM32_NEON_ASM_SALSA2012),1)
 	override DEFS+=-DZT_USE_ARM32_NEON_ASM_SALSA2012
 	override CORE_OBJS+=ext/arm32-neon-salsa2012-asm/salsa2012.o
+	override ASFLAGS+=-meabi=5
 endif
 
 override DEFS+=-DZT_BUILD_PLATFORM=$(ZT_BUILD_PLATFORM) -DZT_BUILD_ARCHITECTURE=$(ZT_ARCHITECTURE) -DZT_SOFTWARE_UPDATE_DEFAULT="\"disable\""
@@ -159,7 +160,7 @@ clean:
 	rm -rf *.a *.o node/*.o controller/*.o osdep/*.o service/*.o ext/http-parser/*.o build-* zerotier-one zerotier-idtool zerotier-selftest zerotier-cli $(ONE_OBJS) $(CORE_OBJS)
 
 debug:	FORCE
-	gmake -j 4 ZT_DEBUG=1
+	$(MAKE) -j ZT_DEBUG=1
 
 install:	one
 	rm -f /usr/local/sbin/zerotier-one

make-linux.mk

@@ -17,6 +17,10 @@ DESTDIR?=
 
 include objects.mk
 ONE_OBJS+=osdep/LinuxEthernetTap.o
+ONE_OBJS+=osdep/LinuxNetLink.o
+
+NLTEST_OBJS+=osdep/LinuxNetLink.o node/InetAddress.o node/Utils.o node/Salsa20.o
+NLTEST_OBJS+=nltest.o
 
 # Auto-detect miniupnpc and nat-pmp as well and use system libs if present,
 # otherwise build into binary as done on Mac and Windows.
@@ -42,8 +46,15 @@ endif
 # Trying to use dynamically linked libhttp-parser causes tons of compatibility problems.
 ONE_OBJS+=ext/http-parser/http_parser.o
 
+# Build with address sanitization library for advanced debugging (clang)
+ifeq ($(ZT_SANITIZE),1)
+	DEFS+=-fsanitize=address -DASAN_OPTIONS=symbolize=1
+endif
+ifeq ($(ZT_DEBUG_TRACE),1)
+	DEFS+=-DZT_DEBUG_TRACE
+endif
 ifeq ($(ZT_TRACE),1)
-	override DEFS+=-DZT_TRACE
+	DEFS+=-DZT_TRACE
 endif
 
 ifeq ($(ZT_RULES_ENGINE_DEBUGGING),1)
@@ -90,6 +101,11 @@ ifeq ($(ZT_USE_TEST_TAP),1)
 	override DEFS+=-DZT_USE_TEST_TAP
 endif
 
+ifeq ($(ZT_VAULT_SUPPORT),1)
+	override DEFS+=-DZT_VAULT_SUPPORT=1
+	override LDLIBS+=-lcurl
+endif
+
 # Uncomment for gprof profile build
 #CFLAGS=-Wall -g -pg -pthread $(INCLUDES) $(DEFS)
 #CXXFLAGS=-Wall -g -pg -pthread $(INCLUDES) $(DEFS)
@@ -306,6 +322,9 @@ debug:	FORCE
 	make ZT_DEBUG=1 one
 	make ZT_DEBUG=1 selftest
 
+nltest: $(NLTEST_OBJS)
+	$(CXX) $(CXXFLAGS) $(LDFLAGS) -o nltest $(NLTEST_OBJS) $(LDLIBS)
+
 # Note: keep the symlinks in /var/lib/zerotier-one to the binaries since these
 # provide backward compatibility with old releases where the binaries actually
 # lived here. Folks got scripts.

make-mac.mk

@@ -43,7 +43,10 @@ ONE_OBJS+=ext/libnatpmp/natpmp.o ext/libnatpmp/getgateway.o ext/miniupnpc/connec
 
 # Build with address sanitization library for advanced debugging (clang)
 ifeq ($(ZT_SANITIZE),1)
-	SANFLAGS+=-fsanitize=address -DASAN_OPTIONS=symbolize=1
+	DEFS+=-fsanitize=address -DASAN_OPTIONS=symbolize=1
+endif
+ifeq ($(ZT_DEBUG_TRACE),1)
+	DEFS+=-DZT_DEBUG_TRACE
 endif
 # Debug mode -- dump trace output, build binary with -g
 ifeq ($(ZT_DEBUG),1)
@@ -63,6 +66,11 @@ ifeq ($(ZT_TRACE),1)
 	DEFS+=-DZT_TRACE
 endif
 
+ifeq ($(ZT_VAULT_SUPPORT),1)
+	DEFS+=-DZT_VAULT_SUPPORT=1
+	LIBS+=-lcurl
+endif
+
 CXXFLAGS=$(CFLAGS) -std=c++11 -stdlib=libc++ 
 
 all: one macui

nltest.cpp

@@ -0,0 +1,13 @@
+#include "osdep/LinuxNetLink.hpp"
+
+using namespace ZeroTier;
+
+int main(int argc, char **argv)
+{
+    LinuxNetLink &nl = LinuxNetLink::getInstance();
+
+
+    while(true) {
+        Thread::sleep(1000);
+    }
+}

node/Capability.hpp

@@ -52,7 +52,7 @@ class RuntimeEnvironment;
  * (1) Evaluates its capabilities in ascending order of ID to determine
  *     which capability allows it to transmit this packet.
  * (2) If it has not done so lately, it then sends this capability to the
- *     receving peer ("presents" it).
+ *     receiving peer ("presents" it).
  * (3) The sender then sends the packet.
  *
  * On the receiving side the receiver evaluates the capabilities presented
@@ -64,7 +64,7 @@ class RuntimeEnvironment;
  *
  * Capabilities support a chain of custody. This is currently unused but
  * in the future would allow the publication of capabilities that can be
- * handed off between nodes. Limited transferrability of capabilities is
+ * handed off between nodes. Limited transferability of capabilities is
  * a feature of true capability based security.
  */
 class Capability : public Credential
@@ -81,7 +81,7 @@ public:
 	 * @param id Capability ID
 	 * @param nwid Network ID
 	 * @param ts Timestamp (at controller)
-	 * @param mccl Maximum custody chain length (1 to create non-transferrable capability)
+	 * @param mccl Maximum custody chain length (1 to create non-transferable capability)
 	 * @param rules Network flow rules for this capability
 	 * @param ruleCount Number of flow rules
 	 */

node/CertificateOfMembership.hpp

@@ -243,7 +243,7 @@ public:
 	 * Compare two certificates for parameter agreement
 	 *
 	 * This compares this certificate with the other and returns true if all
-	 * paramters in this cert are present in the other and if they agree to
+	 * parameters in this cert are present in the other and if they agree to
 	 * within this cert's max delta value for each given parameter.
 	 *
 	 * Tuples present in other but not in this cert are ignored, but any

node/Constants.hpp

@@ -275,6 +275,189 @@
  */
 #define ZT_PING_CHECK_INVERVAL 5000
 
+/**
+ * How often the local.conf file is checked for changes
+ */
+#define ZT_LOCAL_CONF_FILE_CHECK_INTERVAL 10000
+
+/**
+ * How frequently to check for changes to the system's network interfaces. When
+ * the service decides to use this constant it's because we want to react more
+ * quickly to new interfaces that pop up or go down.
+ */
+#define ZT_MULTIPATH_BINDER_REFRESH_PERIOD 5000
+
+/**
+ * Packets are only used for QoS/ACK statistical sampling if their packet ID is divisible by
+ * this integer. This is to provide a mechanism for both peers to agree on which packets need
+ * special treatment without having to exchange information. Changing this value would be
+ * a breaking change and would necessitate a protocol version upgrade. Since each incoming and
+ * outgoing packet ID is checked against this value its evaluation is of the form:
+ * (id & (divisor - 1)) == 0, thus the divisor must be a power of 2.
+ *
+ * This value is set at (16) so that given a normally-distributed RNG output we will sample
+ * 1/16th (or ~6.25%) of packets.
+ */
+#define ZT_PATH_QOS_ACK_PROTOCOL_DIVISOR 0x10
+
+/**
+ * Time horizon for VERB_QOS_MEASUREMENT and VERB_ACK packet processing cutoff
+ */
+#define ZT_PATH_QOS_ACK_CUTOFF_TIME 30000
+
+/**
+ * Maximum number of VERB_QOS_MEASUREMENT and VERB_ACK packets allowed to be
+ * processed within cutoff time. Separate totals are kept for each type but
+ * the limit is the same for both.
+ *
+ * This limits how often this peer will compute statistical estimates
+ * of various QoS measures from a VERB_QOS_MEASUREMENT or VERB_ACK packets to
+ * CUTOFF_LIMIT times per CUTOFF_TIME milliseconds per peer to prevent
+ * this from being useful for DOS amplification attacks.
+ */
+#define ZT_PATH_QOS_ACK_CUTOFF_LIMIT 128
+
+/**
+ * Path choice history window size. This is used to keep track of which paths were
+ * previously selected so that we can maintain a target allocation over time.
+ */
+#define ZT_MULTIPATH_PROPORTION_WIN_SZ 128
+
+/**
+ * How often we will sample packet latency. Should be at least greater than ZT_PING_CHECK_INVERVAL
+ * since we will record a 0 bit/s measurement if no valid latency measurement was made within this
+ * window of time.
+ */
+#define ZT_PATH_LATENCY_SAMPLE_INTERVAL ZT_MULTIPATH_PEER_PING_PERIOD * 2
+
+/**
+ * Interval used for rate-limiting the computation of path quality estimates.
+ */
+#define ZT_PATH_QUALITY_COMPUTE_INTERVAL 1000
+
+/**
+ * Number of samples to consider when computing real-time path statistics
+ */
+#define ZT_PATH_QUALITY_METRIC_REALTIME_CONSIDERATION_WIN_SZ 128
+
+/**
+ * Number of samples to consider when computing performing long-term path quality analysis.
+ * By default this value is set to ZT_PATH_QUALITY_METRIC_REALTIME_CONSIDERATION_WIN_SZ but can
+ * be set to any value greater than that to observe longer-term path quality behavior.
+ */
+#define ZT_PATH_QUALITY_METRIC_WIN_SZ ZT_PATH_QUALITY_METRIC_REALTIME_CONSIDERATION_WIN_SZ
+
+/**
+ * Maximum acceptable Packet Delay Variance (PDV) over a path
+ */
+#define ZT_PATH_MAX_PDV 1000
+
+/**
+ * Maximum acceptable time interval between expectation and receipt of at least one ACK over a path
+ */
+#define ZT_PATH_MAX_AGE 30000
+
+/**
+ * Maximum acceptable mean latency over a path
+ */
+#define ZT_PATH_MAX_MEAN_LATENCY 1000
+
+/**
+ * How much each factor contributes to the "stability" score of a path
+ */
+#define ZT_PATH_CONTRIB_PDV                    1.0 / 3.0
+#define ZT_PATH_CONTRIB_LATENCY                1.0 / 3.0
+#define ZT_PATH_CONTRIB_THROUGHPUT_DISTURBANCE 1.0 / 3.0
+
+/**
+ * How much each factor contributes to the "quality" score of a path
+ */
+#define ZT_PATH_CONTRIB_STABILITY  0.75 / 3.0
+#define ZT_PATH_CONTRIB_THROUGHPUT 1.50 / 3.0
+#define ZT_PATH_CONTRIB_SCOPE      0.75 / 3.0
+
+/**
+ * How often a QoS packet is sent
+ */
+#define ZT_PATH_QOS_INTERVAL 3000
+
+/**
+ * Min and max acceptable sizes for a VERB_QOS_MEASUREMENT packet
+ */
+#define ZT_PATH_MIN_QOS_PACKET_SZ 8 + 1
+#define ZT_PATH_MAX_QOS_PACKET_SZ 1400
+
+/**
+ * How many ID:sojourn time pairs in a single QoS packet
+ */
+#define ZT_PATH_QOS_TABLE_SIZE (ZT_PATH_MAX_QOS_PACKET_SZ * 8) / (64 + 16)
+
+/**
+ * Maximum number of outgoing packets we monitor for QoS information
+ */
+#define ZT_PATH_MAX_OUTSTANDING_QOS_RECORDS 128
+
+/**
+ * Timeout for QoS records
+ */
+#define ZT_PATH_QOS_TIMEOUT ZT_PATH_QOS_INTERVAL * 2
+
+/**
+ * How often the service tests the path throughput
+ */
+#define ZT_PATH_THROUGHPUT_MEASUREMENT_INTERVAL ZT_PATH_ACK_INTERVAL * 8
+
+/**
+ * Minimum amount of time between each ACK packet
+ */
+#define ZT_PATH_ACK_INTERVAL 1000
+
+/**
+ * How often an aggregate link statistics report is emitted into this tracing system
+ */
+#define ZT_PATH_AGGREGATE_STATS_REPORT_INTERVAL 60000
+
+/**
+ * How much an aggregate link's component paths can vary from their target allocation
+ * before the link is considered to be in a state of imbalance.
+ */
+#define ZT_PATH_IMBALANCE_THRESHOLD 0.20
+
+/**
+ * Max allowable time spent in any queue
+ */
+#define ZT_QOS_TARGET 5 // ms
+
+/**
+ * Time period where the time spent in the queue by a packet should fall below
+ * target at least once
+ */
+#define ZT_QOS_INTERVAL 100 // ms
+
+/**
+ * The number of bytes that each queue is allowed to send during each DRR cycle.
+ * This approximates a single-byte-based fairness queuing scheme
+ */
+#define ZT_QOS_QUANTUM ZT_DEFAULT_MTU
+
+/**
+ * The maximum total number of packets that can be queued among all
+ * active/inactive, old/new queues
+ */
+#define ZT_QOS_MAX_ENQUEUED_PACKETS 1024
+
+/**
+ * Number of QoS queues (buckets)
+ */
+#define ZT_QOS_NUM_BUCKETS 9
+
+/**
+ * All unspecified traffic is put in this bucket. Anything in a bucket with a smaller
+ * value is de-prioritized. Anything in a bucket with a higher value is prioritized over
+ * other traffic.
+ */
+#define ZT_QOS_DEFAULT_BUCKET 0
+
 /**
  * How frequently to send heartbeats over in-use paths
  */
@@ -290,6 +473,13 @@
  */
 #define ZT_PEER_PING_PERIOD 60000
 
+/**
+ * Delay between full-fledge pings of directly connected peers.
+ * With multipath bonding enabled ping peers more often to measure
+ * packet loss and latency.
+ */
+#define ZT_MULTIPATH_PEER_PING_PERIOD 5000
+
 /**
  * Paths are considered expired if they have not sent us a real packet in this long
  */

node/Hashtable.hpp

@@ -399,6 +399,10 @@ private:
 	{
 		return ((unsigned long)i * (unsigned long)0x9e3779b1);
 	}
+	static inline unsigned long _hc(const int i)
+	{
+		return ((unsigned long)i * (unsigned long)0x9e3379b1);
+	}
 
 	inline void _grow()
 	{

node/Identity.cpp

@@ -50,8 +50,8 @@ static inline void _computeMemoryHardHash(const void *publicKey,unsigned int pub
 	SHA512::hash(digest,publicKey,publicKeyBytes);
 
 	// Initialize genmem[] using Salsa20 in a CBC-like configuration since
-	// ordinary Salsa20 is randomly seekable. This is good for a cipher
-	// but is not what we want for sequential memory-harndess.
+	// ordinary Salsa20 is randomly seek-able. This is good for a cipher
+	// but is not what we want for sequential memory-hardness.
 	memset(genmem,0,ZT_IDENTITY_GEN_MEMORY);
 	Salsa20 s20(digest,(char *)digest + 32);
 	s20.crypt20((char *)genmem,(char *)genmem,64);

node/IncomingPacket.cpp

@@ -80,6 +80,7 @@ bool IncomingPacket::tryDecode(const RuntimeEnvironment *RR,void *tPtr)
 			if (!trusted) {
 				if (!dearmor(peer->key())) {
 					RR->t->incomingPacketMessageAuthenticationFailure(tPtr,_path,packetId(),sourceAddress,hops(),"invalid MAC");
+					_path->recordInvalidPacket();
 					return true;
 				}
 			}
@@ -93,9 +94,11 @@ bool IncomingPacket::tryDecode(const RuntimeEnvironment *RR,void *tPtr)
 			switch(v) {
 				//case Packet::VERB_NOP:
 				default: // ignore unknown verbs, but if they pass auth check they are "received"
-					peer->received(tPtr,_path,hops(),packetId(),v,0,Packet::VERB_NOP,false,0);
+					peer->received(tPtr,_path,hops(),packetId(),payloadLength(),v,0,Packet::VERB_NOP,false,0);
 					return true;
 				case Packet::VERB_HELLO:                      return _doHELLO(RR,tPtr,true);
+				case Packet::VERB_ACK:                        return _doACK(RR,tPtr,peer);
+				case Packet::VERB_QOS_MEASUREMENT:            return _doQOS_MEASUREMENT(RR,tPtr,peer);
 				case Packet::VERB_ERROR:                      return _doERROR(RR,tPtr,peer);
 				case Packet::VERB_OK:                         return _doOK(RR,tPtr,peer);
 				case Packet::VERB_WHOIS:                      return _doWHOIS(RR,tPtr,peer);
@@ -194,7 +197,57 @@ bool IncomingPacket::_doERROR(const RuntimeEnvironment *RR,void *tPtr,const Shar
 		default: break;
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_ERROR,inRePacketId,inReVerb,false,networkId);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_ERROR,inRePacketId,inReVerb,false,networkId);
+
+	return true;
+}
+
+bool IncomingPacket::_doACK(const RuntimeEnvironment *RR,void *tPtr,const SharedPtr<Peer> &peer)
+{
+	if (!peer->rateGateACK(RR->node->now()))
+		return true;
+	/* Dissect incoming ACK packet. From this we can estimate current throughput of the path, establish known
+	 * maximums and detect packet loss. */
+	if (peer->localMultipathSupport()) {
+		int32_t ackedBytes;
+		if (payloadLength() != sizeof(ackedBytes)) {
+			return true; // ignore
+		}
+		memcpy(&ackedBytes, payload(), sizeof(ackedBytes));
+		_path->receivedAck(RR->node->now(), Utils::ntoh(ackedBytes));
+		peer->inferRemoteMultipathEnabled();
+	}
+
+	return true;
+}
+bool IncomingPacket::_doQOS_MEASUREMENT(const RuntimeEnvironment *RR,void *tPtr,const SharedPtr<Peer> &peer)
+{
+	if (!peer->rateGateQoS(RR->node->now()))
+		return true;
+	/* Dissect incoming QoS packet. From this we can compute latency values and their variance.
+	 * The latency variance is used as a measure of "jitter". */
+	if (peer->localMultipathSupport()) {
+		if (payloadLength() > ZT_PATH_MAX_QOS_PACKET_SZ || payloadLength() < ZT_PATH_MIN_QOS_PACKET_SZ) {
+			return true; // ignore
+		}
+		const int64_t now = RR->node->now();
+		uint64_t rx_id[ZT_PATH_QOS_TABLE_SIZE];
+		uint16_t rx_ts[ZT_PATH_QOS_TABLE_SIZE];
+		char *begin = (char *)payload();
+		char *ptr = begin;
+		int count = 0;
+		int len = payloadLength();
+		// Read packet IDs and latency compensation intervals for each packet tracked by this QoS packet
+		while (ptr < (begin + len) && (count < ZT_PATH_QOS_TABLE_SIZE)) {
+			memcpy((void*)&rx_id[count], ptr, sizeof(uint64_t));
+			ptr+=sizeof(uint64_t);
+			memcpy((void*)&rx_ts[count], ptr, sizeof(uint16_t));
+			ptr+=sizeof(uint16_t);
+			count++;
+		}
+		_path->receivedQoS(now, count, rx_id, rx_ts);
+		peer->inferRemoteMultipathEnabled();
+	}
 
 	return true;
 }
@@ -395,7 +448,7 @@ bool IncomingPacket::_doHELLO(const RuntimeEnvironment *RR,void *tPtr,const bool
 	_path->send(RR,tPtr,outp.data(),outp.size(),now);
 
 	peer->setRemoteVersion(protoVersion,vMajor,vMinor,vRevision); // important for this to go first so received() knows the version
-	peer->received(tPtr,_path,hops(),pid,Packet::VERB_HELLO,0,Packet::VERB_NOP,false,0);
+	peer->received(tPtr,_path,hops(),pid,payloadLength(),Packet::VERB_HELLO,0,Packet::VERB_NOP,false,0);
 
 	return true;
 }
@@ -445,8 +498,9 @@ bool IncomingPacket::_doOK(const RuntimeEnvironment *RR,void *tPtr,const SharedP
 				}
 			}
 
-			if (!hops())
-				_path->updateLatency((unsigned int)latency);
+			if (!hops() && (RR->node->getMultipathMode() != ZT_MULTIPATH_NONE)) {
+				_path->updateLatency((unsigned int)latency, RR->node->now());
+			}
 
 			peer->setRemoteVersion(vProto,vMajor,vMinor,vRevision);
 
@@ -507,7 +561,7 @@ bool IncomingPacket::_doOK(const RuntimeEnvironment *RR,void *tPtr,const SharedP
 		default: break;
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_OK,inRePacketId,inReVerb,false,networkId);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_OK,inRePacketId,inReVerb,false,networkId);
 
 	return true;
 }
@@ -542,7 +596,7 @@ bool IncomingPacket::_doWHOIS(const RuntimeEnvironment *RR,void *tPtr,const Shar
 		_path->send(RR,tPtr,outp.data(),outp.size(),RR->node->now());
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_WHOIS,0,Packet::VERB_NOP,false,0);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_WHOIS,0,Packet::VERB_NOP,false,0);
 
 	return true;
 }
@@ -566,7 +620,7 @@ bool IncomingPacket::_doRENDEZVOUS(const RuntimeEnvironment *RR,void *tPtr,const
 		}
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_RENDEZVOUS,0,Packet::VERB_NOP,false,0);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_RENDEZVOUS,0,Packet::VERB_NOP,false,0);
 
 	return true;
 }
@@ -595,7 +649,7 @@ bool IncomingPacket::_doFRAME(const RuntimeEnvironment *RR,void *tPtr,const Shar
 		_sendErrorNeedCredentials(RR,tPtr,peer,nwid);
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_FRAME,0,Packet::VERB_NOP,trustEstablished,nwid);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_FRAME,0,Packet::VERB_NOP,trustEstablished,nwid);
 
 	return true;
 }
@@ -618,7 +672,7 @@ bool IncomingPacket::_doEXT_FRAME(const RuntimeEnvironment *RR,void *tPtr,const
 		if (!network->gate(tPtr,peer)) {
 			RR->t->incomingNetworkAccessDenied(tPtr,network,_path,packetId(),size(),peer->address(),Packet::VERB_EXT_FRAME,true);
 			_sendErrorNeedCredentials(RR,tPtr,peer,nwid);
-			peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,false,nwid);
+			peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,false,nwid);
 			return true;
 		}
 
@@ -630,7 +684,7 @@ bool IncomingPacket::_doEXT_FRAME(const RuntimeEnvironment *RR,void *tPtr,const
 			const uint8_t *const frameData = (const uint8_t *)field(comLen + ZT_PROTO_VERB_EXT_FRAME_IDX_PAYLOAD,frameLen);
 
 			if ((!from)||(from == network->mac())) {
-				peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
+				peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
 				return true;
 			}
 
@@ -641,19 +695,19 @@ bool IncomingPacket::_doEXT_FRAME(const RuntimeEnvironment *RR,void *tPtr,const
 							network->learnBridgeRoute(from,peer->address());
 						} else {
 							RR->t->incomingNetworkFrameDropped(tPtr,network,_path,packetId(),size(),peer->address(),Packet::VERB_EXT_FRAME,from,to,"bridging not allowed (remote)");
-							peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
+							peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
 							return true;
 						}
 					} else if (to != network->mac()) {
 						if (to.isMulticast()) {
 							if (network->config().multicastLimit == 0) {
 								RR->t->incomingNetworkFrameDropped(tPtr,network,_path,packetId(),size(),peer->address(),Packet::VERB_EXT_FRAME,from,to,"multicast disabled");
-								peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
+								peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
 								return true;
 							}
 						} else if (!network->config().permitsBridging(RR->identity.address())) {
 							RR->t->incomingNetworkFrameDropped(tPtr,network,_path,packetId(),size(),peer->address(),Packet::VERB_EXT_FRAME,from,to,"bridging not allowed (local)");
-							peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
+							peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
 							return true;
 						}
 					}
@@ -673,9 +727,9 @@ bool IncomingPacket::_doEXT_FRAME(const RuntimeEnvironment *RR,void *tPtr,const
 			_path->send(RR,tPtr,outp.data(),outp.size(),RR->node->now());
 		}
 
-		peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid);
+		peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,true,nwid);
 	} else {
-		peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,false,nwid);
+		peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_EXT_FRAME,0,Packet::VERB_NOP,false,nwid);
 	}
 
 	return true;
@@ -695,7 +749,7 @@ bool IncomingPacket::_doECHO(const RuntimeEnvironment *RR,void *tPtr,const Share
 	outp.armor(peer->key(),true);
 	_path->send(RR,tPtr,outp.data(),outp.size(),RR->node->now());
 
-	peer->received(tPtr,_path,hops(),pid,Packet::VERB_ECHO,0,Packet::VERB_NOP,false,0);
+	peer->received(tPtr,_path,hops(),pid,payloadLength(),Packet::VERB_ECHO,0,Packet::VERB_NOP,false,0);
 
 	return true;
 }
@@ -740,7 +794,7 @@ bool IncomingPacket::_doMULTICAST_LIKE(const RuntimeEnvironment *RR,void *tPtr,c
 		}
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_MULTICAST_LIKE,0,Packet::VERB_NOP,trustEstablished,(network) ? network->id() : 0);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_MULTICAST_LIKE,0,Packet::VERB_NOP,trustEstablished,(network) ? network->id() : 0);
 
 	return true;
 }
@@ -863,7 +917,7 @@ bool IncomingPacket::_doNETWORK_CREDENTIALS(const RuntimeEnvironment *RR,void *t
 		}
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_NETWORK_CREDENTIALS,0,Packet::VERB_NOP,trustEstablished,(network) ? network->id() : 0);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_NETWORK_CREDENTIALS,0,Packet::VERB_NOP,trustEstablished,(network) ? network->id() : 0);
 
 	return true;
 }
@@ -889,7 +943,7 @@ bool IncomingPacket::_doNETWORK_CONFIG_REQUEST(const RuntimeEnvironment *RR,void
 		_path->send(RR,tPtr,outp.data(),outp.size(),RR->node->now());
 	}
 
-	peer->received(tPtr,_path,hopCount,requestPacketId,Packet::VERB_NETWORK_CONFIG_REQUEST,0,Packet::VERB_NOP,false,nwid);
+	peer->received(tPtr,_path,hopCount,requestPacketId,payloadLength(),Packet::VERB_NETWORK_CONFIG_REQUEST,0,Packet::VERB_NOP,false,nwid);
 
 	return true;
 }
@@ -910,7 +964,7 @@ bool IncomingPacket::_doNETWORK_CONFIG(const RuntimeEnvironment *RR,void *tPtr,c
 		}
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_NETWORK_CONFIG,0,Packet::VERB_NOP,false,(network) ? network->id() : 0);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_NETWORK_CONFIG,0,Packet::VERB_NOP,false,(network) ? network->id() : 0);
 
 	return true;
 }
@@ -953,7 +1007,7 @@ bool IncomingPacket::_doMULTICAST_GATHER(const RuntimeEnvironment *RR,void *tPtr
 		}
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_MULTICAST_GATHER,0,Packet::VERB_NOP,trustEstablished,nwid);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_MULTICAST_GATHER,0,Packet::VERB_NOP,trustEstablished,nwid);
 
 	return true;
 }
@@ -979,7 +1033,7 @@ bool IncomingPacket::_doMULTICAST_FRAME(const RuntimeEnvironment *RR,void *tPtr,
 		if (!network->gate(tPtr,peer)) {
 			RR->t->incomingNetworkAccessDenied(tPtr,network,_path,packetId(),size(),peer->address(),Packet::VERB_MULTICAST_FRAME,true);
 			_sendErrorNeedCredentials(RR,tPtr,peer,nwid);
-			peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,false,nwid);
+			peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,false,nwid);
 			return true;
 		}
 
@@ -1003,19 +1057,19 @@ bool IncomingPacket::_doMULTICAST_FRAME(const RuntimeEnvironment *RR,void *tPtr,
 
 		if (network->config().multicastLimit == 0) {
 			RR->t->incomingNetworkFrameDropped(tPtr,network,_path,packetId(),size(),peer->address(),Packet::VERB_MULTICAST_FRAME,from,to.mac(),"multicast disabled");
-			peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,false,nwid);
+			peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,false,nwid);
 			return true;
 		}
 
 		if ((frameLen > 0)&&(frameLen <= ZT_MAX_MTU)) {
 			if (!to.mac().isMulticast()) {
 				RR->t->incomingPacketInvalid(tPtr,_path,packetId(),source(),hops(),Packet::VERB_MULTICAST_FRAME,"destination not multicast");
-				peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
+				peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
 				return true;
 			}
 			if ((!from)||(from.isMulticast())||(from == network->mac())) {
 				RR->t->incomingPacketInvalid(tPtr,_path,packetId(),source(),hops(),Packet::VERB_MULTICAST_FRAME,"invalid source MAC");
-				peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
+				peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
 				return true;
 			}
 
@@ -1029,7 +1083,7 @@ bool IncomingPacket::_doMULTICAST_FRAME(const RuntimeEnvironment *RR,void *tPtr,
 					network->learnBridgeRoute(from,peer->address());
 				} else {
 					RR->t->incomingNetworkFrameDropped(tPtr,network,_path,packetId(),size(),peer->address(),Packet::VERB_MULTICAST_FRAME,from,to.mac(),"bridging not allowed (remote)");
-					peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
+					peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,true,nwid); // trustEstablished because COM is okay
 					return true;
 				}
 			}
@@ -1052,10 +1106,10 @@ bool IncomingPacket::_doMULTICAST_FRAME(const RuntimeEnvironment *RR,void *tPtr,
 			}
 		}
 
-		peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,true,nwid);
+		peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,true,nwid);
 	} else {
 		_sendErrorNeedCredentials(RR,tPtr,peer,nwid);
-		peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,false,nwid);
+		peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_MULTICAST_FRAME,0,Packet::VERB_NOP,false,nwid);
 	}
 
 	return true;
@@ -1067,7 +1121,7 @@ bool IncomingPacket::_doPUSH_DIRECT_PATHS(const RuntimeEnvironment *RR,void *tPt
 
 	// First, subject this to a rate limit
 	if (!peer->rateGatePushDirectPaths(now)) {
-		peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_PUSH_DIRECT_PATHS,0,Packet::VERB_NOP,false,0);
+		peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_PUSH_DIRECT_PATHS,0,Packet::VERB_NOP,false,0);
 		return true;
 	}
 
@@ -1120,7 +1174,7 @@ bool IncomingPacket::_doPUSH_DIRECT_PATHS(const RuntimeEnvironment *RR,void *tPt
 		ptr += addrLen;
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_PUSH_DIRECT_PATHS,0,Packet::VERB_NOP,false,0);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_PUSH_DIRECT_PATHS,0,Packet::VERB_NOP,false,0);
 
 	return true;
 }
@@ -1136,7 +1190,7 @@ bool IncomingPacket::_doUSER_MESSAGE(const RuntimeEnvironment *RR,void *tPtr,con
 		RR->node->postEvent(tPtr,ZT_EVENT_USER_MESSAGE,reinterpret_cast<const void *>(&um));
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_USER_MESSAGE,0,Packet::VERB_NOP,false,0);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_USER_MESSAGE,0,Packet::VERB_NOP,false,0);
 
 	return true;
 }
@@ -1160,7 +1214,7 @@ bool IncomingPacket::_doREMOTE_TRACE(const RuntimeEnvironment *RR,void *tPtr,con
 		}
 	}
 
-	peer->received(tPtr,_path,hops(),packetId(),Packet::VERB_REMOTE_TRACE,0,Packet::VERB_NOP,false,0);
+	peer->received(tPtr,_path,hops(),packetId(),payloadLength(),Packet::VERB_REMOTE_TRACE,0,Packet::VERB_NOP,false,0);
 
 	return true;
 }

node/IncomingPacket.hpp

@@ -125,6 +125,8 @@ private:
 	// been authenticated, decrypted, decompressed, and classified.
 	bool _doERROR(const RuntimeEnvironment *RR,void *tPtr,const SharedPtr<Peer> &peer);
 	bool _doHELLO(const RuntimeEnvironment *RR,void *tPtr,const bool alreadyAuthenticated);
+	bool _doACK(const RuntimeEnvironment *RR,void *tPtr,const SharedPtr<Peer> &peer);
+	bool _doQOS_MEASUREMENT(const RuntimeEnvironment *RR,void *tPtr,const SharedPtr<Peer> &peer);
 	bool _doOK(const RuntimeEnvironment *RR,void *tPtr,const SharedPtr<Peer> &peer);
 	bool _doWHOIS(const RuntimeEnvironment *RR,void *tPtr,const SharedPtr<Peer> &peer);
 	bool _doRENDEZVOUS(const RuntimeEnvironment *RR,void *tPtr,const SharedPtr<Peer> &peer);

node/InetAddress.cpp

@@ -5,7 +5,7 @@
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
  * the Free Software Foundation, either version 3 of the License, or
- * (at your oion) any later version.
+ * (at your option) any later version.
  *
  * This program is distributed in the hope that it will be useful,
  * but WITHOUT ANY WARRANTY; without even the implied warranty of

node/MulticastGroup.hpp

@@ -68,7 +68,7 @@ public:
 	 * Derive the multicast group used for address resolution (ARP/NDP) for an IP
 	 *
 	 * @param ip IP address (port field is ignored)
-	 * @return Multicat group for ARP/NDP
+	 * @return Multicast group for ARP/NDP
 	 */
 	static inline MulticastGroup deriveMulticastGroupForAddressResolution(const InetAddress &ip)
 	{

node/Multicaster.cpp

@@ -190,7 +190,7 @@ void Multicaster::send(
 				for(unsigned int i=0;i<multicastReplicatorCount;++i) {
 					const SharedPtr<Peer> p(RR->topology->getPeerNoCache(multicastReplicators[i]));
 					if ((p)&&(p->isAlive(now))) {
-						const SharedPtr<Path> pp(p->getBestPath(now,false));
+						const SharedPtr<Path> pp(p->getAppropriatePath(now,false));
 						if ((pp)&&(pp->latency() < bestMulticastReplicatorLatency)) {
 							bestMulticastReplicatorLatency = pp->latency();
 							bestMulticastReplicatorPath = pp;

node/Network.cpp

@@ -29,6 +29,8 @@
 #include <stdlib.h>
 #include <math.h>
 
+#include "../include/ZeroTierDebug.h"
+
 #include "Constants.hpp"
 #include "../version.h"
 #include "Network.hpp"
@@ -106,7 +108,8 @@ static _doZtFilterResult _doZtFilter(
 	const unsigned int ruleCount,
 	Address &cc, // MUTABLE -- set to TEE destination if TEE action is taken or left alone otherwise
 	unsigned int &ccLength, // MUTABLE -- set to length of packet payload to TEE
-	bool &ccWatch) // MUTABLE -- set to true for WATCH target as opposed to normal TEE
+	bool &ccWatch, // MUTABLE -- set to true for WATCH target as opposed to normal TEE
+	uint8_t &qosBucket) // MUTABLE -- set to the value of the argument provided to PRIORITY
 {
 	// Set to true if we are a TEE/REDIRECT/WATCH target
 	bool superAccept = false;
@@ -124,6 +127,10 @@ static _doZtFilterResult _doZtFilter(
 		if ((unsigned int)rt <= (unsigned int)ZT_NETWORK_RULE_ACTION__MAX_ID) {
 			if (thisSetMatches) {
 				switch(rt) {
+					case ZT_NETWORK_RULE_ACTION_PRIORITY:
+						qosBucket = (rules[rn].v.qosBucket >= 0 || rules[rn].v.qosBucket <= 8) ? rules[rn].v.qosBucket : 4; // 4 = default bucket (no priority)
+						return DOZTFILTER_ACCEPT;
+
 					case ZT_NETWORK_RULE_ACTION_DROP:
 						return DOZTFILTER_DROP;
 
@@ -621,7 +628,8 @@ bool Network::filterOutgoingPacket(
 	const uint8_t *frameData,
 	const unsigned int frameLen,
 	const unsigned int etherType,
-	const unsigned int vlanId)
+	const unsigned int vlanId,
+	uint8_t &qosBucket)
 {
 	const int64_t now = RR->node->now();
 	Address ztFinalDest(ztDest);
@@ -636,7 +644,7 @@ bool Network::filterOutgoingPacket(
 
 	Membership *const membership = (ztDest) ? _memberships.get(ztDest) : (Membership *)0;
 
-	switch(_doZtFilter(RR,rrl,_config,membership,false,ztSource,ztFinalDest,macSource,macDest,frameData,frameLen,etherType,vlanId,_config.rules,_config.ruleCount,cc,ccLength,ccWatch)) {
+	switch(_doZtFilter(RR,rrl,_config,membership,false,ztSource,ztFinalDest,macSource,macDest,frameData,frameLen,etherType,vlanId,_config.rules,_config.ruleCount,cc,ccLength,ccWatch,qosBucket)) {
 
 		case DOZTFILTER_NO_MATCH: {
 			for(unsigned int c=0;c<_config.capabilityCount;++c) {
@@ -644,7 +652,7 @@ bool Network::filterOutgoingPacket(
 				Address cc2;
 				unsigned int ccLength2 = 0;
 				bool ccWatch2 = false;
-				switch (_doZtFilter(RR,crrl,_config,membership,false,ztSource,ztFinalDest,macSource,macDest,frameData,frameLen,etherType,vlanId,_config.capabilities[c].rules(),_config.capabilities[c].ruleCount(),cc2,ccLength2,ccWatch2)) {
+				switch (_doZtFilter(RR,crrl,_config,membership,false,ztSource,ztFinalDest,macSource,macDest,frameData,frameLen,etherType,vlanId,_config.capabilities[c].rules(),_config.capabilities[c].ruleCount(),cc2,ccLength2,ccWatch2,qosBucket)) {
 					case DOZTFILTER_NO_MATCH:
 					case DOZTFILTER_DROP: // explicit DROP in a capability just terminates its evaluation and is an anti-pattern
 						break;
@@ -759,11 +767,13 @@ int Network::filterIncomingPacket(
 	bool ccWatch = false;
 	const Capability *c = (Capability *)0;
 
+	uint8_t qosBucket = 255; // For incoming packets this is a dummy value
+
 	Mutex::Lock _l(_lock);
 
 	Membership &membership = _membership(sourcePeer->address());
 
-	switch (_doZtFilter(RR,rrl,_config,&membership,true,sourcePeer->address(),ztFinalDest,macSource,macDest,frameData,frameLen,etherType,vlanId,_config.rules,_config.ruleCount,cc,ccLength,ccWatch)) {
+	switch (_doZtFilter(RR,rrl,_config,&membership,true,sourcePeer->address(),ztFinalDest,macSource,macDest,frameData,frameLen,etherType,vlanId,_config.rules,_config.ruleCount,cc,ccLength,ccWatch,qosBucket)) {
 
 		case DOZTFILTER_NO_MATCH: {
 			Membership::CapabilityIterator mci(membership,_config);
@@ -772,7 +782,7 @@ int Network::filterIncomingPacket(
 				Address cc2;
 				unsigned int ccLength2 = 0;
 				bool ccWatch2 = false;
-				switch(_doZtFilter(RR,crrl,_config,&membership,true,sourcePeer->address(),ztFinalDest,macSource,macDest,frameData,frameLen,etherType,vlanId,c->rules(),c->ruleCount(),cc2,ccLength2,ccWatch2)) {
+				switch(_doZtFilter(RR,crrl,_config,&membership,true,sourcePeer->address(),ztFinalDest,macSource,macDest,frameData,frameLen,etherType,vlanId,c->rules(),c->ruleCount(),cc2,ccLength2,ccWatch2,qosBucket)) {
 					case DOZTFILTER_NO_MATCH:
 					case DOZTFILTER_DROP: // explicit DROP in a capability just terminates its evaluation and is an anti-pattern
 						break;

node/Network.hpp

@@ -132,7 +132,8 @@ public:
 		const uint8_t *frameData,
 		const unsigned int frameLen,
 		const unsigned int etherType,
-		const unsigned int vlanId);
+		const unsigned int vlanId,
+		uint8_t &qosBucket);
 
 	/**
 	 * Apply filters to an incoming packet
@@ -297,6 +298,13 @@ public:
 	 */
 	void learnBridgeRoute(const MAC &mac,const Address &addr);
 
+	/**
+	 * Whether QoS is in effect for this network
+	 */
+	bool QoSEnabled() {
+		return false;
+	}
+
 	/**
 	 * Learn a multicast group that is bridged to our tap device
 	 *
@@ -438,6 +446,6 @@ private:
 	AtomicCounter __refCount;
 };
 
-} // naemspace ZeroTier
+} // namespace ZeroTier
 
 #endif

node/NetworkConfig.hpp

@@ -562,7 +562,7 @@ public:
 	char name[ZT_MAX_NETWORK_SHORT_NAME_LENGTH + 1];
 
 	/**
-	 * Certficiate of membership (for private networks)
+	 * Certificate of membership (for private networks)
 	 */
 	CertificateOfMembership com;
 };

node/Node.cpp

@@ -234,7 +234,7 @@ public:
 			}
 
 			if ((!contacted)&&(_bestCurrentUpstream)) {
-				const SharedPtr<Path> up(_bestCurrentUpstream->getBestPath(_now,true));
+				const SharedPtr<Path> up(_bestCurrentUpstream->getAppropriatePath(_now,true));
 				if (up)
 					p->sendHELLO(_tPtr,up->localSocket(),up->address(),_now);
 			}
@@ -368,6 +368,7 @@ ZT_ResultCode Node::leave(uint64_t nwid,void **uptr,void *tptr)
 	{
 		Mutex::Lock _l(_networks_m);
 		SharedPtr<Network> *nw = _networks.get(nwid);
+		RR->sw->removeNetworkQoSControlBlock(nwid);
 		if (!nw)
 			return ZT_RESULT_OK;
 		if (uptr)
@@ -450,6 +451,7 @@ ZT_PeerList *Node::peers() const
 	for(std::vector< std::pair< Address,SharedPtr<Peer> > >::iterator pi(peers.begin());pi!=peers.end();++pi) {
 		ZT_Peer *p = &(pl->peers[pl->peerCount++]);
 		p->address = pi->second->address().toInt();
+		p->hadAggregateLink = 0;
 		if (pi->second->remoteVersionKnown()) {
 			p->versionMajor = pi->second->remoteVersionMajor();
 			p->versionMinor = pi->second->remoteVersionMinor();
@@ -465,7 +467,8 @@ ZT_PeerList *Node::peers() const
 		p->role = RR->topology->role(pi->second->identity().address());
 
 		std::vector< SharedPtr<Path> > paths(pi->second->paths(_now));
-		SharedPtr<Path> bestp(pi->second->getBestPath(_now,false));
+		SharedPtr<Path> bestp(pi->second->getAppropriatePath(_now,false));
+		p->hadAggregateLink |= pi->second->hasAggregateLink();
 		p->pathCount = 0;
 		for(std::vector< SharedPtr<Path> >::iterator path(paths.begin());path!=paths.end();++path) {
 			ZT_FAST_MEMCPY(&(p->paths[p->pathCount].address),&((*path)->address()),sizeof(struct sockaddr_storage));
@@ -474,6 +477,17 @@ ZT_PeerList *Node::peers() const
 			p->paths[p->pathCount].trustedPathId = RR->topology->getOutboundPathTrust((*path)->address());
 			p->paths[p->pathCount].expired = 0;
 			p->paths[p->pathCount].preferred = ((*path) == bestp) ? 1 : 0;
+			p->paths[p->pathCount].latency = (*path)->latency();
+			p->paths[p->pathCount].packetDelayVariance = (*path)->packetDelayVariance(); 
+			p->paths[p->pathCount].throughputDisturbCoeff = (*path)->throughputDisturbanceCoefficient();
+			p->paths[p->pathCount].packetErrorRatio = (*path)->packetErrorRatio();
+			p->paths[p->pathCount].packetLossRatio = (*path)->packetLossRatio();
+			p->paths[p->pathCount].stability = (*path)->lastComputedStability();
+			p->paths[p->pathCount].throughput = (*path)->meanThroughput();
+			p->paths[p->pathCount].maxThroughput = (*path)->maxLifetimeThroughput();
+			p->paths[p->pathCount].allocation = (float)(*path)->allocation() / (float)255;
+			p->paths[p->pathCount].ifname = (*path)->getName();
+
 			++p->pathCount;
 		}
 	}

node/Node.hpp

@@ -260,6 +260,9 @@ public:
 	inline const Address &remoteTraceTarget() const { return _remoteTraceTarget; }
 	inline Trace::Level remoteTraceLevel() const { return _remoteTraceLevel; }
 
+	inline void setMultipathMode(uint8_t mode) { _multipathMode = mode; }
+	inline uint8_t getMultipathMode() { return _multipathMode; }
+
 private:
 	RuntimeEnvironment _RR;
 	RuntimeEnvironment *RR;
@@ -284,6 +287,8 @@ private:
 	Address _remoteTraceTarget;
 	enum Trace::Level _remoteTraceLevel;
 
+	uint8_t _multipathMode;
+
 	volatile int64_t _now;
 	int64_t _lastPingCheck;
 	int64_t _lastHousekeepingRun;

node/OutboundMulticast.cpp

@@ -85,7 +85,8 @@ void OutboundMulticast::sendOnly(const RuntimeEnvironment *RR,void *tPtr,const A
 {
 	const SharedPtr<Network> nw(RR->node->network(_nwid));
 	const Address toAddr2(toAddr);
-	if ((nw)&&(nw->filterOutgoingPacket(tPtr,true,RR->identity.address(),toAddr2,_macSrc,_macDest,_frameData,_frameLen,_etherType,0))) {
+	uint8_t QoSBucket = 255; // Dummy value
+	if ((nw)&&(nw->filterOutgoingPacket(tPtr,true,RR->identity.address(),toAddr2,_macSrc,_macDest,_frameData,_frameLen,_etherType,0,QoSBucket))) {
 		_packet.newInitializationVector();
 		_packet.setDestination(toAddr2);
 		RR->node->expectReplyTo(_packet.packetId());

node/Packet.hpp

@@ -150,7 +150,7 @@
  *
  * In cryptography, a "break" means something different from what it means in
  * common discussion. If a cipher is 256 bits strong and someone finds a way
- * to reduce key search to 254 bits, this constitues a "break" in the academic
+ * to reduce key search to 254 bits, this constitutes a "break" in the academic
  * literature. 254 bits is still far beyond what can be leveraged to accomplish
  * a "break" as most people would understand it -- the actual decryption and
  * reading of traffic.
@@ -249,7 +249,7 @@
  */
 #define ZT_PROTO_MIN_FRAGMENT_LENGTH ZT_PACKET_FRAGMENT_IDX_PAYLOAD
 
-// Field incides for parsing verbs -------------------------------------------
+// Field indices for parsing verbs -------------------------------------------
 
 // Some verbs have variable-length fields. Those aren't fully defined here
 // yet-- instead they are parsed using relative indexes in IncomingPacket.
@@ -734,7 +734,7 @@ public:
 		 * Credentials can be for any number of networks.
 		 *
 		 * The use of a zero byte to terminate the COM section is for legacy
-		 * backward compatiblity. Newer fields are prefixed with a length.
+		 * backward compatibility. Newer fields are prefixed with a length.
 		 *
 		 * OK/ERROR are not generated.
 		 */
@@ -751,7 +751,7 @@ public:
 		 * This message requests network configuration from a node capable of
 		 * providing it.
 		 *
-		 * Respones to this are always whole configs intended for the recipient.
+		 * Responses to this are always whole configs intended for the recipient.
 		 * For patches and other updates a NETWORK_CONFIG is sent instead.
 		 *
 		 * It would be valid and correct as of 1.2.0 to use NETWORK_CONFIG always,
@@ -884,7 +884,7 @@ public:
 		 *   <[6] MAC address of multicast group>
 		 *   <[4] 32-bit ADI for multicast group>
 		 *   <[1] flags>
-		 *  [<[...] network certficate of membership (DEPRECATED)>]
+		 *  [<[...] network certificate of membership (DEPRECATED)>]
 		 *  [<[...] implicit gather results if flag 0x01 is set>]
 		 *
 		 * OK flags (same bits as request flags):
@@ -930,7 +930,53 @@ public:
 		 */
 		VERB_PUSH_DIRECT_PATHS = 0x10,
 
-		// 0x11, 0x12 -- deprecated
+		// 0x11 -- deprecated
+
+		/**
+		 * An acknowledgment of receipt of a series of recent packets from another
+		 * peer. This is used to calculate relative throughput values and to detect
+		 * packet loss. Only VERB_FRAME and VERB_EXT_FRAME packets are counted.
+		 *
+		 * ACK response format:
+		 *  <[4] 32-bit number of bytes received since last ACK>
+		 *
+		 * Upon receipt of this packet, the local peer will verify that the correct
+		 * number of bytes were received by the remote peer. If these values do
+		 * not agree that could be an indicator of packet loss.
+		 *
+		 * Additionally, the local peer knows the interval of time that has
+		 * elapsed since the last received ACK. With this information it can compute
+		 * a rough estimate of the current throughput.
+		 *
+		 * This is sent at a maximum rate of once per every ZT_PATH_ACK_INTERVAL
+		 */
+		VERB_ACK = 0x12,
+
+		/**
+		 * A packet containing timing measurements useful for estimating path quality.
+		 * Composed of a list of <packet ID:internal sojourn time> pairs for an
+		 * arbitrary set of recent packets. This is used to sample for latency and
+		 * packet delay variance (PDV, "jitter").
+		 *
+		 * QoS record format:
+		 *
+		 *  <[8] 64-bit packet ID of previously-received packet>
+		 *  <[1] 8-bit packet sojourn time>
+		 *  <...repeat until end of max 1400 byte packet...>
+		 *
+		 * The number of possible records per QoS packet is: (1400 * 8) / 72 = 155
+		 * This packet should be sent very rarely (every few seconds) as it can be
+		 * somewhat large if the connection is saturated. Future versions might use
+		 * a bloom table to probabilistically determine these values in a vastly
+		 * more space-efficient manner.
+		 *
+		 * Note: The 'internal packet sojourn time' is a slight misnomer as it is a
+		 * measure of the amount of time between when a packet was received and the
+		 * egress time of its tracking QoS packet.
+		 *
+		 * This is sent at a maximum rate of once per every ZT_PATH_QOS_INTERVAL
+		 */
+		VERB_QOS_MEASUREMENT = 0x13,
 
 		/**
 		 * A message with arbitrary user-definable content:
@@ -954,7 +1000,7 @@ public:
 		 *
 		 * This message contains a remote trace event. Remote trace events can
 		 * be sent to observers configured at the network level for those that
-		 * pertain directly to actiity on a network, or to global observers if
+		 * pertain directly to activity on a network, or to global observers if
 		 * locally configured.
 		 *
 		 * The instance ID is a random 64-bit value generated by each ZeroTier
@@ -1251,7 +1297,7 @@ public:
 	 * Encrypt/decrypt a separately armored portion of a packet
 	 *
 	 * This is currently only used to mask portions of HELLO as an extra
-	 * security precation since most of that message is sent in the clear.
+	 * security precaution since most of that message is sent in the clear.
 	 *
 	 * This must NEVER be used more than once in the same packet, as doing
 	 * so will result in re-use of the same key stream.

node/Path.hpp

@@ -39,6 +39,10 @@
 #include "SharedPtr.hpp"
 #include "AtomicCounter.hpp"
 #include "Utils.hpp"
+#include "RingBuffer.hpp"
+#include "Packet.hpp"
+
+#include "../osdep/Phy.hpp"
 
 /**
  * Maximum return value of preferenceRank()
@@ -55,6 +59,7 @@ class RuntimeEnvironment;
 class Path
 {
 	friend class SharedPtr<Path>;
+	Phy<Path *> *_phy;
 
 public:
 	/**
@@ -93,22 +98,77 @@ public:
 		_lastOut(0),
 		_lastIn(0),
 		_lastTrustEstablishedPacketReceived(0),
+		_lastPathQualityComputeTime(0),
 		_localSocket(-1),
 		_latency(0xffff),
 		_addr(),
-		_ipScope(InetAddress::IP_SCOPE_NONE)
+		_ipScope(InetAddress::IP_SCOPE_NONE),
+		_lastAck(0),
+		_lastThroughputEstimation(0),
+		_lastQoSMeasurement(0),
+		_lastQoSRecordPurge(0),
+		_unackedBytes(0),
+		_expectingAckAsOf(0),
+		_packetsReceivedSinceLastAck(0),
+		_packetsReceivedSinceLastQoS(0),
+		_maxLifetimeThroughput(0),
+		_lastComputedMeanThroughput(0),
+		_bytesAckedSinceLastThroughputEstimation(0),
+		_lastComputedMeanLatency(0.0),
+		_lastComputedPacketDelayVariance(0.0),
+		_lastComputedPacketErrorRatio(0.0),
+		_lastComputedPacketLossRatio(0),
+		_lastComputedStability(0.0),
+		_lastComputedRelativeQuality(0),
+		_lastComputedThroughputDistCoeff(0.0),
+		_lastAllocation(0)
 	{
+		prepareBuffers();
 	}
 
 	Path(const int64_t localSocket,const InetAddress &addr) :
 		_lastOut(0),
 		_lastIn(0),
 		_lastTrustEstablishedPacketReceived(0),
+		_lastPathQualityComputeTime(0),
 		_localSocket(localSocket),
 		_latency(0xffff),
 		_addr(addr),
-		_ipScope(addr.ipScope())
+		_ipScope(addr.ipScope()),
+		_lastAck(0),
+		_lastThroughputEstimation(0),
+		_lastQoSMeasurement(0),
+		_lastQoSRecordPurge(0),
+		_unackedBytes(0),
+		_expectingAckAsOf(0),
+		_packetsReceivedSinceLastAck(0),
+		_packetsReceivedSinceLastQoS(0),
+		_maxLifetimeThroughput(0),
+		_lastComputedMeanThroughput(0),
+		_bytesAckedSinceLastThroughputEstimation(0),
+		_lastComputedMeanLatency(0.0),
+		_lastComputedPacketDelayVariance(0.0),
+		_lastComputedPacketErrorRatio(0.0),
+		_lastComputedPacketLossRatio(0),
+		_lastComputedStability(0.0),
+		_lastComputedRelativeQuality(0),
+		_lastComputedThroughputDistCoeff(0.0),
+		_lastAllocation(0)
 	{
+		prepareBuffers();
+		_phy->getIfName((PhySocket *)((uintptr_t)_localSocket), _ifname, 16);
+	}
+
+	~Path()
+	{
+		delete _throughputSamples;
+		delete _latencySamples;
+		delete _packetValiditySamples;
+		delete _throughputDisturbanceSamples;
+		_throughputSamples = NULL;
+		_latencySamples = NULL;
+		_packetValiditySamples = NULL;
+		_throughputDisturbanceSamples = NULL;
 	}
 
 	/**
@@ -147,12 +207,16 @@ public:
 	 *
 	 * @param l Measured latency
 	 */
-	inline void updateLatency(const unsigned int l)
+	inline void updateLatency(const unsigned int l, int64_t now)
 	{
 		unsigned int pl = _latency;
-		if (pl < 0xffff)
+		if (pl < 0xffff) {
 			_latency = (pl + l) / 2;
-		else _latency = l;
+		}
+		else {
+			_latency = l;
+		}
+		_latencySamples->push(l);
 	}
 
 	/**
@@ -240,6 +304,324 @@ public:
 		return (((age < (ZT_PATH_HEARTBEAT_PERIOD + 5000)) ? l : (l + 0xffff + age)) * (long)((ZT_INETADDRESS_MAX_SCOPE - _ipScope) + 1));
 	}
 
+	/**
+	 * Record statistics on outgoing packets. Used later to estimate QoS metrics.
+	 *
+	 * @param now Current time
+	 * @param packetId ID of packet
+	 * @param payloadLength Length of payload
+	 * @param verb Packet verb
+	 */
+	inline void recordOutgoingPacket(int64_t now, int64_t packetId, uint16_t payloadLength, Packet::Verb verb)
+	{
+		Mutex::Lock _l(_statistics_m);
+		if (verb != Packet::VERB_ACK && verb != Packet::VERB_QOS_MEASUREMENT) {
+			if ((packetId & (ZT_PATH_QOS_ACK_PROTOCOL_DIVISOR - 1)) == 0) {
+				_unackedBytes += payloadLength;
+				// Take note that we're expecting a VERB_ACK on this path as of a specific time
+				_expectingAckAsOf = ackAge(now) > ZT_PATH_ACK_INTERVAL ? _expectingAckAsOf : now;
+				if (_outQoSRecords.size() < ZT_PATH_MAX_OUTSTANDING_QOS_RECORDS) {
+					_outQoSRecords[packetId] = now;
+				}
+			}
+		}
+	}
+
+	/**
+	 * Record statistics on incoming packets. Used later to estimate QoS metrics.
+	 *
+	 * @param now Current time
+	 * @param packetId ID of packet
+	 * @param payloadLength Length of payload
+	 * @param verb Packet verb
+	 */
+	inline void recordIncomingPacket(int64_t now, int64_t packetId, uint16_t payloadLength, Packet::Verb verb)
+	{
+		Mutex::Lock _l(_statistics_m);
+		if (verb != Packet::VERB_ACK && verb != Packet::VERB_QOS_MEASUREMENT) {
+			if ((packetId & (ZT_PATH_QOS_ACK_PROTOCOL_DIVISOR - 1)) == 0) {
+				_inACKRecords[packetId] = payloadLength;
+				_packetsReceivedSinceLastAck++;
+				_inQoSRecords[packetId] = now;
+				_packetsReceivedSinceLastQoS++;
+			}
+			_packetValiditySamples->push(true);
+		}
+	}
+
+	/**
+	 * Record that we've received a VERB_ACK on this path, also compute throughput if required.
+	 *
+	 * @param now Current time
+	 * @param ackedBytes Number of bytes acknowledged by other peer
+	 */
+	inline void receivedAck(int64_t now, int32_t ackedBytes)
+	{
+		_expectingAckAsOf = 0;
+		_unackedBytes = (ackedBytes > _unackedBytes) ? 0 : _unackedBytes - ackedBytes;
+		int64_t timeSinceThroughputEstimate = (now - _lastThroughputEstimation);
+		if (timeSinceThroughputEstimate >= ZT_PATH_THROUGHPUT_MEASUREMENT_INTERVAL) {
+			uint64_t throughput = (float)(_bytesAckedSinceLastThroughputEstimation * 8) / ((float)timeSinceThroughputEstimate / (float)1000);
+			_throughputSamples->push(throughput);
+			_maxLifetimeThroughput = throughput > _maxLifetimeThroughput ? throughput : _maxLifetimeThroughput;
+			_lastThroughputEstimation = now;
+			_bytesAckedSinceLastThroughputEstimation = 0;
+		} else {
+			_bytesAckedSinceLastThroughputEstimation += ackedBytes;
+		}
+	}
+
+	/**
+	 * @return Number of bytes this peer is responsible for ACKing since last ACK
+	 */
+	inline int32_t bytesToAck()
+	{
+		Mutex::Lock _l(_statistics_m);
+		int32_t bytesToAck = 0;
+		std::map<uint64_t,uint16_t>::iterator it = _inACKRecords.begin();
+		while (it != _inACKRecords.end()) {
+			bytesToAck += it->second;
+			it++;
+		}
+		return bytesToAck;
+	}
+
+	/**
+	 * @return Number of bytes thus far sent that have not been acknowledged by the remote peer
+	 */
+	inline int64_t unackedSentBytes()
+	{
+		return _unackedBytes;
+	}
+
+	/**
+	 * Account for the fact that an ACK was just sent. Reset counters, timers, and clear statistics buffers
+	 *
+	 * @param Current time
+	 */
+	inline void sentAck(int64_t now)
+	{
+		Mutex::Lock _l(_statistics_m);
+		_inACKRecords.clear();
+		_packetsReceivedSinceLastAck = 0;
+		_lastAck = now;
+	}
+
+	/**
+	 * Receive QoS data, match with recorded egress times from this peer, compute latency
+	 * estimates.
+	 *
+	 * @param now Current time
+	 * @param count Number of records
+	 * @param rx_id table of packet IDs
+	 * @param rx_ts table of holding times
+	 */
+	inline void receivedQoS(int64_t now, int count, uint64_t *rx_id, uint16_t *rx_ts)
+	{
+		Mutex::Lock _l(_statistics_m);
+		// Look up egress times and compute latency values for each record
+		std::map<uint64_t,uint64_t>::iterator it;
+		for (int j=0; j<count; j++) {
+			it = _outQoSRecords.find(rx_id[j]);
+			if (it != _outQoSRecords.end()) {
+				uint16_t rtt = (uint16_t)(now - it->second);
+				uint16_t rtt_compensated = rtt - rx_ts[j];
+				float latency = rtt_compensated / 2.0;
+				updateLatency(latency, now);
+				_outQoSRecords.erase(it);
+			}
+		}
+	}
+
+	/**
+	 * Generate the contents of a VERB_QOS_MEASUREMENT packet.
+	 *
+	 * @param now Current time
+	 * @param qosBuffer destination buffer
+	 * @return Size of payload
+	 */
+	inline int32_t generateQoSPacket(int64_t now, char *qosBuffer)
+	{
+		Mutex::Lock _l(_statistics_m);
+		int32_t len = 0;
+		std::map<uint64_t,uint64_t>::iterator it = _inQoSRecords.begin();
+		int i=0;
+		while (i<_packetsReceivedSinceLastQoS && it != _inQoSRecords.end()) {
+			uint64_t id = it->first;
+			memcpy(qosBuffer, &id, sizeof(uint64_t));
+			qosBuffer+=sizeof(uint64_t);
+			uint16_t holdingTime = (now - it->second);
+			memcpy(qosBuffer, &holdingTime, sizeof(uint16_t));
+			qosBuffer+=sizeof(uint16_t);
+			len+=sizeof(uint64_t)+sizeof(uint16_t);
+			_inQoSRecords.erase(it++);
+			i++;
+		}
+		return len;
+	}
+
+	/**
+	 * Account for the fact that a VERB_QOS_MEASUREMENT was just sent. Reset timers.
+	 *
+	 * @param Current time
+	 */
+	inline void sentQoS(int64_t now) {
+		_packetsReceivedSinceLastQoS = 0;
+		_lastQoSMeasurement = now;
+	}
+
+	/**
+	 * @param now Current time
+	 * @return Whether an ACK (VERB_ACK) packet needs to be emitted at this time
+	 */
+	inline bool needsToSendAck(int64_t now) {
+		return ((now - _lastAck) >= ZT_PATH_ACK_INTERVAL ||
+			(_packetsReceivedSinceLastAck == ZT_PATH_QOS_TABLE_SIZE)) && _packetsReceivedSinceLastAck;
+	}
+
+	/**
+	 * @param now Current time
+	 * @return Whether a QoS (VERB_QOS_MEASUREMENT) packet needs to be emitted at this time
+	 */
+	inline bool needsToSendQoS(int64_t now) {
+		return ((_packetsReceivedSinceLastQoS >= ZT_PATH_QOS_TABLE_SIZE) ||
+			((now - _lastQoSMeasurement) > ZT_PATH_QOS_INTERVAL)) && _packetsReceivedSinceLastQoS;
+	}
+
+	/**
+	 * How much time has elapsed since we've been expecting a VERB_ACK on this path. This value
+	 * is used to determine a more relevant path "age". This lets us penalize paths which are no
+	 * longer ACKing, but not those that simple aren't being used to carry traffic at the
+	 * current time.
+	 */
+	inline int64_t ackAge(int64_t now) { return _expectingAckAsOf ? now - _expectingAckAsOf : 0; }
+
+	/**
+	 * The maximum observed throughput (in bits/s) for this path
+	 */
+	inline uint64_t maxLifetimeThroughput() { return _maxLifetimeThroughput; }
+
+	/**
+	 * @return The mean throughput (in bits/s) of this link
+	 */
+	inline uint64_t meanThroughput() { return _lastComputedMeanThroughput; }
+
+	/**
+	 * Assign a new relative quality value for this path in the aggregate link
+	 *
+	 * @param rq Quality of this path in comparison to other paths available to this peer
+	 */
+	inline void updateRelativeQuality(float rq) { _lastComputedRelativeQuality = rq; }
+
+	/**
+	 * @return Quality of this path compared to others in the aggregate link
+	 */
+	inline float relativeQuality() { return _lastComputedRelativeQuality; }
+
+	/**
+	 * Assign a new allocation value for this path in the aggregate link
+	 *
+	 * @param allocation Percentage of traffic to be sent over this path to a peer
+	 */
+	inline void updateComponentAllocationOfAggregateLink(unsigned char allocation) { _lastAllocation = allocation; }
+
+	/**
+	 * @return Percentage of traffic allocated to this path in the aggregate link
+	 */
+	inline unsigned char allocation() { return _lastAllocation; }
+
+	/**
+	 * @return Stability estimates can become expensive to compute, we cache the most recent result.
+	 */
+	inline float lastComputedStability() { return _lastComputedStability; }
+
+	/**
+	 * @return A pointer to a cached copy of the human-readable name of the interface this Path's localSocket is bound to
+	 */
+	inline char *getName() { return _ifname; }
+
+	/**
+	 * @return Packet delay variance
+	 */
+	inline float packetDelayVariance() { return _lastComputedPacketDelayVariance; }
+
+	/**
+	 * @return Previously-computed mean latency
+	 */
+	inline float meanLatency() { return _lastComputedMeanLatency; }
+
+	/**
+	 * @return Packet loss rate (PLR)
+	 */
+	inline float packetLossRatio() { return _lastComputedPacketLossRatio; }
+
+	/**
+	 * @return Packet error ratio (PER)
+	 */
+	inline float packetErrorRatio() { return _lastComputedPacketErrorRatio; }
+
+	/**
+	 * Record an invalid incoming packet. This packet failed MAC/compression/cipher checks and will now
+	 * contribute to a Packet Error Ratio (PER).
+	 */
+	inline void recordInvalidPacket() { _packetValiditySamples->push(false); }
+
+	/**
+	 * @return A pointer to a cached copy of the address string for this Path (For debugging only)
+	 */
+	inline char *getAddressString() { return _addrString; }
+
+	/**
+	 * @return The current throughput disturbance coefficient
+	 */
+	inline float throughputDisturbanceCoefficient() { return _lastComputedThroughputDistCoeff; }
+
+	/**
+	 * Compute and cache stability and performance metrics. The resultant stability coefficient is a measure of how "well behaved"
+	 * this path is. This figure is substantially different from (but required for the estimation of the path's overall "quality".
+	 *
+	 * @param now Current time
+	 */
+	inline void processBackgroundPathMeasurements(int64_t now) {
+		if (now - _lastPathQualityComputeTime > ZT_PATH_QUALITY_COMPUTE_INTERVAL) {
+			Mutex::Lock _l(_statistics_m);
+			_lastPathQualityComputeTime = now;
+			address().toString(_addrString);
+			_lastComputedMeanLatency = _latencySamples->mean();
+			_lastComputedPacketDelayVariance = _latencySamples->stddev(); // Similar to "jitter" (SEE: RFC 3393, RFC 4689)
+			_lastComputedMeanThroughput = (uint64_t)_throughputSamples->mean();
+			// If no packet validity samples, assume PER==0
+			_lastComputedPacketErrorRatio = 1 - (_packetValiditySamples->count() ? _packetValiditySamples->mean() : 1);
+			// Compute path stability
+			// Normalize measurements with wildly different ranges into a reasonable range
+			float normalized_pdv = Utils::normalize(_lastComputedPacketDelayVariance, 0, ZT_PATH_MAX_PDV, 0, 10);
+			float normalized_la = Utils::normalize(_lastComputedMeanLatency, 0, ZT_PATH_MAX_MEAN_LATENCY, 0, 10);
+			float throughput_cv = _throughputSamples->mean() > 0 ? _throughputSamples->stddev() / _throughputSamples->mean() : 1;
+			// Form an exponential cutoff and apply contribution weights
+			float pdv_contrib = exp((-1)*normalized_pdv) * ZT_PATH_CONTRIB_PDV;
+			float latency_contrib = exp((-1)*normalized_la) * ZT_PATH_CONTRIB_LATENCY;
+			// Throughput Disturbance Coefficient
+			float throughput_disturbance_contrib = exp((-1)*throughput_cv) * ZT_PATH_CONTRIB_THROUGHPUT_DISTURBANCE;
+			_throughputDisturbanceSamples->push(throughput_cv);
+			_lastComputedThroughputDistCoeff = _throughputDisturbanceSamples->mean();
+			// Obey user-defined ignored contributions
+			pdv_contrib = ZT_PATH_CONTRIB_PDV > 0.0 ? pdv_contrib : 1;
+			latency_contrib = ZT_PATH_CONTRIB_LATENCY > 0.0 ? latency_contrib : 1;
+			throughput_disturbance_contrib = ZT_PATH_CONTRIB_THROUGHPUT_DISTURBANCE > 0.0 ? throughput_disturbance_contrib : 1;
+			// Stability
+			_lastComputedStability = pdv_contrib + latency_contrib + throughput_disturbance_contrib;
+			_lastComputedStability *= 1 - _lastComputedPacketErrorRatio;
+			// Prevent QoS records from sticking around for too long
+			std::map<uint64_t,uint64_t>::iterator it = _outQoSRecords.begin();
+			while (it != _outQoSRecords.end()) {
+				// Time since egress of tracked packet
+				if ((now - it->second) >= ZT_PATH_QOS_TIMEOUT) {
+					_outQoSRecords.erase(it++);
+				} else { it++; }
+			}
+		}
+	}
+
 	/**
 	 * @return True if this path is alive (receiving heartbeats)
 	 */
@@ -265,15 +647,71 @@ public:
 	 */
 	inline int64_t lastTrustEstablishedPacketReceived() const { return _lastTrustEstablishedPacketReceived; }
 
+	/**
+	 * Initialize statistical buffers
+	 */
+	inline void prepareBuffers() {
+		_throughputSamples = new RingBuffer<uint64_t>(ZT_PATH_QUALITY_METRIC_WIN_SZ);
+		_latencySamples = new RingBuffer<uint32_t>(ZT_PATH_QUALITY_METRIC_WIN_SZ);
+		_packetValiditySamples = new RingBuffer<bool>(ZT_PATH_QUALITY_METRIC_WIN_SZ);
+		_throughputDisturbanceSamples = new RingBuffer<float>(ZT_PATH_QUALITY_METRIC_WIN_SZ);
+		memset(_ifname, 0, 16);
+		memset(_addrString, 0, sizeof(_addrString));
+	}
+
 private:
+	Mutex _statistics_m;
+
 	volatile int64_t _lastOut;
 	volatile int64_t _lastIn;
 	volatile int64_t _lastTrustEstablishedPacketReceived;
+	volatile int64_t _lastPathQualityComputeTime;
 	int64_t _localSocket;
 	volatile unsigned int _latency;
 	InetAddress _addr;
 	InetAddress::IpScope _ipScope; // memoize this since it's a computed value checked often
 	AtomicCounter __refCount;
+
+	std::map<uint64_t, uint64_t> _outQoSRecords; // id:egress_time
+	std::map<uint64_t, uint64_t> _inQoSRecords; // id:now
+	std::map<uint64_t, uint16_t> _inACKRecords; // id:len
+
+	int64_t _lastAck;
+	int64_t _lastThroughputEstimation;
+	int64_t _lastQoSMeasurement;
+	int64_t _lastQoSRecordPurge;
+
+	int64_t _unackedBytes;
+	int64_t _expectingAckAsOf;
+	int16_t _packetsReceivedSinceLastAck;
+	int16_t _packetsReceivedSinceLastQoS;
+
+	uint64_t _maxLifetimeThroughput;
+	uint64_t _lastComputedMeanThroughput;
+	uint64_t _bytesAckedSinceLastThroughputEstimation;
+
+	float _lastComputedMeanLatency;
+	float _lastComputedPacketDelayVariance;
+
+	float _lastComputedPacketErrorRatio;
+	float _lastComputedPacketLossRatio;
+
+	// cached estimates
+	float _lastComputedStability;
+	float _lastComputedRelativeQuality;
+	float _lastComputedThroughputDistCoeff;
+	unsigned char _lastAllocation;
+
+
+
+	// cached human-readable strings for tracing purposes
+	char _ifname[16];
+	char _addrString[256];
+
+	RingBuffer<uint64_t> *_throughputSamples;
+	RingBuffer<uint32_t> *_latencySamples;
+	RingBuffer<bool> *_packetValiditySamples;
+	RingBuffer<float> *_throughputDisturbanceSamples;
 };
 
 } // namespace ZeroTier

node/Peer.cpp

@@ -25,7 +25,6 @@
  */
 
 #include "../version.h"
-
 #include "Constants.hpp"
 #include "Peer.hpp"
 #include "Node.hpp"
@@ -35,6 +34,8 @@
 #include "Packet.hpp"
 #include "Trace.hpp"
 #include "InetAddress.hpp"
+#include "RingBuffer.hpp"
+#include "Utils.hpp"
 
 namespace ZeroTier {
 
@@ -53,16 +54,31 @@ Peer::Peer(const RuntimeEnvironment *renv,const Identity &myIdentity,const Ident
 	_lastCredentialsReceived(0),
 	_lastTrustEstablishedPacketReceived(0),
 	_lastSentFullHello(0),
+	_lastACKWindowReset(0),
+	_lastQoSWindowReset(0),
+	_lastMultipathCompatibilityCheck(0),
+	_freeRandomByte(0),
+	_uniqueAlivePathCount(0),
+	_localMultipathSupported(false),
+	_remoteMultipathSupported(false),
+	_canUseMultipath(false),
 	_vProto(0),
 	_vMajor(0),
 	_vMinor(0),
 	_vRevision(0),
 	_id(peerIdentity),
 	_directPathPushCutoffCount(0),
-	_credentialsCutoffCount(0)
+	_credentialsCutoffCount(0),
+	_linkIsBalanced(false),
+	_linkIsRedundant(false),
+	_remotePeerMultipathEnabled(false),
+	_lastAggregateStatsReport(0),
+	_lastAggregateAllocation(0)
 {
+	Utils::getSecureRandom(&_freeRandomByte, 1);
 	if (!myIdentity.agree(peerIdentity,_key,ZT_PEER_SECRET_KEY_LENGTH))
 		throw ZT_EXCEPTION_INVALID_ARGUMENT;
+	_pathChoiceHist = new RingBuffer<int>(ZT_MULTIPATH_PROPORTION_WIN_SZ);
 }
 
 void Peer::received(
@@ -70,6 +86,7 @@ void Peer::received(
 	const SharedPtr<Path> &path,
 	const unsigned int hops,
 	const uint64_t packetId,
+	const unsigned int payloadLength,
 	const Packet::Verb verb,
 	const uint64_t inRePacketId,
 	const Packet::Verb inReVerb,
@@ -95,9 +112,25 @@ void Peer::received(
 		path->trustedPacketReceived(now);
 	}
 
+	{
+		Mutex::Lock _l(_paths_m);
+
+		recordIncomingPacket(tPtr, path, packetId, payloadLength, verb, now);
+
+		if (_canUseMultipath) {
+			if (path->needsToSendQoS(now)) {
+				sendQOS_MEASUREMENT(tPtr, path, path->localSocket(), path->address(), now);
+			}
+			for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+				if (_paths[i].p) {
+					_paths[i].p->processBackgroundPathMeasurements(now);
+				}
+			}
+		}
+	}
+
 	if (hops == 0) {
 		// If this is a direct packet (no hops), update existing paths or learn new ones
-
 		bool havePath = false;
 		{
 			Mutex::Lock _l(_paths_m);
@@ -116,20 +149,26 @@ void Peer::received(
 		if ((!havePath)&&(RR->node->shouldUsePathForZeroTierTraffic(tPtr,_id.address(),path->localSocket(),path->address()))) {
 			Mutex::Lock _l(_paths_m);
 
-			// Paths are redunant if they duplicate an alive path to the same IP or
+			// Paths are redundant if they duplicate an alive path to the same IP or
 			// with the same local socket and address family.
 			bool redundant = false;
+			unsigned int replacePath = ZT_MAX_PEER_NETWORK_PATHS;
 			for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
 				if (_paths[i].p) {
 					if ( (_paths[i].p->alive(now)) && ( ((_paths[i].p->localSocket() == path->localSocket())&&(_paths[i].p->address().ss_family == path->address().ss_family)) || (_paths[i].p->address().ipsEqual2(path->address())) ) ) {
 						redundant = true;
 						break;
 					}
+					// If the path is the same address and port, simply assume this is a replacement
+					if ( (_paths[i].p->address().ipsEqual2(path->address()) && (_paths[i].p->address().port() == path->address().port()))) {
+						replacePath = i;
+						break;
+					}
 				} else break;
 			}
-
-			if (!redundant) {
-				unsigned int replacePath = ZT_MAX_PEER_NETWORK_PATHS;
+			// If the path isn't a duplicate of the same localSocket AND we haven't already determined a replacePath,
+			// then find the worst path and replace it.
+			if (!redundant && replacePath == ZT_MAX_PEER_NETWORK_PATHS) {
 				int replacePathQuality = 0;
 				for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
 					if (_paths[i].p) {
@@ -143,7 +182,7 @@ void Peer::received(
 						break;
 					}
 				}
-
+			}
 			if (replacePath != ZT_MAX_PEER_NETWORK_PATHS) {
 				if (verb == Packet::VERB_OK) {
 					RR->t->peerLearnedNewPath(tPtr,networkId,*this,path,packetId);
@@ -155,7 +194,6 @@ void Peer::received(
 				}
 			}
 		}
-		}
 
 		if (attemptToContact) {
 			attemptToContactAt(tPtr,path->localSocket(),path->address(),now,true);
@@ -232,11 +270,137 @@ void Peer::received(
 	}
 }
 
-SharedPtr<Path> Peer::getBestPath(int64_t now,bool includeExpired) const
+void Peer::recordOutgoingPacket(const SharedPtr<Path> &path, const uint64_t packetId,
+	uint16_t payloadLength, const Packet::Verb verb, int64_t now)
+{
+	// Grab second byte from packetId to use as a source of entropy in the next path selection
+	_freeRandomByte = (packetId & 0xFF00) >> 8;
+	if (_canUseMultipath) {
+		path->recordOutgoingPacket(now, packetId, payloadLength, verb);
+	}
+}
+
+void Peer::recordIncomingPacket(void *tPtr, const SharedPtr<Path> &path, const uint64_t packetId,
+	uint16_t payloadLength, const Packet::Verb verb, int64_t now)
+{
+	if (_canUseMultipath) {
+		if (path->needsToSendAck(now)) {
+			sendACK(tPtr, path, path->localSocket(), path->address(), now);
+		}
+		path->recordIncomingPacket(now, packetId, payloadLength, verb);
+	}
+}
+
+void Peer::computeAggregateProportionalAllocation(int64_t now)
+{
+	float maxStability = 0;
+	float totalRelativeQuality = 0;
+	float maxThroughput = 1;
+	float maxScope = 0;
+	float relStability[ZT_MAX_PEER_NETWORK_PATHS];
+	float relThroughput[ZT_MAX_PEER_NETWORK_PATHS];
+	memset(&relStability, 0, sizeof(relStability));
+	memset(&relThroughput, 0, sizeof(relThroughput));
+	// Survey all paths
+	for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+		if (_paths[i].p) {
+			relStability[i] = _paths[i].p->lastComputedStability();
+			relThroughput[i] = _paths[i].p->maxLifetimeThroughput();
+			maxStability = relStability[i] > maxStability ? relStability[i] : maxStability;
+			maxThroughput = relThroughput[i] > maxThroughput ? relThroughput[i] : maxThroughput;
+			maxScope = _paths[i].p->ipScope() > maxScope ? _paths[i].p->ipScope() : maxScope;
+		}
+	}
+	// Convert to relative values
+	for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+		if (_paths[i].p) {
+			relStability[i] /= maxStability ? maxStability : 1;
+			relThroughput[i] /= maxThroughput ? maxThroughput : 1;
+			float normalized_ma = Utils::normalize(_paths[i].p->ackAge(now), 0, ZT_PATH_MAX_AGE, 0, 10);
+			float age_contrib = exp((-1)*normalized_ma);
+			float relScope = ((float)(_paths[i].p->ipScope()+1) / (maxScope + 1));
+			float relQuality =
+				(relStability[i] * ZT_PATH_CONTRIB_STABILITY)
+				+ (fmax(1, relThroughput[i]) * ZT_PATH_CONTRIB_THROUGHPUT)
+				+ relScope * ZT_PATH_CONTRIB_SCOPE;
+			relQuality *= age_contrib;
+			// Arbitrary cutoffs
+			relQuality = relQuality > (1.00 / 100.0) ? relQuality : 0.0;
+			relQuality = relQuality < (99.0 / 100.0) ? relQuality : 1.0;
+			totalRelativeQuality += relQuality;
+			_paths[i].p->updateRelativeQuality(relQuality);
+		}
+	}
+	// Convert set of relative performances into an allocation set
+	for(uint16_t i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+		if (_paths[i].p) {
+			_paths[i].p->updateComponentAllocationOfAggregateLink((_paths[i].p->relativeQuality() / totalRelativeQuality) * 255);
+		}
+	}
+}
+
+int Peer::computeAggregateLinkPacketDelayVariance()
+{
+	float pdv = 0.0;
+	for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+		if (_paths[i].p) {
+			pdv += _paths[i].p->relativeQuality() * _paths[i].p->packetDelayVariance();
+		}
+	}
+	return pdv;
+}
+
+int Peer::computeAggregateLinkMeanLatency()
+{
+	int ml = 0;
+	int pathCount = 0;
+	for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+		if (_paths[i].p) {
+			pathCount++;
+			ml += _paths[i].p->relativeQuality() * _paths[i].p->meanLatency();
+		}
+	}
+	return ml / pathCount;
+}
+
+int Peer::aggregateLinkPhysicalPathCount()
+{
+	std::map<std::string, bool> ifnamemap;
+	int pathCount = 0;
+	int64_t now = RR->node->now();
+	for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+		if (_paths[i].p && _paths[i].p->alive(now)) {
+			if (!ifnamemap[_paths[i].p->getName()]) {
+				ifnamemap[_paths[i].p->getName()] = true;
+				pathCount++;
+			}
+		}
+	}
+	return pathCount;
+}
+
+int Peer::aggregateLinkLogicalPathCount()
+{
+	int pathCount = 0;
+	int64_t now = RR->node->now();
+	for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+		if (_paths[i].p && _paths[i].p->alive(now)) {
+			pathCount++;
+		}
+	}
+	return pathCount;
+}
+
+SharedPtr<Path> Peer::getAppropriatePath(int64_t now, bool includeExpired)
 {
 	Mutex::Lock _l(_paths_m);
-
 	unsigned int bestPath = ZT_MAX_PEER_NETWORK_PATHS;
+
+	/**
+	 * Send traffic across the highest quality path only. This algorithm will still
+	 * use the old path quality metric from protocol version 9.
+	 */
+	if (!_canUseMultipath) {
 		long bestPathQuality = 2147483647;
 		for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
 			if (_paths[i].p) {
@@ -249,12 +413,122 @@ SharedPtr<Path> Peer::getBestPath(int64_t now,bool includeExpired) const
 				}
 			} else break;
 		}
-
-	if (bestPath != ZT_MAX_PEER_NETWORK_PATHS)
+		if (bestPath != ZT_MAX_PEER_NETWORK_PATHS) {
 			return _paths[bestPath].p;
+		}
 		return SharedPtr<Path>();
 	}
 
+	for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+		if (_paths[i].p) {
+			_paths[i].p->processBackgroundPathMeasurements(now);
+		}
+	}
+
+	/**
+	 * Randomly distribute traffic across all paths
+	 */
+	int numAlivePaths = 0;
+	int numStalePaths = 0;
+	if (RR->node->getMultipathMode() == ZT_MULTIPATH_RANDOM) {
+		int alivePaths[ZT_MAX_PEER_NETWORK_PATHS];
+		int stalePaths[ZT_MAX_PEER_NETWORK_PATHS];
+		memset(&alivePaths, -1, sizeof(alivePaths));
+		memset(&stalePaths, -1, sizeof(stalePaths));
+		for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+			if (_paths[i].p) {
+				if (_paths[i].p->alive(now)) {
+					alivePaths[numAlivePaths] = i;
+					numAlivePaths++;
+				}
+				else {
+					stalePaths[numStalePaths] = i;
+					numStalePaths++;
+				}
+			}
+		}
+		unsigned int r = _freeRandomByte;
+		if (numAlivePaths > 0) {
+			int rf = r % numAlivePaths;
+			return _paths[alivePaths[rf]].p;
+		}
+		else if(numStalePaths > 0) {
+			// Resort to trying any non-expired path
+			int rf = r % numStalePaths;
+			return _paths[stalePaths[rf]].p;
+		}
+	}
+
+	/**
+	 * Proportionally allocate traffic according to dynamic path quality measurements
+	 */
+	if (RR->node->getMultipathMode() == ZT_MULTIPATH_PROPORTIONALLY_BALANCED) {
+		if ((now - _lastAggregateAllocation) >= ZT_PATH_QUALITY_COMPUTE_INTERVAL) {
+			_lastAggregateAllocation = now;
+			computeAggregateProportionalAllocation(now);
+		}
+		// Randomly choose path according to their allocations
+		float rf = _freeRandomByte;
+		for(int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+			if (_paths[i].p) {
+				if (rf < _paths[i].p->allocation()) {
+					bestPath = i;
+					_pathChoiceHist->push(bestPath); // Record which path we chose
+					break;
+				}
+				rf -= _paths[i].p->allocation();
+			}
+		}
+		if (bestPath < ZT_MAX_PEER_NETWORK_PATHS) {
+			return _paths[bestPath].p;
+		}
+	}
+	return SharedPtr<Path>();
+}
+
+char *Peer::interfaceListStr()
+{
+	std::map<std::string, int> ifnamemap;
+	char tmp[32];
+	const int64_t now = RR->node->now();
+	char *ptr = _interfaceListStr;
+	bool imbalanced = false;
+	memset(_interfaceListStr, 0, sizeof(_interfaceListStr));
+	int alivePathCount = aggregateLinkLogicalPathCount();
+	for(unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+		if (_paths[i].p && _paths[i].p->alive(now)) {
+			int ipv = _paths[i].p->address().isV4();
+			// If this is acting as an aggregate link, check allocations
+			float targetAllocation = 1.0 / alivePathCount;
+			float currentAllocation = 1.0;
+			if (alivePathCount > 1) {
+				currentAllocation = (float)_pathChoiceHist->countValue(i) / (float)_pathChoiceHist->count();
+				if (fabs(targetAllocation - currentAllocation) > ZT_PATH_IMBALANCE_THRESHOLD) {
+					imbalanced = true;
+				}
+			}
+			char *ipvStr = ipv ? (char*)"ipv4" : (char*)"ipv6";
+			sprintf(tmp, "(%s, %s, %.3f)", _paths[i].p->getName(), ipvStr, currentAllocation);
+			// Prevent duplicates
+			if(ifnamemap[_paths[i].p->getName()] != ipv) {
+				memcpy(ptr, tmp, strlen(tmp));
+				ptr += strlen(tmp);
+				*ptr = ' ';
+				ptr++;
+				ifnamemap[_paths[i].p->getName()] = ipv;
+			}
+		}
+	}
+	ptr--; // Overwrite trailing space
+	if (imbalanced) {
+		sprintf(tmp, ", is asymmetrical");
+		memcpy(ptr, tmp, sizeof(tmp));
+	} else {
+		*ptr = '\0';
+	}
+	return _interfaceListStr;
+}
+
 void Peer::introduce(void *const tPtr,const int64_t now,const SharedPtr<Peer> &other) const
 {
 	unsigned int myBestV4ByScope[ZT_INETADDRESS_MAX_SCOPE+1];
@@ -376,6 +650,71 @@ void Peer::introduce(void *const tPtr,const int64_t now,const SharedPtr<Peer> &o
 	}
 }
 
+inline void Peer::processBackgroundPeerTasks(int64_t now)
+{
+	// Determine current multipath compatibility with other peer
+	if ((now - _lastMultipathCompatibilityCheck) >= ZT_PATH_QUALITY_COMPUTE_INTERVAL) {
+		// Cache number of available paths so that we can short-circuit multipath logic elsewhere
+		//
+		// We also take notice of duplicate paths (same IP only) because we may have
+		// recently received a direct path push from a peer and our list might contain
+		// a dead path which hasn't been fully recognized as such. In this case we
+		// don't want the duplicate to trigger execution of multipath code prematurely.
+		//
+		// This is done to support the behavior of auto multipath enable/disable
+		// without user intervention.
+		int currAlivePathCount = 0;
+		int duplicatePathsFound = 0;
+		for (unsigned int i=0;i<ZT_MAX_PEER_NETWORK_PATHS;++i) {
+			if (_paths[i].p) {
+				currAlivePathCount++;
+				for (unsigned int j=0;j<ZT_MAX_PEER_NETWORK_PATHS;++j) {
+					if (_paths[i].p && _paths[j].p && _paths[i].p->address().ipsEqual2(_paths[j].p->address()) && i != j) {
+						duplicatePathsFound+=1;
+						break;
+					}
+				}
+			}
+		}
+		_uniqueAlivePathCount = (currAlivePathCount - (duplicatePathsFound / 2));
+		_lastMultipathCompatibilityCheck = now;
+		_localMultipathSupported = ((RR->node->getMultipathMode() != ZT_MULTIPATH_NONE) && (ZT_PROTO_VERSION > 9));
+		_remoteMultipathSupported = _vProto > 9;
+		// If both peers support multipath and more than one path exist, we can use multipath logic
+		_canUseMultipath = _localMultipathSupported && _remoteMultipathSupported && (_uniqueAlivePathCount > 1);
+	}
+}
+
+void Peer::sendACK(void *tPtr,const SharedPtr<Path> &path,const int64_t localSocket,const InetAddress &atAddress,int64_t now)
+{
+	Packet outp(_id.address(),RR->identity.address(),Packet::VERB_ACK);
+	uint32_t bytesToAck = path->bytesToAck();
+	outp.append<uint32_t>(bytesToAck);
+	if (atAddress) {
+		outp.armor(_key,false);
+		RR->node->putPacket(tPtr,localSocket,atAddress,outp.data(),outp.size());
+	} else {
+		RR->sw->send(tPtr,outp,false);
+	}
+	path->sentAck(now);
+}
+
+void Peer::sendQOS_MEASUREMENT(void *tPtr,const SharedPtr<Path> &path,const int64_t localSocket,const InetAddress &atAddress,int64_t now)
+{
+	const int64_t _now = RR->node->now();
+	Packet outp(_id.address(),RR->identity.address(),Packet::VERB_QOS_MEASUREMENT);
+	char qosData[ZT_PATH_MAX_QOS_PACKET_SZ];
+	int16_t len = path->generateQoSPacket(_now,qosData);
+	outp.append(qosData,len);
+	if (atAddress) {
+		outp.armor(_key,false);
+		RR->node->putPacket(tPtr,localSocket,atAddress,outp.data(),outp.size());
+	} else {
+		RR->sw->send(tPtr,outp,false);
+	}
+	path->sentQoS(now);
+}
+
 void Peer::sendHELLO(void *tPtr,const int64_t localSocket,const InetAddress &atAddress,int64_t now)
 {
 	Packet outp(_id.address(),RR->identity.address(),Packet::VERB_HELLO);
@@ -444,12 +783,30 @@ void Peer::tryMemorizedPath(void *tPtr,int64_t now)
 unsigned int Peer::doPingAndKeepalive(void *tPtr,int64_t now)
 {
 	unsigned int sent = 0;
-
 	Mutex::Lock _l(_paths_m);
 
 	const bool sendFullHello = ((now - _lastSentFullHello) >= ZT_PEER_PING_PERIOD);
 	_lastSentFullHello = now;
 
+	processBackgroundPeerTasks(now);
+
+	// Emit traces regarding aggregate link status
+	if (_canUseMultipath) {
+		int alivePathCount = aggregateLinkPhysicalPathCount();
+		if ((now - _lastAggregateStatsReport) > ZT_PATH_AGGREGATE_STATS_REPORT_INTERVAL) {
+			_lastAggregateStatsReport = now;
+			if (alivePathCount) {
+				RR->t->peerLinkAggregateStatistics(NULL,*this);
+			}
+		} if (alivePathCount < 2 && _linkIsRedundant) {
+			_linkIsRedundant = !_linkIsRedundant;
+			RR->t->peerLinkNoLongerRedundant(NULL,*this);
+		} if (alivePathCount > 1 && !_linkIsRedundant) {
+			_linkIsRedundant = !_linkIsRedundant;
+			RR->t->peerLinkNowRedundant(NULL,*this);
+		}
+	}
+
 	// Right now we only keep pinging links that have the maximum priority. The
 	// priority is used to track cluster redirections, meaning that when a cluster
 	// redirects us its redirect target links override all other links and we
@@ -477,13 +834,14 @@ unsigned int Peer::doPingAndKeepalive(void *tPtr,int64_t now)
 			}
 		} else break;
 	}
+	if (canUseMultipath()) {
 		while(j < ZT_MAX_PEER_NETWORK_PATHS) {
 			_paths[j].lr = 0;
 			_paths[j].p.zero();
 			_paths[j].priority = 1;
 			++j;
 		}
-
+	}
 	return sent;
 }
 

node/Peer.hpp

@@ -27,18 +27,13 @@
 #ifndef ZT_PEER_HPP
 #define ZT_PEER_HPP
 
-#include <stdint.h>
-
-#include "Constants.hpp"
-
-#include <algorithm>
-#include <utility>
 #include <vector>
-#include <stdexcept>
 
 #include "../include/ZeroTierOne.h"
 
+#include "Constants.hpp"
 #include "RuntimeEnvironment.hpp"
+#include "Node.hpp"
 #include "Path.hpp"
 #include "Address.hpp"
 #include "Utils.hpp"
@@ -65,7 +60,11 @@ private:
 	Peer() {} // disabled to prevent bugs -- should not be constructed uninitialized
 
 public:
-	~Peer() { Utils::burn(_key,sizeof(_key)); }
+	~Peer() {
+		Utils::burn(_key,sizeof(_key));
+		delete _pathChoiceHist;
+		_pathChoiceHist = NULL;
+	}
 
 	/**
 	 * Construct a new peer
@@ -108,6 +107,7 @@ public:
 		const SharedPtr<Path> &path,
 		const unsigned int hops,
 		const uint64_t packetId,
+		const unsigned int payloadLength,
 		const Packet::Verb verb,
 		const uint64_t inRePacketId,
 		const Packet::Verb inReVerb,
@@ -145,20 +145,95 @@ public:
 	 */
 	inline bool sendDirect(void *tPtr,const void *data,unsigned int len,int64_t now,bool force)
 	{
-		SharedPtr<Path> bp(getBestPath(now,force));
+		SharedPtr<Path> bp(getAppropriatePath(now,force));
 		if (bp)
 			return bp->send(RR,tPtr,data,len,now);
 		return false;
 	}
 
 	/**
-	 * Get the best current direct path
+	 * Record statistics on outgoing packets
+	 *
+	 * @param path Path over which packet was sent
+	 * @param id Packet ID
+	 * @param len Length of packet payload
+	 * @param verb Packet verb
+	 * @param now Current time
+	 */
+	void recordOutgoingPacket(const SharedPtr<Path> &path, const uint64_t packetId, uint16_t payloadLength, const Packet::Verb verb, int64_t now);
+
+	/**
+	 * Record statistics on incoming packets
+	 *
+	 * @param path Path over which packet was sent
+	 * @param id Packet ID
+	 * @param len Length of packet payload
+	 * @param verb Packet verb
+	 * @param now Current time
+	 */
+	void recordIncomingPacket(void *tPtr, const SharedPtr<Path> &path, const uint64_t packetId, uint16_t payloadLength, const Packet::Verb verb, int64_t now);
+
+	/**
+	 * Send an ACK to peer for the most recent packets received
+	 *
+	 * @param tPtr Thread pointer to be handed through to any callbacks called as a result of this call
+	 * @param localSocket Raw socket the ACK packet will be sent over
+	 * @param atAddress Destination for the ACK packet
+	 * @param now Current time
+	 */
+	void sendACK(void *tPtr, const SharedPtr<Path> &path, const int64_t localSocket,const InetAddress &atAddress,int64_t now);
+
+	/**
+	 * Send a QoS packet to peer so that it can evaluate the quality of this link
+	 *
+	 * @param tPtr Thread pointer to be handed through to any callbacks called as a result of this call
+	 * @param localSocket Raw socket the QoS packet will be sent over
+	 * @param atAddress Destination for the QoS packet
+	 * @param now Current time
+	 */
+	void sendQOS_MEASUREMENT(void *tPtr, const SharedPtr<Path> &path, const int64_t localSocket,const InetAddress &atAddress,int64_t now);
+
+	/**
+	 * Compute relative quality values and allocations for the components of the aggregate link
+	 *
+	 * @param now Current time
+	 */
+	void computeAggregateProportionalAllocation(int64_t now);
+
+	/**
+	 * @return The aggregate link Packet Delay Variance (PDV)
+	 */
+	int computeAggregateLinkPacketDelayVariance();
+
+	/**
+	 * @return The aggregate link mean latency
+	 */
+	int computeAggregateLinkMeanLatency();
+
+	/**
+	 * @return The number of currently alive "physical" paths in the aggregate link
+	 */
+	int aggregateLinkPhysicalPathCount();
+
+	/**
+	 * @return The number of currently alive "logical" paths in the aggregate link
+	 */
+	int aggregateLinkLogicalPathCount();
+
+	/**
+	 * Get the most appropriate direct path based on current multipath and QoS configuration
 	 *
 	 * @param now Current time
 	 * @param includeExpired If true, include even expired paths
 	 * @return Best current path or NULL if none
 	 */
-	SharedPtr<Path> getBestPath(int64_t now,bool includeExpired) const;
+	SharedPtr<Path> getAppropriatePath(int64_t now, bool includeExpired);
+
+	/**
+	 * Generate a human-readable string of interface names making up the aggregate link, also include
+	 * moving allocation and IP version number for each (for tracing)
+	 */
+	char *interfaceListStr();
 
 	/**
 	 * Send VERB_RENDEZVOUS to this and another peer via the best common IP scope and path
@@ -212,6 +287,16 @@ public:
 	 */
 	unsigned int doPingAndKeepalive(void *tPtr,int64_t now);
 
+	/**
+	 * Clear paths whose localSocket(s) are in a CLOSED state or have an otherwise INVALID state.
+	 * This should be called frequently so that we can detect and remove unproductive or invalid paths.
+	 *
+	 * Under the hood this is done periodically based on ZT_CLOSED_PATH_PRUNING_INTERVAL.
+	 *
+	 * @return Number of paths that were pruned this round
+	 */
+	unsigned int prunePaths();
+
 	/**
 	 * Process a cluster redirect sent by this peer
 	 *
@@ -268,15 +353,19 @@ public:
 	inline int64_t isActive(int64_t now) const { return ((now - _lastNontrivialReceive) < ZT_PEER_ACTIVITY_TIMEOUT); }
 
 	/**
-	 * @return Latency in milliseconds of best path or 0xffff if unknown / no paths
+	 * @return Latency in milliseconds of best/aggregate path or 0xffff if unknown / no paths
 	 */
-	inline unsigned int latency(const int64_t now) const
+	inline unsigned int latency(const int64_t now)
 	{
-		SharedPtr<Path> bp(getBestPath(now,false));
+		if (_canUseMultipath) {
+			return (int)computeAggregateLinkMeanLatency();
+		} else {
+			SharedPtr<Path> bp(getAppropriatePath(now,false));
 			if (bp)
 				return bp->latency();
 			return 0xffff;
 		}
+	}
 
 	/**
 	 * This computes a quality score for relays and root servers
@@ -289,7 +378,7 @@ public:
 	 *
 	 * @return Relay quality score computed from latency and other factors, lower is better
 	 */
-	inline unsigned int relayQuality(const int64_t now) const
+	inline unsigned int relayQuality(const int64_t now)
 	{
 		const uint64_t tsr = now - _lastReceive;
 		if (tsr >= ZT_PEER_ACTIVITY_TIMEOUT)
@@ -328,6 +417,37 @@ public:
 
 	inline bool remoteVersionKnown() const { return ((_vMajor > 0)||(_vMinor > 0)||(_vRevision > 0)); }
 
+	/**
+	 * Periodically update known multipath activation constraints. This is done so that we know when and when
+	 * not to use multipath logic. Doing this once every few seconds is sufficient.
+	 *
+	 * @param now Current time
+	 */
+	inline void processBackgroundPeerTasks(int64_t now);
+
+	/**
+	 * Record that the remote peer does have multipath enabled. As is evident by the receipt of a VERB_ACK
+	 * or a VERB_QOS_MEASUREMENT packet at some point in the past. Until this flag is set, the local client
+	 * shall assume that multipath is not enabled and should only use classical Protocol 9 logic.
+	 */
+	inline void inferRemoteMultipathEnabled() { _remotePeerMultipathEnabled = true; }
+
+	/**
+	 * @return Whether the local client supports and is configured to use multipath
+	 */
+	inline bool localMultipathSupport() { return _localMultipathSupported; }
+
+	/**
+	 * @return Whether the remote peer supports and is configured to use multipath
+	 */
+	inline bool remoteMultipathSupport() { return _remoteMultipathSupported; }
+
+	/**
+	 * @return Whether this client can use multipath to communicate with this peer. True if both peers are using
+	 * the correct protocol and if both peers have multipath enabled. False if otherwise.
+	 */
+	inline bool canUseMultipath() { return _canUseMultipath; }
+
 	/**
 	 * @return True if peer has received a trust established packet (e.g. common network membership) in the past ZT_TRUST_EXPIRATION ms
 	 */
@@ -417,6 +537,41 @@ public:
 		return false;
 	}
 
+	/**
+	 * Rate limit gate for VERB_ACK
+	 */
+	inline bool rateGateACK(const int64_t now)
+	{
+		if ((now - _lastACKWindowReset) >= ZT_PATH_QOS_ACK_CUTOFF_TIME) {
+			_lastACKWindowReset = now;
+			_ACKCutoffCount = 0;
+		} else {
+			++_ACKCutoffCount;
+		}
+		return (_ACKCutoffCount < ZT_PATH_QOS_ACK_CUTOFF_LIMIT);
+	}
+
+	/**
+	 * Rate limit gate for VERB_QOS_MEASUREMENT
+	 */
+	inline bool rateGateQoS(const int64_t now)
+	{
+		if ((now - _lastQoSWindowReset) >= ZT_PATH_QOS_ACK_CUTOFF_TIME) {
+			_lastQoSWindowReset = now;
+			_QoSCutoffCount = 0;
+		} else {
+			++_QoSCutoffCount;
+		}
+		return (_QoSCutoffCount < ZT_PATH_QOS_ACK_CUTOFF_LIMIT);
+	}
+
+	/**
+	 * @return Whether this peer is reachable via an aggregate link
+	 */
+	inline bool hasAggregateLink() {
+		return _localMultipathSupported && _remoteMultipathSupported && _remotePeerMultipathEnabled;
+	}
+
 	/**
 	 * Serialize a peer for storage in local cache
 	 *
@@ -515,6 +670,18 @@ private:
 	int64_t _lastCredentialsReceived;
 	int64_t _lastTrustEstablishedPacketReceived;
 	int64_t _lastSentFullHello;
+	int64_t _lastPathPrune;
+	int64_t _lastACKWindowReset;
+	int64_t _lastQoSWindowReset;
+	int64_t _lastMultipathCompatibilityCheck;
+
+	unsigned char _freeRandomByte;
+
+	int _uniqueAlivePathCount;
+
+	bool _localMultipathSupported;
+	bool _remoteMultipathSupported;
+	bool _canUseMultipath;
 
 	uint16_t _vProto;
 	uint16_t _vMajor;
@@ -528,8 +695,21 @@ private:
 
 	unsigned int _directPathPushCutoffCount;
 	unsigned int _credentialsCutoffCount;
+	unsigned int _QoSCutoffCount;
+	unsigned int _ACKCutoffCount;
 
 	AtomicCounter __refCount;
+
+	RingBuffer<int> *_pathChoiceHist;
+
+	bool _linkIsBalanced;
+	bool _linkIsRedundant;
+	bool _remotePeerMultipathEnabled;
+
+	int64_t _lastAggregateStatsReport;
+	int64_t _lastAggregateAllocation;
+
+	char _interfaceListStr[256]; // 16 characters * 16 paths in a link
 };
 
 } // namespace ZeroTier

node/RingBuffer.hpp

@@ -0,0 +1,345 @@
+/*
+ * ZeroTier One - Network Virtualization Everywhere
+ * Copyright (C) 2011-2018  ZeroTier, Inc.  https://www.zerotier.com/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * You can be released from the requirements of the license by purchasing
+ * a commercial license. Buying such a license is mandatory as soon as you
+ * develop commercial closed-source software that incorporates or links
+ * directly against ZeroTier software without disclosing the source code
+ * of your own application.
+ */
+
+#ifndef ZT_RINGBUFFER_H
+#define ZT_RINGBUFFER_H
+
+#include <typeinfo>
+#include <cstdint>
+#include <stdlib.h>
+#include <memory.h>
+#include <algorithm>
+#include <math.h>
+
+namespace ZeroTier {
+
+/**
+ * A circular buffer
+ *
+ * For fast handling of continuously-evolving variables (such as path quality metrics).
+ * Using this, we can maintain longer sliding historical windows for important path 
+ * metrics without the need for potentially expensive calls to memcpy/memmove.
+ *
+ * Some basic statistical functionality is implemented here in an attempt
+ * to reduce the complexity of code needed to interact with this type of buffer.
+ */
+
+template <class T>
+class RingBuffer
+{
+private:
+	T * buf;
+	size_t size;
+	size_t begin;
+	size_t end;
+	bool wrap;
+
+public:
+
+	/**
+	 * create a RingBuffer with space for up to size elements.
+	 */
+	explicit RingBuffer(size_t size)
+		: size(size),
+		begin(0),
+		end(0),
+		wrap(false)
+	{
+		buf = new T[size];
+		memset(buf, 0, sizeof(T) * size);
+	}
+
+	/**
+	 * @return A pointer to the underlying buffer
+	 */
+	T* get_buf()
+	{
+		return buf + begin;
+	}
+
+	/** 
+	 * Adjust buffer index pointer as if we copied data in
+	 * @param n Number of elements to copy in
+	 * @return Number of elements we copied in
+	 */
+	size_t produce(size_t n)
+	{
+		n = std::min(n, getFree());
+		if (n == 0) {
+			return n;
+		}
+		const size_t first_chunk = std::min(n, size - end);
+		end = (end + first_chunk) % size;
+		if (first_chunk < n) {
+			const size_t second_chunk = n - first_chunk;
+			end = (end + second_chunk) % size;
+		}
+		if (begin == end) {
+			wrap = true;
+		}
+		return n;
+	}
+
+	/** 
+	 * Fast erase, O(1). 
+	 * Merely reset the buffer pointer, doesn't erase contents
+	 */
+	void reset()
+	{
+		consume(count());
+	}
+
+	/** 
+	 * adjust buffer index pointer as if we copied data out
+	 * @param n Number of elements we copied from the buffer
+	 * @return Number of elements actually available from the buffer
+	 */
+	size_t consume(size_t n)
+	{
+		n = std::min(n, count());
+		if (n == 0) {
+			return n;
+		}
+		if (wrap) {
+			wrap = false;
+		}
+		const size_t first_chunk = std::min(n, size - begin);
+		begin = (begin + first_chunk) % size;
+		if (first_chunk < n) {
+			const size_t second_chunk = n - first_chunk;
+			begin = (begin + second_chunk) % size;
+		}
+		return n;
+	}
+
+	/**
+	 * @param data Buffer that is to be written to the ring
+	 * @param n Number of elements to write to the buffer
+	 */
+	size_t write(const T * data, size_t n)
+	{
+		n = std::min(n, getFree());
+		if (n == 0) {
+			return n;
+		}
+		const size_t first_chunk = std::min(n, size - end);
+		memcpy(buf + end, data, first_chunk * sizeof(T));
+		end = (end + first_chunk) % size;
+		if (first_chunk < n) {
+			const size_t second_chunk = n - first_chunk;
+			memcpy(buf + end, data + first_chunk, second_chunk * sizeof(T));
+			end = (end + second_chunk) % size;
+		}
+		if (begin == end) {
+			wrap = true;
+		}
+		return n;
+	}
+
+	/**
+	 * Place a single value on the buffer. If the buffer is full, consume a value first.
+	 *
+	 * @param value A single value to be placed in the buffer
+	 */
+	void push(const T value)
+	{
+		if (count() == size) {
+			consume(1);
+		}
+		const size_t first_chunk = std::min((size_t)1, size - end);
+		*(buf + end) = value;
+		end = (end + first_chunk) % size;
+		if (begin == end) {
+			wrap = true;
+		}
+	}
+
+	/**
+	 * @return The most recently pushed element on the buffer
+	 */
+	T get_most_recent() { return *(buf + end); }
+
+	/**
+	 * @param dest Destination buffer
+	 * @param n Size (in terms of number of elements) of the destination buffer
+	 * @return Number of elements read from the buffer
+	 */
+	size_t read(T * dest, size_t n)
+	{
+		n = std::min(n, count());
+		if (n == 0) {
+			return n;
+		}
+		if (wrap) {
+			wrap = false;
+		}
+		const size_t first_chunk = std::min(n, size - begin);
+		memcpy(dest, buf + begin, first_chunk * sizeof(T));
+		begin = (begin + first_chunk) % size;
+		if (first_chunk < n) {
+			const size_t second_chunk = n - first_chunk;
+			memcpy(dest + first_chunk, buf + begin, second_chunk * sizeof(T));
+			begin = (begin + second_chunk) % size;
+		}
+		return n;
+	}
+
+	/**
+	 * Return how many elements are in the buffer, O(1).
+	 *
+	 * @return The number of elements in the buffer
+	 */
+	size_t count()
+	{
+		if (end == begin) {
+			return wrap ? size : 0;
+		}
+		else if (end > begin) {
+			return end - begin;
+		}
+		else {
+			return size + end - begin;
+		}
+	}
+
+	/**
+	 * @return The number of slots that are unused in the buffer
+	 */
+	size_t getFree() { return size - count(); }
+
+	/**
+	 * @return The arithmetic mean of the contents of the buffer
+	 */
+	float mean()
+	{
+		size_t iterator = begin;
+		float subtotal = 0;
+		size_t curr_cnt = count();
+		for (size_t i=0; i<curr_cnt; i++) {
+			iterator = (iterator + size - 1) % curr_cnt;
+			subtotal += (float)*(buf + iterator);
+		}
+		return curr_cnt ? subtotal / (float)curr_cnt : 0;
+	}
+
+	/**
+	 * @return The arithmetic mean of the most recent 'n' elements of the buffer
+	 */
+	float mean(size_t n)
+	{
+		n = n < size ? n : size;
+		size_t iterator = begin;
+		float subtotal = 0;
+		size_t curr_cnt = count();
+		for (size_t i=0; i<n; i++) {
+			iterator = (iterator + size - 1) % curr_cnt;
+			subtotal += (float)*(buf + iterator);
+		}
+		return curr_cnt ? subtotal / (float)curr_cnt : 0;
+	}
+
+	/**
+	 * @return The sample standard deviation of element values
+	 */
+	float stddev() { return sqrt(variance()); }
+
+	/**
+	 * @return The variance of element values
+	 */
+	float variance()
+	{
+		size_t iterator = begin;
+		float cached_mean = mean();
+		size_t curr_cnt = count();
+		if (size) {
+			T sum_of_squared_deviations = 0;
+			for (size_t i=0; i<curr_cnt; i++) {
+				iterator = (iterator + size - 1) % curr_cnt;
+				float deviation = (buf[i] - cached_mean);
+				sum_of_squared_deviations += (deviation*deviation);
+			}
+			float variance = (float)sum_of_squared_deviations / (float)(size - 1);
+			return variance;
+		}
+		return 0;
+	}
+
+	/**
+	 * @return The number of elements of zero value
+	 */
+	size_t zeroCount()
+	{
+		size_t iterator = begin;
+		size_t zeros = 0;
+		size_t curr_cnt = count();
+		for (size_t i=0; i<curr_cnt; i++) {
+			iterator = (iterator + size - 1) % curr_cnt;
+			if (*(buf + iterator) == 0) {
+				zeros++;
+			}
+		}
+		return zeros;
+	}
+
+	/**
+	 * @param value Value to match against in buffer
+	 * @return The number of values held in the ring buffer which match a given value
+	 */
+	size_t countValue(T value)
+	{
+		size_t iterator = begin;
+		size_t cnt = 0;
+		size_t curr_cnt = count();
+		for (size_t i=0; i<curr_cnt; i++) {
+			iterator = (iterator + size - 1) % curr_cnt;
+			if (*(buf + iterator) == value) {
+				cnt++;
+			}
+		}
+		return cnt;
+	}
+
+	/**
+	 * Print the contents of the buffer
+	 */
+	void dump()
+	{
+		size_t iterator = begin;
+		for (size_t i=0; i<size; i++) {
+			iterator = (iterator + size - 1) % size;
+			if (typeid(T) == typeid(int)) {
+				 //DEBUG_INFO("buf[%2zu]=%2d", iterator, (int)*(buf + iterator));
+			}
+			else {
+				 //DEBUG_INFO("buf[%2zu]=%2f", iterator, (float)*(buf + iterator));
+			}
+		}
+	}
+};
+
+} // namespace ZeroTier
+
+#endif

node/Salsa20.cpp

@@ -14,7 +14,7 @@
 #define XOR(v,w) ((v) ^ (w))
 #define PLUS(v,w) ((uint32_t)((v) + (w)))
 
-// Set up laod/store macros with appropriate endianness (we don't use these in SSE mode)
+// Set up load/store macros with appropriate endianness (we don't use these in SSE mode)
 #ifndef ZT_SALSA20_SSE
 
 #if __BYTE_ORDER == __LITTLE_ENDIAN

node/SelfAwareness.cpp

@@ -145,7 +145,7 @@ std::vector<InetAddress> SelfAwareness::getSymmetricNatPredictions()
 	 *
 	 * Since flows are encrypted and authenticated they could not actually
 	 * read or modify traffic, but they could gather meta-data for forensics
-	 * purpsoes or use this as a DOS attack vector. */
+	 * purposes or use this as a DOS attack vector. */
 
 	std::map< uint32_t,unsigned int > maxPortByIp;
 	InetAddress theOneTrueSurface;

node/Switch.cpp

@@ -269,6 +269,8 @@ void Switch::onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const
 		}
 	}
 
+	uint8_t qosBucket = ZT_QOS_DEFAULT_BUCKET;
+
 	if (to.isMulticast()) {
 		MulticastGroup multicastGroup(to,0);
 
@@ -386,7 +388,7 @@ void Switch::onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const
 			network->learnBridgedMulticastGroup(tPtr,multicastGroup,RR->node->now());
 
 		// First pass sets noTee to false, but noTee is set to true in OutboundMulticast to prevent duplicates.
-		if (!network->filterOutgoingPacket(tPtr,false,RR->identity.address(),Address(),from,to,(const uint8_t *)data,len,etherType,vlanId)) {
+		if (!network->filterOutgoingPacket(tPtr,false,RR->identity.address(),Address(),from,to,(const uint8_t *)data,len,etherType,vlanId,qosBucket)) {
 			RR->t->outgoingNetworkFrameDropped(tPtr,network,from,to,etherType,vlanId,len,"filter blocked");
 			return;
 		}
@@ -410,7 +412,7 @@ void Switch::onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const
 		Address toZT(to.toAddress(network->id())); // since in-network MACs are derived from addresses and network IDs, we can reverse this
 		SharedPtr<Peer> toPeer(RR->topology->getPeer(tPtr,toZT));
 
-		if (!network->filterOutgoingPacket(tPtr,false,RR->identity.address(),toZT,from,to,(const uint8_t *)data,len,etherType,vlanId)) {
+		if (!network->filterOutgoingPacket(tPtr,false,RR->identity.address(),toZT,from,to,(const uint8_t *)data,len,etherType,vlanId,qosBucket)) {
 			RR->t->outgoingNetworkFrameDropped(tPtr,network,from,to,etherType,vlanId,len,"filter blocked");
 			return;
 		}
@@ -425,7 +427,7 @@ void Switch::onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const
 			outp.append(data,len);
 			if (!network->config().disableCompression())
 				outp.compress();
-			send(tPtr,outp,true);
+			aqm_enqueue(tPtr,network,outp,true,qosBucket);
 		} else {
 			Packet outp(toZT,RR->identity.address(),Packet::VERB_FRAME);
 			outp.append(network->id());
@@ -433,7 +435,7 @@ void Switch::onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const
 			outp.append(data,len);
 			if (!network->config().disableCompression())
 				outp.compress();
-			send(tPtr,outp,true);
+			aqm_enqueue(tPtr,network,outp,true,qosBucket);
 		}
 
 	} else {
@@ -442,7 +444,7 @@ void Switch::onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const
 		// We filter with a NULL destination ZeroTier address first. Filtrations
 		// for each ZT destination are also done below. This is the same rationale
 		// and design as for multicast.
-		if (!network->filterOutgoingPacket(tPtr,false,RR->identity.address(),Address(),from,to,(const uint8_t *)data,len,etherType,vlanId)) {
+		if (!network->filterOutgoingPacket(tPtr,false,RR->identity.address(),Address(),from,to,(const uint8_t *)data,len,etherType,vlanId,qosBucket)) {
 			RR->t->outgoingNetworkFrameDropped(tPtr,network,from,to,etherType,vlanId,len,"filter blocked");
 			return;
 		}
@@ -480,7 +482,7 @@ void Switch::onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const
 		}
 
 		for(unsigned int b=0;b<numBridges;++b) {
-			if (network->filterOutgoingPacket(tPtr,true,RR->identity.address(),bridges[b],from,to,(const uint8_t *)data,len,etherType,vlanId)) {
+			if (network->filterOutgoingPacket(tPtr,true,RR->identity.address(),bridges[b],from,to,(const uint8_t *)data,len,etherType,vlanId,qosBucket)) {
 				Packet outp(bridges[b],RR->identity.address(),Packet::VERB_EXT_FRAME);
 				outp.append(network->id());
 				outp.append((uint8_t)0x00);
@@ -490,7 +492,7 @@ void Switch::onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const
 				outp.append(data,len);
 				if (!network->config().disableCompression())
 					outp.compress();
-				send(tPtr,outp,true);
+				aqm_enqueue(tPtr,network,outp,true,qosBucket);
 			} else {
 				RR->t->outgoingNetworkFrameDropped(tPtr,network,from,to,etherType,vlanId,len,"filter blocked (bridge replication)");
 			}
@@ -498,6 +500,263 @@ void Switch::onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const
 	}
 }
 
+void Switch::aqm_enqueue(void *tPtr, const SharedPtr<Network> &network, Packet &packet,bool encrypt,int qosBucket)
+{
+	if(!network->QoSEnabled()) {
+		send(tPtr, packet, encrypt);
+		return;
+	}
+	NetworkQoSControlBlock *nqcb = _netQueueControlBlock[network->id()];
+	if (!nqcb) {
+		// DEBUG_INFO("creating network QoS control block (NQCB) for network %llx", network->id());
+		nqcb = new NetworkQoSControlBlock();
+		_netQueueControlBlock[network->id()] = nqcb;
+		// Initialize ZT_QOS_NUM_BUCKETS queues and place them in the INACTIVE list
+		// These queues will be shuffled between the new/old/inactive lists by the enqueue/dequeue algorithm
+		for (int i=0; i<ZT_QOS_NUM_BUCKETS; i++) {
+			nqcb->inactiveQueues.push_back(new ManagedQueue(i));
+		}
+	}
+
+	if (packet.verb() != Packet::VERB_FRAME && packet.verb() != Packet::VERB_EXT_FRAME) {
+		// DEBUG_INFO("skipping, no QoS for this packet, verb=%x", packet.verb());
+		// just send packet normally, no QoS for ZT protocol traffic
+		send(tPtr, packet, encrypt);
+	} 
+
+	_aqm_m.lock();
+
+	// Enqueue packet and move queue to appropriate list
+
+	const Address dest(packet.destination());
+	TXQueueEntry *txEntry = new TXQueueEntry(dest,RR->node->now(),packet,encrypt);
+	
+	ManagedQueue *selectedQueue = nullptr;
+	for (int i=0; i<ZT_QOS_NUM_BUCKETS; i++) {
+		if (i < nqcb->oldQueues.size()) { // search old queues first (I think this is best since old would imply most recent usage of the queue)
+			if (nqcb->oldQueues[i]->id == qosBucket) {
+				selectedQueue = nqcb->oldQueues[i];
+			}
+		} if (i < nqcb->newQueues.size()) { // search new queues (this would imply not often-used queues)
+			if (nqcb->newQueues[i]->id == qosBucket) {
+				selectedQueue = nqcb->newQueues[i];
+			}
+		} if (i < nqcb->inactiveQueues.size()) { // search inactive queues
+			if (nqcb->inactiveQueues[i]->id == qosBucket) {
+				selectedQueue = nqcb->inactiveQueues[i];
+				// move queue to end of NEW queue list
+				selectedQueue->byteCredit = ZT_QOS_QUANTUM;
+				// DEBUG_INFO("moving q=%p from INACTIVE to NEW list", selectedQueue);
+				nqcb->newQueues.push_back(selectedQueue);
+				nqcb->inactiveQueues.erase(nqcb->inactiveQueues.begin() + i);
+			}
+		}
+	}
+	if (!selectedQueue) {
+		return;
+	}
+
+	selectedQueue->q.push_back(txEntry);
+	selectedQueue->byteLength+=txEntry->packet.payloadLength();
+	nqcb->_currEnqueuedPackets++;
+
+	// DEBUG_INFO("nq=%2lu, oq=%2lu, iq=%2lu, nqcb.size()=%3d, bucket=%2d, q=%p", nqcb->newQueues.size(), nqcb->oldQueues.size(), nqcb->inactiveQueues.size(), nqcb->_currEnqueuedPackets, qosBucket, selectedQueue);
+
+	// Drop a packet if necessary
+	ManagedQueue *selectedQueueToDropFrom = nullptr;
+	if (nqcb->_currEnqueuedPackets > ZT_QOS_MAX_ENQUEUED_PACKETS)
+	{
+		// DEBUG_INFO("too many enqueued packets (%d), finding packet to drop", nqcb->_currEnqueuedPackets);
+		int maxQueueLength = 0;
+		for (int i=0; i<ZT_QOS_NUM_BUCKETS; i++) {
+			if (i < nqcb->oldQueues.size()) {
+				if (nqcb->oldQueues[i]->byteLength > maxQueueLength) {
+					maxQueueLength = nqcb->oldQueues[i]->byteLength;
+					selectedQueueToDropFrom = nqcb->oldQueues[i];
+				}
+			} if (i < nqcb->newQueues.size()) {
+				if (nqcb->newQueues[i]->byteLength > maxQueueLength) {
+					maxQueueLength = nqcb->newQueues[i]->byteLength;
+					selectedQueueToDropFrom = nqcb->newQueues[i];
+				}
+			} if (i < nqcb->inactiveQueues.size()) {
+				if (nqcb->inactiveQueues[i]->byteLength > maxQueueLength) {
+					maxQueueLength = nqcb->inactiveQueues[i]->byteLength;
+					selectedQueueToDropFrom = nqcb->inactiveQueues[i];
+				}
+			}
+		}
+		if (selectedQueueToDropFrom) {
+			// DEBUG_INFO("dropping packet from head of largest queue (%d payload bytes)", maxQueueLength);
+			int sizeOfDroppedPacket = selectedQueueToDropFrom->q.front()->packet.payloadLength();
+			delete selectedQueueToDropFrom->q.front();
+			selectedQueueToDropFrom->q.pop_front();
+			selectedQueueToDropFrom->byteLength-=sizeOfDroppedPacket;
+			nqcb->_currEnqueuedPackets--;
+		}
+	}
+	_aqm_m.unlock();
+	aqm_dequeue(tPtr);
+}
+
+uint64_t Switch::control_law(uint64_t t, int count)
+{
+	return t + ZT_QOS_INTERVAL / sqrt(count);
+}
+
+Switch::dqr Switch::dodequeue(ManagedQueue *q, uint64_t now) 
+{
+	dqr r;
+	r.ok_to_drop = false;
+	r.p = q->q.front();
+
+	if (r.p == NULL) {
+		q->first_above_time = 0;
+		return r;
+	}
+	uint64_t sojourn_time = now - r.p->creationTime;
+	if (sojourn_time < ZT_QOS_TARGET || q->byteLength <= ZT_DEFAULT_MTU) {
+		// went below - stay below for at least interval
+		q->first_above_time = 0;
+	} else {
+		if (q->first_above_time == 0) {
+			// just went above from below. if still above at
+			// first_above_time, will say it's ok to drop.
+			q->first_above_time = now + ZT_QOS_INTERVAL;
+		} else if (now >= q->first_above_time) {
+			r.ok_to_drop = true;
+		}
+	}
+	return r;
+}
+
+Switch::TXQueueEntry * Switch::CoDelDequeue(ManagedQueue *q, bool isNew, uint64_t now)
+{
+	dqr r = dodequeue(q, now);
+
+	if (q->dropping) {
+		if (!r.ok_to_drop) {
+			q->dropping = false;
+		}
+		while (now >= q->drop_next && q->dropping) {
+			q->q.pop_front(); // drop
+			r = dodequeue(q, now);
+			if (!r.ok_to_drop) {
+				// leave dropping state
+				q->dropping = false;
+			} else {
+				++(q->count);
+				// schedule the next drop.
+				q->drop_next = control_law(q->drop_next, q->count);
+			}
+		}
+	} else if (r.ok_to_drop) {
+		q->q.pop_front(); // drop
+		r = dodequeue(q, now);
+		q->dropping = true;
+		q->count = (q->count > 2 && now - q->drop_next < 8*ZT_QOS_INTERVAL)?
+		q->count - 2 : 1;
+		q->drop_next = control_law(now, q->count);
+	}
+	return r.p;
+}
+
+void Switch::aqm_dequeue(void *tPtr)
+{
+	// Cycle through network-specific QoS control blocks
+	for(std::map<uint64_t,NetworkQoSControlBlock*>::iterator nqcb(_netQueueControlBlock.begin());nqcb!=_netQueueControlBlock.end();) {
+		if (!(*nqcb).second->_currEnqueuedPackets) {
+			return;
+		}
+
+		uint64_t now = RR->node->now();
+		TXQueueEntry *entryToEmit = nullptr;
+		std::vector<ManagedQueue*> *currQueues = &((*nqcb).second->newQueues);
+		std::vector<ManagedQueue*> *oldQueues = &((*nqcb).second->oldQueues);
+		std::vector<ManagedQueue*> *inactiveQueues = &((*nqcb).second->inactiveQueues);
+
+		_aqm_m.lock();
+
+		// Attempt dequeue from queues in NEW list
+		bool examiningNewQueues = true;
+		while (currQueues->size()) {
+			ManagedQueue *queueAtFrontOfList = currQueues->front();
+			if (queueAtFrontOfList->byteCredit < 0) {
+				queueAtFrontOfList->byteCredit += ZT_QOS_QUANTUM;
+				// Move to list of OLD queues
+				// DEBUG_INFO("moving q=%p from NEW to OLD list", queueAtFrontOfList);
+				oldQueues->push_back(queueAtFrontOfList);
+				currQueues->erase(currQueues->begin());
+			} else {
+				entryToEmit = CoDelDequeue(queueAtFrontOfList, examiningNewQueues, now);
+				if (!entryToEmit) {
+					// Move to end of list of OLD queues
+					// DEBUG_INFO("moving q=%p from NEW to OLD list", queueAtFrontOfList);
+					oldQueues->push_back(queueAtFrontOfList);
+					currQueues->erase(currQueues->begin());
+				}
+				else {
+					int len = entryToEmit->packet.payloadLength();
+					queueAtFrontOfList->byteLength -= len;
+					queueAtFrontOfList->byteCredit -= len;
+					// Send the packet!
+					queueAtFrontOfList->q.pop_front();
+					send(tPtr, entryToEmit->packet, entryToEmit->encrypt);
+					(*nqcb).second->_currEnqueuedPackets--;
+				}
+				if (queueAtFrontOfList) {
+					//DEBUG_INFO("dequeuing from q=%p, len=%lu in NEW list (byteCredit=%d)", queueAtFrontOfList, queueAtFrontOfList->q.size(), queueAtFrontOfList->byteCredit);
+				}
+				break;
+			}
+		}
+
+		// Attempt dequeue from queues in OLD list
+		examiningNewQueues = false;
+		currQueues = &((*nqcb).second->oldQueues);
+		while (currQueues->size()) {
+			ManagedQueue *queueAtFrontOfList = currQueues->front();
+			if (queueAtFrontOfList->byteCredit < 0) {
+				queueAtFrontOfList->byteCredit += ZT_QOS_QUANTUM;
+				oldQueues->push_back(queueAtFrontOfList);
+				currQueues->erase(currQueues->begin());
+			} else {
+				entryToEmit = CoDelDequeue(queueAtFrontOfList, examiningNewQueues, now);
+				if (!entryToEmit) {
+					//DEBUG_INFO("moving q=%p from OLD to INACTIVE list", queueAtFrontOfList);
+					// Move to inactive list of queues
+					inactiveQueues->push_back(queueAtFrontOfList);
+					currQueues->erase(currQueues->begin());
+				}
+				else {
+					int len = entryToEmit->packet.payloadLength();
+					queueAtFrontOfList->byteLength -= len;
+					queueAtFrontOfList->byteCredit -= len;
+					queueAtFrontOfList->q.pop_front();
+					send(tPtr, entryToEmit->packet, entryToEmit->encrypt);
+					(*nqcb).second->_currEnqueuedPackets--;
+				}
+				if (queueAtFrontOfList) {
+					//DEBUG_INFO("dequeuing from q=%p, len=%lu in OLD list (byteCredit=%d)", queueAtFrontOfList, queueAtFrontOfList->q.size(), queueAtFrontOfList->byteCredit);
+				}
+				break;
+			}
+		}
+		nqcb++;
+		_aqm_m.unlock();
+	}
+}
+
+void Switch::removeNetworkQoSControlBlock(uint64_t nwid)
+{
+	NetworkQoSControlBlock *nq = _netQueueControlBlock[nwid];
+	if (nq) {
+		_netQueueControlBlock.erase(nwid);
+		delete nq;
+		nq = NULL;
+	}
+}
+
 void Switch::send(void *tPtr,Packet &packet,bool encrypt)
 {
 	const Address dest(packet.destination());
@@ -557,6 +816,7 @@ void Switch::doAnythingWaitingForPeer(void *tPtr,const SharedPtr<Peer> &peer)
 
 	{
 		Mutex::Lock _l(_txQueue_m);
+
 		for(std::list< TXQueueEntry >::iterator txi(_txQueue.begin());txi!=_txQueue.end();) {
 			if (txi->dest == peer->address()) {
 				if (_trySend(tPtr,txi->packet,txi->encrypt)) {
@@ -581,6 +841,7 @@ unsigned long Switch::doTimerTasks(void *tPtr,int64_t now)
 	std::vector<Address> needWhois;
 	{
 		Mutex::Lock _l(_txQueue_m);
+
 		for(std::list< TXQueueEntry >::iterator txi(_txQueue.begin());txi!=_txQueue.end();) {
 			if (_trySend(tPtr,txi->packet,txi->encrypt)) {
 				_txQueue.erase(txi++);
@@ -654,12 +915,12 @@ bool Switch::_trySend(void *tPtr,Packet &packet,bool encrypt)
 
 	const SharedPtr<Peer> peer(RR->topology->getPeer(tPtr,destination));
 	if (peer) {
-		viaPath = peer->getBestPath(now,false);
+		viaPath = peer->getAppropriatePath(now,false);
 		if (!viaPath) {
 			peer->tryMemorizedPath(tPtr,now); // periodically attempt memorized or statically defined paths, if any are known
 			const SharedPtr<Peer> relay(RR->topology->getUpstreamPeer());
-			if ( (!relay) || (!(viaPath = relay->getBestPath(now,false))) ) {
-				if (!(viaPath = peer->getBestPath(now,true)))
+			if ( (!relay) || (!(viaPath = relay->getAppropriatePath(now,false))) ) {
+				if (!(viaPath = peer->getAppropriatePath(now,true)))
 					return false;
 			}
 		}
@@ -674,6 +935,8 @@ bool Switch::_trySend(void *tPtr,Packet &packet,bool encrypt)
 	unsigned int chunkSize = std::min(packet.size(),mtu);
 	packet.setFragmented(chunkSize < packet.size());
 
+	peer->recordOutgoingPacket(viaPath, packet.packetId(), packet.payloadLength(), packet.verb(), now);
+
 	if (trustedPathId) {
 		packet.setTrusted(trustedPathId);
 	} else {

node/Switch.hpp

@@ -59,6 +59,14 @@ class Peer;
  */
 class Switch
 {
+	struct ManagedQueue;
+	struct TXQueueEntry;
+
+	typedef struct {
+		TXQueueEntry *p;
+		bool ok_to_drop;
+	} dqr;
+
 public:
 	Switch(const RuntimeEnvironment *renv);
 
@@ -87,6 +95,62 @@ public:
 	 */
 	void onLocalEthernet(void *tPtr,const SharedPtr<Network> &network,const MAC &from,const MAC &to,unsigned int etherType,unsigned int vlanId,const void *data,unsigned int len);
 
+	/**
+	 * Determines the next drop schedule for packets in the TX queue
+	 *
+	 * @param t Current time
+	 * @param count Number of packets dropped this round
+	 */
+	uint64_t control_law(uint64_t t, int count);
+
+	/**
+	 * Selects a packet eligible for transmission from a TX queue. According to the control law, multiple packets
+	 * may be intentionally dropped before a packet is returned to the AQM scheduler.
+	 *
+	 * @param q The TX queue that is being dequeued from
+	 * @param now Current time
+	 */
+	dqr dodequeue(ManagedQueue *q, uint64_t now);
+
+	/**
+	 * Presents a packet to the AQM scheduler.
+	 *
+	 * @param tPtr Thread pointer to be handed through to any callbacks called as a result of this call
+	 * @param network Network that the packet shall be sent over
+	 * @param packet Packet to be sent
+	 * @param encrypt Encrypt packet payload? (always true except for HELLO)
+	 * @param qosBucket Which bucket the rule-system determined this packet should fall into
+	 */
+	void aqm_enqueue(void *tPtr, const SharedPtr<Network> &network, Packet &packet,bool encrypt,int qosBucket);
+
+	/**
+	 * Performs a single AQM cycle and dequeues and transmits all eligible packets on all networks
+	 *
+	 * @param tPtr Thread pointer to be handed through to any callbacks called as a result of this call
+	 */
+	void aqm_dequeue(void *tPtr);
+
+	/**
+	 * Calls the dequeue mechanism and adjust queue state variables
+	 *
+	 * @param q The TX queue that is being dequeued from
+	 * @param isNew Whether or not this queue is in the NEW list
+	 * @param now Current time
+	 */
+	Switch::TXQueueEntry * CoDelDequeue(ManagedQueue *q, bool isNew, uint64_t now);
+
+	/**
+	 * Removes QoS Queues and flow state variables for a specific network. These queues are created
+	 * automatically upon the transmission of the first packet from this peer to another peer on the
+	 * given network.
+	 *
+	 * The reason for existence of queues and flow state variables specific to each network is so that
+	 * each network's QoS rules function independently.
+	 *
+	 * @param nwid Network ID
+	 */
+	void removeNetworkQoSControlBlock(uint64_t nwid);
+
 	/**
 	 * Send a packet to a ZeroTier address (destination in packet)
 	 *
@@ -200,6 +264,7 @@ private:
 	};
 	std::list< TXQueueEntry > _txQueue;
 	Mutex _txQueue_m;
+	Mutex _aqm_m;
 
 	// Tracks sending of VERB_RENDEZVOUS to relaying peers
 	struct _LastUniteKey
@@ -221,6 +286,35 @@ private:
 	};
 	Hashtable< _LastUniteKey,uint64_t > _lastUniteAttempt; // key is always sorted in ascending order, for set-like behavior
 	Mutex _lastUniteAttempt_m;
+
+	// Queue with additional flow state variables
+	struct ManagedQueue
+	{
+		ManagedQueue(int id) :
+			id(id),
+			byteCredit(ZT_QOS_QUANTUM),
+			byteLength(0),
+			dropping(false)
+		{}
+		int id;
+		int byteCredit;
+		int byteLength;
+		uint64_t first_above_time;
+		uint32_t count;
+		uint64_t drop_next;
+		bool dropping;
+		uint64_t drop_next_time;
+		std::list< TXQueueEntry *> q;
+	};
+	// To implement fq_codel we need to maintain a queue of queues
+	struct NetworkQoSControlBlock
+	{
+		int _currEnqueuedPackets;
+		std::vector<ManagedQueue *> newQueues;
+		std::vector<ManagedQueue *> oldQueues;
+		std::vector<ManagedQueue *> inactiveQueues;
+	};
+	std::map<uint64_t,NetworkQoSControlBlock*> _netQueueControlBlock;
 };
 
 } // namespace ZeroTier

node/Tag.hpp

@@ -58,7 +58,7 @@ class RuntimeEnvironment;
  * values.
  *
  * Unlike capabilities tags are signed only by the issuer and are never
- * transferrable.
+ * transferable.
  */
 class Tag : public Credential
 {

node/Topology.cpp

@@ -138,7 +138,7 @@ SharedPtr<Peer> Topology::getPeer(void *tPtr,const Address &zta)
 			}
 			return SharedPtr<Peer>();
 		}
-	} catch ( ... ) {} // ignore invalid identities or other strage failures
+	} catch ( ... ) {} // ignore invalid identities or other strange failures
 
 	return SharedPtr<Peer>();
 }

node/Topology.hpp

@@ -299,7 +299,7 @@ public:
 		Address *a = (Address *)0;
 		SharedPtr<Peer> *p = (SharedPtr<Peer> *)0;
 		while (i.next(a,p)) {
-			const SharedPtr<Path> pp((*p)->getBestPath(now,false));
+			const SharedPtr<Path> pp((*p)->getAppropriatePath(now,false));
 			if (pp)
 				++cnt;
 		}

node/Trace.cpp

@@ -40,6 +40,7 @@
 #include "Tag.hpp"
 #include "Capability.hpp"
 #include "Revocation.hpp"
+#include "../include/ZeroTierDebug.h"
 
 namespace ZeroTier {
 
@@ -106,6 +107,26 @@ void Trace::peerConfirmingUnknownPath(void *const tPtr,const uint64_t networkId,
 	}
 }
 
+void Trace::peerLinkNowRedundant(void *const tPtr,Peer &peer)
+{
+	ZT_LOCAL_TRACE(tPtr,RR,"link to peer %.10llx is fully redundant",peer.address().toInt());
+}
+
+void Trace::peerLinkNoLongerRedundant(void *const tPtr,Peer &peer)
+{
+	ZT_LOCAL_TRACE(tPtr,RR,"link to peer %.10llx is no longer redundant",peer.address().toInt());
+}
+
+void Trace::peerLinkAggregateStatistics(void *const tPtr,Peer &peer)
+{
+	ZT_LOCAL_TRACE(tPtr,RR,"link to peer %.10llx is composed of (%d) physical paths %s, has packet delay variance (%.0f ms), mean latency (%.0f ms)",
+		peer.address().toInt(),
+		peer.aggregateLinkPhysicalPathCount(),
+		peer.interfaceListStr(),
+		peer.computeAggregateLinkPacketDelayVariance(),
+		peer.computeAggregateLinkMeanLatency());
+}
+
 void Trace::peerLearnedNewPath(void *const tPtr,const uint64_t networkId,Peer &peer,const SharedPtr<Path> &newPath,const uint64_t packetId)
 {
 	char tmp[128];

node/Trace.hpp

@@ -121,6 +121,12 @@ public:
 	void resettingPathsInScope(void *const tPtr,const Address &reporter,const InetAddress &reporterPhysicalAddress,const InetAddress &myPhysicalAddress,const InetAddress::IpScope scope);
 
 	void peerConfirmingUnknownPath(void *const tPtr,const uint64_t networkId,Peer &peer,const SharedPtr<Path> &path,const uint64_t packetId,const Packet::Verb verb);
+
+	void peerLinkNowRedundant(void *const tPtr,Peer &peer);
+	void peerLinkNoLongerRedundant(void *const tPtr,Peer &peer);
+
+	void peerLinkAggregateStatistics(void *const tPtr,Peer &peer);
+
 	void peerLearnedNewPath(void *const tPtr,const uint64_t networkId,Peer &peer,const SharedPtr<Path> &newPath,const uint64_t packetId);
 	void peerRedirected(void *const tPtr,const uint64_t networkId,Peer &peer,const SharedPtr<Path> &newPath);
 

node/Utils.hpp

@@ -38,6 +38,10 @@
 #include <vector>
 #include <map>
 
+#if defined(__FreeBSD__)
+#include <sys/endian.h>
+#endif
+
 #include "Constants.hpp"
 
 #ifdef __LINUX__
@@ -261,6 +265,14 @@ public:
 		return l;
 	}
 
+	static inline float normalize(float value, int64_t bigMin, int64_t bigMax, int32_t targetMin, int32_t targetMax)
+	{
+	    int64_t bigSpan = bigMax - bigMin;
+	    int64_t smallSpan = targetMax - targetMin;
+	    float valueScaled = (value - (float)bigMin) / (float)bigSpan;
+	    return (float)targetMin + valueScaled * (float)smallSpan;
+	}
+
 	/**
 	 * Generate secure random bytes
 	 *
@@ -411,8 +423,12 @@ public:
 	static inline uint64_t hton(uint64_t n)
 	{
 #if __BYTE_ORDER == __LITTLE_ENDIAN
-#if defined(__GNUC__) && (!defined(__OpenBSD__))
+#if defined(__GNUC__)
+#if defined(__FreeBSD__)
+		return bswap64(n);
+#elif (!defined(__OpenBSD__))
 		return __builtin_bswap64(n);
+#endif
 #else
 		return (
 			((n & 0x00000000000000FFULL) << 56) |
@@ -440,8 +456,12 @@ public:
 	static inline uint64_t ntoh(uint64_t n)
 	{
 #if __BYTE_ORDER == __LITTLE_ENDIAN
-#if defined(__GNUC__) && !defined(__OpenBSD__)
+#if defined(__GNUC__)
+#if defined(__FreeBSD__)
+		return bswap64(n);
+#elif (!defined(__OpenBSD__))
 		return __builtin_bswap64(n);
+#endif
 #else
 		return (
 			((n & 0x00000000000000FFULL) << 56) |

osdep/Binder.hpp

@@ -389,6 +389,7 @@ public:
 						_bindings[_bindingCount].udpSock = udps;
 						_bindings[_bindingCount].tcpListenSock = tcps;
 						_bindings[_bindingCount].address = ii->first;
+						phy.setIfName(udps, (char*)ii->second.c_str(), ii->second.length());
 						++_bindingCount;
 					}
 				} else {

osdep/LinuxEthernetTap.cpp

@@ -56,6 +56,7 @@
 #include "../node/Dictionary.hpp"
 #include "OSUtils.hpp"
 #include "LinuxEthernetTap.hpp"
+#include "LinuxNetLink.hpp"
 
 // ff:ff:ff:ff:ff:ff with no ADI
 static const ZeroTier::MulticastGroup _blindWildcardMulticastGroup(ZeroTier::MAC(0xff),0);
@@ -97,6 +98,9 @@ LinuxEthernetTap::LinuxEthernetTap(
 	char procpath[128],nwids[32];
 	struct stat sbuf;
 
+	// ensure netlink connection is started
+	(void)LinuxNetLink::getInstance();
+	
 	OSUtils::ztsnprintf(nwids,sizeof(nwids),"%.16llx",nwid);
 
 	Mutex::Lock _l(__tapCreateLock); // create only one tap at a time, globally
@@ -263,18 +267,8 @@ bool LinuxEthernetTap::enabled() const
 
 static bool ___removeIp(const std::string &_dev,const InetAddress &ip)
 {
-	long cpid = (long)vfork();
-	if (cpid == 0) {
-		OSUtils::redirectUnixOutputs("/dev/null",(const char *)0);
-		setenv("PATH", "/sbin:/bin:/usr/sbin:/usr/bin", 1);
-		char iptmp[128];
-		::execlp("ip","ip","addr","del",ip.toString(iptmp),"dev",_dev.c_str(),(const char *)0);
-		::_exit(-1);
-	} else {
-		int exitcode = -1;
-		::waitpid(cpid,&exitcode,0);
-		return (exitcode == 0);
-	}
+	LinuxNetLink::getInstance().removeAddress(ip, _dev.c_str());
+	return true;
 }
 
 #ifdef __SYNOLOGY__
@@ -285,12 +279,6 @@ bool LinuxEthernetTap::addIpSyn(std::vector<InetAddress> ips)
 	std::string cfg_contents = "DEVICE="+_dev+"\nBOOTPROTO=static";
 	int ip4=0,ip6=0,ip4_tot=0,ip6_tot=0;
 
-	long cpid = (long)vfork();
-	if (cpid == 0) {
-		OSUtils::redirectUnixOutputs("/dev/null",(const char *)0);
-		setenv("PATH", "/sbin:/bin:/usr/sbin:/usr/bin", 1);
-		// We must know if there is at least (one) of each protocol version so we
-		// can properly enumerate address/netmask combinations in the ifcfg-dev file
 	for(int i=0; i<(int)ips.size(); i++) {
 		if (ips[i].isV4())
 			ip4_tot++;
@@ -305,8 +293,7 @@ bool LinuxEthernetTap::addIpSyn(std::vector<InetAddress> ips)
 			cfg_contents += "\nIPADDR"+numstr4+"="+ips[i].toIpString(iptmp)
 				+ "\nNETMASK"+numstr4+"="+ips[i].netmask().toIpString(iptmp2)+"\n";
 			ip4++;
-			}
-			else {
+		} else {
 			char iptmp[64],iptmp2[64];
 			std::string numstr6 = ip6_tot > 1 ? std::to_string(ip6) : "";
 			cfg_contents += "\nIPV6ADDR"+numstr6+"="+ips[i].toIpString(iptmp)
@@ -317,17 +304,7 @@ bool LinuxEthernetTap::addIpSyn(std::vector<InetAddress> ips)
 	OSUtils::writeFile(filepath.c_str(), cfg_contents.c_str(), cfg_contents.length());
 	// Finaly, add IPs
 	for(int i=0; i<(int)ips.size(); i++){
-			char iptmp[128],iptmp2[128];
-			if (ips[i].isV4())
-				::execlp("ip","ip","addr","add",ips[i].toString(iptmp),"broadcast",ips[i].broadcast().toIpString(iptmp2),"dev",_dev.c_str(),(const char *)0);
-			else
-				::execlp("ip","ip","addr","add",ips[i].toString(iptmp),"dev",_dev.c_str(),(const char *)0);
-		}
-		::_exit(-1);
-	} else if (cpid > 0) {
-		int exitcode = -1;
-		::waitpid(cpid,&exitcode,0);
-		return (exitcode == 0);
+		LinuxNetLink::getInstance().addAddress(ips[i], _dev.c_str());
 	}
 	return true;
 }
@@ -348,24 +325,9 @@ bool LinuxEthernetTap::addIp(const InetAddress &ip)
 			___removeIp(_dev,*i);
 	}
 
-	long cpid = (long)vfork();
-	if (cpid == 0) {
-		OSUtils::redirectUnixOutputs("/dev/null",(const char *)0);
-		setenv("PATH", "/sbin:/bin:/usr/sbin:/usr/bin", 1);
-		char iptmp[128],iptmp2[128];
-		if (ip.isV4()) {
-			::execlp("ip","ip","addr","add",ip.toString(iptmp),"broadcast",ip.broadcast().toIpString(iptmp2),"dev",_dev.c_str(),(const char *)0);
-		} else {
-			::execlp("ip","ip","addr","add",ip.toString(iptmp),"dev",_dev.c_str(),(const char *)0);
-		}
-		::_exit(-1);
-	} else if (cpid > 0) {
-		int exitcode = -1;
-		::waitpid(cpid,&exitcode,0);
-		return (exitcode == 0);
-	}
+	LinuxNetLink::getInstance().addAddress(ip, _dev.c_str());
 
-	return false;
+	return true;
 }
 
 bool LinuxEthernetTap::removeIp(const InetAddress &ip)

osdep/LinuxNetLink.hpp

@@ -0,0 +1,129 @@
+/*
+ * ZeroTier One - Network Virtualization Everywhere
+ * Copyright (C) 2011-2018  ZeroTier, Inc.  https://www.zerotier.com/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ *
+ * --
+ *
+ * You can be released from the requirements of the license by purchasing
+ * a commercial license. Buying such a license is mandatory as soon as you
+ * develop commercial closed-source software that incorporates or links
+ * directly against ZeroTier software without disclosing the source code
+ * of your own application.
+ */
+
+#ifndef ZT_LINUX_NETLINK_HPP
+#define ZT_LINUX_NETLINK_HPP
+
+#include <vector>
+
+#include <sys/socket.h>
+#include <asm/types.h>
+#include <linux/rtnetlink.h>
+#include <sys/socket.h>
+#include <linux/if.h>
+
+#include "../node/InetAddress.hpp"
+#include "../node/MAC.hpp"
+#include "Thread.hpp"
+#include "../node/Hashtable.hpp"
+#include "../node/Mutex.hpp"
+
+
+namespace ZeroTier {
+
+struct route_entry {
+	InetAddress target;
+	InetAddress via;
+	int if_index;
+	char iface[IFNAMSIZ];
+};
+typedef std::vector<route_entry> RouteList;
+
+/**
+ * Interface with Linux's RTNETLINK 
+ */
+class LinuxNetLink
+{
+private:
+    LinuxNetLink();
+    ~LinuxNetLink();
+
+public:
+    static LinuxNetLink& getInstance()
+    {
+        static LinuxNetLink instance;
+        return instance;
+    }
+
+    LinuxNetLink(LinuxNetLink const&) = delete;
+    void operator=(LinuxNetLink const&) = delete;
+
+    void addRoute(const InetAddress &target, const InetAddress &via, const InetAddress &src, const char *ifaceName);
+    void delRoute(const InetAddress &target, const InetAddress &via, const InetAddress &src, const char *ifaceName);
+    RouteList getIPV4Routes() const;
+    RouteList getIPV6Routes() const;
+
+    void addAddress(const InetAddress &addr, const char *iface);
+    void removeAddress(const InetAddress &addr, const char *iface);
+
+    void threadMain() throw();
+private:
+    int _doRecv(int fd);
+
+    void _processMessage(struct nlmsghdr *nlp, int nll);
+    void _routeAdded(struct nlmsghdr *nlp);
+    void _routeDeleted(struct nlmsghdr *nlp);
+    void _linkAdded(struct nlmsghdr *nlp);
+    void _linkDeleted(struct nlmsghdr *nlp);
+    void _ipAddressAdded(struct nlmsghdr *nlp);
+    void _ipAddressDeleted(struct nlmsghdr *nlp);
+
+    void _requestInterfaceList();
+    void _requestIPv4Routes();
+    void _requestIPv6Routes();
+
+    int _indexForInterface(const char *iface);
+
+    void _setSocketTimeout(int fd, int seconds = 1);
+
+    Thread _t;
+    bool _running;
+
+    RouteList _routes_ipv4;
+    Mutex _rv4_m;
+    RouteList _routes_ipv6;
+    Mutex _rv6_m;
+
+    uint32_t _seq;
+
+    struct iface_entry {
+        int index;
+        char ifacename[IFNAMSIZ];
+        char mac[18];
+        char mac_bin[6];
+        unsigned int mtu;
+    };
+    Hashtable<int, iface_entry> _interfaces;
+    Mutex _if_m;
+
+    // socket communication vars;
+    int _fd;
+    struct sockaddr_nl _la;
+};    
+
+}
+
+#endif // ZT_LINUX_NETLINK_HPPS

osdep/ManagedRoute.hpp

@@ -49,14 +49,20 @@ class ManagedRoute
 	friend class SharedPtr<ManagedRoute>;
 
 public:
-	ManagedRoute(const InetAddress &target,const InetAddress &via,const char *device)
+	ManagedRoute(const InetAddress &target,const InetAddress &via,const InetAddress &src,const char *device)
 	{
 		_target = target;
 		_via = via;
+		_src = src;
 		if (via.ss_family == AF_INET)
 			_via.setPort(32);
 		else if (via.ss_family == AF_INET6)
 			_via.setPort(128);
+		if (src.ss_family == AF_INET) {
+			_src.setPort(32);
+		} else if (src.ss_family == AF_INET6) {
+			_src.setPort(128);
+		}
 		Utils::scopy(_device,sizeof(_device),device);
 		_systemDevice[0] = (char)0;
 	}
@@ -87,6 +93,7 @@ public:
 
 	inline const InetAddress &target() const { return _target; }
 	inline const InetAddress &via() const { return _via; }
+	inline const InetAddress &src() const { return _src; }
 	inline const char *device() const { return _device; }
 
 private:
@@ -95,6 +102,7 @@ private:
 
 	InetAddress _target;
 	InetAddress _via;
+	InetAddress _src;
 	InetAddress _systemVia; // for route overrides
 	std::map<InetAddress,bool> _applied; // routes currently applied
 	char _device[128];

osdep/OSUtils.cpp

@@ -29,6 +29,7 @@
 #include <stdlib.h>
 #include <stdarg.h>
 #include <sys/stat.h>
+#include <stdlib.h>
 
 #include "../node/Constants.hpp"
 #include "../node/Utils.hpp"
@@ -366,7 +367,7 @@ std::vector<std::string> OSUtils::split(const char *s,const char *const sep,cons
 				if (buf.size() > 0) {
 					fields.push_back(buf);
 					buf.clear();
-				} // else skip runs of seperators
+				} // else skip runs of separators
 			} else buf.push_back(*s);
 		}
 		++s;
@@ -398,6 +399,25 @@ std::string OSUtils::platformDefaultHomePath()
     return homeDir;
 #endif
 
+#ifdef __SYNOLOGY__
+	return std::string("/var/packages/zerotier/target/var");
+#endif
+
+    // Check for user-defined environment variable before using defaults
+#ifdef __WINDOWS__
+	DWORD bufferSize = 65535;
+	std::string userDefinedPath;
+	bufferSize = GetEnvironmentVariable("ZEROTIER_HOME", &userDefinedPath[0], bufferSize);
+	if (bufferSize) {
+		return userDefinedPath;
+	}
+#else
+	if(const char* userDefinedPath = getenv("ZEROTIER_HOME")) {
+		return std::string(userDefinedPath);
+	}
+#endif
+
+	// Finally, resort to using default paths if no user-defined path was provided
 #ifdef __UNIX_LIKE__
 
 #ifdef __APPLE__

osdep/Phy.hpp

@@ -152,12 +152,13 @@ private:
 		ZT_PHY_SOCKET_UNIX_LISTEN = 0x08
 	};
 
-	struct PhySocketImpl
-	{
+	struct PhySocketImpl {
+		PhySocketImpl() { memset(ifname, 0, sizeof(ifname)); }
 		PhySocketType type;
 		ZT_PHY_SOCKFD_TYPE sock;
 		void *uptr; // user-settable pointer
 		ZT_PHY_SOCKADDR_STORAGE_TYPE saddr; // remote for TCP_OUT and TCP_IN, local for TCP_LISTEN, RAW, and UDP
+		char ifname[16];
 	};
 
 	std::list<PhySocketImpl> _socks;
@@ -249,6 +250,70 @@ public:
 	 */
 	static inline void** getuptr(PhySocket *s) throw() { return &(reinterpret_cast<PhySocketImpl *>(s)->uptr); }
 
+	/**
+	 * @param s Socket object
+	 * @param nameBuf Buffer to store name of interface which this Socket object is bound to
+	 * @param buflen Length of buffer to copy name into
+	 */
+	static inline void getIfName(PhySocket *s, char *nameBuf, int buflen)
+	{
+		if (s) {
+			memcpy(nameBuf, reinterpret_cast<PhySocketImpl *>(s)->ifname, buflen);
+		}
+	}
+
+	/**
+	 * @param s Socket object
+	 * @param ifname Buffer containing name of interface that this Socket object is bound to
+	 * @param len Length of name of interface
+	 */
+	static inline void setIfName(PhySocket *s, char *ifname, int len)
+	{
+		if (s) {
+			memcpy(&(reinterpret_cast<PhySocketImpl *>(s)->ifname), ifname, len);
+		}
+	}
+
+	/**
+	 * Whether or not the socket object is in a closed state
+	 *
+	 * @param s Socket object
+	 * @return true if socket is closed, false if otherwise
+	 */
+	inline bool isClosed(PhySocket *s)
+	{
+		PhySocketImpl *sws = (reinterpret_cast<PhySocketImpl *>(s));
+		return sws->type == ZT_PHY_SOCKET_CLOSED;
+	}
+
+	/**
+	 * Get state of socket object
+	 *
+	 * @param s Socket object
+	 * @return State of socket
+	 */
+	inline int getState(PhySocket *s)
+	{
+		PhySocketImpl *sws = (reinterpret_cast<PhySocketImpl *>(s));
+		return sws->type;
+	}
+
+	/**
+	 * In the event that this socket is erased, we need a way to convey to the multipath logic
+	 * that this path is no longer valid.
+	 *
+	 * @param s Socket object
+	 * @return Whether the state of this socket is within an acceptable range of values
+	 */
+	inline bool isValidState(PhySocket *s)
+	{
+		if (s) {
+			PhySocketImpl *sws = (reinterpret_cast<PhySocketImpl *>(s));
+			return sws->type >= ZT_PHY_SOCKET_CLOSED && sws->type <= ZT_PHY_SOCKET_UNIX_LISTEN;
+		}
+		return false;
+	}
+
 	/**
 	 * Cause poll() to stop waiting immediately
 	 *
@@ -985,7 +1050,7 @@ public:
 					ZT_PHY_SOCKFD_TYPE sock = s->sock; // if closed, s->sock becomes invalid as s is no longer dereferencable
 					if ((FD_ISSET(sock,&wfds))&&(FD_ISSET(sock,&_writefds))) {
 						try {
-							_handler->phyOnUnixWritable((PhySocket *)&(*s),&(s->uptr),false);
+							_handler->phyOnUnixWritable((PhySocket *)&(*s),&(s->uptr));
 						} catch ( ... ) {}
 					}
 					if (FD_ISSET(sock,&rfds)) {

osdep/WindowsEthernetTap.hpp

@@ -71,7 +71,7 @@ public:
 	static std::string destroyAllPersistentTapDevices();
 
 	/**
-	 * Uninstall a specific persistent tap device by instance ID
+	 * Uninstalls a specific persistent tap device by instance ID
 	 *
 	 * @param instanceId Device instance ID
 	 * @return Empty string on success, otherwise an error message

rule-compiler/rule-compiler.js

@@ -65,7 +65,8 @@ const OPEN_BLOCK_KEYWORDS = {
 	'tee': true,
 	'watch': true,
 	'redirect': true,
-	'break': true
+	'break': true,
+	'priority': true
 };
 
 // Reserved words that can't be used as tag, capability, or rule set names
@@ -81,6 +82,7 @@ const RESERVED_WORDS = {
 	'watch': true,
 	'redirect': true,
 	'break': true,
+	'priority': true,
 
 	'ztsrc': true,
 	'ztdest': true,
@@ -131,6 +133,7 @@ const KEYWORD_TO_API_MAP = {
 	'watch': 'ACTION_WATCH',
 	'redirect': 'ACTION_REDIRECT',
 	'break': 'ACTION_BREAK',
+	'priority': 'ACTION_PRIORITY',
 
 	'ztsrc': 'MATCH_SOURCE_ZEROTIER_ADDRESS',
 	'ztdest': 'MATCH_DEST_ZEROTIER_ADDRESS',

version.h

@@ -40,7 +40,7 @@
 /**
  * Revision
  */
-#define ZEROTIER_ONE_VERSION_REVISION 12
+#define ZEROTIER_ONE_VERSION_REVISION 13
 
 /**
  * Build version