1.3.13

Fix #739

CHANGELOG

@@ -1,5 +1,162 @@
 # Change Log
 
+## 1.3.13 11/12/2015
+
+* Update links for new website.
+
+## 1.3.12 11/12/2015
+
+* Contributing instructions
+* Correctly display log messages.
+* Tentative fix for "WinError 64 The specified network name is no longer available" issues.
+* Fix minor errors reported by codacy.com
+* Add doc on how to got code coverage
+* Raise an error when you use a port outside the ranges
+* Fix asyncio error when closing the app
+* Release UDP ports when closing a Qemu VM. Fixes #323.
+
+## 1.3.11 07/10/2015
+
+* Escape other usage of glob
+* Fix Dynamips identifier is already used by another router
+* Protect dynamips against bad glob
+* Catch ProcessLookupError in Qemu VM.
+* Use the correct UDP tunnel Qemu syntax for version > 1.1.0 when legacy networking is enabled.
+* Prevent launching a packet capture with a non-ASCII path when using Dynamips.
+* Do not automatically delete Dynamips bootflash file because they are necessary to restore VLANs on the c3600 platform.
+* Fix dynamips configuration lost when you delete a node
+* Clarify error message when we got UTF-8 chars in the iourc file
+* Check for valid FR or ATM switch mappings. Fixes #300.
+
+## 1.3.10 04/09/2015
+
+* Catch exception when a process cannot be killed. Fixes #296.
+* Backport: fixes NAT NIO for Qemu VMs (do not launch any legacy scripts)
+* Fixes Unicode error. Fixes #290.
+* Don't delete Dynamips ROM files. They are used to restore the nvram.
+
+## 1.3.9 03/08/2015
+
+* Backport: removes code that deletes IOS router instance files.
+
+## 1.3.8 27/07/2015
+
+* Catch ProcessLookupError when updating iouyap config. Fixes #255.
+* Fixes IOS adapters and WICS cannot be removed. Fixes #282.
+* Makes sure the loop is running when closing the app.
+* Catch GeneratorExit exception. Fixes #231.
+* Fixes missing chipset info for VirtualBox VM. Fixes #254.
+* Fixes IOURC upload.
+* Restore images & projects tarballs
+* Allow users to backup projects and images.
+* Update gns3.conf.upstart.
+* Fix incorrect vboxmanage sudo command.
+* Backport from 1.4: option to drop nvram & disk files for IOS routers in order to save disk space.
+* Backport from 1.4: Remove timeout to wait for connections to finish.
+* Backport from 1.4: Fixes RuntimeError: Event loop is closed.
+* Backport from 1.4: Bind host on 0.0.0.0 when checking for a free UDP port.
+
+## 1.3.7 22/06/2015
+
+* Prevent install on Python 2
+
+## 1.3.6 16/06/2015
+
+* Fix an issue with 1.4dev compatibility
+
+## 1.3.5 16/06/15
+
+* Ignore invalid characters when reading the output of a process
+* Turn on / off authentication
+* Ensure no colored output on Windows
+* Do not stop saving IOS router configs when there is an exception while a project is committed.
+* Create a private config file if expected
+* Distribute our own version of netifaces working with python 3
+* Fix crash if a private config exist in IOS but no private config file
+* Basic Auth support
+* Fix crash when virtualbox list of VMS return an empty line
+
+## 1.3.4 02/06/15
+
+* Drop useless dependencie dateutil
+* Check if port or adapter is connected before starting/stopping a packet capture. Fixes #196.
+* Prevent users to add links to running Qemu VMs and start a capture on running VirtualBox VMs.
+* Fixes bug: couldn't set PCMCIA disk1 size for IOS routers.
+* Fix crash if you pass an invalid hostname
+* Catch VPCS kill errors
+* Raise a VirtualBox error if adapter doesn't exists
+* Ignore VirtualBox VM Name with a carriage return in name
+* Cleanup the temporary project after modules have been notified of the path change
+* Do not return error if we can't remove the old project directory
+* Catch encoding errors in windows logger
+* Use setter for the qemu_path (allow to pass only the binary name)
+* Fixes TAP connection when using VPCS.
+* Fix crash launching qemu on OSX from another location.
+* Adds NAT NIO in device schema validation so they can return an error that it is not supported.
+
+## 1.3.3 14/05/15
+
+* Check for empty iourc path.
+* Fixes bugs with IOS router configs. Fixes #354.
+* Use a temporary directory as egg cache
+* Catch crash error in IOU in case of permission denied
+
+## 1.3.3rc1 07/05/2015
+
+* Return an error if an adapter slot doesn't exist on an IOS router.
+* NIO NAT support for VirtualBox VMs.
+* NIO NAT support for QEMU VMs (user mode back-end is used).
+* Throw an error if user put an invalid port range in config file
+* Turn off configuration parser interpolation
+* Catch configuration file parsing errors
+* Force closing the event loop to avoid warning with Python 3.4.3
+* Catch error when you can't mark a project as no longer temporary
+* Catch BrokenPipeError for OSX frozen server
+* Match how IOU initial-config is set for VPCS VM.
+* Refactors how startup-config and private-config are handled for IOS routers.
+* Catch the "WinError 0 The operation completed successfully" exception at a higher level.
+* Fix temporary project not cleanup with save as
+* If image is not found in VM directory look in images folder
+* Ordered MAC addresses for QEMU based VMs.
+* Merge remote-tracking branch 'origin/master'
+* Force utf-8 configuraton files reading
+* Do not list file starting with a . in upload handler
+* Do not crash when closing a project if VirtualBox is not accessible
+* Catch connection reset errors
+
+
+## 1.3.2 28/04/2015
+
+* Cleanup the VirtualBox Media Manager after closing a project.
+* Avoid Cygwin warning with VPCS on Windows.
+* Close VirtualBox VM linked clone disks after the VM is unregistered.
+* TAP interface support for QEMU VMs.
+* Return an explicit error when a NIO type is not supported by a VM.
+* Do not erase the IOU config
+* Explicit utf-8 decoding.
+* Check NIO exists when stopping an IOU capture.
+* Fixes c7200 NPE setting.
+* Fixes VPCS process termination.
+* Catch FileNotFoundError exception in os.getcwd()
+* Explicit utf-8 encoding where necessary to avoid Unicode errors on Windows (we require/set an utf-8 locale on other systems).
+* Fixes #270. Relative paths management with empty ones.
+* New crash report key and doesn't send report for developers
+* Catch COM errors when connecting to WMI.
+* Don't assume the PATH environment variable exists.
+* Use UUIDs instead of the VM names for VirtualBox pipe paths.
+* Add --log options for daemon support
+* Basic upstart script
+* Add qemu-kvm to the list of binary
+* Fix IOU licence check flag
+* Config paths are not used when updating Dynamips or IOU VM settings.
+* Fixes initial-configs that were not restored when opening a project containing IOU VMs.
+* Prevent parallel execution of VBox commands
+* Fix a crash when in some cases you can't access to VBOX state
+* Fix crash if VirtualBox doesn't return API version
+* Fix a crash in VirtualBox vm creation
+* Allocate random names for Dynamips NIOs.
+* Explicitly delete Dynamips NIOs and unmap VCs for ATM and Frame-Relay switches.
+
 ## 1.3.1 11/04/2015
 
 * Release

CONTRIBUTING.md

@@ -0,0 +1,50 @@
+# Contributing to GNS3
+
+We welcome contributions and bugs reports from everyone.
+We are friendly so don't be afraid to ask questions.
+
+## Bug reports
+
+Before reporting an issue:
+* check our website over at https://gns3.com
+* check if an issue already exists on https://github.com/GNS3/gns3-gui
+* check if an issue already exists on https://github.com/GNS3/gns3-server
+
+Please post on our community website if you are unsure you found a bug,
+you will get faster support and be able to exchange with more users.
+
+If you are unsure which project you should create an issue for, just do 
+it on https://github.com/GNS3/gns3-gui we will take care of the triage.
+
+For bugs specific to the GNS3 VM, please report on https://github.com/GNS3/gns3-vm
+
+## Asking for new features
+
+The best is to start a discussion on the community website in order to get feedback
+from the whole community.
+
+
+## Contributing code
+
+We welcome code contribution from everyone including beginners.
+Don't be afraid to submit a half finished or mediocre contribution and we will help you.
+
+Don't hesitate to share your plans before starting working on a contribution, we can help
+you to find the best approach.
+
+### Contributors License Agreements
+
+We at GNS3 are eager to work with you. For small changes — little bugfixes, correcting typos, and the like — please just submit pull requests to any of our projects. For larger changes, though, we have to ask you to jump through a little hoop.
+
+In particular, in order for us to accept any major patches from you, you will have to electronically sign a statement that indicates two things:
+
+- You are willingly licensing your contributions under the terms of the open source license of the project that you’re contributing to.
+- You are legally able to license your contributions as stated.
+
+The reason we do this is to ensure, to the extent possible, that we don’t “taint” the projects we manage with contributions that turn out to be improper. This protects everyone who wants to use the projects, including you!
+
+More information there: https://github.com/GNS3/cla
+
+### Pull requests
+
+Creating a pull request is the easiest way to contribute code. Do not hesitate to create one early when  contributing for new feature in order to get our feedback.

MANIFEST.in

@@ -4,7 +4,7 @@ include INSTALL
 include LICENSE
 include MANIFEST.in
 include tox.ini
-recursive-exclude tests *
+recursive-include tests *
 recursive-exclude docs *
 recursive-include gns3server *
 recursive-exclude * __pycache__

README.rst

@@ -26,6 +26,12 @@ unstable
 ********
 *Never* use this branch for production. Major new features pull requests goes here. 
 
+Linux
+-----
+
+GNS3 is perhaps packaged for your distribution:
+* Gentoo: https://packages.gentoo.org/package/net-misc/gns3-server
+
 
 Linux (Debian based)
 --------------------
@@ -62,11 +68,34 @@ To run tests use:
 
    py.test -v
 
+
+Run as daemon 
+***************
+
+You will found init sample script for various systems
+inside the init directory.
+
+upstart
+~~~~~~~
+
+For ubuntu < 15.04
+
+You need to copy init/gns3.conf.upstart to /etc/init/gns3.conf
+
+.. code:: bash
+
+    sudo chown root /etc/init/gns3.conf
+    sudo service gns3 start
+
+
 Windows
 -------
 
 Please use our all-in-one installer.
 
+If you install it via source you need to install also:
+https://sourceforge.net/projects/pywin32/
+
 Mac OS X
 --------
 
@@ -84,4 +113,17 @@ and homebrew: http://brew.sh/.
    python3 setup.py install
    gns3server
 
+Running tests
+*************
 
+Just run:
+
+.. code:: bash
+
+    py.test -vv
+
+If you want test coverage:
+
+.. code:: bash
+
+    py.test --cov=gns3server

dev-requirements.txt

@@ -6,4 +6,3 @@ pep8==1.5.7
 pytest-timeout
 pytest-capturelog
 pytest-cov
-python-coveralls

docs/index.rst

@@ -2,8 +2,8 @@ Welcome to API documentation!
 ======================================
 
 .. WARNING::
-   The API is not stable, feel free to send comment on GNS3 Jungle
-   https://community.gns3.com/
+   The API is not stable, feel free to post comments on our website
+   https://gns3.com/
 
 .. toctree::
     general

gns3server.bat

@@ -0,0 +1,2 @@
+SET PYTHONPATH=.
+python.exe gns3server/main.py --debug --local

gns3server/config.py

@@ -92,7 +92,7 @@ class Config(object):
 
     def clear(self):
         """Restart with a clean config"""
-        self._config = configparser.ConfigParser()
+        self._config = configparser.RawConfigParser()
         # Override config from command line even if we modify the config file and live reload it.
         self._override_config = {}
 
@@ -135,7 +135,11 @@ class Config(object):
         Read the configuration files.
         """
 
-        parsed_files = self._config.read(self._files)
+        try:
+            parsed_files = self._config.read(self._files, encoding="utf-8")
+        except configparser.Error as e:
+            log.error("Can't parse configuration file: %s", str(e))
+            return
         if not parsed_files:
             log.warning("No configuration file could be found or read")
         else:

gns3server/crash_report.py

@@ -20,6 +20,7 @@ import sys
 import struct
 import platform
 
+
 try:
     import raven
     RAVEN_AVAILABLE = True
@@ -34,13 +35,23 @@ import logging
 log = logging.getLogger(__name__)
 
 
+# Dev build
+if __version__[4] != 0:
+    import faulthandler
+
+    # Display a traceback in case of segfault crash. Usefull when frozen
+    # Not enabled by default for security reason
+    log.info("Enable catching segfault")
+    faulthandler.enable()
+
+
 class CrashReport:
 
     """
     Report crash to a third party service
     """
 
-    DSN = "sync+https://90fe04368a3e4109b584a116ee03ca87:268ef86c65cf4e8fa41d4c7fb1c70b72@app.getsentry.com/38482"
+    DSN = "sync+https://e47fab7019fe4fb3b699609bb9f4fefc:d1d3008a6c8a443ba34f336434c2da34@app.getsentry.com/38482"
     if hasattr(sys, "frozen"):
         cacert = os.path.join(os.getcwd(), "cacert.pem")
         if os.path.isfile(cacert):
@@ -55,6 +66,9 @@ class CrashReport:
     def capture_exception(self, request=None):
         if not RAVEN_AVAILABLE:
             return
+        if os.path.exists(".git"):
+            log.warning("A .git directory exist crash report is turn off for developers")
+            return
         server_config = Config.instance().get_section_config("Server")
         if server_config.getboolean("report_errors"):
             if self._client is None:

gns3server/handlers/__init__.py

@@ -31,4 +31,6 @@ from gns3server.handlers.upload_handler import UploadHandler
 from gns3server.handlers.index_handler import IndexHandler
 
 if sys.platform.startswith("linux") or hasattr(sys, "_called_from_test") or os.environ.get("PYTEST_BUILD_DOCUMENTATION") == "1":
-    from gns3server.handlers.api.iou_handler import IOUHandler
+    # IOU runs only on Linux but testsuite work on UNIX platform
+    if not sys.platform.startswith("win"):
+        from gns3server.handlers.api.iou_handler import IOUHandler

gns3server/handlers/api/dynamips_device_handler.py

@@ -181,10 +181,8 @@ class DynamipsDeviceHandler:
         dynamips_manager = Dynamips.instance()
         device = dynamips_manager.get_device(request.match_info["device_id"], project_id=request.match_info["project_id"])
         port_number = int(request.match_info["port_number"])
-        if asyncio.iscoroutinefunction(device.remove_nio):
-            yield from device.remove_nio(port_number)
-        else:
-            device.remove_nio(port_number)
+        nio = yield from device.remove_nio(port_number)
+        yield from nio.delete()
         response.set_status(204)
 
     @Route.post(

gns3server/handlers/api/dynamips_vm_handler.py

@@ -16,16 +16,18 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
+import sys
 import base64
 
 from ...web.route import Route
+from ...schemas.nio import NIO_SCHEMA
 from ...schemas.dynamips_vm import VM_CREATE_SCHEMA
 from ...schemas.dynamips_vm import VM_UPDATE_SCHEMA
 from ...schemas.dynamips_vm import VM_CAPTURE_SCHEMA
 from ...schemas.dynamips_vm import VM_OBJECT_SCHEMA
-from ...schemas.dynamips_vm import VM_NIO_SCHEMA
 from ...schemas.dynamips_vm import VM_CONFIGS_SCHEMA
 from ...modules.dynamips import Dynamips
+from ...modules.dynamips.dynamips_error import DynamipsError
 from ...modules.project_manager import ProjectManager
 
 DEFAULT_CHASSIS = {
@@ -256,8 +258,8 @@ class DynamipsVMHandler:
             404: "Instance doesn't exist"
         },
         description="Add a NIO to a Dynamips VM instance",
-        input=VM_NIO_SCHEMA,
-        output=VM_NIO_SCHEMA)
+        input=NIO_SCHEMA,
+        output=NIO_SCHEMA)
     def create_nio(request, response):
 
         dynamips_manager = Dynamips.instance()
@@ -290,7 +292,8 @@ class DynamipsVMHandler:
         vm = dynamips_manager.get_vm(request.match_info["vm_id"], project_id=request.match_info["project_id"])
         slot_number = int(request.match_info["adapter_number"])
         port_number = int(request.match_info["port_number"])
-        yield from vm.slot_remove_nio_binding(slot_number, port_number)
+        nio = yield from vm.slot_remove_nio_binding(slot_number, port_number)
+        yield from nio.delete()
         response.set_status(204)
 
     @Route.post(
@@ -315,6 +318,14 @@ class DynamipsVMHandler:
         slot_number = int(request.match_info["adapter_number"])
         port_number = int(request.match_info["port_number"])
         pcap_file_path = os.path.join(vm.project.capture_working_directory(), request.json["capture_file_name"])
+
+        if sys.platform.startswith('win'):
+            #FIXME: Dynamips (Cygwin actually) doesn't like non ascii paths on Windows
+            try:
+                pcap_file_path.encode('ascii')
+            except UnicodeEncodeError:
+                raise DynamipsError('The capture file path "{}" must only contain ASCII (English) characters'.format(pcap_file_path))
+
         yield from vm.start_capture(slot_number, port_number, pcap_file_path, request.json["data_link_type"])
         response.json({"pcap_file_path": pcap_file_path})
 
@@ -357,13 +368,41 @@ class DynamipsVMHandler:
                                      project_id=request.match_info["project_id"])
 
         startup_config_base64, private_config_base64 = yield from vm.extract_config()
+        module_workdir = vm.project.module_working_directory(dynamips_manager.module_name.lower())
         result = {}
         if startup_config_base64:
-            startup_config_content = base64.b64decode(startup_config_base64).decode(errors='replace')
+            startup_config_content = base64.b64decode(startup_config_base64).decode("utf-8", errors='replace')
             result["startup_config_content"] = startup_config_content
+        else:
+            # nvram doesn't contain anything if the router has not been started at least once
+            # in this case just use the startup-config file
+            if vm.startup_config:
+                startup_config_path = os.path.join(module_workdir, vm.startup_config)
+                if os.path.isfile(startup_config_path):
+                    try:
+                        with open(startup_config_path, "rb") as f:
+                            content = f.read().decode("utf-8", errors='replace')
+                            if content:
+                                result["startup_config_content"] = content
+                    except OSError as e:
+                        raise DynamipsError("Could not read the startup-config {}: {}".format(startup_config_path, e))
+
         if private_config_base64:
-            private_config_content = base64.b64decode(private_config_base64).decode(errors='replace')
+            private_config_content = base64.b64decode(private_config_base64).decode("utf-8", errors='replace')
             result["private_config_content"] = private_config_content
+        else:
+            # nvram doesn't contain anything if the router has not been started at least once
+            # in this case just use the private-config file
+            if vm.private_config:
+                private_config_path = os.path.join(module_workdir, vm.private_config)
+                if os.path.isfile(private_config_path):
+                    try:
+                        with open(private_config_path, "rb") as f:
+                            content = f.read().decode("utf-8", errors='replace')
+                            if content:
+                                result["private_config_content"] = content
+                    except OSError as e:
+                        raise DynamipsError("Could not read the private-config {}: {}".format(private_config_path, e))
 
         response.set_status(200)
         response.json(result)

gns3server/handlers/api/iou_handler.py

@@ -19,11 +19,10 @@ import os
 from aiohttp.web import HTTPConflict
 
 from ...web.route import Route
-from ...modules.port_manager import PortManager
+from ...schemas.nio import NIO_SCHEMA
 from ...schemas.iou import IOU_CREATE_SCHEMA
 from ...schemas.iou import IOU_UPDATE_SCHEMA
 from ...schemas.iou import IOU_OBJECT_SCHEMA
-from ...schemas.iou import IOU_NIO_SCHEMA
 from ...schemas.iou import IOU_CAPTURE_SCHEMA
 from ...schemas.iou import IOU_INITIAL_CONFIG_SCHEMA
 from ...modules.iou import IOU
@@ -52,20 +51,16 @@ class IOUHandler:
     def create(request, response):
 
         iou = IOU.instance()
-        vm = yield from iou.create_vm(request.json["name"],
+        vm = yield from iou.create_vm(request.json.pop("name"),
                                       request.match_info["project_id"],
                                       request.json.get("vm_id"),
-                                      console=request.json.get("console"),
-                                      serial_adapters=request.json.get("serial_adapters"),
-                                      ethernet_adapters=request.json.get("ethernet_adapters"),
-                                      ram=request.json.get("ram"),
-                                      nvram=request.json.get("nvram"),
-                                      use_default_iou_values=request.json.get("use_default_iou_values"),
-                                      l1_keepalives=request.json.get("l1_keepalives"),
-                                      initial_config=request.json.get("initial_config_content"),
-                                      iourc_content=request.json.get("iourc_content")
-                                      )
-        vm.path = request.json.get("path", vm.path)
+                                      console=request.json.get("console"))
+
+        for name, value in request.json.items():
+            if hasattr(vm, name) and getattr(vm, name) != value:
+                if name == "initial_config_content" and (vm.initial_config_content and len(vm.initial_config_content) > 0):
+                    continue
+                setattr(vm, name, value)
         response.set_status(201)
         response.json(vm)
 
@@ -109,18 +104,10 @@ class IOUHandler:
 
         iou_manager = IOU.instance()
         vm = iou_manager.get_vm(request.match_info["vm_id"], project_id=request.match_info["project_id"])
-        vm.name = request.json.get("name", vm.name)
-        vm.console = request.json.get("console", vm.console)
-        vm.path = request.json.get("path", vm.path)
-        vm.ethernet_adapters = request.json.get("ethernet_adapters", vm.ethernet_adapters)
-        vm.serial_adapters = request.json.get("serial_adapters", vm.serial_adapters)
-        vm.ram = request.json.get("ram", vm.ram)
-        vm.nvram = request.json.get("nvram", vm.nvram)
-        vm.use_default_iou_values = request.json.get("use_default_iou_values", vm.use_default_iou_values)
-        vm.l1_keepalives = request.json.get("l1_keepalives", vm.l1_keepalives)
-        vm.initial_config = request.json.get("initial_config_content", vm.initial_config)
-        vm.iourc_content = request.json.get("iourc_content", None)
 
+        for name, value in request.json.items():
+            if hasattr(vm, name) and getattr(vm, name) != value:
+                setattr(vm, name, value)
         response.json(vm)
 
     @classmethod
@@ -215,12 +202,15 @@ class IOUHandler:
             404: "Instance doesn't exist"
         },
         description="Add a NIO to a IOU instance",
-        input=IOU_NIO_SCHEMA,
-        output=IOU_NIO_SCHEMA)
+        input=NIO_SCHEMA,
+        output=NIO_SCHEMA)
     def create_nio(request, response):
 
         iou_manager = IOU.instance()
         vm = iou_manager.get_vm(request.match_info["vm_id"], project_id=request.match_info["project_id"])
+        nio_type = request.json["type"]
+        if nio_type not in ("nio_udp", "nio_tap", "nio_generic_ethernet"):
+            raise HTTPConflict(text="NIO of type {} is not supported".format(nio_type))
         nio = iou_manager.create_nio(vm.iouyap_path, request.json)
         vm.adapter_add_nio_binding(int(request.match_info["adapter_number"]), int(request.match_info["port_number"]), nio)
         response.set_status(201)
@@ -273,7 +263,7 @@ class IOUHandler:
         pcap_file_path = os.path.join(vm.project.capture_working_directory(), request.json["capture_file_name"])
 
         if not vm.is_running():
-            raise HTTPConflict(text="You can't capture the traffic on a non started VM")
+            raise HTTPConflict(text="Cannot capture traffic on a non started VM")
         yield from vm.start_capture(adapter_number, port_number, pcap_file_path, request.json["data_link_type"])
         response.json({"pcap_file_path": str(pcap_file_path)})
 
@@ -298,7 +288,7 @@ class IOUHandler:
         vm = iou_manager.get_vm(request.match_info["vm_id"], project_id=request.match_info["project_id"])
 
         if not vm.is_running():
-            raise HTTPConflict(text="You can't capture the traffic on a non started VM")
+            raise HTTPConflict(text="Cannot capture traffic on a non started VM")
 
         adapter_number = int(request.match_info["adapter_number"])
         port_number = int(request.match_info["port_number"])
@@ -320,4 +310,4 @@ class IOUHandler:
         vm = iou_manager.get_vm(request.match_info["vm_id"],
                                 project_id=request.match_info["project_id"])
         response.set_status(200)
-        response.json({"content": vm.initial_config})
+        response.json({"content": vm.initial_config_content})

gns3server/handlers/api/project_handler.py

@@ -81,13 +81,16 @@ class ProjectHandler:
 
         pm = ProjectManager.instance()
         project = pm.get_project(request.match_info["project_id"])
-        project.temporary = request.json.get("temporary", project.temporary)
         project.name = request.json.get("name", project.name)
         project_path = request.json.get("path", project.path)
         if project_path != project.path:
+            old_path = project.path
             project.path = project_path
             for module in MODULES:
                 yield from module.instance().project_moved(project)
+            yield from project.clean_old_path(old_path)
+        # Very important we need to remove temporary flag after moving the project
+        project.temporary = request.json.get("temporary", project.temporary)
         response.json(project)
 
     @classmethod

gns3server/handlers/api/qemu_handler.py

@@ -15,11 +15,12 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+from aiohttp.web import HTTPConflict
 from ...web.route import Route
+from ...schemas.nio import NIO_SCHEMA
 from ...schemas.qemu import QEMU_CREATE_SCHEMA
 from ...schemas.qemu import QEMU_UPDATE_SCHEMA
 from ...schemas.qemu import QEMU_OBJECT_SCHEMA
-from ...schemas.qemu import QEMU_NIO_SCHEMA
 from ...schemas.qemu import QEMU_BINARY_LIST_SCHEMA
 from ...modules.qemu import Qemu
 
@@ -41,24 +42,21 @@ class QEMUHandler:
             400: "Invalid request",
             409: "Conflict"
         },
-        description="Create a new Qemu.instance",
+        description="Create a new Qemu VM instance",
         input=QEMU_CREATE_SCHEMA,
         output=QEMU_OBJECT_SCHEMA)
     def create(request, response):
 
         qemu = Qemu.instance()
-        vm = yield from qemu.create_vm(request.json["name"],
+        vm = yield from qemu.create_vm(request.json.pop("name"),
                                        request.match_info["project_id"],
                                        request.json.get("vm_id"),
                                        qemu_path=request.json.get("qemu_path"),
                                        console=request.json.get("console"))
 
-        # Clear already used keys
-        map(request.json.__delitem__, ["name", "project_id", "vm_id",
-                                       "qemu_path", "console"])
-
-        for field in request.json:
-            setattr(vm, field, request.json[field])
+        for name, value in request.json.items():
+            if hasattr(vm, name) and getattr(vm, name) != value:
+                setattr(vm, name, value)
 
         response.set_status(201)
         response.json(vm)
@@ -75,7 +73,7 @@ class QEMUHandler:
             400: "Invalid request",
             404: "Instance doesn't exist"
         },
-        description="Get a Qemu.instance",
+        description="Get a Qemu VM instance",
         output=QEMU_OBJECT_SCHEMA)
     def show(request, response):
 
@@ -96,15 +94,17 @@ class QEMUHandler:
             404: "Instance doesn't exist",
             409: "Conflict"
         },
-        description="Update a Qemu.instance",
+        description="Update a Qemu VM instance",
         input=QEMU_UPDATE_SCHEMA,
         output=QEMU_OBJECT_SCHEMA)
     def update(request, response):
 
         qemu_manager = Qemu.instance()
         vm = qemu_manager.get_vm(request.match_info["vm_id"], project_id=request.match_info["project_id"])
-        for field in request.json:
-            setattr(vm, field, request.json[field])
+
+        for name, value in request.json.items():
+            if hasattr(vm, name) and getattr(vm, name) != value:
+                setattr(vm, name, value)
 
         response.json(vm)
 
@@ -120,7 +120,7 @@ class QEMUHandler:
             400: "Invalid request",
             404: "Instance doesn't exist"
         },
-        description="Delete a Qemu.instance")
+        description="Delete a Qemu VM instance")
     def delete(request, response):
 
         yield from Qemu.instance().delete_vm(request.match_info["vm_id"])
@@ -138,7 +138,7 @@ class QEMUHandler:
             400: "Invalid request",
             404: "Instance doesn't exist"
         },
-        description="Start a Qemu.instance")
+        description="Start a Qemu VM instance")
     def start(request, response):
 
         qemu_manager = Qemu.instance()
@@ -158,7 +158,7 @@ class QEMUHandler:
             400: "Invalid request",
             404: "Instance doesn't exist"
         },
-        description="Stop a Qemu.instance")
+        description="Stop a Qemu VM instance")
     def stop(request, response):
 
         qemu_manager = Qemu.instance()
@@ -178,7 +178,7 @@ class QEMUHandler:
             400: "Invalid request",
             404: "Instance doesn't exist"
         },
-        description="Reload a Qemu.instance")
+        description="Reload a Qemu VM instance")
     def reload(request, response):
 
         qemu_manager = Qemu.instance()
@@ -198,7 +198,7 @@ class QEMUHandler:
             400: "Invalid request",
             404: "Instance doesn't exist"
         },
-        description="Suspend a Qemu.instance")
+        description="Suspend a Qemu VM instance")
     def suspend(request, response):
 
         qemu_manager = Qemu.instance()
@@ -218,7 +218,7 @@ class QEMUHandler:
             400: "Invalid request",
             404: "Instance doesn't exist"
         },
-        description="Resume a Qemu.instance")
+        description="Resume a Qemu VM instance")
     def resume(request, response):
 
         qemu_manager = Qemu.instance()
@@ -239,13 +239,16 @@ class QEMUHandler:
             400: "Invalid request",
             404: "Instance doesn't exist"
         },
-        description="Add a NIO to a Qemu.instance",
-        input=QEMU_NIO_SCHEMA,
-        output=QEMU_NIO_SCHEMA)
+        description="Add a NIO to a Qemu VM instance",
+        input=NIO_SCHEMA,
+        output=NIO_SCHEMA)
     def create_nio(request, response):
 
         qemu_manager = Qemu.instance()
         vm = qemu_manager.get_vm(request.match_info["vm_id"], project_id=request.match_info["project_id"])
+        nio_type = request.json["type"]
+        if nio_type not in ("nio_udp", "nio_tap", "nio_nat"):
+            raise HTTPConflict(text="NIO of type {} is not supported".format(nio_type))
         nio = qemu_manager.create_nio(vm.qemu_path, request.json)
         yield from vm.adapter_add_nio_binding(int(request.match_info["adapter_number"]), nio)
         response.set_status(201)
@@ -265,7 +268,7 @@ class QEMUHandler:
             400: "Invalid request",
             404: "Instance doesn't exist"
         },
-        description="Remove a NIO from a Qemu.instance")
+        description="Remove a NIO from a Qemu VM instance")
     def delete_nio(request, response):
 
         qemu_manager = Qemu.instance()

gns3server/handlers/api/virtualbox_handler.py

@@ -16,10 +16,11 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
+from aiohttp.web import HTTPConflict
 from ...web.route import Route
+from ...schemas.nio import NIO_SCHEMA
 from ...schemas.virtualbox import VBOX_CREATE_SCHEMA
 from ...schemas.virtualbox import VBOX_UPDATE_SCHEMA
-from ...schemas.virtualbox import VBOX_NIO_SCHEMA
 from ...schemas.virtualbox import VBOX_CAPTURE_SCHEMA
 from ...schemas.virtualbox import VBOX_OBJECT_SCHEMA
 from ...modules.virtualbox import VirtualBox
@@ -79,8 +80,9 @@ class VirtualBoxHandler:
                 yield from vm.set_ram(ram)
 
         for name, value in request.json.items():
-            if hasattr(vm, name) and getattr(vm, name) != value:
-                setattr(vm, name, value)
+            if name != "vm_id":
+                if hasattr(vm, name) and getattr(vm, name) != value:
+                    setattr(vm, name, value)
 
         response.set_status(201)
         response.json(vm)
@@ -285,12 +287,15 @@ class VirtualBoxHandler:
             404: "Instance doesn't exist"
         },
         description="Add a NIO to a VirtualBox VM instance",
-        input=VBOX_NIO_SCHEMA,
-        output=VBOX_NIO_SCHEMA)
+        input=NIO_SCHEMA,
+        output=NIO_SCHEMA)
     def create_nio(request, response):
 
         vbox_manager = VirtualBox.instance()
         vm = vbox_manager.get_vm(request.match_info["vm_id"], project_id=request.match_info["project_id"])
+        nio_type = request.json["type"]
+        if nio_type not in ("nio_udp", "nio_nat"):
+            raise HTTPConflict(text="NIO of type {} is not supported".format(nio_type))
         nio = vbox_manager.create_nio(vbox_manager.vboxmanage_path, request.json)
         yield from vm.adapter_add_nio_binding(int(request.match_info["adapter_number"]), nio)
         response.set_status(201)
@@ -339,7 +344,7 @@ class VirtualBoxHandler:
         vm = vbox_manager.get_vm(request.match_info["vm_id"], project_id=request.match_info["project_id"])
         adapter_number = int(request.match_info["adapter_number"])
         pcap_file_path = os.path.join(vm.project.capture_working_directory(), request.json["capture_file_name"])
-        vm.start_capture(adapter_number, pcap_file_path)
+        yield from vm.start_capture(adapter_number, pcap_file_path)
         response.json({"pcap_file_path": pcap_file_path})
 
     @Route.post(

gns3server/handlers/api/vpcs_handler.py

@@ -15,11 +15,12 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+from aiohttp.web import HTTPConflict
 from ...web.route import Route
+from ...schemas.nio import NIO_SCHEMA
 from ...schemas.vpcs import VPCS_CREATE_SCHEMA
 from ...schemas.vpcs import VPCS_UPDATE_SCHEMA
 from ...schemas.vpcs import VPCS_OBJECT_SCHEMA
-from ...schemas.vpcs import VPCS_NIO_SCHEMA
 from ...modules.vpcs import VPCS
 
 
@@ -191,12 +192,15 @@ class VPCSHandler:
             404: "Instance doesn't exist"
         },
         description="Add a NIO to a VPCS instance",
-        input=VPCS_NIO_SCHEMA,
-        output=VPCS_NIO_SCHEMA)
+        input=NIO_SCHEMA,
+        output=NIO_SCHEMA)
     def create_nio(request, response):
 
         vpcs_manager = VPCS.instance()
         vm = vpcs_manager.get_vm(request.match_info["vm_id"], project_id=request.match_info["project_id"])
+        nio_type = request.json["type"]
+        if nio_type not in ("nio_udp", "nio_tap"):
+            raise HTTPConflict(text="NIO of type {} is not supported".format(nio_type))
         nio = vpcs_manager.create_nio(vm.vpcs_path, request.json)
         vm.port_add_nio_binding(int(request.match_info["port_number"]), nio)
         response.set_status(201)

gns3server/handlers/upload_handler.py

@@ -18,6 +18,9 @@
 import os
 import aiohttp
 import stat
+import io
+import tarfile
+import asyncio
 
 from ..config import Config
 from ..web.route import Route
@@ -36,8 +39,9 @@ class UploadHandler:
         try:
             for root, _, files in os.walk(UploadHandler.image_directory()):
                 for filename in files:
-                    image_file = os.path.join(root, filename)
-                    uploaded_files.append(image_file)
+                    if not filename.startswith("."):
+                        image_file = os.path.join(root, filename)
+                        uploaded_files.append(image_file)
         except OSError:
             pass
         iourc_path = os.path.join(os.path.expanduser("~/"), ".iourc")
@@ -58,29 +62,97 @@ class UploadHandler:
             response.redirect("/upload")
             return
 
-        if data["type"] not in ["IOU", "IOURC", "QEMU", "IOS"]:
-            raise aiohttp.web.HTTPForbidden("You are not authorized to upload this kind of image {}".format(data["type"]))
+        if data["type"] not in ["IOU", "IOURC", "QEMU", "IOS", "IMAGES", "PROJECTS"]:
+            raise aiohttp.web.HTTPForbidden(text="You are not authorized to upload this kind of image {}".format(data["type"]))
 
-        if data["type"] == "IOURC":
-            destination_dir = os.path.expanduser("~/")
-            destination_path = os.path.join(destination_dir, ".iourc")
-        else:
-            destination_dir = os.path.join(UploadHandler.image_directory(), data["type"])
-            destination_path = os.path.join(destination_dir, data["file"].filename)
         try:
-            os.makedirs(destination_dir, exist_ok=True)
-            with open(destination_path, "wb+") as f:
-                chunk = data["file"].file.read()
-                f.write(chunk)
-            st = os.stat(destination_path)
-            os.chmod(destination_path, st.st_mode | stat.S_IXUSR)
+            if data["type"] == "IMAGES":
+                UploadHandler._restore_directory(data["file"], UploadHandler.image_directory())
+            elif data["type"] == "PROJECTS":
+                UploadHandler._restore_directory(data["file"], UploadHandler.project_directory())
+            else:
+                if data["type"] == "IOURC":
+                    destination_dir = os.path.expanduser("~/")
+                    destination_path = os.path.join(destination_dir, ".iourc")
+                else:
+                    destination_dir = os.path.join(UploadHandler.image_directory(), data["type"])
+                    destination_path = os.path.join(destination_dir, data["file"].filename)
+                os.makedirs(destination_dir, exist_ok=True)
+                with open(destination_path, "wb+") as f:
+                    chunk = data["file"].file.read()
+                    f.write(chunk)
+                st = os.stat(destination_path)
+                os.chmod(destination_path, st.st_mode | stat.S_IXUSR)
         except OSError as e:
             response.html("Could not upload file: {}".format(e))
             response.set_status(200)
             return
         response.redirect("/upload")
 
+    @classmethod
+    @Route.get(
+        r"/backup/images.tar",
+        description="Backup GNS3 images",
+        api_version=None
+    )
+    def backup_images(request, response):
+        yield from UploadHandler._backup_directory(request, response, UploadHandler.image_directory())
+
+    @classmethod
+    @Route.get(
+        r"/backup/projects.tar",
+        description="Backup GNS3 projects",
+        api_version=None
+    )
+    def backup_images(request, response):
+        yield from UploadHandler._backup_directory(request, response, UploadHandler.project_directory())
+
+    @staticmethod
+    def _restore_directory(file, directory):
+        """
+        Extract from HTTP stream the content of a tar
+        """
+        destination_path = os.path.join(directory, "archive.tar")
+        os.makedirs(directory, exist_ok=True)
+        with open(destination_path, "wb+") as f:
+            chunk = file.file.read()
+            f.write(chunk)
+        t = tarfile.open(destination_path)
+        t.extractall(directory)
+        t.close()
+        os.remove(destination_path)
+
+    @staticmethod
+    @asyncio.coroutine
+    def _backup_directory(request, response, directory):
+        """
+        Return a tar archive from a directory
+        """
+        response.content_type = 'application/x-gtar'
+        response.set_status(200)
+        response.enable_chunked_encoding()
+        # Very important: do not send a content length otherwise QT close the connection but curl can consume the Feed
+        response.content_length = None
+        response.start(request)
+
+        buffer = io.BytesIO()
+        with tarfile.open('arch.tar', 'w', fileobj=buffer) as tar:
+            for root, dirs, files in os.walk(directory):
+                for file in files:
+                    path = os.path.join(root, file)
+                    tar.add(os.path.join(root, file), arcname=os.path.relpath(path, directory))
+                    response.write(buffer.getvalue())
+                    yield from response.drain()
+                    buffer.truncate(0)
+                    buffer.seek(0)
+        yield from response.write_eof()
+
     @staticmethod
     def image_directory():
         server_config = Config.instance().get_section_config("Server")
         return os.path.expanduser(server_config.get("images_path", "~/GNS3/images"))
+
+    @staticmethod
+    def project_directory():
+        server_config = Config.instance().get_section_config("Server")
+        return os.path.expanduser(server_config.get("projects_path", "~/GNS3/projects"))

gns3server/main.py

@@ -16,6 +16,16 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+
+# WARNING
+# Due to buggy user machines we choose to put this as the first loading modules
+# otherwise the egg cache is initialized in his standard location and
+# if is not writetable the application crash. It's the user fault
+# because one day the user as used sudo to run an egg and break his
+# filesystem permissions, but it's a common mistake.
+import gns3server.utils.get_resource
+
+
 import os
 import datetime
 import sys
@@ -92,6 +102,7 @@ def parse_arguments(argv, config):
         "quiet": config.getboolean("quiet", False),
         "debug": config.getboolean("debug", False),
         "live": config.getboolean("live", False),
+        "logfile": config.getboolean("logfile", ""),
     }
 
     parser = argparse.ArgumentParser(description="GNS3 server version {}".format(__version__))
@@ -109,6 +120,7 @@ def parse_arguments(argv, config):
     parser.add_argument("-d", "--debug", action="store_true", help="show debug logs")
     parser.add_argument("--live", action="store_true", help="enable code live reload")
     parser.add_argument("--shell", action="store_true", help="start a shell inside the server (debugging purpose only you need to install ptpython before)")
+    parser.add_argument("--log", help="send output to logfile instead of console")
 
     return parser.parse_args(argv)
 
@@ -141,7 +153,7 @@ def main():
     if args.debug:
         level = logging.DEBUG
 
-    user_log = init_logger(level, quiet=args.quiet)
+    user_log = init_logger(level, logfile=args.log, quiet=args.quiet)
     user_log.info("GNS3 server version {}".format(__version__))
     current_year = datetime.date.today().year
     user_log.info("Copyright (c) 2007-{} GNS3 Technologies Inc.".format(current_year))
@@ -174,11 +186,21 @@ def main():
     Project.clean_project_directory()
 
     CrashReport.instance()
-    host = server_config["host"]
+
+    try:
+        host = server_config["host"].encode("idna").decode()
+    except UnicodeError:
+        log.critical("Invalid hostname %s", server_config["host"])
+        return
+
     port = int(server_config["port"])
     server = Server.instance(host, port)
     try:
         server.run()
+    except OSError as e:
+        # This is to ignore OSError: [WinError 0] The operation completed successfully exception on Windows.
+        if not sys.platform.startswith("win") and not e.winerror == 0:
+            raise
     except Exception as e:
         log.critical("Critical error while running the server: {}".format(e), exc_info=1)
         CrashReport.instance().capture_exception()

gns3server/modules/__init__.py

@@ -25,6 +25,8 @@ from .qemu import Qemu
 MODULES = [VPCS, VirtualBox, Dynamips, Qemu]
 
 if sys.platform.startswith("linux") or hasattr(sys, "_called_from_test") or os.environ.get("PYTEST_BUILD_DOCUMENTATION") == "1":
-    # IOU runs only on Linux
-    from .iou import IOU
-    MODULES.append(IOU)
+
+    # IOU runs only on Linux but testsuite work on UNIX platform
+    if not sys.platform.startswith("win"):
+        from .iou import IOU
+        MODULES.append(IOU)

gns3server/modules/base_manager.py

@@ -34,6 +34,7 @@ from .project_manager import ProjectManager
 
 from .nios.nio_udp import NIOUDP
 from .nios.nio_tap import NIOTAP
+from .nios.nio_nat import NIONAT
 from .nios.nio_generic_ethernet import NIOGenericEthernet
 
 
@@ -112,7 +113,7 @@ class BaseManager:
             for future in done:
                 try:
                     future.result()
-                except Exception as e:
+                except (Exception, GeneratorExit) as e:
                     log.error("Could not close VM {}".format(e), exc_info=1)
                     continue
 
@@ -236,7 +237,7 @@ class BaseManager:
     @asyncio.coroutine
     def close_vm(self, vm_id):
         """
-        Delete a VM
+        Close a VM
 
         :param vm_id: VM identifier
 
@@ -304,7 +305,8 @@ class BaseManager:
 
         vm = yield from self.close_vm(vm_id)
         vm.project.mark_vm_for_destruction(vm)
-        del self._vms[vm.id]
+        if vm.id in self._vms:
+            del self._vms[vm.id]
         return vm
 
     @staticmethod
@@ -364,10 +366,63 @@ class BaseManager:
             nio = NIOUDP(lport, rhost, rport)
         elif nio_settings["type"] == "nio_tap":
             tap_device = nio_settings["tap_device"]
-            if not self._has_privileged_access(executable):
-                raise aiohttp.web.HTTPForbidden(text="{} has no privileged access to {}.".format(executable, tap_device))
+            #FIXME: check for permissions on tap device
+            #if not self._has_privileged_access(executable):
+            #    raise aiohttp.web.HTTPForbidden(text="{} has no privileged access to {}.".format(executable, tap_device))
             nio = NIOTAP(tap_device)
         elif nio_settings["type"] == "nio_generic_ethernet":
             nio = NIOGenericEthernet(nio_settings["ethernet_device"])
+        elif nio_settings["type"] == "nio_nat":
+            nio = NIONAT()
         assert nio is not None
         return nio
+
+    def get_abs_image_path(self, path):
+        """
+        Get the absolute path of an image
+
+        :param path: file path
+        :return: file path
+        """
+
+        if not path:
+            return ""
+        img_directory = self.get_images_directory()
+        if not os.path.isabs(path):
+            s = os.path.split(path)
+            path = os.path.normpath(os.path.join(img_directory, *s))
+
+            # Compatibility with old topologies we look in parent directory
+            # We look at first in new location
+            if not os.path.exists(path):
+                old_path = os.path.normpath(os.path.join(img_directory, '..', *s))
+                if os.path.exists(old_path):
+                    return old_path
+
+            return path
+        return path
+
+    def get_relative_image_path(self, path):
+        """
+        Get a path relative to images directory path
+        or an abspath if the path is not located inside
+        image directory
+
+        :param path: file path
+        :return: file path
+        """
+
+        if not path:
+            return ""
+        img_directory = self.get_images_directory()
+        path = self.get_abs_image_path(path)
+        if os.path.dirname(path) == img_directory:
+            return os.path.basename(path)
+        return path
+
+    def get_images_directory(self):
+        """
+        Get the image directory on disk
+        """
+
+        raise NotImplementedError

gns3server/modules/base_vm.py

@@ -61,7 +61,6 @@ class BaseVM:
 
     def __del__(self):
 
-        self.close()
         if self._temporary_directory is not None:
             if os.path.exists(self._temporary_directory):
                 shutil.rmtree(self._temporary_directory, ignore_errors=True)

gns3server/modules/dynamips/__init__.py

@@ -27,13 +27,14 @@ import socket
 import time
 import asyncio
 import tempfile
-import glob
 import logging
+import glob
 
 log = logging.getLogger(__name__)
 
 from gns3server.utils.interfaces import get_windows_interfaces
 from gns3server.utils.asyncio import wait_run_in_executor
+from gns3server.utils.glob import glob_escape
 from pkg_resources import parse_version
 from uuid import UUID, uuid4
 from ..base_manager import BaseManager
@@ -115,6 +116,43 @@ class Dynamips(BaseManager):
         self._devices = {}
         self._ghost_files = set()
         self._dynamips_path = None
+        self._dynamips_ids = {}
+
+    def get_dynamips_id(self, project_id):
+        """
+        :param project_id: UUID of the project
+        :returns: a free dynamips id
+        """
+        self._dynamips_ids.setdefault(project_id, set())
+        dynamips_id = 0
+        for dynamips_id in range(1, 4097):
+            if dynamips_id not in self._dynamips_ids[project_id]:
+                self._dynamips_ids[project_id].add(dynamips_id)
+                return dynamips_id
+        raise DynamipsError("Maximum number of Dynamips instances reached")
+
+    def take_dynamips_id(self, project_id, dynamips_id):
+        """
+        Reserve a dynamips id or raise an error
+
+        :param project_id: UUID of the project
+        :param dynamips_id: Asked id
+        """
+        self._dynamips_ids.setdefault(project_id, set())
+        if dynamips_id in self._dynamips_ids[project_id]:
+            raise DynamipsError("Dynamips identifier {} is already used by another router".format(dynamips_id))
+        self._dynamips_ids[project_id].add(dynamips_id)
+
+    def release_dynamips_id(self, project_id, dynamips_id):
+        """
+        A dynamips id can be reused by another vm
+
+        :param project_id: UUID of the project
+        :param dynamips_id: Asked id
+        """
+        self._dynamips_ids.setdefault(project_id, set())
+        if dynamips_id in self._dynamips_ids[project_id]:
+            self._dynamips_ids[project_id].remove(dynamips_id)
 
     @asyncio.coroutine
     def unload(self):
@@ -130,7 +168,7 @@ class Dynamips(BaseManager):
             for future in done:
                 try:
                     future.result()
-                except Exception as e:
+                except (Exception, GeneratorExit) as e:
                     log.error("Could not stop device hypervisor {}".format(e), exc_info=1)
                     continue
 
@@ -154,7 +192,7 @@ class Dynamips(BaseManager):
             for future in done:
                 try:
                     future.result()
-                except Exception as e:
+                except (Exception, GeneratorExit) as e:
                     log.error("Could not delete device {}".format(e), exc_info=1)
 
     @asyncio.coroutine
@@ -164,15 +202,14 @@ class Dynamips(BaseManager):
 
         :param project: Project instance
         """
-
         yield from super().project_closed(project)
         # delete useless Dynamips files
         project_dir = project.module_working_path(self.module_name.lower())
-        files = glob.glob(os.path.join(project_dir, "*.ghost"))
-        files += glob.glob(os.path.join(project_dir, "*_lock"))
-        files += glob.glob(os.path.join(project_dir, "ilt_*"))
-        files += glob.glob(os.path.join(project_dir, "c[0-9][0-9][0-9][0-9]_*_rommon_vars"))
-        files += glob.glob(os.path.join(project_dir, "c[0-9][0-9][0-9][0-9]_*_ssa"))
+        files = glob.glob(os.path.join(glob_escape(project_dir), "*.ghost"))
+        files += glob.glob(os.path.join(glob_escape(project_dir), "*_lock"))
+        files += glob.glob(os.path.join(glob_escape(project_dir), "ilt_*"))
+        files += glob.glob(os.path.join(glob_escape(project_dir), "c[0-9][0-9][0-9][0-9]_i[0-9]*_rommon_vars"))
+        files += glob.glob(os.path.join(glob_escape(project_dir), "c[0-9][0-9][0-9][0-9]_i[0-9]*_log.txt"))
         for file in files:
             try:
                 log.debug("Deleting file {}".format(file))
@@ -183,6 +220,11 @@ class Dynamips(BaseManager):
                 log.warn("Could not delete file {}: {}".format(file, e))
                 continue
 
+        # Release the dynamips ids if we want to reload the same project
+        # later
+        if project.id in self._dynamips_ids:
+            del self._dynamips_ids[project.id]
+
     @asyncio.coroutine
     def project_moved(self, project):
         """
@@ -208,9 +250,13 @@ class Dynamips(BaseManager):
         """
 
         # save the configs when the project is committed
-        for vm in self._vms.values():
+        for vm in self._vms.copy().values():
             if vm.project.id == project.id:
-                yield from vm.save_configs()
+                try:
+                    yield from vm.save_configs()
+                except DynamipsError as e:
+                    log.warning(e)
+                    continue
 
     @property
     def dynamips_path(self):
@@ -353,7 +399,10 @@ class Dynamips(BaseManager):
         ghost_ios_support = self.config.get_section_config("Dynamips").getboolean("ghost_ios_support", True)
         if ghost_ios_support:
             with (yield from Dynamips._ghost_ios_lock):
-                yield from self._set_ghost_ios(vm)
+                try:
+                    yield from self._set_ghost_ios(vm)
+                except GeneratorExit:
+                    log.warning("Could not create ghost IOS image {} (GeneratorExit)".format(vm.name))
 
     @asyncio.coroutine
     def create_nio(self, node, nio_settings):
@@ -410,6 +459,8 @@ class Dynamips(BaseManager):
             nio = NIOVDE(node.hypervisor, control_file, local_file)
         elif nio_settings["type"] == "nio_null":
             nio = NIONull(node.hypervisor)
+        else:
+            raise aiohttp.web.HTTPConflict(text="NIO of type {} is not supported".format(nio_settings["type"]))
 
         yield from nio.create()
         return nio
@@ -471,31 +522,42 @@ class Dynamips(BaseManager):
                 if hasattr(vm, "set_{}".format(name)):
                     setter = getattr(vm, "set_{}".format(name))
                     yield from setter(value)
-
             elif name.startswith("slot") and value in ADAPTER_MATRIX:
                 slot_id = int(name[-1])
                 adapter_name = value
                 adapter = ADAPTER_MATRIX[adapter_name]()
-                if vm.slots[slot_id] and not isinstance(vm.slots[slot_id], type(adapter)):
-                    yield from vm.slot_remove_binding(slot_id)
-                if not isinstance(vm.slots[slot_id], type(adapter)):
-                    yield from vm.slot_add_binding(slot_id, adapter)
+                try:
+                    if vm.slots[slot_id] and not isinstance(vm.slots[slot_id], type(adapter)):
+                        yield from vm.slot_remove_binding(slot_id)
+                    if not isinstance(vm.slots[slot_id], type(adapter)):
+                        yield from vm.slot_add_binding(slot_id, adapter)
+                except IndexError:
+                    raise DynamipsError("Slot {} doesn't exist on this router".format(slot_id))
             elif name.startswith("slot") and value is None:
                 slot_id = int(name[-1])
-                if vm.slots[slot_id]:
-                    yield from vm.slot_remove_binding(slot_id)
+                try:
+                    if vm.slots[slot_id]:
+                        yield from vm.slot_remove_binding(slot_id)
+                except IndexError:
+                    raise DynamipsError("Slot {} doesn't exist on this router".format(slot_id))
             elif name.startswith("wic") and value in WIC_MATRIX:
                 wic_slot_id = int(name[-1])
                 wic_name = value
                 wic = WIC_MATRIX[wic_name]()
-                if vm.slots[0].wics[wic_slot_id] and not isinstance(vm.slots[0].wics[wic_slot_id], type(wic)):
-                    yield from vm.uninstall_wic(wic_slot_id)
-                if not isinstance(vm.slots[0].wics[wic_slot_id], type(wic)):
-                    yield from vm.install_wic(wic_slot_id, wic)
+                try:
+                    if vm.slots[0].wics[wic_slot_id] and not isinstance(vm.slots[0].wics[wic_slot_id], type(wic)):
+                        yield from vm.uninstall_wic(wic_slot_id)
+                    if not isinstance(vm.slots[0].wics[wic_slot_id], type(wic)):
+                        yield from vm.install_wic(wic_slot_id, wic)
+                except IndexError:
+                    raise DynamipsError("WIC slot {} doesn't exist on this router".format(wic_slot_id))
             elif name.startswith("wic") and value is None:
                 wic_slot_id = int(name[-1])
-                if vm.slots[0].wics and vm.slots[0].wics[wic_slot_id]:
-                    yield from vm.uninstall_wic(wic_slot_id)
+                try:
+                    if vm.slots[0].wics and vm.slots[0].wics[wic_slot_id]:
+                        yield from vm.uninstall_wic(wic_slot_id)
+                except IndexError:
+                    raise DynamipsError("WIC slot {} doesn't exist on this router".format(wic_slot_id))
 
         mmap_support = self.config.get_section_config("Dynamips").getboolean("mmap_support", True)
         if mmap_support is False:
@@ -521,38 +583,34 @@ class Dynamips(BaseManager):
         default_startup_config_path = os.path.join(module_workdir, "configs", "i{}_startup-config.cfg".format(vm.dynamips_id))
         default_private_config_path = os.path.join(module_workdir, "configs", "i{}_private-config.cfg".format(vm.dynamips_id))
 
+        startup_config_path = settings.get("startup_config")
         startup_config_content = settings.get("startup_config_content")
-        if startup_config_content:
-            startup_config_path = self._create_config(vm, startup_config_content, default_startup_config_path)
+        if startup_config_path:
+            yield from vm.set_configs(startup_config_path)
+        elif startup_config_content:
+            startup_config_path = self._create_config(vm, default_startup_config_path, startup_config_content)
             yield from vm.set_configs(startup_config_path)
-        else:
-            startup_config_path = settings.get("startup_config")
-            if startup_config_path:
-                yield from vm.set_configs(startup_config_path)
 
+        private_config_path = settings.get("private_config")
         private_config_content = settings.get("private_config_content")
-        if private_config_content:
-            private_config_path = self._create_config(vm, private_config_content, default_private_config_path)
+        if private_config_path:
+            yield from vm.set_configs(vm.startup_config, private_config_path)
+        elif private_config_content:
+            private_config_path = self._create_config(vm, default_private_config_path, private_config_content)
             yield from vm.set_configs(vm.startup_config, private_config_path)
-        else:
-            private_config_path = settings.get("private_config")
-            if private_config_path:
-                yield from vm.set_configs(vm.startup_config, private_config_path)
 
-    def _create_config(self, vm, content, path):
+    def _create_config(self, vm, path, content=None):
         """
         Creates a config file.
 
         :param vm: VM instance
-        :param content: config content
         :param path: path to the destination config file
+        :param content: config content
 
         :returns: relative path to the created config file
         """
 
         log.info("Creating config file {}".format(path))
-        content = "!\n" + content.replace("\r", "")
-        content = content.replace('%h', vm.name)
         config_dir = os.path.dirname(path)
         try:
             os.makedirs(config_dir, exist_ok=True)
@@ -561,7 +619,10 @@ class Dynamips(BaseManager):
 
         try:
             with open(path, "wb") as f:
-                f.write(content.encode("utf-8"))
+                if content:
+                    content = "!\n" + content.replace("\r", "")
+                    content = content.replace('%h', vm.name)
+                    f.write(content.encode("utf-8"))
         except OSError as e:
             raise DynamipsError("Could not create config file {}: {}".format(path, e))
 
@@ -616,3 +677,9 @@ class Dynamips(BaseManager):
             if was_auto_started:
                 yield from vm.stop()
         return validated_idlepc
+
+    def get_images_directory(self):
+        """
+        Return the full path of the images directory on disk
+        """
+        return os.path.join(os.path.expanduser(self.config.get_section_config("Server").get("images_path", "~/GNS3/images")), "IOS")

gns3server/modules/dynamips/dynamips_hypervisor.py

@@ -267,8 +267,8 @@ class DynamipsHypervisor:
                 self._writer.write(command.encode())
                 yield from self._writer.drain()
             except OSError as e:
-                raise DynamipsError("Lost communication with {host}:{port} :{error}, Dynamips process running: {run}"
-                                    .format(host=self._host, port=self._port, error=e, run=self.is_running()))
+                raise DynamipsError("Could not send Dynamips command '{command}' to {host}:{port}: {error}, process running: {run}"
+                                    .format(command=command.strip(), host=self._host, port=self._port, error=e, run=self.is_running()))
 
             # Now retrieve the result
             data = []
@@ -276,18 +276,25 @@ class DynamipsHypervisor:
             while True:
                 try:
                     try:
-                        line = yield from self._reader.readline()
+                        #line = yield from self._reader.readline()  # this can lead to ValueError: Line is too long
+                        chunk = yield from self._reader.read(1024)  # match to Dynamips' buffer size
                     except asyncio.CancelledError:
                         # task has been canceled but continue to read
                         # any remaining data sent by the hypervisor
                         continue
-                    if not line:
+                    except ConnectionResetError as e:
+                        # Sometimes WinError 64 (ERROR_NETNAME_DELETED) is returned here on Windows.
+                        # These happen if connection reset is received before IOCP could complete
+                        # a previous operation. Ignore and try again....
+                        log.warning("Connection reset received while reading Dynamips response: {}".format(e))
+                        continue
+                    if not chunk:
                         raise DynamipsError("No data returned from {host}:{port}, Dynamips process running: {run}"
                                             .format(host=self._host, port=self._port, run=self.is_running()))
-                    buf += line.decode()
+                    buf += chunk.decode("utf-8", errors="ignore")
                 except OSError as e:
-                    raise DynamipsError("Lost communication with {host}:{port} :{error}, Dynamips process running: {run}"
-                                        .format(host=self._host, port=self._port, error=e, run=self.is_running()))
+                    raise DynamipsError("Could not read response for '{command}' from {host}:{port}: {error}, process running: {run}"
+                                        .format(command=command.strip(), host=self._host, port=self._port, error=e, run=self.is_running()))
 
                 # If the buffer doesn't end in '\n' then we can't be done
                 try:

gns3server/modules/dynamips/hypervisor.py

@@ -140,7 +140,12 @@ class Hypervisor(DynamipsHypervisor):
             except asyncio.TimeoutError:
                 if self._process.returncode is None:
                     log.warn("Dynamips process {} is still running... killing it".format(self._process.pid))
-                    self._process.kill()
+                    try:
+                        self._process.kill()
+                    except OSError as e:
+                        log.error("Cannot stop the Dynamips process: {}".format(e))
+                    if self._process.returncode is None:
+                        log.warn('Dynamips hypervisor with PID={} is still running'.format(self._process.pid))
 
         if self._stdout_file and os.access(self._stdout_file, os.W_OK):
             try:
@@ -158,8 +163,8 @@ class Hypervisor(DynamipsHypervisor):
         output = ""
         if self._stdout_file and os.access(self._stdout_file, os.R_OK):
             try:
-                with open(self._stdout_file, errors="replace") as file:
-                    output = file.read()
+                with open(self._stdout_file, "rb") as file:
+                    output = file.read().decode("utf-8", errors="replace")
             except OSError as e:
                 log.warn("could not read {}: {}".format(self._stdout_file, e))
         return output

gns3server/modules/dynamips/nios/nio_fifo.py

@@ -20,6 +20,7 @@ Interface for FIFO NIOs.
 """
 
 import asyncio
+import uuid
 from .nio import NIO
 
 import logging
@@ -34,24 +35,12 @@ class NIOFIFO(NIO):
     :param hypervisor: Dynamips hypervisor instance
     """
 
-    _instance_count = 0
-
     def __init__(self, hypervisor):
 
-        # create an unique ID and name
-        nio_id = NIOFIFO._instance_count
-        NIOFIFO._instance_count += 1
-        name = 'nio_fifo' + str(nio_id)
+        # create an unique name
+        name = 'fifo-{}'.format(uuid.uuid4())
         super().__init__(name, hypervisor)
 
-    @classmethod
-    def reset(cls):
-        """
-        Reset the instance count.
-        """
-
-        cls._instance_count = 0
-
     @asyncio.coroutine
     def create(self):
 

gns3server/modules/dynamips/nios/nio_generic_ethernet.py

@@ -20,6 +20,7 @@ Interface for generic Ethernet NIOs (PCAP library).
 """
 
 import asyncio
+import uuid
 from .nio import NIO
 
 import logging
@@ -35,25 +36,13 @@ class NIOGenericEthernet(NIO):
     :param ethernet_device: Ethernet device name (e.g. eth0)
     """
 
-    _instance_count = 0
-
     def __init__(self, hypervisor, ethernet_device):
 
-        # create an unique ID and name
-        nio_id = NIOGenericEthernet._instance_count
-        NIOGenericEthernet._instance_count += 1
-        name = 'nio_gen_eth' + str(nio_id)
+        # create an unique name
+        name = 'generic_ethernet-{}'.format(uuid.uuid4())
         self._ethernet_device = ethernet_device
         super().__init__(name, hypervisor)
 
-    @classmethod
-    def reset(cls):
-        """
-        Reset the instance count.
-        """
-
-        cls._instance_count = 0
-
     @asyncio.coroutine
     def create(self):
 

gns3server/modules/dynamips/nios/nio_linux_ethernet.py

@@ -20,6 +20,7 @@ Interface for Linux Ethernet NIOs (Linux only).
 """
 
 import asyncio
+import uuid
 from .nio import NIO
 
 import logging
@@ -35,25 +36,12 @@ class NIOLinuxEthernet(NIO):
     :param ethernet_device: Ethernet device name (e.g. eth0)
     """
 
-    _instance_count = 0
-
     def __init__(self, hypervisor, ethernet_device):
-
-        # create an unique ID and name
-        nio_id = NIOLinuxEthernet._instance_count
-        NIOLinuxEthernet._instance_count += 1
-        name = 'nio_linux_eth' + str(nio_id)
+        # create an unique name
+        name = 'linux_ethernet-{}'.format(uuid.uuid4())
         self._ethernet_device = ethernet_device
         super().__init__(name, hypervisor)
 
-    @classmethod
-    def reset(cls):
-        """
-        Reset the instance count.
-        """
-
-        cls._instance_count = 0
-
     @asyncio.coroutine
     def create(self):
 

gns3server/modules/dynamips/nios/nio_mcast.py

@@ -20,6 +20,7 @@ Interface for multicast NIOs.
 """
 
 import asyncio
+import uuid
 from .nio import NIO
 
 import logging
@@ -36,27 +37,15 @@ class NIOMcast(NIO):
     :param port: port for binding
     """
 
-    _instance_count = 0
-
     def __init__(self, hypervisor, group, port):
 
-        # create an unique ID and name
-        nio_id = NIOMcast._instance_count
-        NIOMcast._instance_count += 1
-        name = 'nio_mcast' + str(nio_id)
+        # create an unique name
+        name = 'mcast-{}'.format(uuid.uuid4())
         self._group = group
         self._port = port
         self._ttl = 1  # default TTL
         super().__init__(name, hypervisor)
 
-    @classmethod
-    def reset(cls):
-        """
-        Reset the instance count.
-        """
-
-        cls._instance_count = 0
-
     @asyncio.coroutine
     def create(self):
 

gns3server/modules/dynamips/nios/nio_null.py

@@ -20,6 +20,7 @@ Interface for dummy NIOs (mostly for tests).
 """
 
 import asyncio
+import uuid
 from .nio import NIO
 
 import logging
@@ -34,24 +35,12 @@ class NIONull(NIO):
     :param hypervisor: Dynamips hypervisor instance
     """
 
-    _instance_count = 0
-
     def __init__(self, hypervisor):
 
-        # create an unique ID and name
-        nio_id = NIONull._instance_count
-        NIONull._instance_count += 1
-        name = 'nio_null' + str(nio_id)
+        # create an unique name
+        name = 'null-{}'.format(uuid.uuid4())
         super().__init__(name, hypervisor)
 
-    @classmethod
-    def reset(cls):
-        """
-        Reset the instance count.
-        """
-
-        cls._instance_count = 0
-
     @asyncio.coroutine
     def create(self):
 

gns3server/modules/dynamips/nios/nio_tap.py

@@ -20,6 +20,7 @@ Interface for TAP NIOs (UNIX based OSes only).
 """
 
 import asyncio
+import uuid
 from .nio import NIO
 
 import logging
@@ -35,25 +36,13 @@ class NIOTAP(NIO):
     :param tap_device: TAP device name (e.g. tap0)
     """
 
-    _instance_count = 0
-
     def __init__(self, hypervisor, tap_device):
 
-        # create an unique ID and name
-        nio_id = NIOTAP._instance_count
-        NIOTAP._instance_count += 1
-        name = 'nio_tap' + str(nio_id)
+        # create an unique name
+        name = 'tap-{}'.format(uuid.uuid4())
         self._tap_device = tap_device
         super().__init__(name, hypervisor)
 
-    @classmethod
-    def reset(cls):
-        """
-        Reset the instance count.
-        """
-
-        cls._instance_count = 0
-
     @asyncio.coroutine
     def create(self):
 

gns3server/modules/dynamips/nios/nio_udp.py

@@ -20,6 +20,7 @@ Interface for UDP NIOs.
 """
 
 import asyncio
+import uuid
 from .nio import NIO
 
 import logging
@@ -37,27 +38,15 @@ class NIOUDP(NIO):
     :param rport: remote port number
     """
 
-    _instance_count = 0
-
     def __init__(self, hypervisor, lport, rhost, rport):
 
-        # create an unique ID and name
-        nio_id = NIOUDP._instance_count
-        NIOUDP._instance_count += 1
-        name = 'nio_udp' + str(nio_id)
+        # create an unique name
+        name = 'udp-{}'.format(uuid.uuid4())
         self._lport = lport
         self._rhost = rhost
         self._rport = rport
         super().__init__(name, hypervisor)
 
-    @classmethod
-    def reset(cls):
-        """
-        Reset the instance count.
-        """
-
-        cls._instance_count = 0
-
     @asyncio.coroutine
     def create(self):
 

gns3server/modules/dynamips/nios/nio_unix.py

@@ -20,6 +20,7 @@ Interface for UNIX NIOs (Unix based OSes only).
 """
 
 import asyncio
+import uuid
 from .nio import NIO
 
 import logging
@@ -36,26 +37,14 @@ class NIOUNIX(NIO):
     :param remote_file: remote UNIX socket filename
     """
 
-    _instance_count = 0
-
     def __init__(self, hypervisor, local_file, remote_file):
 
-        # create an unique ID and name
-        nio_id = NIOUNIX._instance_count
-        NIOUNIX._instance_count += 1
-        name = 'nio_unix' + str(nio_id)
+        # create an unique name
+        name = 'unix-{}'.format(uuid.uuid4())
         self._local_file = local_file
         self._remote_file = remote_file
         super().__init__(name, hypervisor)
 
-    @classmethod
-    def reset(cls):
-        """
-        Reset the instance count.
-        """
-
-        cls._instance_count = 0
-
     @asyncio.coroutine
     def create(self):
 

gns3server/modules/dynamips/nios/nio_vde.py

@@ -20,6 +20,7 @@ Interface for VDE (Virtual Distributed Ethernet) NIOs (Unix based OSes only).
 """
 
 import asyncio
+import uuid
 from .nio import NIO
 
 import logging
@@ -36,26 +37,14 @@ class NIOVDE(NIO):
     :param local_file: VDE local filename
     """
 
-    _instance_count = 0
-
     def __init__(self, hypervisor, control_file, local_file):
 
-        # create an unique ID and name
-        nio_id = NIOVDE._instance_count
-        NIOVDE._instance_count += 1
-        name = 'nio_vde' + str(nio_id)
+        # create an unique name
+        name = 'vde-{}'.format(uuid.uuid4())
         self._control_file = control_file
         self._local_file = local_file
         super().__init__(name, hypervisor)
 
-    @classmethod
-    def reset(cls):
-        """
-        Reset the instance count.
-        """
-
-        cls._instance_count = 0
-
     @asyncio.coroutine
     def create(self):
 

gns3server/modules/dynamips/nodes/atm_switch.py

@@ -51,10 +51,14 @@ class ATMSwitch(Device):
 
     def __json__(self):
 
+        mappings = {}
+        for source, destination in self._mappings.items():
+            mappings[str(source)] = str(destination)
+
         return {"name": self.name,
                 "device_id": self.id,
                 "project_id": self.project.id,
-                "mappings": self._mappings}
+                "mappings": mappings}
 
     @asyncio.coroutine
     def create(self):
@@ -150,6 +154,7 @@ class ATMSwitch(Device):
 
         self._nios[port_number] = nio
 
+    @asyncio.coroutine
     def remove_nio(self, port_number):
         """
         Removes the specified NIO as member of this ATM switch.
@@ -160,6 +165,23 @@ class ATMSwitch(Device):
         if port_number not in self._nios:
             raise DynamipsError("Port {} is not allocated".format(port_number))
 
+        # remove VCs mapped with the port
+        for source, destination in self._mappings.copy().items():
+            if len(source) == 3 and len(destination) == 3:
+                # remove the virtual channels mapped with this port/nio
+                source_port, source_vpi, source_vci = source
+                destination_port, destination_vpi, destination_vci = destination
+                if port_number == source_port:
+                    yield from self.unmap_pvc(source_port, source_vpi, source_vci, destination_port, destination_vpi, destination_vci)
+                    yield from self.unmap_pvc(destination_port, destination_vpi, destination_vci, source_port, source_vpi, source_vci)
+            else:
+                # remove the virtual paths mapped with this port/nio
+                source_port, source_vpi = source
+                destination_port, destination_vpi = destination
+                if port_number == source_port:
+                    yield from self.unmap_vp(source_port, source_vpi, destination_port, destination_vpi)
+                    yield from self.unmap_vp(destination_port, destination_vpi, source_port, source_vpi)
+
         nio = self._nios[port_number]
         if isinstance(nio, NIOUDP):
             self.manager.port_manager.release_udp_port(nio.lport, self._project)
@@ -181,6 +203,8 @@ class ATMSwitch(Device):
 
         pvc_entry = re.compile(r"""^([0-9]*):([0-9]*):([0-9]*)$""")
         for source, destination in mappings.items():
+            if not isinstance(source, str) or not isinstance(destination, str):
+                raise DynamipsError("Invalid ATM mappings")
             match_source_pvc = pvc_entry.search(source)
             match_destination_pvc = pvc_entry.search(destination)
             if match_source_pvc and match_destination_pvc:

gns3server/modules/dynamips/nodes/c7200.py

@@ -120,7 +120,7 @@ class C7200(Router):
         npe-225, npe-300, npe-400 and npe-g2 (PowerPC c7200 only)
         """
 
-        if self.is_running():
+        if (yield from self.is_running()):
             raise DynamipsError("Cannot change NPE on running router")
 
         yield from self._hypervisor.send('c7200 set_npe "{name}" {npe}'.format(name=self._name, npe=npe))

gns3server/modules/dynamips/nodes/ethernet_switch.py

@@ -298,6 +298,9 @@ class EthernetSwitch(Device):
 
         nio = self._nios[port_number]
 
+        if not nio:
+            raise DynamipsError("Port {} is not connected".format(port_number))
+
         data_link_type = data_link_type.lower()
         if data_link_type.startswith("dlt_"):
             data_link_type = data_link_type[4:]
@@ -324,6 +327,10 @@ class EthernetSwitch(Device):
             raise DynamipsError("Port {} is not allocated".format(port_number))
 
         nio = self._nios[port_number]
+
+        if not nio:
+            raise DynamipsError("Port {} is not connected".format(port_number))
+
         yield from nio.unbind_filter("both")
         log.info('Ethernet switch "{name}" [{id}]: stopping packet capture on port {port}'.format(name=self._name,
                                                                                                   id=self._id,

gns3server/modules/dynamips/nodes/frame_relay_switch.py

@@ -50,10 +50,14 @@ class FrameRelaySwitch(Device):
 
     def __json__(self):
 
+        mappings = {}
+        for source, destination in self._mappings.items():
+            mappings[str(source)] = str(destination)
+
         return {"name": self.name,
                 "device_id": self.id,
                 "project_id": self.project.id,
-                "mappings": self._mappings}
+                "mappings": mappings}
 
     @asyncio.coroutine
     def create(self):
@@ -149,6 +153,7 @@ class FrameRelaySwitch(Device):
 
         self._nios[port_number] = nio
 
+    @asyncio.coroutine
     def remove_nio(self, port_number):
         """
         Removes the specified NIO as member of this Frame Relay switch.
@@ -161,6 +166,14 @@ class FrameRelaySwitch(Device):
         if port_number not in self._nios:
             raise DynamipsError("Port {} is not allocated".format(port_number))
 
+        # remove VCs mapped with the port
+        for source, destination in self._mappings.copy().items():
+            source_port, source_dlci = source
+            destination_port, destination_dlci = destination
+            if port_number == source_port:
+                yield from self.unmap_vc(source_port, source_dlci, destination_port, destination_dlci)
+                yield from self.unmap_vc(destination_port, destination_dlci, source_port, source_dlci)
+
         nio = self._nios[port_number]
         if isinstance(nio, NIOUDP):
             self.manager.port_manager.release_udp_port(nio.lport, self._project)
@@ -182,6 +195,8 @@ class FrameRelaySwitch(Device):
         """
 
         for source, destination in mappings.items():
+            if not isinstance(source, str) or not isinstance(destination, str):
+                raise DynamipsError("Invalid Frame-Relay mappings")
             source_port, source_dlci = map(int, source.split(':'))
             destination_port, destination_dlci = map(int, destination.split(':'))
             if self.has_port(destination_port):

gns3server/modules/dynamips/nodes/router.py

@@ -34,6 +34,7 @@ log = logging.getLogger(__name__)
 from ...base_vm import BaseVM
 from ..dynamips_error import DynamipsError
 from ..nios.nio_udp import NIOUDP
+from ....utils.glob import glob_escape
 
 from gns3server.utils.asyncio import wait_run_in_executor
 
@@ -53,7 +54,6 @@ class Router(BaseVM):
     :param platform: Platform of this router
     """
 
-    _dynamips_ids = {}
     _status = {0: "inactive",
                1: "shutting down",
                2: "running",
@@ -87,6 +87,7 @@ class Router(BaseVM):
             self._exec_area = 64  # 64 MB on other systems
         self._disk0 = 0  # Megabytes
         self._disk1 = 0  # Megabytes
+        self._auto_delete_disks = False
         self._aux = aux
         self._mac_addr = ""
         self._system_id = "FTX0945W0MY"  # processor board ID in IOS
@@ -94,20 +95,11 @@ class Router(BaseVM):
         self._ghost_flag = ghost_flag
 
         if not ghost_flag:
-            self._dynamips_ids.setdefault(project.id, list())
             if not dynamips_id:
-                # find a Dynamips ID if none is provided (0 < id <= 4096)
-                self._dynamips_id = 0
-                for identifier in range(1, 4097):
-                    if identifier not in self._dynamips_ids[project.id]:
-                        self._dynamips_id = identifier
-                        break
-                if self._dynamips_id == 0:
-                    raise DynamipsError("Maximum number of Dynamips instances reached")
+                self._dynamips_id = manager.get_dynamips_id(project.id)
             else:
-                if dynamips_id in self._dynamips_ids[project.id]:
-                    raise DynamipsError("Dynamips identifier {} is already used by another router".format(dynamips_id))
-            self._dynamips_ids[project.id].append(self._dynamips_id)
+                self._dynamips_id = dynamips_id
+                manager.take_dynamips_id(project.id, dynamips_id)
 
             if self._aux is not None:
                 self._aux = self._manager.port_manager.reserve_tcp_port(self._aux, self._project)
@@ -145,23 +137,21 @@ class Router(BaseVM):
                        "exec_area": self._exec_area,
                        "disk0": self._disk0,
                        "disk1": self._disk1,
+                       "auto_delete_disks": self._auto_delete_disks,
                        "console": self._console,
                        "aux": self._aux,
                        "mac_addr": self._mac_addr,
                        "system_id": self._system_id}
 
         # return the relative path if the IOS image is in the images_path directory
-        server_config = self.manager.config.get_section_config("Server")
-        relative_image = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), "IOS", self._image)
-        if os.path.exists(relative_image):
-            router_info["image"] = os.path.basename(self._image)
+        router_info["image"] = self.manager.get_relative_image_path(self._image)
 
         # add the slots
         slot_number = 0
         for slot in self._slots:
             if slot:
                 slot = str(slot)
-                router_info["slot" + str(slot_number)] = slot
+            router_info["slot" + str(slot_number)] = slot
             slot_number += 1
 
         # add the wics
@@ -169,17 +159,11 @@ class Router(BaseVM):
             for wic_slot_number in range(0, len(self._slots[0].wics)):
                 if self._slots[0].wics[wic_slot_number]:
                     router_info["wic" + str(wic_slot_number)] = str(self._slots[0].wics[wic_slot_number])
+                else:
+                    router_info["wic" + str(wic_slot_number)] = None
 
         return router_info
 
-    @classmethod
-    def reset(cls):
-        """
-        Resets the instance count and the allocated instances list.
-        """
-
-        cls._dynamips_ids.clear()
-
     @property
     def dynamips_id(self):
         """
@@ -325,8 +309,6 @@ class Router(BaseVM):
             return
 
         log.debug('Router "{name}" [{id}] is closing'.format(name=self._name, id=self._id))
-        if self._dynamips_id in self._dynamips_ids[self._project.id]:
-            self._dynamips_ids[self._project.id].remove(self._dynamips_id)
 
         if self._console:
             self._manager.port_manager.release_tcp_port(self._console, self._project)
@@ -353,6 +335,23 @@ class Router(BaseVM):
                 pass
             yield from self.hypervisor.stop()
 
+        if self._auto_delete_disks:
+            # delete nvram and disk files
+            project_dir = os.path.join(self.project.module_working_directory(self.manager.module_name.lower()))
+            files = glob.glob(os.path.join(glob_escape(project_dir), "{}_i{}_disk[0-1]".format(self.platform, self.dynamips_id)))
+            files += glob.glob(os.path.join(glob_escape(project_dir), "{}_i{}_slot[0-1]".format(self.platform, self.dynamips_id)))
+            files += glob.glob(os.path.join(glob_escape(project_dir), "{}_i{}_nvram".format(self.platform, self.dynamips_id)))
+            files += glob.glob(os.path.join(glob_escape(project_dir), "{}_i{}_flash[0-1]".format(self.platform, self.dynamips_id)))
+            files += glob.glob(os.path.join(glob_escape(project_dir), "{}_i{}_rom".format(self.platform, self.dynamips_id)))
+            files += glob.glob(os.path.join(glob_escape(project_dir), "{}_i{}_bootflash".format(self.platform, self.dynamips_id)))
+            files += glob.glob(os.path.join(glob_escape(project_dir), "{}_i{}_ssa").format(self.platform, self.dynamips_id))
+            for file in files:
+                try:
+                    log.debug("Deleting file {}".format(file))
+                    yield from wait_run_in_executor(os.remove, file)
+                except OSError as e:
+                    log.warn("Could not delete file {}: {}".format(file, e))
+                    continue
         self._closed = True
 
     @property
@@ -427,9 +426,7 @@ class Router(BaseVM):
         :param image: path to IOS image file
         """
 
-        if not os.path.isabs(image):
-            server_config = self.manager.config.get_section_config("Server")
-            image = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), "IOS", image)
+        image = self.manager.get_abs_image_path(image)
 
         if not os.path.isfile(image):
             raise DynamipsError("IOS image '{}' is not accessible".format(image))
@@ -849,7 +846,7 @@ class Router(BaseVM):
         return self._disk1
 
     @asyncio.coroutine
-    def disk1(self, disk1):
+    def set_disk1(self, disk1):
         """
         Sets the size (MB) for PCMCIA disk1.
 
@@ -864,6 +861,30 @@ class Router(BaseVM):
                                                                                                     new_disk1=disk1))
         self._disk1 = disk1
 
+    @property
+    def auto_delete_disks(self):
+        """
+        Returns True if auto delete disks is enabled on this router.
+
+        :returns: boolean either auto delete disks is activated or not
+        """
+
+        return self._auto_delete_disks
+
+    @asyncio.coroutine
+    def set_auto_delete_disks(self, auto_delete_disks):
+        """
+        Enable/disable use of auto delete disks
+
+        :param auto_delete_disks: activate/deactivate auto delete disks (boolean)
+        """
+
+        if auto_delete_disks:
+            log.info('Router "{name}" [{id}]: auto delete disks enabled'.format(name=self._name, id=self._id))
+        else:
+            log.info('Router "{name}" [{id}]: auto delete disks disabled'.format(name=self._name, id=self._id))
+        self._auto_delete_disks = auto_delete_disks
+
     @asyncio.coroutine
     def set_console(self, console):
         """
@@ -1191,11 +1212,20 @@ class Router(BaseVM):
         if not adapter.port_exists(port_number):
             raise DynamipsError("Port {port_number} does not exist in adapter {adapter}".format(adapter=adapter,
                                                                                                 port_number=port_number))
-
-        yield from self._hypervisor.send('vm slot_add_nio_binding "{name}" {slot_number} {port_number} {nio}'.format(name=self._name,
-                                                                                                                     slot_number=slot_number,
-                                                                                                                     port_number=port_number,
-                                                                                                                     nio=nio))
+        try:
+            yield from self._hypervisor.send('vm slot_add_nio_binding "{name}" {slot_number} {port_number} {nio}'.format(name=self._name,
+                                                                                                                         slot_number=slot_number,
+                                                                                                                         port_number=port_number,
+                                                                                                                         nio=nio))
+        except DynamipsError:
+            # in case of error try to remove and add the nio binding
+            yield from self._hypervisor.send('vm slot_remove_nio_binding "{name}" {slot_number} {port_number}'.format(name=self._name,
+                                                                                                                      slot_number=slot_number,
+                                                                                                                      port_number=port_number))
+            yield from self._hypervisor.send('vm slot_add_nio_binding "{name}" {slot_number} {port_number} {nio}'.format(name=self._name,
+                                                                                                                         slot_number=slot_number,
+                                                                                                                         port_number=port_number,
+                                                                                                                         nio=nio))
 
         log.info('Router "{name}" [{id}]: NIO {nio_name} bound to port {slot_number}/{port_number}'.format(name=self._name,
                                                                                                            id=self._id,
@@ -1316,6 +1346,10 @@ class Router(BaseVM):
 
         nio = adapter.get_nio(port_number)
 
+        if not nio:
+            raise DynamipsError("Port {slot_number}/{port_number} is not connected".format(slot_number=slot_number,
+                                                                                           port_number=port_number))
+
         if nio.input_filter[0] is not None and nio.output_filter[0] is not None:
             raise DynamipsError("Port {port_number} has already a filter applied on {adapter}".format(adapter=adapter,
                                                                                                       port_number=port_number))
@@ -1348,6 +1382,11 @@ class Router(BaseVM):
                                                                                                 port_number=port_number))
 
         nio = adapter.get_nio(port_number)
+
+        if not nio:
+            raise DynamipsError("Port {slot_number}/{port_number} is not connected".format(slot_number=slot_number,
+                                                                                           port_number=port_number))
+
         yield from nio.unbind_filter("both")
 
         log.info('Router "{name}" [{id}]: stopping packet capture on port {slot_number}/{port_number}'.format(name=self._name,
@@ -1409,7 +1448,7 @@ class Router(BaseVM):
             startup_config_path = os.path.join(module_workdir, "configs", "i{}_startup-config.cfg".format(self._dynamips_id))
             if os.path.isfile(startup_config_path):
                 try:
-                    with open(startup_config_path, "r+", errors="replace") as f:
+                    with open(startup_config_path, "r+", encoding="utf-8", errors="replace") as f:
                         old_config = f.read()
                         new_config = old_config.replace(self.name, new_name)
                         f.seek(0)
@@ -1422,7 +1461,7 @@ class Router(BaseVM):
             private_config_path = os.path.join(module_workdir, "configs", "i{}_private-config.cfg".format(self._dynamips_id))
             if os.path.isfile(private_config_path):
                 try:
-                    with open(private_config_path, "r+", errors="replace") as f:
+                    with open(private_config_path, "r+", encoding="utf-8", errors="replace") as f:
                         old_config = f.read()
                         new_config = old_config.replace(self.name, new_name)
                         f.seek(0)
@@ -1449,6 +1488,17 @@ class Router(BaseVM):
         private_config = private_config.replace("\\", '/')
 
         if self._startup_config != startup_config or self._private_config != private_config:
+            self._startup_config = startup_config
+            self._private_config = private_config
+
+            module_workdir = self.project.module_working_directory(self.manager.module_name.lower())
+            private_config_path = os.path.join(module_workdir, private_config)
+            try:
+                if not os.path.getsize(private_config_path):
+                    # an empty private-config can prevent a router to boot.
+                    private_config = ''
+            except OSError as e:
+                raise DynamipsError("Cannot access the private-config {}: {}".format(private_config_path, e))
 
             yield from self._hypervisor.send('vm set_config "{name}" "{startup}" "{private}"'.format(name=self._name,
                                                                                                      startup=startup_config,
@@ -1458,15 +1508,11 @@ class Router(BaseVM):
                                                                                                 id=self._id,
                                                                                                 startup=startup_config))
 
-            self._startup_config = startup_config
-
             if private_config:
                 log.info('Router "{name}" [{id}]: has a new private-config set: "{private}"'.format(name=self._name,
                                                                                                     id=self._id,
                                                                                                     private=private_config))
 
-                self._private_config = private_config
-
     @asyncio.coroutine
     def extract_config(self):
         """
@@ -1496,8 +1542,11 @@ class Router(BaseVM):
             module_workdir = self.project.module_working_directory(self.manager.module_name.lower())
             startup_config_base64, private_config_base64 = yield from self.extract_config()
             if startup_config_base64:
+                if not self.startup_config:
+                    self._startup_config = os.path.join("configs", "i{}_startup-config.cfg".format(self._dynamips_id))
+
                 try:
-                    config = base64.b64decode(startup_config_base64).decode(errors='replace')
+                    config = base64.b64decode(startup_config_base64).decode("utf-8", errors="replace")
                     config = "!\n" + config.replace("\r", "")
                     config_path = os.path.join(module_workdir, self.startup_config)
                     with open(config_path, "wb") as f:
@@ -1507,8 +1556,11 @@ class Router(BaseVM):
                     raise DynamipsError("Could not save the startup configuration {}: {}".format(config_path, e))
 
             if private_config_base64:
+                if not self.private_config:
+                    self._private_config = os.path.join("configs", "i{}_private-config.cfg".format(self._dynamips_id))
+
                 try:
-                    config = base64.b64decode(private_config_base64).decode(errors='replace')
+                    config = base64.b64decode(private_config_base64).decode("utf-8", errors="replace")
                     config = "!\n" + config.replace("\r", "")
                     config_path = os.path.join(module_workdir, self.private_config)
                     with open(config_path, "wb") as f:
@@ -1547,6 +1599,8 @@ class Router(BaseVM):
                 log.warn("Could not delete file {}: {}".format(file, e))
                 continue
 
+        self.manager.release_dynamips_id(self._project.id, self._dynamips_id)
+
     @asyncio.coroutine
     def clean_delete(self):
         """

gns3server/modules/iou/__init__.py

@@ -91,3 +91,9 @@ class IOU(BaseManager):
         """
 
         return os.path.join("iou", "device-{}".format(legacy_vm_id))
+
+    def get_images_directory(self):
+        """
+        Return the full path of the images directory on disk
+        """
+        return os.path.join(os.path.expanduser(self.config.get_section_config("Server").get("images_path", "~/GNS3/images")), "IOU")

gns3server/modules/iou/iou_vm.py

@@ -41,6 +41,7 @@ from ..nios.nio_udp import NIOUDP
 from ..nios.nio_tap import NIOTAP
 from ..nios.nio_generic_ethernet import NIOGenericEthernet
 from ..base_vm import BaseVM
+from ...utils.glob import glob_escape
 from .ioucon import start_ioucon
 import gns3server.utils.asyncio
 
@@ -61,25 +62,9 @@ class IOUVM(BaseVM):
     :param project: Project instance
     :param manager: Manager instance
     :param console: TCP console port
-    :params ethernet_adapters: number of ethernet adapters
-    :params serial_adapters: number of serial adapters
-    :params ram: amount of RAM in MB
-    :params nvram: amount of NVRAM in KB
-    :params l1_keepalives: always keep the Ethernet interfaces up
-    :params initial_config: content of the initial configuration file
-    :params iourc_content: content of the iourc file if no licence is installed on the machine
     """
 
-    def __init__(self, name, vm_id, project, manager,
-                 console=None,
-                 ram=None,
-                 nvram=None,
-                 use_default_iou_values=None,
-                 ethernet_adapters=None,
-                 serial_adapters=None,
-                 l1_keepalives=None,
-                 initial_config=None,
-                 iourc_content=None):
+    def __init__(self, name, vm_id, project, manager, console=None):
 
         super().__init__(name, vm_id, project, manager, console=console)
 
@@ -94,17 +79,13 @@ class IOUVM(BaseVM):
         # IOU settings
         self._ethernet_adapters = []
         self._serial_adapters = []
-        self.ethernet_adapters = 2 if ethernet_adapters is None else ethernet_adapters  # one adapter = 4 interfaces
-        self.serial_adapters = 2 if serial_adapters is None else serial_adapters  # one adapter = 4 interfaces
-        self._use_default_iou_values = True if use_default_iou_values is None else use_default_iou_values  # for RAM & NVRAM values
-        self._nvram = 128 if nvram is None else nvram  # Kilobytes
+        self.ethernet_adapters = 2  # one adapter = 4 interfaces
+        self.serial_adapters = 2  # one adapter = 4 interfaces
+        self._use_default_iou_values = True  # for RAM & NVRAM values
+        self._nvram = 128  # Kilobytes
         self._initial_config = ""
-        self._ram = 256 if ram is None else ram  # Megabytes
-        self._l1_keepalives = False if l1_keepalives is None else l1_keepalives  # used to overcome the always-up Ethernet interfaces (not supported by all IOSes).
-
-        self.iourc_content = iourc_content
-        if initial_config is not None:
-            self.initial_config = initial_config
+        self._ram = 256  # Megabytes
+        self._l1_keepalives = False  # used to overcome the always-up Ethernet interfaces (not supported by all IOSes).
 
     @asyncio.coroutine
     def close(self):
@@ -145,14 +126,7 @@ class IOUVM(BaseVM):
         :param path: path to the IOU image executable
         """
 
-        if not os.path.isabs(path):
-            server_config = self.manager.config.get_section_config("Server")
-            relative_path = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), path)
-            if not os.path.exists(relative_path):
-                relative_path = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), "IOU", path)
-            path = relative_path
-
-        self._path = path
+        self._path = self.manager.get_abs_image_path(path)
 
         # In 1.2 users uploaded images to the images roots
         # after the migration their images are inside images/IOU
@@ -236,15 +210,11 @@ class IOUVM(BaseVM):
                        "nvram": self._nvram,
                        "l1_keepalives": self._l1_keepalives,
                        "initial_config": self.relative_initial_config_file,
-                       "use_default_iou_values": self._use_default_iou_values,
-                       "iourc_path": self.iourc_path}
+                       "iourc_path": self.iourc_path,
+                       "use_default_iou_values": self._use_default_iou_values}
 
         # return the relative path if the IOU image is in the images_path directory
-        server_config = self.manager.config.get_section_config("Server")
-        relative_image = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), "IOU", self.path)
-        if os.path.exists(relative_image):
-            iou_vm_info["path"] = os.path.basename(self.path)
-
+        iou_vm_info["path"] = self.manager.get_relative_image_path(self.path)
         return iou_vm_info
 
     @property
@@ -348,9 +318,9 @@ class IOUVM(BaseVM):
         """
 
         if self.initial_config_file:
-            content = self.initial_config
+            content = self.initial_config_content
             content = content.replace(self._name, new_name)
-            self.initial_config = content
+            self.initial_config_content = content
 
         super(IOUVM, IOUVM).name.__set__(self, new_name)
 
@@ -363,8 +333,8 @@ class IOUVM(BaseVM):
     def iourc_content(self):
 
         try:
-            with open(os.path.join(self.temporary_directory, "iourc")) as f:
-                return f.read()
+            with open(os.path.join(self.temporary_directory, "iourc"), "rb") as f:
+                return f.read().decode("utf-8")
         except OSError:
             return None
 
@@ -374,8 +344,8 @@ class IOUVM(BaseVM):
         if value is not None:
             path = os.path.join(self.temporary_directory, "iourc")
             try:
-                with open(path, "w+") as f:
-                    f.write(value)
+                with open(path, "wb+") as f:
+                    f.write(value.encode("utf-8"))
             except OSError as e:
                 raise IOUError("Could not write the iourc file {}: {}".format(path, e))
 
@@ -403,18 +373,20 @@ class IOUVM(BaseVM):
         Checks for a valid IOU key in the iourc file (paranoid mode).
         """
 
-        license_check = self._manager.config.get_section_config("IOU").getboolean("license_check", False)
-        if license_check:
+        license_check = self._manager.config.get_section_config("IOU").getboolean("license_check", True)
+        if license_check is False:
             return
 
         config = configparser.ConfigParser()
         try:
-            with open(self.iourc_path) as f:
+            with open(self.iourc_path, encoding="utf-8") as f:
                 config.read_file(f)
         except OSError as e:
             raise IOUError("Could not open iourc file {}: {}".format(self.iourc_path, e))
         except configparser.Error as e:
             raise IOUError("Could not parse iourc file {}: {}".format(self.iourc_path, e))
+        except UnicodeDecodeError as e:
+            raise IOUError("Non ascii characters in iourc file {}, please remove them: {}".format(self.iourc_path, e))
         if "license" not in config:
             raise IOUError("License section not found in iourc file {}".format(self.iourc_path))
         hostname = socket.gethostname()
@@ -462,11 +434,14 @@ class IOUVM(BaseVM):
 
             yield from self._library_check()
 
-            self._rename_nvram_file()
+            try:
+                self._rename_nvram_file()
+            except OSError as e:
+                raise IOUError("Could not rename nvram files: {}".format(e))
 
             iourc_path = self.iourc_path
-            if iourc_path is None:
-                raise IOUError("Could not find a iourc file (IOU license)")
+            if not iourc_path:
+                raise IOUError("Could not find an iourc file (IOU license)")
             if not os.path.isfile(iourc_path):
                 raise IOUError("The iourc path '{}' is not a regular file".format(iourc_path))
 
@@ -486,7 +461,7 @@ class IOUVM(BaseVM):
                 log.info("Starting IOU: {}".format(self._command))
                 self._iou_stdout_file = os.path.join(self.working_dir, "iou.log")
                 log.info("Logging to {}".format(self._iou_stdout_file))
-                with open(self._iou_stdout_file, "w") as fd:
+                with open(self._iou_stdout_file, "w", encoding="utf-8") as fd:
                     self._iou_process = yield from asyncio.create_subprocess_exec(*self._command,
                                                                                   stdout=fd,
                                                                                   stderr=subprocess.STDOUT,
@@ -512,10 +487,10 @@ class IOUVM(BaseVM):
         """
 
         destination = os.path.join(self.working_dir, "nvram_{:05d}".format(self.application_id))
-        for file_path in glob.glob(os.path.join(self.working_dir, "nvram_*")):
+        for file_path in glob.glob(os.path.join(glob_escape(self.working_dir), "nvram_*")):
             shutil.move(file_path, destination)
         destination = os.path.join(self.working_dir, "vlan.dat-{:05d}".format(self.application_id))
-        for file_path in glob.glob(os.path.join(self.working_dir, "vlan.dat-*")):
+        for file_path in glob.glob(os.path.join(glob_escape(self.working_dir), "vlan.dat-*")):
             shutil.move(file_path, destination)
 
     @asyncio.coroutine
@@ -530,7 +505,7 @@ class IOUVM(BaseVM):
             log.info("starting iouyap: {}".format(command))
             self._iouyap_stdout_file = os.path.join(self.working_dir, "iouyap.log")
             log.info("logging to {}".format(self._iouyap_stdout_file))
-            with open(self._iouyap_stdout_file, "w") as fd:
+            with open(self._iouyap_stdout_file, "w", encoding="utf-8") as fd:
                 self._iouyap_process = yield from asyncio.create_subprocess_exec(*command,
                                                                                  stdout=fd,
                                                                                  stderr=subprocess.STDOUT,
@@ -596,7 +571,7 @@ class IOUVM(BaseVM):
             bay_id += 1
 
         try:
-            with open(iouyap_ini, "w") as config_file:
+            with open(iouyap_ini, "w", encoding="utf-8") as config_file:
                 config.write(config_file)
             log.info("IOU {name} [id={id}]: iouyap.ini updated".format(name=self._name,
                                                                        id=self._id))
@@ -618,7 +593,6 @@ class IOUVM(BaseVM):
                 self._ioucon_thread = None
 
             self._terminate_process_iou()
-
             if self._iou_process.returncode is None:
                 try:
                     yield from gns3server.utils.asyncio.wait_for_process_termination(self._iou_process, timeout=3)
@@ -702,7 +676,7 @@ class IOUVM(BaseVM):
 
         netmap_path = os.path.join(self.working_dir, "NETMAP")
         try:
-            with open(netmap_path, "w") as f:
+            with open(netmap_path, "w", encoding="utf-8") as f:
                 for bay in range(0, 16):
                     for unit in range(0, 4):
                         f.write("{iouyap_id}:{bay}/{unit}{iou_id:>5d}:{bay}/{unit}\n".format(iouyap_id=str(self.application_id + 512),
@@ -772,8 +746,8 @@ class IOUVM(BaseVM):
         output = ""
         if self._iou_stdout_file:
             try:
-                with open(self._iou_stdout_file, errors="replace") as file:
-                    output = file.read()
+                with open(self._iou_stdout_file, "rb") as file:
+                    output = file.read().decode("utf-8", errors="replace")
             except OSError as e:
                 log.warn("could not read {}: {}".format(self._iou_stdout_file, e))
         return output
@@ -787,8 +761,8 @@ class IOUVM(BaseVM):
         output = ""
         if self._iouyap_stdout_file:
             try:
-                with open(self._iouyap_stdout_file, errors="replace") as file:
-                    output = file.read()
+                with open(self._iouyap_stdout_file, "rb") as file:
+                    output = file.read().decode("utf-8", errors="replace")
             except OSError as e:
                 log.warn("could not read {}: {}".format(self._iouyap_stdout_file, e))
         return output
@@ -889,7 +863,10 @@ class IOUVM(BaseVM):
                                                                                              port_number=port_number))
         if self.is_iouyap_running():
             self._update_iouyap_config()
-            os.kill(self._iouyap_process.pid, signal.SIGHUP)
+            try:
+                os.kill(self._iouyap_process.pid, signal.SIGHUP)
+            except ProcessLookupError:
+                log.error("Could not update iouyap configuration: process (PID={}) not found".format(self._iouyap_process.pid))
 
     def adapter_remove_nio_binding(self, adapter_number, port_number):
         """
@@ -921,8 +898,10 @@ class IOUVM(BaseVM):
                                                                                                  port_number=port_number))
         if self.is_iouyap_running():
             self._update_iouyap_config()
-            os.kill(self._iouyap_process.pid, signal.SIGHUP)
-
+            try:
+                os.kill(self._iouyap_process.pid, signal.SIGHUP)
+            except ProcessLookupError:
+                log.error("Could not update iouyap configuration: process (PID={}) not found".format(self._iouyap_process.pid))
         return nio
 
     @property
@@ -970,7 +949,7 @@ class IOUVM(BaseVM):
             log.warn("could not determine if layer 1 keepalive messages are supported by {}: {}".format(os.path.basename(self._path), e))
 
     @property
-    def initial_config(self):
+    def initial_config_content(self):
         """
         Returns the content of the current initial-config file.
         """
@@ -980,13 +959,13 @@ class IOUVM(BaseVM):
             return None
 
         try:
-            with open(config_file) as f:
-                return f.read()
+            with open(config_file, "rb") as f:
+                return f.read().decode("utf-8", errors="replace")
         except OSError as e:
-            raise IOUError("Can't read configuration file '{}'".format(config_file))
+            raise IOUError("Can't read configuration file '{}': {}".format(config_file, e))
 
-    @initial_config.setter
-    def initial_config(self, initial_config):
+    @initial_config_content.setter
+    def initial_config_content(self, initial_config):
         """
         Update the initial config
 
@@ -994,23 +973,23 @@ class IOUVM(BaseVM):
         """
 
         try:
-            script_file = os.path.join(self.working_dir, "initial-config.cfg")
+            initial_config_path = os.path.join(self.working_dir, "initial-config.cfg")
 
             if initial_config is None:
                 initial_config = ''
 
             # We disallow erasing the initial config file
-            if len(initial_config) == 0 and os.path.exists(script_file):
+            if len(initial_config) == 0 and os.path.exists(initial_config_path):
                 return
 
-            with open(script_file, 'w+') as f:
+            with open(initial_config_path, 'w+', encoding='utf-8') as f:
                 if len(initial_config) == 0:
                     f.write('')
                 else:
                     initial_config = initial_config.replace("%h", self._name)
                     f.write(initial_config)
         except OSError as e:
-            raise IOUError("Can't write initial configuration file '{}': {}".format(script_file, e))
+            raise IOUError("Can't write initial configuration file '{}': {}".format(initial_config_path, e))
 
     @property
     def initial_config_file(self):
@@ -1079,7 +1058,10 @@ class IOUVM(BaseVM):
 
         if self.is_iouyap_running():
             self._update_iouyap_config()
-            os.kill(self._iouyap_process.pid, signal.SIGHUP)
+            try:
+                os.kill(self._iouyap_process.pid, signal.SIGHUP)
+            except ProcessLookupError:
+                log.error("Could not update iouyap configuration: process (PID={}) not found".format(self._iouyap_process.pid))
 
     @asyncio.coroutine
     def stop_capture(self, adapter_number, port_number):
@@ -1101,6 +1083,10 @@ class IOUVM(BaseVM):
                                                                                            port_number=port_number))
 
         nio = adapter.get_nio(port_number)
+        if not nio:
+            raise IOUError("NIO {port_number} does not exist in adapter {adapter}".format(adapter=adapter,
+                                                                                          port_number=port_number))
+
         nio.stopPacketCapture()
         log.info('IOU "{name}" [{id}]: stopping packet capture on {adapter_number}/{port_number}'.format(name=self._name,
                                                                                                          id=self._id,
@@ -1108,4 +1094,7 @@ class IOUVM(BaseVM):
                                                                                                          port_number=port_number))
         if self.is_iouyap_running():
             self._update_iouyap_config()
-            os.kill(self._iouyap_process.pid, signal.SIGHUP)
+            try:
+                os.kill(self._iouyap_process.pid, signal.SIGHUP)
+            except ProcessLookupError:
+                log.error("Could not update iouyap configuration: process (PID={}) not found".format(self._iouyap_process.pid))

gns3server/modules/nios/nio_nat.py

@@ -0,0 +1,41 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2013 GNS3 Technologies Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+Interface for NAT NIOs.
+"""
+
+from .nio import NIO
+
+
+class NIONAT(NIO):
+
+    """
+    NAT NIO.
+    """
+
+    def __init__(self):
+
+        super().__init__()
+
+    def __str__(self):
+
+        return "NIO TAP"
+
+    def __json__(self):
+
+        return {"type": "nio_nat"}

gns3server/modules/port_manager.py

@@ -34,8 +34,9 @@ class PortManager:
     def __init__(self, host="127.0.0.1"):
 
         self._console_host = host
-        self._udp_host = host
 
+        # UDP host must be 0.0.0.0, reason: https://github.com/GNS3/gns3-server/issues/265
+        self._udp_host = "0.0.0.0"
         self._used_tcp_ports = set()
         self._used_udp_ports = set()
 
@@ -90,7 +91,7 @@ class PortManager:
 
         return self._console_port_range
 
-    @console_host.setter
+    @console_port_range.setter
     def console_port_range(self, new_range):
 
         assert isinstance(new_range, tuple)
@@ -111,7 +112,7 @@ class PortManager:
 
         return self._udp_port_range
 
-    @udp_host.setter
+    @udp_port_range.setter
     def udp_port_range(self, new_range):
 
         assert isinstance(new_range, tuple)
@@ -140,7 +141,7 @@ class PortManager:
         """
 
         if end_port < start_port:
-            raise Exception("Invalid port range {}-{}".format(start_port, end_port))
+            raise HTTPConflict(text="Invalid port range {}-{}".format(start_port, end_port))
 
         if socket_type == "UDP":
             socket_type = socket.SOCK_DGRAM
@@ -202,6 +203,8 @@ class PortManager:
 
         if port in self._used_tcp_ports:
             raise HTTPConflict(text="TCP port {} already in use on host".format(port, self._console_host))
+        if port < self._console_port_range[0] or port > self._console_port_range[1]:
+            raise HTTPConflict(text="TCP port {} is outside the range {}-{}".format(port, self._console_port_range[0], self._console_port_range[1]))
         self._used_tcp_ports.add(port)
         project.record_tcp_port(port)
         log.debug("TCP port {} has been reserved".format(port))
@@ -248,23 +251,20 @@ class PortManager:
 
         if port in self._used_udp_ports:
             raise HTTPConflict(text="UDP port {} already in use on host".format(port, self._console_host))
+        if port < self._udp_port_range[0] or port > self._udp_port_range[1]:
+            raise HTTPConflict(text="UDP port {} is outside the range {}-{}".format(port, self._udp_port_range[0], self._udp_port_range[1]))
         self._used_udp_ports.add(port)
         project.record_udp_port(port)
         log.debug("UDP port {} has been reserved".format(port))
 
-    def release_udp_port(self, port, project, force_remove=False):
+    def release_udp_port(self, port, project):
         """
         Release a specific UDP port number
 
         :param port: UDP port number
         :param project: Project instance
-        :param force_remove: Force port removal even on Darwnin
         """
 
-        # A bug with Dynamips on Darwin which doesn't correctly free UDP ports, they are freed only when changing the project
-        if sys.platform.startswith("darwin") and force_remove is False:
-            return
-
         if port in self._used_udp_ports:
             self._used_udp_ports.remove(port)
             project.remove_udp_port(port)

gns3server/modules/project.py

@@ -102,6 +102,7 @@ class Project:
 
         server_config = Config.instance().get_section_config("Server")
         path = os.path.expanduser(server_config.get("projects_path", "~/GNS3/projects"))
+        path = os.path.normpath(path)
         try:
             os.makedirs(path, exist_ok=True)
         except OSError as e:
@@ -138,9 +139,25 @@ class Project:
             if path != self._path and self.is_local() is False:
                 raise aiohttp.web.HTTPForbidden(text="You are not allowed to modify the project directory location")
 
+        old_path = None
+        if hasattr(self, "_path"):
+            old_path = self._path
+
         self._path = path
         self._update_temporary_file()
 
+    @asyncio.coroutine
+    def clean_old_path(self, old_path):
+        """
+        Called after a project location change. All the modules should
+        have been notified before
+        """
+        if self._temporary:
+            try:
+                yield from wait_run_in_executor(shutil.rmtree, old_path)
+            except OSError as e:
+                log.warn("Can't remove temporary directory {}: {}".format(old_path, e))
+
     @property
     def name(self):
 
@@ -227,7 +244,10 @@ class Project:
                 raise aiohttp.web.HTTPInternalServerError(text="Could not create temporary project: {}".format(e))
         else:
             if os.path.exists(os.path.join(self._path, ".gns3_temporary")):
-                os.remove(os.path.join(self._path, ".gns3_temporary"))
+                try:
+                    os.remove(os.path.join(self._path, ".gns3_temporary"))
+                except OSError as e:
+                    raise aiohttp.web.HTTPInternalServerError(text="Could not mark project as no longer temporary: {}".format(e))
 
     def module_working_directory(self, module_name):
         """
@@ -343,7 +363,7 @@ class Project:
             for future in done:
                 try:
                     future.result()
-                except Exception as e:
+                except (Exception, GeneratorExit) as e:
                     log.error("Could not close VM or device {}".format(e), exc_info=1)
 
         if cleanup and os.path.exists(self.path):
@@ -365,7 +385,7 @@ class Project:
         for port in self._used_tcp_ports.copy():
             port_manager.release_tcp_port(port, self)
         for port in self._used_udp_ports.copy():
-            port_manager.release_udp_port(port, self, force_remove=True)
+            port_manager.release_udp_port(port, self)
 
     @asyncio.coroutine
     def commit(self):

gns3server/modules/qemu/__init__.py

@@ -30,6 +30,9 @@ from ..base_manager import BaseManager
 from .qemu_error import QemuError
 from .qemu_vm import QemuVM
 
+import logging
+log = logging.getLogger(__name__)
+
 
 class Qemu(BaseManager):
 
@@ -44,7 +47,15 @@ class Qemu(BaseManager):
         """
 
         qemus = []
-        paths = [os.getcwd()] + os.environ["PATH"].split(os.pathsep)
+        paths = set()
+        try:
+            paths.add(os.getcwd())
+        except FileNotFoundError:
+            log.warning("The current working directory doesn't exist")
+        if "PATH" in os.environ:
+            paths.update(os.environ["PATH"].split(os.pathsep))
+        else:
+            log.warning("The PATH environment variable doesn't exist")
         # look for Qemu binaries in the current working directory and $PATH
         if sys.platform.startswith("win"):
             # add specific Windows paths
@@ -53,25 +64,29 @@ class Qemu(BaseManager):
                 exec_dir = os.path.dirname(os.path.abspath(sys.executable))
                 for f in os.listdir(exec_dir):
                     if f.lower().startswith("qemu"):
-                        paths.append(os.path.join(exec_dir, f))
+                        paths.add(os.path.join(exec_dir, f))
 
             if "PROGRAMFILES(X86)" in os.environ and os.path.exists(os.environ["PROGRAMFILES(X86)"]):
-                paths.append(os.path.join(os.environ["PROGRAMFILES(X86)"], "qemu"))
+                paths.add(os.path.join(os.environ["PROGRAMFILES(X86)"], "qemu"))
             if "PROGRAMFILES" in os.environ and os.path.exists(os.environ["PROGRAMFILES"]):
-                paths.append(os.path.join(os.environ["PROGRAMFILES"], "qemu"))
+                paths.add(os.path.join(os.environ["PROGRAMFILES"], "qemu"))
         elif sys.platform.startswith("darwin"):
             # add specific locations on Mac OS X regardless of what's in $PATH
-            paths.extend(["/usr/local/bin", "/opt/local/bin"])
+            paths.update(["/usr/local/bin", "/opt/local/bin"])
             if hasattr(sys, "frozen"):
-                paths.append(os.path.abspath(os.path.join(os.getcwd(), "../../../qemu/bin/")))
+                try:
+                    paths.add(os.path.abspath(os.path.join(os.getcwd(), "../../../qemu/bin/")))
+                # If the user run the server by hand from outside
+                except FileNotFoundError:
+                    paths.add("/Applications/GNS3.app/Contents/Resources/qemu/bin")
         for path in paths:
             try:
                 for f in os.listdir(path):
-                    if (f.startswith("qemu-system") or f == "qemu" or f == "qemu.exe") and \
+                    if (f.startswith("qemu-system") or f.startswith("qemu-kvm") or f == "qemu" or f == "qemu.exe") and \
                             os.access(os.path.join(path, f), os.X_OK) and \
                             os.path.isfile(os.path.join(path, f)):
                         qemu_path = os.path.join(path, f)
-                        version = yield from Qemu._get_qemu_version(qemu_path)
+                        version = yield from Qemu.get_qemu_version(qemu_path)
                         qemus.append({"path": qemu_path, "version": version})
             except OSError:
                 continue
@@ -80,7 +95,7 @@ class Qemu(BaseManager):
 
     @staticmethod
     @asyncio.coroutine
-    def _get_qemu_version(qemu_path):
+    def get_qemu_version(qemu_path):
         """
         Gets the Qemu version.
 
@@ -88,17 +103,30 @@ class Qemu(BaseManager):
         """
 
         if sys.platform.startswith("win"):
+            # Qemu on Windows doesn't return anything with parameter -version
+            # look for a version number in version.txt file in the same directory instead
+            version_file = os.path.join(os.path.dirname(qemu_path), "version.txt")
+            if os.path.isfile(version_file):
+                try:
+                    with open(version_file, "rb") as file:
+                        version = file.read().decode("utf-8").strip()
+                        match = re.search("[0-9\.]+", version)
+                        if match:
+                            return version
+                except (UnicodeDecodeError, OSError) as e:
+                    log.warn("could not read {}: {}".format(version_file, e))
             return ""
-        try:
-            output = yield from subprocess_check_output(qemu_path, "-version")
-            match = re.search("version\s+([0-9a-z\-\.]+)", output)
-            if match:
-                version = match.group(1)
-                return version
-            else:
-                raise QemuError("Could not determine the Qemu version for {}".format(qemu_path))
-        except subprocess.SubprocessError as e:
-            raise QemuError("Error while looking for the Qemu version: {}".format(e))
+        else:
+            try:
+                output = yield from subprocess_check_output(qemu_path, "-version")
+                match = re.search("version\s+([0-9a-z\-\.]+)", output)
+                if match:
+                    version = match.group(1)
+                    return version
+                else:
+                    raise QemuError("Could not determine the Qemu version for {}".format(qemu_path))
+            except subprocess.SubprocessError as e:
+                raise QemuError("Error while looking for the Qemu version: {}".format(e))
 
     @staticmethod
     def get_legacy_vm_workdir(legacy_vm_id, name):
@@ -112,3 +140,9 @@ class Qemu(BaseManager):
         """
 
         return os.path.join("qemu", "vm-{}".format(legacy_vm_id))
+
+    def get_images_directory(self):
+        """
+        Return the full path of the images directory on disk
+        """
+        return os.path.join(os.path.expanduser(self.config.get_section_config("Server").get("images_path", "~/GNS3/images")), "QEMU")

gns3server/modules/qemu/qemu_vm.py

@@ -23,15 +23,17 @@ order to run a QEMU VM.
 import sys
 import os
 import shutil
-import random
 import subprocess
 import shlex
 import asyncio
 import socket
 
+from pkg_resources import parse_version
 from .qemu_error import QemuError
 from ..adapters.ethernet_adapter import EthernetAdapter
 from ..nios.nio_udp import NIOUDP
+from ..nios.nio_tap import NIOTAP
+from ..nios.nio_nat import NIONAT
 from ..base_vm import BaseVM
 from ...schemas.qemu import QEMU_OBJECT_SCHEMA
 
@@ -68,7 +70,7 @@ class QemuVM(BaseVM):
         self._stdout_file = ""
 
         # QEMU VM settings
-        self._qemu_path = qemu_path
+        self.qemu_path = qemu_path
         self._hda_disk_image = ""
         self._hdb_disk_image = ""
         self._hdc_disk_image = ""
@@ -148,14 +150,10 @@ class QemuVM(BaseVM):
         :param hda_disk_image: QEMU hda disk image path
         """
 
-        if not os.path.isabs(hda_disk_image):
-            server_config = self.manager.config.get_section_config("Server")
-            hda_disk_image = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), "QEMU", hda_disk_image)
-
+        self._hda_disk_image = self.manager.get_abs_image_path(hda_disk_image)
         log.info('QEMU VM "{name}" [{id}] has set the QEMU hda disk image path to {disk_image}'.format(name=self._name,
                                                                                                        id=self._id,
-                                                                                                       disk_image=hda_disk_image))
-        self._hda_disk_image = hda_disk_image
+                                                                                                       disk_image=self._hda_disk_image))
 
     @property
     def hdb_disk_image(self):
@@ -175,14 +173,10 @@ class QemuVM(BaseVM):
         :param hdb_disk_image: QEMU hdb disk image path
         """
 
-        if not os.path.isabs(hdb_disk_image):
-            server_config = self.manager.config.get_section_config("Server")
-            hdb_disk_image = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), "QEMU", hdb_disk_image)
-
+        self._hdb_disk_image = self.manager.get_abs_image_path(hdb_disk_image)
         log.info('QEMU VM "{name}" [{id}] has set the QEMU hdb disk image path to {disk_image}'.format(name=self._name,
                                                                                                        id=self._id,
-                                                                                                       disk_image=hdb_disk_image))
-        self._hdb_disk_image = hdb_disk_image
+                                                                                                       disk_image=self._hdb_disk_image))
 
     @property
     def hdc_disk_image(self):
@@ -202,14 +196,10 @@ class QemuVM(BaseVM):
         :param hdc_disk_image: QEMU hdc disk image path
         """
 
-        if not os.path.isabs(hdc_disk_image):
-            server_config = self.manager.config.get_section_config("Server")
-            hdc_disk_image = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), "QEMU", hdc_disk_image)
-
+        self._hdc_disk_image = self.manager.get_abs_image_path(hdc_disk_image)
         log.info('QEMU VM "{name}" [{id}] has set the QEMU hdc disk image path to {disk_image}'.format(name=self._name,
                                                                                                        id=self._id,
-                                                                                                       disk_image=hdc_disk_image))
-        self._hdc_disk_image = hdc_disk_image
+                                                                                                       disk_image=self._hdc_disk_image))
 
     @property
     def hdd_disk_image(self):
@@ -229,14 +219,11 @@ class QemuVM(BaseVM):
         :param hdd_disk_image: QEMU hdd disk image path
         """
 
-        if not os.path.isabs(hdd_disk_image):
-            server_config = self.manager.config.get_section_config("Server")
-            hdd_disk_image = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), "QEMU", hdd_disk_image)
-
+        self._hdd_disk_image = hdd_disk_image
+        self._hdd_disk_image = self.manager.get_abs_image_path(hdd_disk_image)
         log.info('QEMU VM "{name}" [{id}] has set the QEMU hdd disk image path to {disk_image}'.format(name=self._name,
                                                                                                        id=self._id,
-                                                                                                       disk_image=hdd_disk_image))
-        self._hdd_disk_image = hdd_disk_image
+                                                                                                       disk_image=self._hdd_disk_image))
 
     @property
     def adapters(self):
@@ -589,7 +576,7 @@ class QemuVM(BaseVM):
                 log.info("Starting QEMU: {}".format(self._command))
                 self._stdout_file = os.path.join(self.working_dir, "qemu.log")
                 log.info("logging to {}".format(self._stdout_file))
-                with open(self._stdout_file, "w") as fd:
+                with open(self._stdout_file, "w", encoding="utf-8") as fd:
                     self._process = yield from asyncio.create_subprocess_exec(*self._command,
                                                                               stdout=fd,
                                                                               stderr=subprocess.STDOUT,
@@ -601,7 +588,7 @@ class QemuVM(BaseVM):
                 log.error("Could not start QEMU {}: {}\n{}".format(self.qemu_path, e, stdout))
                 raise QemuError("Could not start QEMU {}: {}\n{}".format(self.qemu_path, e, stdout))
 
-            self._set_process_priority()
+            yield from self._set_process_priority()
             if self._cpu_throttling:
                 self._set_cpu_throttling()
 
@@ -616,11 +603,16 @@ class QemuVM(BaseVM):
             log.info('Stopping QEMU VM "{}" PID={}'.format(self._name, self._process.pid))
             try:
                 self._process.terminate()
-                self._process.wait()
+                yield from self._process.wait()
             except subprocess.TimeoutExpired:
-                self._process.kill()
+                try:
+                    self._process.kill()
+                except OSError as e:
+                    log.error("Cannot stop the Qemu process: {}".format(e))
                 if self._process.returncode is None:
-                    log.warn('QEMU VM "{}" PID={} is still running'.format(self._name, self._process.pid))
+                    log.warn('QEMU VM "{}" with PID={} is still running'.format(self._name, self._process.pid))
+            except ProcessLookupError:
+                pass
         self._process = None
         self._started = False
         self._stop_cpulimit()
@@ -659,7 +651,7 @@ class QemuVM(BaseVM):
                             break
                         for expect in expected:
                             if expect in line:
-                                result = line.decode().strip()
+                                result = line.decode("utf-8").strip()
                                 break
                 except EOFError as e:
                     log.warn("Could not read from QEMU monitor: {}".format(e))
@@ -673,11 +665,19 @@ class QemuVM(BaseVM):
         """
 
         log.debug('QEMU VM "{name}" [{id}] is closing'.format(name=self._name, id=self._id))
-        yield from self.stop()
+
         if self._console:
             self._manager.port_manager.release_tcp_port(self._console, self._project)
             self._console = None
 
+        for adapter in self._ethernet_adapters:
+            if adapter is not None:
+                for nio in adapter.ports.values():
+                    if nio and isinstance(nio, NIOUDP):
+                        self.manager.port_manager.release_udp_port(nio.lport, self._project)
+
+        yield from self.stop()
+
     @asyncio.coroutine
     def _get_vm_status(self):
         """
@@ -755,6 +755,8 @@ class QemuVM(BaseVM):
                                                                                                  adapter_number=adapter_number))
 
         if self.is_running():
+            raise QemuError("Sorry, adding a link to a started Qemu VM is not supported.")
+            # FIXME: does the code below work? very undocumented feature...
             # dynamically configure an UDP tunnel on the QEMU VM adapter
             if nio and isinstance(nio, NIOUDP):
                 if self._legacy_networking:
@@ -765,7 +767,6 @@ class QemuVM(BaseVM):
                                                                                                                           nio.rport,
                                                                                                                           nio.rhost))
                 else:
-                    # FIXME: does it work? very undocumented feature...
                     # Apparently there is a bug in Qemu...
                     # netdev_add [user|tap|socket|hubport|netmap],id=str[,prop=value][,...] -- add host network device
                     # netdev_del id -- remove host network device
@@ -778,9 +779,9 @@ class QemuVM(BaseVM):
 
         adapter.add_nio(0, nio)
         log.info('QEMU VM "{name}" [{id}]: {nio} added to adapter {adapter_number}'.format(name=self._name,
-                                                                                          id=self._id,
-                                                                                          nio=nio,
-                                                                                          adapter_number=adapter_number))
+                                                                                           id=self._id,
+                                                                                           nio=nio,
+                                                                                           adapter_number=adapter_number))
 
     @asyncio.coroutine
     def adapter_remove_nio_binding(self, adapter_number):
@@ -799,6 +800,7 @@ class QemuVM(BaseVM):
                                                                                                  adapter_number=adapter_number))
 
         if self.is_running():
+            # FIXME: does the code below work? very undocumented feature...
             # dynamically disable the QEMU VM adapter
             yield from self._control_vm("host_net_remove {} gns3-{}".format(adapter_number, adapter_number))
             yield from self._control_vm("host_net_add user vlan={},name=gns3-{}".format(adapter_number, adapter_number))
@@ -832,8 +834,8 @@ class QemuVM(BaseVM):
         output = ""
         if self._stdout_file:
             try:
-                with open(self._stdout_file, errors="replace") as file:
-                    output = file.read()
+                with open(self._stdout_file, "rb") as file:
+                    output = file.read().decode("utf-8", errors="replace")
             except OSError as e:
                 log.warn("Could not read {}: {}".format(self._stdout_file, e))
         return output
@@ -995,40 +997,66 @@ class QemuVM(BaseVM):
 
         return options
 
-    def _get_random_mac(self, adapter_number):
-        # TODO: let users specify a base mac address
-        return "00:00:ab:%02x:%02x:%02d" % (random.randint(0x00, 0xff), random.randint(0x00, 0xff), adapter_number)
-
+    @asyncio.coroutine
     def _network_options(self):
 
         network_options = []
-        adapter_number = 0
-        for adapter in self._ethernet_adapters:
-            mac = self._get_random_mac(adapter_number)
-            if self._legacy_networking:
-                network_options.extend(["-net", "nic,vlan={},macaddr={},model={}".format(adapter_number, mac, self._adapter_type)])
-            else:
-                network_options.extend(["-device", "{},mac={},netdev=gns3-{}".format(self._adapter_type, mac, adapter_number)])
+        network_options.extend(["-net", "none"])  # we do not want any user networking back-end if no adapter is connected.
+
+        patched_qemu = False
+        if self._legacy_networking:
+            version = yield from self.manager.get_qemu_version(self.qemu_path)
+            if version and parse_version(version) < parse_version("1.1.0"):
+                # this is a patched Qemu if version is below 1.1.0
+                patched_qemu = True
+
+        for adapter_number, adapter in enumerate(self._ethernet_adapters):
+            # TODO: let users specify a base mac address
+            mac = "00:00:ab:%s:%s:%02x" % (self.id[-4:-2], self.id[-2:], adapter_number)
             nio = adapter.get_nio(0)
-            if nio and isinstance(nio, NIOUDP):
-                if self._legacy_networking:
-                    network_options.extend(["-net", "udp,vlan={},name=gns3-{},sport={},dport={},daddr={}".format(adapter_number,
-                                                                                                                 adapter_number,
-                                                                                                                 nio.lport,
-                                                                                                                 nio.rport,
-                                                                                                                 nio.rhost)])
+            if self._legacy_networking:
+                # legacy QEMU networking syntax (-net)
+                if nio:
+                    network_options.extend(["-net", "nic,vlan={},macaddr={},model={}".format(adapter_number, mac, self._adapter_type)])
+                    if isinstance(nio, NIOUDP):
+                        if patched_qemu:
+                            # use patched Qemu syntax
+                            network_options.extend(["-net", "udp,vlan={},name=gns3-{},sport={},dport={},daddr={}".format(adapter_number,
+                                                                                                                         adapter_number,
+                                                                                                                         nio.lport,
+                                                                                                                         nio.rport,
+                                                                                                                         nio.rhost)])
+                        else:
+                            # use UDP tunnel support added in Qemu 1.1.0
+                            network_options.extend(["-net", "socket,vlan={},name=gns3-{},udp={}:{},localaddr={}:{}".format(adapter_number,
+                                                                                                                           adapter_number,
+                                                                                                                           nio.rhost,
+                                                                                                                           nio.rport,
+                                                                                                                           self._host,
+                                                                                                                           nio.lport)])
+                    elif isinstance(nio, NIOTAP):
+                        network_options.extend(["-net", "tap,name=gns3-{},ifname={}".format(adapter_number, nio.tap_device)])
+                    elif isinstance(nio, NIONAT):
+                        network_options.extend(["-net", "user,vlan={},name=gns3-{}".format(adapter_number, adapter_number)])
                 else:
-                    network_options.extend(["-netdev", "socket,id=gns3-{},udp={}:{},localaddr={}:{}".format(adapter_number,
-                                                                                                            nio.rhost,
-                                                                                                            nio.rport,
-                                                                                                            self._host,
-                                                                                                            nio.lport)])
+                    network_options.extend(["-net", "nic,vlan={},macaddr={},model={}".format(adapter_number, mac, self._adapter_type)])
+
             else:
-                if self._legacy_networking:
-                    network_options.extend(["-net", "user,vlan={},name=gns3-{}".format(adapter_number, adapter_number)])
+                # newer QEMU networking syntax
+                if nio:
+                    network_options.extend(["-device", "{},mac={},netdev=gns3-{}".format(self._adapter_type, mac, adapter_number)])
+                    if isinstance(nio, NIOUDP):
+                        network_options.extend(["-netdev", "socket,id=gns3-{},udp={}:{},localaddr={}:{}".format(adapter_number,
+                                                                                                                nio.rhost,
+                                                                                                                nio.rport,
+                                                                                                                self._host,
+                                                                                                                nio.lport)])
+                    elif isinstance(nio, NIOTAP):
+                        network_options.extend(["-netdev", "tap,id=gns3-{},ifname={},script=no,downscript=no".format(adapter_number, nio.tap_device)])
+                    elif isinstance(nio, NIONAT):
+                        network_options.extend(["-netdev", "user,id=gns3-{}".format(adapter_number)])
                 else:
-                    network_options.extend(["-netdev", "user,id=gns3-{}".format(adapter_number)])
-            adapter_number += 1
+                    network_options.extend(["-device", "{},mac={}".format(self._adapter_type, mac)])
 
         return network_options
 
@@ -1053,8 +1081,7 @@ class QemuVM(BaseVM):
         command = [self.qemu_path]
         command.extend(["-name", self._name])
         command.extend(["-m", str(self._ram)])
-        disk_options = yield from self._disk_options()
-        command.extend(disk_options)
+        command.extend((yield from self._disk_options()))
         command.extend(self._linux_boot_options())
         command.extend(self._serial_options())
         command.extend(self._monitor_options())
@@ -1064,27 +1091,10 @@ class QemuVM(BaseVM):
                 command.extend(shlex.split(additional_options))
             except ValueError as e:
                 QemuError("Invalid additional options: {} error {}".format(additional_options, e))
-        command.extend(self._network_options())
+        command.extend((yield from self._network_options()))
         command.extend(self._graphic())
         return command
 
-    def _get_relative_disk_image_path(self, disk_image):
-        """
-        Returns a relative image path if the disk image is in the images directory.
-
-        :param disk_image: path to the disk image
-
-        :returns: relative or full path
-        """
-
-        if disk_image:
-            # return the relative path if the disk image is in the images_path directory
-            server_config = self.manager.config.get_section_config("Server")
-            relative_image = os.path.join(os.path.expanduser(server_config.get("images_path", "~/GNS3/images")), "QEMU", disk_image)
-            if os.path.exists(relative_image):
-                return os.path.basename(disk_image)
-        return disk_image
-
     def __json__(self):
         answer = {
             "project_id": self.project.id,
@@ -1095,11 +1105,11 @@ class QemuVM(BaseVM):
             if field not in answer:
                 answer[field] = getattr(self, field)
 
-        answer["hda_disk_image"] = self._get_relative_disk_image_path(self._hda_disk_image)
-        answer["hdb_disk_image"] = self._get_relative_disk_image_path(self._hdb_disk_image)
-        answer["hdc_disk_image"] = self._get_relative_disk_image_path(self._hdc_disk_image)
-        answer["hdd_disk_image"] = self._get_relative_disk_image_path(self._hdd_disk_image)
-        answer["initrd"] = self._get_relative_disk_image_path(self._initrd)
-        answer["kernel_image"] = self._get_relative_disk_image_path(self._kernel_image)
+        answer["hda_disk_image"] = self.manager.get_relative_image_path(self._hda_disk_image)
+        answer["hdb_disk_image"] = self.manager.get_relative_image_path(self._hdb_disk_image)
+        answer["hdc_disk_image"] = self.manager.get_relative_image_path(self._hdc_disk_image)
+        answer["hdd_disk_image"] = self.manager.get_relative_image_path(self._hdd_disk_image)
+        answer["initrd"] = self.manager.get_relative_image_path(self._initrd)
+        answer["kernel_image"] = self.manager.get_relative_image_path(self._kernel_image)
 
         return answer

gns3server/modules/virtualbox/__init__.py

@@ -41,6 +41,7 @@ class VirtualBox(BaseManager):
 
         super().__init__()
         self._vboxmanage_path = None
+        self._execute_lock = asyncio.Lock()
 
     @property
     def vboxmanage_path(self):
@@ -82,34 +83,81 @@ class VirtualBox(BaseManager):
     @asyncio.coroutine
     def execute(self, subcommand, args, timeout=60):
 
-        vboxmanage_path = self.vboxmanage_path
-        if not vboxmanage_path:
-            vboxmanage_path = self.find_vboxmanage()
-        command = [vboxmanage_path, "--nologo", subcommand]
-        command.extend(args)
-        log.debug("Executing VBoxManage with command: {}".format(command))
+        # We use a lock prevent parallel execution due to strange errors
+        # reported by a user and reproduced by us.
+        # https://github.com/GNS3/gns3-gui/issues/261
+        with (yield from self._execute_lock):
+            vboxmanage_path = self.vboxmanage_path
+            if not vboxmanage_path:
+                vboxmanage_path = self.find_vboxmanage()
+            command = [vboxmanage_path, "--nologo", subcommand]
+            command.extend(args)
+            log.debug("Executing VBoxManage with command: {}".format(command))
+            try:
+                vbox_user = self.config.get_section_config("VirtualBox").get("vbox_user")
+                if vbox_user:
+                    # TODO: test & review this part
+                    sudo_command = "sudo -i -u {} ".format(vbox_user) + " ".join(command)
+                    process = yield from asyncio.create_subprocess_shell(sudo_command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)
+                else:
+                    process = yield from asyncio.create_subprocess_exec(*command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)
+            except (OSError, subprocess.SubprocessError) as e:
+                raise VirtualBoxError("Could not execute VBoxManage: {}".format(e))
+
+            try:
+                stdout_data, stderr_data = yield from asyncio.wait_for(process.communicate(), timeout=timeout)
+            except asyncio.TimeoutError:
+                raise VirtualBoxError("VBoxManage has timed out after {} seconds!".format(timeout))
+
+            if process.returncode:
+                vboxmanage_error = stderr_data.decode("utf-8", errors="ignore")
+                raise VirtualBoxError("VirtualBox has returned an error: {}".format(vboxmanage_error))
+
+            return stdout_data.decode("utf-8", errors="ignore").splitlines()
+
+    @asyncio.coroutine
+    def _find_inaccessible_hdd_files(self):
+        """
+        Finds inaccessible disk files (to clean up the VirtualBox media manager)
+        """
+
+        hdds = []
         try:
-            vbox_user = self.config.get_section_config("VirtualBox").get("vbox_user")
-            if vbox_user:
-                # TODO: test & review this part
-                sudo_command = "sudo -i -u {}".format(vbox_user) + " ".join(command)
-                process = yield from asyncio.create_subprocess_shell(sudo_command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)
-            else:
-                process = yield from asyncio.create_subprocess_exec(*command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE)
-        except (OSError, subprocess.SubprocessError) as e:
-            raise VirtualBoxError("Could not execute VBoxManage: {}".format(e))
+            properties = yield from self.execute("list", ["hdds"])
+        # If VirtualBox is not available we have no inaccessible hdd
+        except VirtualBoxError:
+            return hdds
 
-        try:
-            stdout_data, stderr_data = yield from asyncio.wait_for(process.communicate(), timeout=timeout)
-        except asyncio.TimeoutError:
-            raise VirtualBoxError("VBoxManage has timed out after {} seconds!".format(timeout))
+        flag_inaccessible = False
+        for prop in properties:
+            try:
+                name, value = prop.split(':', 1)
+            except ValueError:
+                continue
+            if name.strip() == "State" and value.strip() == "inaccessible":
+                flag_inaccessible = True
+            if flag_inaccessible and name.strip() == "Location":
+                hdds.append(value.strip())
+                flag_inaccessible = False
+        return reversed(hdds)
 
-        if process.returncode:
-            # only the first line of the output is useful
-            vboxmanage_error = stderr_data.decode("utf-8", errors="ignore")
-            raise VirtualBoxError("VirtualBox has returned an error: {}".format(vboxmanage_error))
+    @asyncio.coroutine
+    def project_closed(self, project):
+        """
+        Called when a project is closed.
 
-        return stdout_data.decode("utf-8", errors="ignore").splitlines()
+        :param project: Project instance
+        """
+
+        yield from super().project_closed(project)
+        hdd_files_to_close = yield from self._find_inaccessible_hdd_files()
+        for hdd_file in hdd_files_to_close:
+            log.info("Closing VirtualBox VM disk file {}".format(os.path.basename(hdd_file)))
+            try:
+                yield from self.execute("closemedium", ["disk", hdd_file])
+            except VirtualBoxError as e:
+                log.warning("Could not close VirtualBox VM disk file {}: {}".format(os.path.basename(hdd_file), e))
+                continue
 
     @asyncio.coroutine
     def get_list(self):
@@ -120,6 +168,8 @@ class VirtualBox(BaseManager):
         vms = []
         result = yield from self.execute("list", ["vms"])
         for line in result:
+            if len(line) == 0 or line[0] != '"' or line[-1:] != "}":
+                continue # Broken output (perhaps a carriage return in VM name
             vmname, _ = line.rsplit(' ', 1)
             vmname = vmname.strip('"')
             if vmname == "<inaccessible>":
@@ -128,6 +178,7 @@ class VirtualBox(BaseManager):
             if not extra_data[0].strip() == "Value: yes":
                 # get the amount of RAM
                 info_results = yield from self.execute("showvminfo", [vmname, "--machinereadable"])
+                ram = 0
                 for info in info_results:
                     try:
                         name, value = info.split('=', 1)

gns3server/modules/virtualbox/virtualbox_vm.py

@@ -31,6 +31,7 @@ import asyncio
 from pkg_resources import parse_version
 from .virtualbox_error import VirtualBoxError
 from ..nios.nio_udp import NIOUDP
+from ..nios.nio_nat import NIONAT
 from ..adapters.ethernet_adapter import EthernetAdapter
 from .telnet_server import TelnetServer  # TODO: port TelnetServer to asyncio
 from ..base_vm import BaseVM
@@ -105,9 +106,10 @@ class VirtualBoxVM(BaseVM):
 
         results = yield from self.manager.execute("showvminfo", [self._vmname, "--machinereadable"])
         for info in results:
-            name, value = info.split('=', 1)
-            if name == "VMState":
-                return value.strip('"')
+            if '=' in info:
+                name, value = info.split('=', 1)
+                if name == "VMState":
+                    return value.strip('"')
         raise VirtualBoxError("Could not get VM state for {}".format(self._vmname))
 
     @asyncio.coroutine
@@ -139,6 +141,8 @@ class VirtualBoxVM(BaseVM):
     def create(self):
 
         yield from self._get_system_properties()
+        if "API version" not in self._system_properties:
+            raise VirtualBoxError("Can't access to VirtualBox API Version")
         if parse_version(self._system_properties["API version"]) < parse_version("4_3"):
             raise VirtualBoxError("The VirtualBox API version is lower than 4.3")
         log.info("VirtualBox VM '{name}' [{id}] created".format(name=self.name, id=self.id))
@@ -206,7 +210,7 @@ class VirtualBoxVM(BaseVM):
             log.info("VirtualBox VM '{name}' [{id}] stopped".format(name=self.name, id=self.id))
             log.debug("Stop result: {}".format(result))
 
-            yield from asyncio.sleep(0.5)  # give some time for VirtualBox to unlock the VM
+            # yield from asyncio.sleep(0.5)  # give some time for VirtualBox to unlock the VM
             try:
                 # deactivate the first serial port
                 yield from self._modify_vm("--uart1 off")
@@ -273,9 +277,9 @@ class VirtualBoxVM(BaseVM):
 
         hdd_info_file = os.path.join(self.working_dir, self._vmname, "hdd_info.json")
         try:
-            with open(hdd_info_file, "r") as f:
+            with open(hdd_info_file, "r", encoding="utf-8") as f:
                 hdd_table = json.load(f)
-        except OSError as e:
+        except (ValueError, OSError) as e:
             raise VirtualBoxError("Could not read HDD info file: {}".format(e))
 
         for hdd_info in hdd_table:
@@ -351,7 +355,7 @@ class VirtualBoxVM(BaseVM):
             if hdd_table:
                 try:
                     hdd_info_file = os.path.join(self.working_dir, self._vmname, "hdd_info.json")
-                    with open(hdd_info_file, "w") as f:
+                    with open(hdd_info_file, "w", encoding="utf-8") as f:
                         json.dump(hdd_table, f, indent=4)
                 except OSError as e:
                     log.warning("VirtualBox VM '{name}' [{id}] could not write HHD info file: {error}".format(name=self.name,
@@ -570,10 +574,11 @@ class VirtualBoxVM(BaseVM):
 
         # check the maximum number of adapters supported by the VM
         vm_info = yield from self._get_vm_info()
-        chipset = vm_info["chipset"]
         maximum_adapters = 8
-        if chipset == "ich9":
-            maximum_adapters = int(self._system_properties["Maximum ICH9 Network Adapter count"])
+        if "chipset" in vm_info:
+            chipset = vm_info["chipset"]
+            if chipset == "ich9":
+                maximum_adapters = int(self._system_properties["Maximum ICH9 Network Adapter count"])
         return maximum_adapters
 
     def _get_pipe_name(self):
@@ -583,12 +588,14 @@ class VirtualBoxVM(BaseVM):
         :returns: pipe path (string)
         """
 
-        p = re.compile('\s+', re.UNICODE)
-        pipe_name = p.sub("_", self._vmname)
         if sys.platform.startswith("win"):
-            pipe_name = r"\\.\pipe\VBOX\{}".format(pipe_name)
+            pipe_name = r"\\.\pipe\gns3_vbox\{}".format(self.id)
         else:
-            pipe_name = os.path.join(tempfile.gettempdir(), "pipe_{}".format(pipe_name))
+            pipe_name = os.path.join(tempfile.gettempdir(), "gns3_vbox", "{}".format(self.id))
+            try:
+                os.makedirs(os.path.dirname(pipe_name), exist_ok=True)
+            except OSError as e:
+                raise VirtualBoxError("Could not create the VirtualBox pipe directory: {}".format(e))
         return pipe_name
 
     @asyncio.coroutine
@@ -654,12 +661,12 @@ class VirtualBoxVM(BaseVM):
                 yield from self._modify_vm("--cableconnected{} off".format(adapter_number + 1))
             nio = self._ethernet_adapters[adapter_number].get_nio(0)
             if nio:
-                if not self._use_any_adapter and attachment not in ("none", "null", "generic"):
+                if not isinstance(nio, NIONAT) and not self._use_any_adapter and attachment not in ("none", "null", "generic"):
                     raise VirtualBoxError("Attachment ({}) already configured on adapter {}. "
                                           "Please set it to 'Not attached' to allow GNS3 to use it.".format(attachment,
                                                                                                             adapter_number + 1))
-                yield from self._modify_vm("--nictrace{} off".format(adapter_number + 1))
 
+                yield from self._modify_vm("--nictrace{} off".format(adapter_number + 1))
                 vbox_adapter_type = "82540EM"
                 if self._adapter_type == "PCnet-PCI II (Am79C970A)":
                     vbox_adapter_type = "Am79C970A"
@@ -676,13 +683,17 @@ class VirtualBoxVM(BaseVM):
                 args = [self._vmname, "--nictype{}".format(adapter_number + 1), vbox_adapter_type]
                 yield from self.manager.execute("modifyvm", args)
 
-                log.debug("setting UDP params on adapter {}".format(adapter_number))
-                yield from self._modify_vm("--nic{} generic".format(adapter_number + 1))
-                yield from self._modify_vm("--nicgenericdrv{} UDPTunnel".format(adapter_number + 1))
-                yield from self._modify_vm("--nicproperty{} sport={}".format(adapter_number + 1, nio.lport))
-                yield from self._modify_vm("--nicproperty{} dest={}".format(adapter_number + 1, nio.rhost))
-                yield from self._modify_vm("--nicproperty{} dport={}".format(adapter_number + 1, nio.rport))
-                yield from self._modify_vm("--cableconnected{} on".format(adapter_number + 1))
+                if isinstance(nio, NIOUDP):
+                    log.debug("setting UDP params on adapter {}".format(adapter_number))
+                    yield from self._modify_vm("--nic{} generic".format(adapter_number + 1))
+                    yield from self._modify_vm("--nicgenericdrv{} UDPTunnel".format(adapter_number + 1))
+                    yield from self._modify_vm("--nicproperty{} sport={}".format(adapter_number + 1, nio.lport))
+                    yield from self._modify_vm("--nicproperty{} dest={}".format(adapter_number + 1, nio.rhost))
+                    yield from self._modify_vm("--nicproperty{} dport={}".format(adapter_number + 1, nio.rport))
+                    yield from self._modify_vm("--cableconnected{} on".format(adapter_number + 1))
+                elif isinstance(nio, NIONAT):
+                    yield from self._modify_vm("--nic{} nat".format(adapter_number + 1))
+                    yield from self._modify_vm("--cableconnected{} on".format(adapter_number + 1))
 
                 if nio.capturing:
                     yield from self._modify_vm("--nictrace{} on".format(adapter_number + 1))
@@ -789,18 +800,22 @@ class VirtualBoxVM(BaseVM):
 
         try:
             adapter = self._ethernet_adapters[adapter_number]
-        except IndexError:
+        except KeyError:
             raise VirtualBoxError("Adapter {adapter_number} doesn't exist on VirtualBox VM '{name}'".format(name=self.name,
                                                                                                             adapter_number=adapter_number))
 
         vm_state = yield from self._get_vm_state()
         if vm_state == "running":
-            # dynamically configure an UDP tunnel on the VirtualBox adapter
-            yield from self._control_vm("nic{} generic UDPTunnel".format(adapter_number + 1))
-            yield from self._control_vm("nicproperty{} sport={}".format(adapter_number + 1, nio.lport))
-            yield from self._control_vm("nicproperty{} dest={}".format(adapter_number + 1, nio.rhost))
-            yield from self._control_vm("nicproperty{} dport={}".format(adapter_number + 1, nio.rport))
-            yield from self._control_vm("setlinkstate{} on".format(adapter_number + 1))
+            if isinstance(nio, NIOUDP):
+                # dynamically configure an UDP tunnel on the VirtualBox adapter
+                yield from self._control_vm("nic{} generic UDPTunnel".format(adapter_number + 1))
+                yield from self._control_vm("nicproperty{} sport={}".format(adapter_number + 1, nio.lport))
+                yield from self._control_vm("nicproperty{} dest={}".format(adapter_number + 1, nio.rhost))
+                yield from self._control_vm("nicproperty{} dport={}".format(adapter_number + 1, nio.rport))
+                yield from self._control_vm("setlinkstate{} on".format(adapter_number + 1))
+            elif isinstance(nio, NIONAT):
+                yield from self._control_vm("nic{} nat".format(adapter_number + 1))
+                yield from self._control_vm("setlinkstate{} on".format(adapter_number + 1))
 
         adapter.add_nio(0, nio)
         log.info("VirtualBox VM '{name}' [{id}]: {nio} added to adapter {adapter_number}".format(name=self.name,
@@ -820,7 +835,7 @@ class VirtualBoxVM(BaseVM):
 
         try:
             adapter = self._ethernet_adapters[adapter_number]
-        except IndexError:
+        except KeyError:
             raise VirtualBoxError("Adapter {adapter_number} doesn't exist on VirtualBox VM '{name}'".format(name=self.name,
                                                                                                             adapter_number=adapter_number))
 
@@ -841,6 +856,7 @@ class VirtualBoxVM(BaseVM):
                                                                                                      adapter_number=adapter_number))
         return nio
 
+    @asyncio.coroutine
     def start_capture(self, adapter_number, output_file):
         """
         Starts a packet capture.
@@ -851,11 +867,19 @@ class VirtualBoxVM(BaseVM):
 
         try:
             adapter = self._ethernet_adapters[adapter_number]
-        except IndexError:
+        except KeyError:
             raise VirtualBoxError("Adapter {adapter_number} doesn't exist on VirtualBox VM '{name}'".format(name=self.name,
                                                                                                             adapter_number=adapter_number))
 
+        vm_state = yield from self._get_vm_state()
+        if vm_state == "running" or vm_state == "paused" or vm_state == "stuck":
+            raise VirtualBoxError("Sorry, packet capturing on a started VirtualBox VM is not supported.")
+
         nio = adapter.get_nio(0)
+
+        if not nio:
+            raise VirtualBoxError("Adapter {} is not connected".format(adapter_number))
+
         if nio.capturing:
             raise VirtualBoxError("Packet capture is already activated on adapter {adapter_number}".format(adapter_number=adapter_number))
 
@@ -873,11 +897,15 @@ class VirtualBoxVM(BaseVM):
 
         try:
             adapter = self._ethernet_adapters[adapter_number]
-        except IndexError:
+        except KeyError:
             raise VirtualBoxError("Adapter {adapter_number} doesn't exist on VirtualBox VM '{name}'".format(name=self.name,
                                                                                                             adapter_number=adapter_number))
 
         nio = adapter.get_nio(0)
+
+        if not nio:
+            raise VirtualBoxError("Adapter {} is not connected".format(adapter_number))
+
         nio.stopPacketCapture()
 
         log.info("VirtualBox VM '{name}' [{id}]: stopping packet capture on adapter {adapter_number}".format(name=self.name,

gns3server/modules/vpcs/__init__.py

@@ -62,9 +62,9 @@ class VPCS(BaseManager):
         """
 
         vm = self.get_vm(vm_id)
-        i = self._used_mac_ids[vm_id]
-        self._free_mac_ids[vm.project.id].insert(0, i)
         if vm_id in self._used_mac_ids:
+            i = self._used_mac_ids[vm_id]
+            self._free_mac_ids[vm.project.id].insert(0, i)
             del self._used_mac_ids[vm_id]
         yield from super().close_vm(vm_id, *args, **kwargs)
         return vm

gns3server/modules/vpcs/vpcs_vm.py

@@ -27,6 +27,7 @@ import signal
 import re
 import asyncio
 import shutil
+import gns3server.utils.asyncio
 
 from pkg_resources import parse_version
 from .vpcs_error import VPCSError
@@ -166,8 +167,8 @@ class VPCSVM(BaseVM):
             return None
 
         try:
-            with open(script_file) as f:
-                return f.read()
+            with open(script_file, "rb") as f:
+                return f.read().decode("utf-8", errors="replace")
         except OSError as e:
             raise VPCSError('Cannot read the startup script file "{}": {}'.format(script_file, e))
 
@@ -180,15 +181,15 @@ class VPCSVM(BaseVM):
         """
 
         try:
-            script_file = os.path.join(self.working_dir, 'startup.vpc')
-            with open(script_file, 'w+') as f:
+            startup_script_path = os.path.join(self.working_dir, 'startup.vpc')
+            with open(startup_script_path, "w+", encoding='utf-8') as f:
                 if startup_script is None:
                     f.write('')
                 else:
                     startup_script = startup_script.replace("%h", self._name)
                     f.write(startup_script)
         except OSError as e:
-            raise VPCSError('Cannot write the startup script file "{}": {}'.format(self.script_file, e))
+            raise VPCSError('Cannot write the startup script file "{}": {}'.format(startup_script_path, e))
 
     @asyncio.coroutine
     def _check_vpcs_version(self):
@@ -226,7 +227,7 @@ class VPCSVM(BaseVM):
                 flags = 0
                 if sys.platform.startswith("win32"):
                     flags = subprocess.CREATE_NEW_PROCESS_GROUP
-                with open(self._vpcs_stdout_file, "w") as fd:
+                with open(self._vpcs_stdout_file, "w", encoding="utf-8") as fd:
                     self._process = yield from asyncio.create_subprocess_exec(*self._command,
                                                                               stdout=fd,
                                                                               stderr=subprocess.STDOUT,
@@ -247,12 +248,17 @@ class VPCSVM(BaseVM):
 
         if self.is_running():
             self._terminate_process()
-            try:
-                yield from asyncio.wait_for(self._process.wait(), timeout=3)
-            except asyncio.TimeoutError:
-                if self._process.returncode is None:
-                    log.warn("VPCS process {} is still running... killing it".format(self._process.pid))
-                    self._process.kill()
+            if self._process.returncode is None:
+                try:
+                    yield from gns3server.utils.asyncio.wait_for_process_termination(self._process, timeout=3)
+                except asyncio.TimeoutError:
+                    if self._process.returncode is None:
+                        try:
+                            self._process.kill()
+                        except OSError as e:
+                            log.error("Cannot stop the VPCS process: {}".format(e))
+                        if self._process.returncode is None:
+                            log.warn('VPCS VM "{}" with PID={} is still running'.format(self._name, self._process.pid))
 
         self._process = None
         self._started = False
@@ -290,8 +296,8 @@ class VPCSVM(BaseVM):
         output = ""
         if self._vpcs_stdout_file:
             try:
-                with open(self._vpcs_stdout_file, errors="replace") as file:
-                    output = file.read()
+                with open(self._vpcs_stdout_file, "rb") as file:
+                    output = file.read().decode("utf-8", errors="replace")
             except OSError as e:
                 log.warn("Could not read {}: {}".format(self._vpcs_stdout_file, e))
         return output
@@ -388,6 +394,9 @@ class VPCSVM(BaseVM):
 
         command = [self.vpcs_path]
         command.extend(["-p", str(self._console)])  # listen to console port
+        command.extend(["-m", str(self._manager.get_mac_id(self.id))])   # the unique ID is used to set the MAC address offset
+        command.extend(["-i", "1"])  # option to start only one VPC instance
+        command.extend(["-F"])  # option to avoid the daemonization of VPCS
 
         nio = self._ethernet_adapter.get_nio(0)
         if nio:
@@ -402,12 +411,8 @@ class VPCSVM(BaseVM):
                 command.extend(["-e"])
                 command.extend(["-d", nio.tap_device])
 
-        command.extend(["-m", str(self._manager.get_mac_id(self.id))])   # the unique ID is used to set the MAC address offset
-        command.extend(["-i", "1"])  # option to start only one VPC instance
-        command.extend(["-F"])  # option to avoid the daemonization of VPCS
-
         if self.script_file:
-            command.extend([self.script_file])
+            command.extend([os.path.basename(self.script_file)])
         return command
 
     @property

gns3server/schemas/dynamips_device.py

@@ -218,6 +218,16 @@ DEVICE_NIO_SCHEMA = {
             "required": ["type", "ethernet_device"],
             "additionalProperties": False
         },
+        "NAT": {
+            "description": "NAT Network Input/Output",
+            "properties": {
+                "type": {
+                    "enum": ["nio_nat"]
+                },
+            },
+            "required": ["type"],
+            "additionalProperties": False
+        },
         "TAP": {
             "description": "TAP Network Input/Output",
             "properties": {
@@ -291,6 +301,7 @@ DEVICE_NIO_SCHEMA = {
                 {"$ref": "#/definitions/UDP"},
                 {"$ref": "#/definitions/Ethernet"},
                 {"$ref": "#/definitions/LinuxEthernet"},
+                {"$ref": "#/definitions/NAT"},
                 {"$ref": "#/definitions/TAP"},
                 {"$ref": "#/definitions/UNIX"},
                 {"$ref": "#/definitions/VDE"},

gns3server/schemas/dynamips_vm.py

@@ -120,6 +120,10 @@ VM_CREATE_SCHEMA = {
             "description": "disk1 size in MB",
             "type": "integer"
         },
+        "auto_delete_disks": {
+            "description": "automatically delete nvram and disk files",
+            "type": "boolean"
+        },
         "console": {
             "description": "console TCP port",
             "type": "integer",
@@ -284,20 +288,10 @@ VM_UPDATE_SCHEMA = {
             "type": "string",
             "minLength": 1,
         },
-        "startup_config": {
-            "description": "path to the IOS startup configuration file",
-            "type": "string",
-            "minLength": 1,
-        },
         "startup_config_content": {
             "description": "Content of IOS startup configuration file",
             "type": "string",
         },
-        "private_config": {
-            "description": "path to the IOS private configuration file",
-            "type": "string",
-            "minLength": 1,
-        },
         "private_config_content": {
             "description": "Content of IOS private configuration file",
             "type": "string",
@@ -347,6 +341,10 @@ VM_UPDATE_SCHEMA = {
             "description": "disk1 size in MB",
             "type": "integer"
         },
+        "auto_delete_disks": {
+            "description": "automatically delete nvram and disk files",
+            "type": "boolean"
+        },
         "console": {
             "description": "console TCP port",
             "type": "integer",
@@ -483,147 +481,6 @@ VM_UPDATE_SCHEMA = {
     "additionalProperties": False,
 }
 
-VM_NIO_SCHEMA = {
-    "$schema": "http://json-schema.org/draft-04/schema#",
-    "description": "Request validation to add a NIO for a Dynamips VM instance",
-    "type": "object",
-    "definitions": {
-        "UDP": {
-            "description": "UDP Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_udp"]
-                },
-                "lport": {
-                    "description": "Local port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                },
-                "rhost": {
-                    "description": "Remote host",
-                    "type": "string",
-                    "minLength": 1
-                },
-                "rport": {
-                    "description": "Remote port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                }
-            },
-            "required": ["type", "lport", "rhost", "rport"],
-            "additionalProperties": False
-        },
-        "Ethernet": {
-            "description": "Generic Ethernet Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_generic_ethernet"]
-                },
-                "ethernet_device": {
-                    "description": "Ethernet device name e.g. eth0",
-                    "type": "string",
-                    "minLength": 1
-                },
-            },
-            "required": ["type", "ethernet_device"],
-            "additionalProperties": False
-        },
-        "LinuxEthernet": {
-            "description": "Linux Ethernet Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_linux_ethernet"]
-                },
-                "ethernet_device": {
-                    "description": "Ethernet device name e.g. eth0",
-                    "type": "string",
-                    "minLength": 1
-                },
-            },
-            "required": ["type", "ethernet_device"],
-            "additionalProperties": False
-        },
-        "TAP": {
-            "description": "TAP Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_tap"]
-                },
-                "tap_device": {
-                    "description": "TAP device name e.g. tap0",
-                    "type": "string",
-                    "minLength": 1
-                },
-            },
-            "required": ["type", "tap_device"],
-            "additionalProperties": False
-        },
-        "UNIX": {
-            "description": "UNIX Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_unix"]
-                },
-                "local_file": {
-                    "description": "path to the UNIX socket file (local)",
-                    "type": "string",
-                    "minLength": 1
-                },
-                "remote_file": {
-                    "description": "path to the UNIX socket file (remote)",
-                    "type": "string",
-                    "minLength": 1
-                },
-            },
-            "required": ["type", "local_file", "remote_file"],
-            "additionalProperties": False
-        },
-        "VDE": {
-            "description": "VDE Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_vde"]
-                },
-                "control_file": {
-                    "description": "path to the VDE control file",
-                    "type": "string",
-                    "minLength": 1
-                },
-                "local_file": {
-                    "description": "path to the VDE control file",
-                    "type": "string",
-                    "minLength": 1
-                },
-            },
-            "required": ["type", "control_file", "local_file"],
-            "additionalProperties": False
-        },
-        "NULL": {
-            "description": "NULL Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_null"]
-                },
-            },
-            "required": ["type"],
-            "additionalProperties": False
-        },
-    },
-    "oneOf": [
-        {"$ref": "#/definitions/UDP"},
-        {"$ref": "#/definitions/Ethernet"},
-        {"$ref": "#/definitions/LinuxEthernet"},
-        {"$ref": "#/definitions/TAP"},
-        {"$ref": "#/definitions/UNIX"},
-        {"$ref": "#/definitions/VDE"},
-        {"$ref": "#/definitions/NULL"},
-    ],
-    "additionalProperties": True,
-    "required": ["type"]
-}
-
 VM_CAPTURE_SCHEMA = {
     "$schema": "http://json-schema.org/draft-04/schema#",
     "description": "Request validation to start a packet capture on a Dynamips VM instance port",
@@ -742,6 +599,10 @@ VM_OBJECT_SCHEMA = {
             "description": "disk1 size in MB",
             "type": "integer"
         },
+        "auto_delete_disks": {
+            "description": "automatically delete nvram and disk files",
+            "type": "boolean"
+        },
         "console": {
             "description": "console TCP port",
             "type": "integer",
@@ -896,5 +757,4 @@ VM_CONFIGS_SCHEMA = {
         },
     },
     "additionalProperties": False,
-    "required": ["startup_config_content", "private_config_content"]
 }

gns3server/schemas/iou.py

@@ -70,8 +70,12 @@ IOU_CREATE_SCHEMA = {
             "description": "Use default IOU values",
             "type": ["boolean", "null"]
         },
+        "initial_config": {
+            "description": "Path to the initial configuration of IOU",
+            "type": ["string", "null"]
+        },
         "initial_config_content": {
-            "description": "Initial configuration of the IOU",
+            "description": "Initial configuration of IOU",
             "type": ["string", "null"]
         },
         "iourc_content": {
@@ -124,7 +128,7 @@ IOU_UPDATE_SCHEMA = {
             "type": ["boolean", "null"]
         },
         "initial_config_content": {
-            "description": "Initial configuration of the IOU",
+            "description": "Initial configuration of IOU",
             "type": ["string", "null"]
         },
         "use_default_iou_values": {
@@ -210,78 +214,6 @@ IOU_OBJECT_SCHEMA = {
     "required": ["name", "vm_id", "console", "project_id", "path", "serial_adapters", "ethernet_adapters", "ram", "nvram", "l1_keepalives", "initial_config", "use_default_iou_values"]
 }
 
-IOU_NIO_SCHEMA = {
-    "$schema": "http://json-schema.org/draft-04/schema#",
-    "description": "Request validation to add a NIO for a IOU instance",
-    "type": "object",
-    "definitions": {
-        "UDP": {
-            "description": "UDP Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_udp"]
-                },
-                "lport": {
-                    "description": "Local port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                },
-                "rhost": {
-                    "description": "Remote host",
-                    "type": "string",
-                    "minLength": 1
-                },
-                "rport": {
-                    "description": "Remote port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                }
-            },
-            "required": ["type", "lport", "rhost", "rport"],
-            "additionalProperties": False
-        },
-        "Ethernet": {
-            "description": "Generic Ethernet Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_generic_ethernet"]
-                },
-                "ethernet_device": {
-                    "description": "Ethernet device name e.g. eth0",
-                    "type": "string",
-                    "minLength": 1
-                },
-            },
-            "required": ["type", "ethernet_device"],
-            "additionalProperties": False
-        },
-        "TAP": {
-            "description": "TAP Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_tap"]
-                },
-                "tap_device": {
-                    "description": "TAP device name e.g. tap0",
-                    "type": "string",
-                    "minLength": 1
-                },
-            },
-            "required": ["type", "tap_device"],
-            "additionalProperties": False
-        },
-    },
-    "oneOf": [
-        {"$ref": "#/definitions/UDP"},
-        {"$ref": "#/definitions/Ethernet"},
-        {"$ref": "#/definitions/TAP"},
-    ],
-    "additionalProperties": True,
-    "required": ["type"]
-}
-
 IOU_CAPTURE_SCHEMA = {
     "$schema": "http://json-schema.org/draft-04/schema#",
     "description": "Request validation to start a packet capture on a IOU instance",

gns3server/schemas/nio.py

@@ -0,0 +1,169 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2015 GNS3 Technologies Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+NIO_SCHEMA = {
+    "$schema": "http://json-schema.org/draft-04/schema#",
+    "description": "Request validation to add a NIO for a VM instance",
+    "type": "object",
+    "definitions": {
+        "UDP": {
+            "description": "UDP Network Input/Output",
+            "properties": {
+                "type": {
+                    "enum": ["nio_udp"]
+                },
+                "lport": {
+                    "description": "Local port",
+                    "type": "integer",
+                    "minimum": 1,
+                    "maximum": 65535
+                },
+                "rhost": {
+                    "description": "Remote host",
+                    "type": "string",
+                    "minLength": 1
+                },
+                "rport": {
+                    "description": "Remote port",
+                    "type": "integer",
+                    "minimum": 1,
+                    "maximum": 65535
+                }
+            },
+            "required": ["type", "lport", "rhost", "rport"],
+            "additionalProperties": False
+        },
+        "Ethernet": {
+            "description": "Generic Ethernet Network Input/Output",
+            "properties": {
+                "type": {
+                    "enum": ["nio_generic_ethernet"]
+                },
+                "ethernet_device": {
+                    "description": "Ethernet device name e.g. eth0",
+                    "type": "string",
+                    "minLength": 1
+                },
+            },
+            "required": ["type", "ethernet_device"],
+            "additionalProperties": False
+        },
+        "LinuxEthernet": {
+            "description": "Linux Ethernet Network Input/Output",
+            "properties": {
+                "type": {
+                    "enum": ["nio_linux_ethernet"]
+                },
+                "ethernet_device": {
+                    "description": "Ethernet device name e.g. eth0",
+                    "type": "string",
+                    "minLength": 1
+                },
+            },
+            "required": ["type", "ethernet_device"],
+            "additionalProperties": False
+        },
+        "NAT": {
+            "description": "NAT Network Input/Output",
+            "properties": {
+                "type": {
+                    "enum": ["nio_nat"]
+                },
+            },
+            "required": ["type"],
+            "additionalProperties": False
+        },
+        "TAP": {
+            "description": "TAP Network Input/Output",
+            "properties": {
+                "type": {
+                    "enum": ["nio_tap"]
+                },
+                "tap_device": {
+                    "description": "TAP device name e.g. tap0",
+                    "type": "string",
+                    "minLength": 1
+                },
+            },
+            "required": ["type", "tap_device"],
+            "additionalProperties": False
+        },
+        "UNIX": {
+            "description": "UNIX Network Input/Output",
+            "properties": {
+                "type": {
+                    "enum": ["nio_unix"]
+                },
+                "local_file": {
+                    "description": "path to the UNIX socket file (local)",
+                    "type": "string",
+                    "minLength": 1
+                },
+                "remote_file": {
+                    "description": "path to the UNIX socket file (remote)",
+                    "type": "string",
+                    "minLength": 1
+                },
+            },
+            "required": ["type", "local_file", "remote_file"],
+            "additionalProperties": False
+        },
+        "VDE": {
+            "description": "VDE Network Input/Output",
+            "properties": {
+                "type": {
+                    "enum": ["nio_vde"]
+                },
+                "control_file": {
+                    "description": "path to the VDE control file",
+                    "type": "string",
+                    "minLength": 1
+                },
+                "local_file": {
+                    "description": "path to the VDE control file",
+                    "type": "string",
+                    "minLength": 1
+                },
+            },
+            "required": ["type", "control_file", "local_file"],
+            "additionalProperties": False
+        },
+        "NULL": {
+            "description": "NULL Network Input/Output",
+            "properties": {
+                "type": {
+                    "enum": ["nio_null"]
+                },
+            },
+            "required": ["type"],
+            "additionalProperties": False
+        },
+    },
+    "oneOf": [
+        {"$ref": "#/definitions/UDP"},
+        {"$ref": "#/definitions/Ethernet"},
+        {"$ref": "#/definitions/LinuxEthernet"},
+        {"$ref": "#/definitions/NAT"},
+        {"$ref": "#/definitions/TAP"},
+        {"$ref": "#/definitions/UNIX"},
+        {"$ref": "#/definitions/VDE"},
+        {"$ref": "#/definitions/NULL"},
+    ],
+    "additionalProperties": True,
+    "required": ["type"]
+}

gns3server/schemas/qemu.py

@@ -211,62 +211,6 @@ QEMU_UPDATE_SCHEMA = {
     "additionalProperties": False,
 }
 
-QEMU_NIO_SCHEMA = {
-    "$schema": "http://json-schema.org/draft-04/schema#",
-    "description": "Request validation to add a NIO for a QEMU instance",
-    "type": "object",
-    "definitions": {
-        "UDP": {
-            "description": "UDP Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_udp"]
-                },
-                "lport": {
-                    "description": "Local port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                },
-                "rhost": {
-                    "description": "Remote host",
-                    "type": "string",
-                    "minLength": 1
-                },
-                "rport": {
-                    "description": "Remote port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                }
-            },
-            "required": ["type", "lport", "rhost", "rport"],
-            "additionalProperties": False
-        },
-        "Ethernet": {
-            "description": "Generic Ethernet Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_generic_ethernet"]
-                },
-                "ethernet_device": {
-                    "description": "Ethernet device name e.g. eth0",
-                    "type": "string",
-                    "minLength": 1
-                },
-            },
-            "required": ["type", "ethernet_device"],
-            "additionalProperties": False
-        },
-    },
-    "oneOf": [
-        {"$ref": "#/definitions/UDP"},
-        {"$ref": "#/definitions/Ethernet"},
-    ],
-    "additionalProperties": True,
-    "required": ["type"]
-}
-
 QEMU_OBJECT_SCHEMA = {
     "$schema": "http://json-schema.org/draft-04/schema#",
     "description": "Request validation for a QEMU VM instance",

gns3server/schemas/virtualbox.py

@@ -139,46 +139,6 @@ VBOX_UPDATE_SCHEMA = {
     "additionalProperties": False,
 }
 
-VBOX_NIO_SCHEMA = {
-    "$schema": "http://json-schema.org/draft-04/schema#",
-    "description": "Request validation to add a NIO for a VirtualBox VM instance",
-    "type": "object",
-    "definitions": {
-        "UDP": {
-            "description": "UDP Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_udp"]
-                },
-                "lport": {
-                    "description": "Local port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                },
-                "rhost": {
-                    "description": "Remote host",
-                    "type": "string",
-                    "minLength": 1
-                },
-                "rport": {
-                    "description": "Remote port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                }
-            },
-            "required": ["type", "lport", "rhost", "rport"],
-            "additionalProperties": False
-        },
-    },
-    "oneOf": [
-        {"$ref": "#/definitions/UDP"},
-    ],
-    "additionalProperties": True,
-    "required": ["type"]
-}
-
 VBOX_CAPTURE_SCHEMA = {
     "$schema": "http://json-schema.org/draft-04/schema#",
     "description": "Request validation to start a packet capture on a VirtualBox VM instance port",

gns3server/schemas/vpcs.py

@@ -75,62 +75,6 @@ VPCS_UPDATE_SCHEMA = {
     "additionalProperties": False,
 }
 
-VPCS_NIO_SCHEMA = {
-    "$schema": "http://json-schema.org/draft-04/schema#",
-    "description": "Request validation to add a NIO for a VPCS instance",
-    "type": "object",
-    "definitions": {
-        "UDP": {
-            "description": "UDP Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_udp"]
-                },
-                "lport": {
-                    "description": "Local port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                },
-                "rhost": {
-                    "description": "Remote host",
-                    "type": "string",
-                    "minLength": 1
-                },
-                "rport": {
-                    "description": "Remote port",
-                    "type": "integer",
-                    "minimum": 1,
-                    "maximum": 65535
-                }
-            },
-            "required": ["type", "lport", "rhost", "rport"],
-            "additionalProperties": False
-        },
-        "TAP": {
-            "description": "TAP Network Input/Output",
-            "properties": {
-                "type": {
-                    "enum": ["nio_tap"]
-                },
-                "tap_device": {
-                    "description": "TAP device name e.g. tap0",
-                    "type": "string",
-                    "minLength": 1
-                },
-            },
-            "required": ["type", "tap_device"],
-            "additionalProperties": False
-        },
-    },
-    "oneOf": [
-        {"$ref": "#/definitions/UDP"},
-        {"$ref": "#/definitions/TAP"},
-    ],
-    "additionalProperties": True,
-    "required": ["type"]
-}
-
 VPCS_OBJECT_SCHEMA = {
     "$schema": "http://json-schema.org/draft-04/schema#",
     "description": "VPCS instance",

gns3server/server.py

@@ -27,6 +27,7 @@ import aiohttp
 import functools
 import types
 import time
+import atexit
 
 from .web.route import Route
 from .web.request_handler import RequestHandler
@@ -85,6 +86,7 @@ class Server:
 
         if self._handler:
             yield from self._handler.finish_connections()
+            self._handler = None
 
         for module in MODULES:
             log.debug("Unloading module {}".format(module.__name__))
@@ -101,10 +103,9 @@ class Server:
 
     def _signal_handling(self):
 
-        @asyncio.coroutine
         def signal_handler(signame):
             log.warning("Server has got signal {}, exiting...".format(signame))
-            yield from self.shutdown_server()
+            asyncio.async(self.shutdown_server())
 
         signals = ["SIGTERM", "SIGINT"]
         if sys.platform.startswith("win"):
@@ -113,7 +114,7 @@ class Server:
             signals.extend(["SIGHUP", "SIGQUIT"])
 
         for signal_name in signals:
-            callback = functools.partial(asyncio.async, signal_handler(signal_name))
+            callback = functools.partial(signal_handler, signal_name)
             if sys.platform.startswith("win"):
                 # add_signal_handler() is not yet supported on Windows
                 signal.signal(getattr(signal, signal_name), callback)
@@ -173,6 +174,22 @@ class Server:
             return
         yield from embed(globals(), locals(), return_asyncio_coroutine=True, patch_stdout=True)
 
+    def _exit_handling(self):
+        """
+        Makes sure the asyncio loop is closed.
+        """
+
+        def close_asyncio_loop():
+            loop = None
+            try:
+                loop = asyncio.get_event_loop()
+            except AttributeError:
+                pass
+            if loop is not None:
+                loop.close()
+
+        atexit.register(close_asyncio_loop)
+
     def run(self):
         """
         Starts the server.
@@ -213,9 +230,12 @@ class Server:
 
         log.info("Starting server on {}:{}".format(self._host, self._port))
         self._handler = app.make_handler(handler=RequestHandler)
-        self._loop.run_until_complete(self._run_application(self._handler, ssl_context))
+        server = self._run_application(self._handler, ssl_context)
+        self._loop.run_until_complete(server)
         self._signal_handling()
 
+        self._exit_handling()
+
         if server_config.getboolean("live"):
             log.info("Code live reload is enabled, watching for file changes")
             self._loop.call_later(1, self._reload_hook)
@@ -230,3 +250,9 @@ class Server:
             # on Windows when the process gets the SIGBREAK signal
             # TypeError: async() takes 1 positional argument but 3 were given
             log.warning("TypeError exception in the loop {}".format(e))
+        finally:
+            if self._handler and self._loop.is_running():
+                self._loop.run_until_complete(self._handler.finish_connections())
+            server.close()
+            if self._loop.is_running():
+                self._loop.run_until_complete(app.finish())

gns3server/templates/index.html

@@ -4,8 +4,8 @@
     Welcome to GNS 3.
 </h3>
 <ul>
-    <li><a href="http://community.gns3.com">Community</a></li>
+    <li><a href="https://gns3.com">Website</a></li>
     <li><a href="http://api.gns3.net">API documentation</a></li>
-    <li><a href="/upload">Upload images</a></li>
+    <li><a href="/upload">Upload images & backup</a></li>
 </ul>
 {% endblock %}

gns3server/templates/layout.html

@@ -4,6 +4,15 @@
 <title>GNS3 Server</title>
 </head>
 <body>
+<div>
+    <a href="/">Home</a>
+    |
+    <a href="/upload">Upload</a>
+    |    
+    <a href="/backup/images.tar">Backup images</a>
+    |
+    <a href="/backup/projects.tar">Backup projects</a>
+</div>
 {% block body %}{% endblock %}
 </body>
 <small>

gns3server/templates/upload.html

@@ -8,6 +8,8 @@
         <option value="IOURC">IOU licence (iourc)</option>
         <option value="IOS">IOS</option>
         <option value="QEMU">Qemu</option>
+        <option value="IMAGES">GNS3 images backup (.tar)</option>
+        <option value="PROJECTS">GNS3 projects backup (.tar)</option>
     </select>
     <br />
     <br />

gns3server/utils/asyncio.py

@@ -51,7 +51,10 @@ def subprocess_check_output(*args, cwd=None, env=None):
     output = yield from proc.stdout.read()
     if output is None:
         return ""
-    return output.decode("utf-8")
+    # If we received garbage we ignore invalid characters
+    # it should happend only when user try to use another binary
+    # and the code of VPCS, dynamips... Will detect it's not the correct binary
+    return output.decode("utf-8", errors="ignore")
 
 
 @asyncio.coroutine

gns3server/utils/get_resource.py

@@ -0,0 +1,41 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2015 GNS3 Technologies Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import tempfile
+import pkg_resources
+import atexit
+import logging
+
+log = logging.getLogger(__name__)
+
+try:
+    egg_cache_dir = tempfile.mkdtemp()
+    pkg_resources.set_extraction_path(egg_cache_dir)
+except ValueError:
+    # If the path is already set the module throw an error
+    pass
+
+
+@atexit.register
+def clean_egg_cache():
+    try:
+        import shutil
+        log.debug("Clean egg cache %s", egg_cache_dir)
+        shutil.rmtree(egg_cache_dir)
+    except Exception:
+        # We don't care if we can not cleanup
+        pass

gns3server/utils/glob.py

@@ -0,0 +1,45 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2013 GNS3 Technologies Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import sys
+import re
+import os
+import glob
+
+
+def glob_escape(pathname):
+    """
+    Escape all special chars for glob.
+    For Python after 3.4 we use the glob.escape method.
+
+    :returns: Escaped path
+    """
+
+    if sys.version_info < (3, 4):
+        # Extracted from Python 3.4 source code
+        # Escaping is done by wrapping any of "*?[" between square brackets.
+        # Metacharacters do not work in the drive part and shouldn't be escaped.
+        magic_check = re.compile('([*?[])')
+        magic_check_bytes = re.compile(b'([*?[])')
+        drive, pathname = os.path.splitdrive(pathname)
+        if isinstance(pathname, bytes):
+            pathname = magic_check_bytes.sub(br'[\1]', pathname)
+        else:
+            pathname = magic_check.sub(r'[\1]', pathname)
+        return drive + pathname
+    else:
+        return glob.escape(pathname)

gns3server/utils/interfaces.py

@@ -58,10 +58,11 @@ def get_windows_interfaces():
 
     import win32com.client
     import pywintypes
-    locator = win32com.client.Dispatch("WbemScripting.SWbemLocator")
-    service = locator.ConnectServer(".", "root\cimv2")
+
     interfaces = []
     try:
+        locator = win32com.client.Dispatch("WbemScripting.SWbemLocator")
+        service = locator.ConnectServer(".", "root\cimv2")
         # more info on Win32_NetworkAdapter: http://msdn.microsoft.com/en-us/library/aa394216%28v=vs.85%29.aspx
         for adapter in service.InstancesOf("Win32_NetworkAdapter"):
             if adapter.NetConnectionStatus == 2 or adapter.NetConnectionStatus == 7:
@@ -70,7 +71,7 @@ def get_windows_interfaces():
                 interfaces.append({"id": npf_interface,
                                    "name": adapter.NetConnectionID})
     except (AttributeError, pywintypes.com_error):
-        log.warn("could not use the COM service to retrieve interface info, trying using the registry...")
+        log.warn("Could not use the COM service to retrieve interface info, trying using the registry...")
         return _get_windows_interfaces_from_registry()
 
     return interfaces

gns3server/version.py

@@ -23,5 +23,5 @@
 # or negative for a release candidate or beta (after the base version
 # number has been incremented)
 
-__version__ = "1.3.1"
-__version_info__ = (1, 3, 1, 0)
+__version__ = "1.3.13"
+__version_info__ = (1, 3, 13, 0)

gns3server/web/logger.py

@@ -34,7 +34,7 @@ class ColouredFormatter(logging.Formatter):
 
         message = super().format(record)
 
-        if not colour:
+        if not colour or sys.platform.startswith("win"):
             return message.replace("#RESET#", "")
 
         level_no = record.levelno
@@ -74,13 +74,36 @@ class ColouredStreamHandler(logging.StreamHandler):
             stream.write(msg)
             stream.write(self.terminator)
             self.flush()
+        # On OSX when frozen flush raise a BrokenPipeError
+        except BrokenPipeError:
+            pass
         except Exception:
             self.handleError(record)
 
 
-def init_logger(level, quiet=False):
-    if sys.platform.startswith("win"):
-        stream_handler = logging.StreamHandler(sys.stdout)
+class WinStreamHandler(logging.StreamHandler):
+
+    def emit(self, record):
+
+        if sys.stdin.encoding != "utf-8":
+            record = record
+
+        stream = self.stream
+        try:
+            msg = self.formatter.format(record, stream.isatty())
+            stream.write(msg.encode(stream.encoding, errors="replace").decode(stream.encoding))
+            stream.write(self.terminator)
+            self.flush()
+        except Exception:
+            self.handleError(record)
+
+
+def init_logger(level, logfile=None, quiet=False):
+    if logfile and len(logfile) > 0:
+        stream_handler = logging.FileHandler(logfile)
+        stream_handler.formatter = ColouredFormatter("{asctime} {levelname} {filename}:{lineno} {message}", "%Y-%m-%d %H:%M:%S", "{")
+    elif sys.platform.startswith("win"):
+        stream_handler = WinStreamHandler(sys.stdout)
         stream_handler.formatter = ColouredFormatter("{asctime} {levelname} {filename}:{lineno} {message}", "%Y-%m-%d %H:%M:%S", "{")
     else:
         stream_handler = ColouredStreamHandler(sys.stdout)

gns3server/web/response.py

@@ -30,10 +30,11 @@ renderer = jinja2.Environment(loader=jinja2.PackageLoader('gns3server', 'templat
 
 class Response(aiohttp.web.Response):
 
-    def __init__(self, route=None, output_schema=None, headers={}, **kwargs):
+    def __init__(self, request=None, route=None, output_schema=None, headers={}, **kwargs):
 
         self._route = route
         self._output_schema = output_schema
+        self._request = request
         headers['X-Route'] = self._route
         headers['Server'] = "Python/{0[0]}.{0[1]} GNS3/{1}".format(sys.version_info, __version__)
         super().__init__(headers=headers, **kwargs)
@@ -70,6 +71,7 @@ class Response(aiohttp.web.Response):
         """
         template = renderer.get_template(template_filename)
         kwargs["gns3_version"] = __version__
+        kwargs["gns3_host"] = self._request.host
         self.html(template.render(**kwargs))
 
     def json(self, answer):

gns3server/web/route.py

@@ -83,6 +83,31 @@ class Route(object):
     def delete(cls, path, *args, **kw):
         return cls._route('DELETE', path, *args, **kw)
 
+    @classmethod
+    def authenticate(cls, request, route, server_config):
+        """
+        Ask user for authentication
+
+        :returns: Response if you need to auth the user otherwise None
+        """
+        if not server_config.getboolean("auth", False):
+            return
+
+        user = server_config.get("user", "").strip()
+        password = server_config.get("password", "").strip()
+
+        if len(user) == 0:
+            return
+
+        if "AUTHORIZATION" in request.headers:
+            if request.headers["AUTHORIZATION"] == aiohttp.helpers.BasicAuth(user, password).encode():
+                return
+
+        response = Response(request=request, route=route)
+        response.set_status(401)
+        response.headers["WWW-Authenticate"] = 'Basic realm="GNS3 server"'
+        return response
+
     @classmethod
     def _route(cls, method, path, *args, **kw):
         # This block is executed only the first time
@@ -118,52 +143,63 @@ class Route(object):
             def control_schema(request):
                 # This block is executed at each method call
 
+                server_config = Config.instance().get_section_config("Server")
+
+                # Authenticate
+                response = cls.authenticate(request, route, server_config)
+                if response:
+                    return response
+
                 # Non API call
                 if api_version is None:
-                    response = Response(route=route, output_schema=output_schema)
+                    response = Response(request=request, route=route, output_schema=output_schema)
                     yield from func(request, response)
                     return response
 
                 # API call
                 try:
                     request = yield from parse_request(request, input_schema)
-                    server_config = Config.instance().get_section_config("Server")
                     record_file = server_config.get("record")
                     if record_file:
                         try:
-                            with open(record_file, "a") as f:
+                            with open(record_file, "a", encoding="utf-8") as f:
                                 f.write("curl -X {} 'http://{}{}' -d '{}'".format(request.method, request.host, request.path_qs, json.dumps(request.json)))
                                 f.write("\n")
                         except OSError as e:
                             log.warn("Could not write to the record file {}: {}".format(record_file, e))
-                    response = Response(route=route, output_schema=output_schema)
+                    response = Response(request=request, route=route, output_schema=output_schema)
                     yield from func(request, response)
                 except aiohttp.web.HTTPBadRequest as e:
-                    response = Response(route=route)
+                    response = Response(request=request, route=route)
                     response.set_status(e.status)
                     response.json({"message": e.text, "status": e.status, "path": route, "request": request.json})
                 except aiohttp.web.HTTPException as e:
-                    response = Response(route=route)
+                    response = Response(request=request, route=route)
                     response.set_status(e.status)
                     response.json({"message": e.text, "status": e.status})
                 except VMError as e:
                     log.error("VM error detected: {type}".format(type=type(e)), exc_info=1)
-                    response = Response(route=route)
+                    response = Response(request=request, route=route)
                     response.set_status(409)
                     response.json({"message": str(e), "status": 409})
                 except asyncio.futures.CancelledError as e:
                     log.error("Request canceled")
-                    response = Response(route=route)
+                    response = Response(request=request, route=route)
                     response.set_status(408)
                     response.json({"message": "Request canceled", "status": 408})
                 except aiohttp.ClientDisconnectedError:
                     log.error("Client disconnected")
-                    response = Response(route=route)
+                    response = Response(request=request, route=route)
                     response.set_status(408)
                     response.json({"message": "Client disconnected", "status": 408})
+                except ConnectionResetError:
+                    log.error("Client connection reset")
+                    response = Response(request=request, route=route)
+                    response.set_status(408)
+                    response.json({"message": "Connection reset", "status": 408})
                 except Exception as e:
                     log.error("Uncaught exception detected: {type}".format(type=type(e)), exc_info=1)
-                    response = Response(route=route)
+                    response = Response(request=request, route=route)
                     response.set_status(500)
                     CrashReport.instance().capture_exception(request)
                     exc_type, exc_value, exc_tb = sys.exc_info()

init/gns3.conf.upstart

@@ -0,0 +1,19 @@
+description "GNS3 server"
+author      "GNS3 Team"
+
+start on filesystem or runlevel [2345]
+stop on shutdown
+
+script
+    echo $$ > /var/run/gns3.pid
+    exec start-stop-daemon --start -c gns3 --exec /usr/local/bin/gns3server -- --log /var/log/gns3.log
+end script
+
+pre-start script
+    echo "[`date`] GNS3 Starting" >> /var/log/gns3.log
+end script
+
+pre-stop script
+    rm /var/run/gns3.pid
+    echo "[`date`] GNS3 Stopping" >> /var/log/gns3.log
+end script

requirements.txt

@@ -1,6 +1,5 @@
-netifaces==0.10.4
+gns3-netifaces==0.10.4.1
 jsonschema==2.4.0
-python-dateutil==2.3
 aiohttp==0.14.4
 Jinja2==2.7.3
 raven==5.2.0

setup.py

@@ -19,6 +19,10 @@ import sys
 from setuptools import setup, find_packages
 from setuptools.command.test import test as TestCommand
 
+# we only support Python 3 version >= 3.3
+if sys.version_info < (3, 3):
+    raise SystemExit("Python 3.3 or higher is required")
+
 
 class PyTest(TestCommand):
 
@@ -34,12 +38,14 @@ class PyTest(TestCommand):
         sys.exit(errcode)
 
 
-dependencies = ["aiohttp>=0.14.4",
-                "jsonschema>=2.4.0",
-                "Jinja2>=2.7.3",
-                "raven>=5.2.0"]
+dependencies = [
+    # "gns3-netifaces>=0.10.4.1",
+    "aiohttp>=0.14.4",
+    "jsonschema>=2.4.0",
+    "Jinja2>=2.7.3",
+    "raven>=5.2.0"]
 
-#if not sys.platform.startswith("win"):
+# if not sys.platform.startswith("win"):
 #    dependencies.append("netifaces==0.10.4")
 
 if sys.version_info == (3, 3):

tests/conftest.py

@@ -73,7 +73,7 @@ def _get_unused_port():
     return port
 
 
-@pytest.fixture(scope="session")
+@pytest.fixture
 def server(request, loop, port_manager, monkeypatch):
     """A GNS3 server"""
 
@@ -136,6 +136,7 @@ def run_around_tests(monkeypatch):
     config = Config.instance()
     config.clear()
     config.set("Server", "project_directory", tmppath)
+    config.set("Server", "auth", False)
 
     # Prevent exectuions of the VM if we forgot to mock something
     config.set("VirtualBox", "vboxmanage_path", tmppath)

tests/handlers/api/base.py

@@ -88,7 +88,10 @@ class Query:
                 except ValueError:
                     response.json = None
             else:
-                response.html = response.body.decode("utf-8")
+                try:
+                    response.html = response.body.decode("utf-8")
+                except UnicodeDecodeError:
+                    response.html = None
         else:
             response.json = {}
             response.html = ""

tests/handlers/api/test_iou.py

@@ -18,10 +18,14 @@
 import pytest
 import os
 import stat
+import sys
+import uuid
 
 from tests.utils import asyncio_patch
 from unittest.mock import patch, MagicMock, PropertyMock
 
+pytestmark = pytest.mark.skipif(sys.platform.startswith("win"), reason="Not supported on Windows")
+
 
 @pytest.fixture
 def fake_iou_bin(tmpdir):
@@ -91,11 +95,32 @@ def test_iou_create_with_params(server, project, base_params):
 
     assert "initial-config.cfg" in response.json["initial_config"]
     with open(initial_config_file(project, response.json)) as f:
-        assert f.read() == params["initial_config_content"]
+        assert f.read() == "hostname test"
 
     assert "iourc" in response.json["iourc_path"]
 
 
+def test_iou_create_initial_config_already_exist(server, project, base_params):
+    """We don't erase an initial config if already exist at project creation"""
+
+    vm_id = str(uuid.uuid4())
+    initial_config_file_path = initial_config_file(project, {'vm_id': vm_id})
+    with open(initial_config_file_path, 'w+') as f:
+        f.write("echo hello")
+
+    params = base_params
+    params["vm_id"] = vm_id
+    params["initial_config_content"] = "hostname test"
+
+    response = server.post("/projects/{project_id}/iou/vms".format(project_id=project.id), params, example=True)
+    assert response.status == 201
+    assert response.route == "/projects/{project_id}/iou/vms"
+
+    assert "initial-config.cfg" in response.json["initial_config"]
+    with open(initial_config_file(project, response.json)) as f:
+        assert f.read() == "echo hello"
+
+
 def test_iou_get(server, project, vm):
     response = server.get("/projects/{project_id}/iou/vms/{vm_id}".format(project_id=vm["project_id"], vm_id=vm["vm_id"]), example=True)
     assert response.status == 200

tests/handlers/api/test_network.py

@@ -15,6 +15,9 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+import os
+import pytest
+
 
 def test_udp_allocation(server, project):
     response = server.post('/projects/{}/ports/udp'.format(project.id), {}, example=True)
@@ -22,6 +25,8 @@ def test_udp_allocation(server, project):
     assert response.json == {'udp_port': 10000}
 
 
+# Netfifaces is not available on Travis
+@pytest.mark.skipif(os.environ.get("TRAVIS", False) is not False, reason="Not supported on Travis")
 def test_interfaces(server):
     response = server.get('/interfaces', example=True)
     assert response.status == 200

tests/handlers/api/test_project.py

@@ -20,6 +20,7 @@ This test suite check /project endpoint
 """
 
 import uuid
+import os
 from unittest.mock import patch
 from tests.utils import asyncio_patch
 
@@ -85,18 +86,43 @@ def test_update_temporary_project(server):
     assert response.json["temporary"] is False
 
 
-def test_update_path_project(server, tmpdir):
+def test_update_path_project_temporary(server, tmpdir):
+
+    os.makedirs(str(tmpdir / "a"))
+    os.makedirs(str(tmpdir / "b"))
 
     with patch("gns3server.modules.project.Project.is_local", return_value=True):
-        response = server.post("/projects", {"name": "first_name"})
+        response = server.post("/projects", {"name": "first_name", "path": str(tmpdir / "a"), "temporary": True})
         assert response.status == 201
         assert response.json["name"] == "first_name"
-        query = {"name": "second_name", "path": str(tmpdir)}
+        query = {"name": "second_name", "path": str(tmpdir / "b")}
         response = server.put("/projects/{project_id}".format(project_id=response.json["project_id"]), query, example=True)
         assert response.status == 200
-        assert response.json["path"] == str(tmpdir)
+        assert response.json["path"] == str(tmpdir / "b")
         assert response.json["name"] == "second_name"
 
+        assert not os.path.exists(str(tmpdir / "a"))
+        assert os.path.exists(str(tmpdir / "b"))
+
+
+def test_update_path_project_non_temporary(server, tmpdir):
+
+    os.makedirs(str(tmpdir / "a"))
+    os.makedirs(str(tmpdir / "b"))
+
+    with patch("gns3server.modules.project.Project.is_local", return_value=True):
+        response = server.post("/projects", {"name": "first_name", "path": str(tmpdir / "a")})
+        assert response.status == 201
+        assert response.json["name"] == "first_name"
+        query = {"name": "second_name", "path": str(tmpdir / "b")}
+        response = server.put("/projects/{project_id}".format(project_id=response.json["project_id"]), query, example=True)
+        assert response.status == 200
+        assert response.json["path"] == str(tmpdir / "b")
+        assert response.json["name"] == "second_name"
+
+        assert os.path.exists(str(tmpdir / "a"))
+        assert os.path.exists(str(tmpdir / "b"))
+
 
 def test_update_path_project_non_local(server, tmpdir):
 

tests/handlers/api/test_qemu.py

@@ -150,10 +150,7 @@ def test_qemu_nio_create_ethernet(server, vm):
                                                                                                                                                      "ethernet_device": "eth0",
                                                                                                                                                      },
                            example=True)
-    assert response.status == 201
-    assert response.route == "/projects/{project_id}/qemu/vms/{vm_id}/adapters/{adapter_number:\d+}/ports/{port_number:\d+}/nio"
-    assert response.json["type"] == "nio_generic_ethernet"
-    assert response.json["ethernet_device"] == "eth0"
+    assert response.status == 409
 
 
 def test_qemu_delete_nio(server, vm):

tests/handlers/api/test_vpcs.py

@@ -42,7 +42,7 @@ def test_vpcs_get(server, project, vm):
     assert response.route == "/projects/{project_id}/vpcs/vms/{vm_id}"
     assert response.json["name"] == "PC TEST 1"
     assert response.json["project_id"] == project.id
-    assert response.json["startup_script_path"] == None
+    assert response.json["startup_script_path"] is None
 
 
 def test_vpcs_create_startup_script(server, project):
@@ -51,7 +51,7 @@ def test_vpcs_create_startup_script(server, project):
     assert response.route == "/projects/{project_id}/vpcs/vms"
     assert response.json["name"] == "PC TEST 1"
     assert response.json["project_id"] == project.id
-    assert response.json["startup_script"] == "ip 192.168.1.2\necho TEST"
+    assert response.json["startup_script"] == os.linesep.join(["ip 192.168.1.2", "echo TEST"])
     assert response.json["startup_script_path"] == "startup.vpc"
 
 

tests/handlers/test_index.py

@@ -27,5 +27,5 @@ def test_index(server):
     response = server.get('/', api_version=None)
     assert response.status == 200
     html = response.html
-    assert "Community" in html
+    assert "Website" in html
     assert __version__ in html

tests/handlers/test_upload.py

@@ -17,10 +17,15 @@
 
 
 import aiohttp
+import asyncio
 import os
+import tarfile
 from unittest.mock import patch
 
 
+from gns3server.config import Config
+
+
 def test_index_upload(server):
     response = server.get('/upload', api_version=None)
     assert response.status == 200
@@ -37,10 +42,141 @@ def test_upload(server, tmpdir):
     body.add_field("type", "QEMU")
     body.add_field("file", open(str(tmpdir / "test"), "rb"), content_type="application/iou", filename="test2")
 
-    with patch("gns3server.config.Config.get_section_config", return_value={"images_path": str(tmpdir)}):
-        response = server.post('/upload', api_version=None, body=body, raw=True)
+    Config.instance().set("Server", "images_path", str(tmpdir))
+    response = server.post('/upload', api_version=None, body=body, raw=True)
 
     with open(str(tmpdir / "QEMU" / "test2")) as f:
         assert f.read() == "TEST"
 
     assert "test2" in response.body.decode("utf-8")
+
+
+def test_upload_images_backup(server, tmpdir):
+    Config.instance().set("Server", "images_path", str(tmpdir / 'images'))
+    os.makedirs(str(tmpdir / 'images' / 'IOU'))
+    # An old IOU image that we need to replace
+    with open(str(tmpdir / 'images' / 'IOU' / 'b.img'), 'w+') as f:
+        f.write('bad')
+
+    os.makedirs(str(tmpdir / 'old' / 'QEMU'))
+    with open(str(tmpdir / 'old' / 'QEMU' / 'a.img'), 'w+') as f:
+        f.write('hello')
+    os.makedirs(str(tmpdir / 'old' / 'IOU'))
+    with open(str(tmpdir / 'old' / 'IOU' / 'b.img'), 'w+') as f:
+        f.write('world')
+
+    os.chdir(str(tmpdir / 'old'))
+    with tarfile.open(str(tmpdir / 'test.tar'), 'w') as tar:
+        tar.add('.', recursive=True)
+
+    body = aiohttp.FormData()
+    body.add_field('type', 'IMAGES')
+    body.add_field('file', open(str(tmpdir / 'test.tar'), 'rb'), content_type='application/x-gtar', filename='test.tar')
+    response = server.post('/upload', api_version=None, body=body, raw=True)
+    assert response.status == 200
+
+    with open(str(tmpdir / 'images' / 'QEMU' / 'a.img')) as f:
+        assert f.read() == 'hello'
+    with open(str(tmpdir / 'images' / 'IOU' / 'b.img')) as f:
+        assert f.read() == 'world'
+
+    assert 'a.img' in response.body.decode('utf-8')
+    assert 'b.img' in response.body.decode('utf-8')
+    assert not os.path.exists(str(tmpdir / 'images' / 'archive.tar'))
+
+
+def test_upload_projects_backup(server, tmpdir):
+    Config.instance().set("Server", "projects_path", str(tmpdir / 'projects'))
+    os.makedirs(str(tmpdir / 'projects' / 'b'))
+    # An old b image that we need to replace
+    with open(str(tmpdir / 'projects' / 'b' / 'b.img'), 'w+') as f:
+        f.write('bad')
+
+    os.makedirs(str(tmpdir / 'old' / 'a'))
+    with open(str(tmpdir / 'old' / 'a' / 'a.img'), 'w+') as f:
+        f.write('hello')
+    os.makedirs(str(tmpdir / 'old' / 'b'))
+    with open(str(tmpdir / 'old' / 'b' / 'b.img'), 'w+') as f:
+        f.write('world')
+
+    os.chdir(str(tmpdir / 'old'))
+    with tarfile.open(str(tmpdir / 'test.tar'), 'w') as tar:
+        tar.add('.', recursive=True)
+
+    body = aiohttp.FormData()
+    body.add_field('type', 'PROJECTS')
+    body.add_field('file', open(str(tmpdir / 'test.tar'), 'rb'), content_type='application/x-gtar', filename='test.tar')
+    response = server.post('/upload', api_version=None, body=body, raw=True)
+    assert response.status == 200
+
+    with open(str(tmpdir / 'projects' / 'a' / 'a.img')) as f:
+        assert f.read() == 'hello'
+    with open(str(tmpdir / 'projects' / 'b' / 'b.img')) as f:
+        assert f.read() == 'world'
+
+    assert 'a.img' not in response.body.decode('utf-8')
+    assert 'b.img' not in response.body.decode('utf-8')
+    assert not os.path.exists(str(tmpdir / 'projects' / 'archive.tar'))
+
+
+def test_backup_images(server, tmpdir, loop):
+    Config.instance().set('Server', 'images_path', str(tmpdir))
+
+    os.makedirs(str(tmpdir / 'QEMU'))
+    with open(str(tmpdir / 'QEMU' / 'a.img'), 'w+') as f:
+        f.write('hello')
+    with open(str(tmpdir / 'QEMU' / 'b.img'), 'w+') as f:
+        f.write('world')
+
+    response = server.get('/backup/images.tar', api_version=None, raw=True)
+    assert response.status == 200
+    assert response.headers['CONTENT-TYPE'] == 'application/x-gtar'
+
+    with open(str(tmpdir / 'images.tar'), 'wb+') as f:
+        print(len(response.body))
+        f.write(response.body)
+
+    tar = tarfile.open(str(tmpdir / 'images.tar'), 'r')
+    os.makedirs(str(tmpdir / 'extract'))
+    os.chdir(str(tmpdir / 'extract'))
+    # Extract to current working directory
+    tar.extractall()
+    tar.close()
+
+    assert os.path.exists(os.path.join('QEMU', 'a.img'))
+    assert open(os.path.join('QEMU', 'a.img')).read() == 'hello'
+
+    assert os.path.exists(os.path.join('QEMU', 'b.img'))
+    assert open(os.path.join('QEMU', 'b.img')).read() == 'world'
+
+
+def test_backup_projects(server, tmpdir, loop):
+    Config.instance().set('Server', 'projects_path', str(tmpdir))
+
+    os.makedirs(str(tmpdir / 'a'))
+    with open(str(tmpdir / 'a' / 'a.gns3'), 'w+') as f:
+        f.write('hello')
+    os.makedirs(str(tmpdir / 'b'))
+    with open(str(tmpdir / 'b' / 'b.gns3'), 'w+') as f:
+        f.write('world')
+
+    response = server.get('/backup/projects.tar', api_version=None, raw=True)
+    assert response.status == 200
+    assert response.headers['CONTENT-TYPE'] == 'application/x-gtar'
+
+    with open(str(tmpdir / 'projects.tar'), 'wb+') as f:
+        print(len(response.body))
+        f.write(response.body)
+
+    tar = tarfile.open(str(tmpdir / 'projects.tar'), 'r')
+    os.makedirs(str(tmpdir / 'extract'))
+    os.chdir(str(tmpdir / 'extract'))
+    # Extract to current working directory
+    tar.extractall()
+    tar.close()
+
+    assert os.path.exists(os.path.join('a', 'a.gns3'))
+    assert open(os.path.join('a', 'a.gns3')).read() == 'hello'
+
+    assert os.path.exists(os.path.join('b', 'b.gns3'))
+    assert open(os.path.join('b', 'b.gns3')).read() == 'world'

tests/modules/dynamips/test_dynamips_manager.py

@@ -18,6 +18,10 @@
 
 import pytest
 import tempfile
+import sys
+import uuid
+import os
+import asyncio
 
 from gns3server.modules.dynamips import Dynamips
 from gns3server.modules.dynamips.dynamips_error import DynamipsError
@@ -37,8 +41,56 @@ def test_vm_invalid_dynamips_path(manager):
             manager.find_dynamips()
 
 
+@pytest.mark.skipif(sys.platform.startswith("win"), reason="Not supported by Windows")
 def test_vm_non_executable_dynamips_path(manager):
     tmpfile = tempfile.NamedTemporaryFile()
     with patch("gns3server.config.Config.get_section_config", return_value={"dynamips_path": tmpfile.name}):
         with pytest.raises(DynamipsError):
             manager.find_dynamips()
+
+
+def test_get_dynamips_id(manager):
+    project_1 = str(uuid.uuid4())
+    project_2 = str(uuid.uuid4())
+    project_3 = str(uuid.uuid4())
+
+    assert manager.get_dynamips_id(project_1) == 1
+    assert manager.get_dynamips_id(project_1) == 2
+    assert manager.get_dynamips_id(project_2) == 1
+    with pytest.raises(DynamipsError):
+        for dynamips_id in range(1, 4098):
+            manager.get_dynamips_id(project_3)
+
+
+def test_take_dynamips_id(manager):
+    project_1 = str(uuid.uuid4())
+
+    manager.take_dynamips_id(project_1, 1)
+    assert manager.get_dynamips_id(project_1) == 2
+    with pytest.raises(DynamipsError):
+        manager.take_dynamips_id(project_1, 1)
+
+
+def test_release_dynamips_id(manager):
+    project_1 = str(uuid.uuid4())
+    project_2 = str(uuid.uuid4())
+
+    manager.take_dynamips_id(project_1, 1)
+    manager.release_dynamips_id(project_1, 1)
+    assert manager.get_dynamips_id(project_1) == 1
+    # Should not crash for 0 id
+    manager.release_dynamips_id(project_2, 0)
+
+
+def test_project_closed(manager, project, loop):
+
+    manager._dynamips_ids[project.id] = set([1, 2, 3])
+
+    project_dir = project.module_working_path(manager.module_name.lower())
+    os.makedirs(project_dir)
+    open(os.path.join(project_dir, "test.ghost"), "w+").close()
+
+    loop.run_until_complete(asyncio.async(manager.project_closed(project)))
+
+    assert not os.path.exists(os.path.join(project_dir, "test.ghost"))
+    assert project.id not in manager._dynamips_ids

tests/modules/iou/test_iou_manager.py

@@ -17,14 +17,29 @@
 
 
 import pytest
+from unittest.mock import patch
 import uuid
+import os
+import sys
 
+pytestmark = pytest.mark.skipif(sys.platform.startswith("win"), reason="Not supported on Windows")
+
+if not sys.platform.startswith("win"):
+    from gns3server.modules.iou import IOU
+    from gns3server.modules.iou.iou_error import IOUError
 
-from gns3server.modules.iou import IOU
-from gns3server.modules.iou.iou_error import IOUError
 from gns3server.modules.project_manager import ProjectManager
 
 
+@pytest.fixture(scope="function")
+def iou(port_manager):
+    # Cleanup the IOU object
+    IOU._instance = None
+    iou = IOU.instance()
+    iou.port_manager = port_manager
+    return iou
+
+
 def test_get_application_id(loop, project, port_manager):
     # Cleanup the IOU object
     IOU._instance = None
@@ -71,3 +86,8 @@ def test_get_application_id_no_id_available(loop, project, port_manager):
             vm_id = str(uuid.uuid4())
             loop.run_until_complete(iou.create_vm("PC {}".format(i), project.id, vm_id))
             assert iou.get_application_id(vm_id) == i
+
+
+def test_get_images_directory(iou, tmpdir):
+    with patch("gns3server.config.Config.get_section_config", return_value={"images_path": str(tmpdir)}):
+        assert iou.get_images_directory() == str(tmpdir / "IOU")

tests/modules/iou/test_iou_vm.py

@@ -21,13 +21,19 @@ import asyncio
 import os
 import stat
 import socket
+import sys
 from tests.utils import asyncio_patch
 
 
 from unittest.mock import patch, MagicMock, PropertyMock
-from gns3server.modules.iou.iou_vm import IOUVM
-from gns3server.modules.iou.iou_error import IOUError
-from gns3server.modules.iou import IOU
+
+pytestmark = pytest.mark.skipif(sys.platform.startswith("win"), reason="Not supported on Windows")
+
+if not sys.platform.startswith("win"):
+    from gns3server.modules.iou.iou_vm import IOUVM
+    from gns3server.modules.iou.iou_error import IOUError
+    from gns3server.modules.iou import IOU
+
 from gns3server.config import Config
 
 
@@ -81,10 +87,11 @@ def test_vm(project, manager):
     assert vm.id == "00010203-0405-0607-0809-0a0b0c0d0e0f"
 
 
-def test_vm_initial_config(project, manager):
-    vm = IOUVM("test", "00010203-0405-0607-0808-0a0b0c0d0e0f", project, manager, initial_config="hostname %h")
+def test_vm_initial_config_content(project, manager):
+    vm = IOUVM("test", "00010203-0405-0607-0808-0a0b0c0d0e0f", project, manager)
+    vm.initial_config_content = "hostname %h"
     assert vm.name == "test"
-    assert vm.initial_config == "hostname test"
+    assert vm.initial_config_content == "hostname test"
     assert vm.id == "00010203-0405-0607-0808-0a0b0c0d0e0f"
 
 
@@ -286,10 +293,10 @@ def test_update_initial_config_empty(vm):
         assert f.read() == content
 
 
-def test_update_initial_config_h(vm):
+def test_update_initial_config_content_hostname(vm):
     content = "hostname %h\n"
     vm.name = "pc1"
-    vm.initial_config = content
+    vm.initial_config_content = content
     with open(vm.initial_config_file) as f:
         assert f.read() == "hostname pc1\n"
 

tests/modules/qemu/test_qemu_manager.py

@@ -18,6 +18,7 @@
 import os
 import stat
 import asyncio
+import sys
 
 from gns3server.modules.qemu import Qemu
 from tests.utils import asyncio_patch
@@ -26,13 +27,16 @@ from tests.utils import asyncio_patch
 def test_get_qemu_version(loop):
 
     with asyncio_patch("gns3server.modules.qemu.subprocess_check_output", return_value="QEMU emulator version 2.2.0, Copyright (c) 2003-2008 Fabrice Bellard") as mock:
-        version = loop.run_until_complete(asyncio.async(Qemu._get_qemu_version("/tmp/qemu-test")))
-        assert version == "2.2.0"
+        version = loop.run_until_complete(asyncio.async(Qemu.get_qemu_version("/tmp/qemu-test")))
+        if sys.platform.startswith("win"):
+            assert version == ""
+        else:
+            assert version == "2.2.0"
 
 
 def test_binary_list(loop):
 
-    files_to_create = ["qemu-system-x86", "qemu-system-x42", "hello"]
+    files_to_create = ["qemu-system-x86", "qemu-system-x42", "qemu-kvm", "hello"]
 
     for file_to_create in files_to_create:
         path = os.path.join(os.environ["PATH"], file_to_create)
@@ -43,11 +47,17 @@ def test_binary_list(loop):
     with asyncio_patch("gns3server.modules.qemu.subprocess_check_output", return_value="QEMU emulator version 2.2.0, Copyright (c) 2003-2008 Fabrice Bellard") as mock:
         qemus = loop.run_until_complete(asyncio.async(Qemu.binary_list()))
 
-        assert {"path": os.path.join(os.environ["PATH"], "qemu-system-x86"), "version": "2.2.0"} in qemus
-        assert {"path": os.path.join(os.environ["PATH"], "qemu-system-x42"), "version": "2.2.0"} in qemus
-        assert {"path": os.path.join(os.environ["PATH"], "hello"), "version": "2.2.0"} not in qemus
+        if sys.platform.startswith("win"):
+            version = ""
+        else:
+            version = "2.2.0"
+
+        assert {"path": os.path.join(os.environ["PATH"], "qemu-system-x86"), "version": version} in qemus
+        assert {"path": os.path.join(os.environ["PATH"], "qemu-kvm"), "version": version} in qemus
+        assert {"path": os.path.join(os.environ["PATH"], "qemu-system-x42"), "version": version} in qemus
+        assert {"path": os.path.join(os.environ["PATH"], "hello"), "version": version} not in qemus
 
 
 def test_get_legacy_vm_workdir():
 
-    assert Qemu.get_legacy_vm_workdir(42, "bla") == "qemu/vm-42"
+    assert Qemu.get_legacy_vm_workdir(42, "bla") == os.path.join("qemu", "vm-42")

tests/modules/qemu/test_qemu_vm.py

@@ -19,6 +19,7 @@ import pytest
 import aiohttp
 import asyncio
 import os
+import sys
 import stat
 import re
 from tests.utils import asyncio_patch
@@ -51,7 +52,10 @@ def fake_qemu_img_binary():
 @pytest.fixture
 def fake_qemu_binary():
 
-    bin_path = os.path.join(os.environ["PATH"], "qemu_x42")
+    if sys.platform.startswith("win"):
+        bin_path = os.path.join(os.environ["PATH"], "qemu_x42.EXE")
+    else:
+        bin_path = os.path.join(os.environ["PATH"], "qemu_x42")
     with open(bin_path, "w+") as f:
         f.write("1")
     os.chmod(bin_path, stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR)
@@ -178,8 +182,9 @@ def test_set_qemu_path(vm, tmpdir, fake_qemu_binary):
         f.write("1")
 
     # Raise because file is not executable
-    with pytest.raises(QemuError):
-        vm.qemu_path = path
+    if not sys.platform.startswith("win"):
+        with pytest.raises(QemuError):
+            vm.qemu_path = path
 
     os.chmod(path, stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR)
 
@@ -204,6 +209,7 @@ def test_disk_options(vm, loop, fake_qemu_img_binary):
         assert args == (fake_qemu_img_binary, "create", "-f", "qcow2", os.path.join(vm.working_dir, "flash.qcow2"), "256M")
 
 
+@pytest.mark.skipif(sys.platform.startswith("win"), reason="Not supported on Windows")
 def test_set_process_priority(vm, loop, fake_qemu_img_binary):
 
     with asyncio_patch("asyncio.create_subprocess_exec", return_value=MagicMock()) as process:
@@ -254,26 +260,26 @@ def test_control_vm_expect_text(vm, loop, running_subprocess_mock):
 def test_build_command(vm, loop, fake_qemu_binary, port_manager):
 
     os.environ["DISPLAY"] = "0:0"
-    with patch("gns3server.modules.qemu.qemu_vm.QemuVM._get_random_mac", return_value="00:00:ab:7e:b5:00"):
-        with asyncio_patch("asyncio.create_subprocess_exec", return_value=MagicMock()) as process:
-            cmd = loop.run_until_complete(asyncio.async(vm._build_command()))
-            assert cmd == [
-                fake_qemu_binary,
-                "-name",
-                "test",
-                "-m",
-                "256",
-                "-hda",
-                os.path.join(vm.working_dir, "flash.qcow2"),
-                "-serial",
-                "telnet:127.0.0.1:{},server,nowait".format(vm.console),
-                "-device",
-                "e1000,mac=00:00:ab:7e:b5:00,netdev=gns3-0",
-                "-netdev",
-                "user,id=gns3-0"
-            ]
+    with asyncio_patch("asyncio.create_subprocess_exec", return_value=MagicMock()) as process:
+        cmd = loop.run_until_complete(asyncio.async(vm._build_command()))
+        assert cmd == [
+            fake_qemu_binary,
+            "-name",
+            "test",
+            "-m",
+            "256",
+            "-hda",
+            os.path.join(vm.working_dir, "flash.qcow2"),
+            "-serial",
+            "telnet:127.0.0.1:{},server,nowait".format(vm.console),
+            "-net",
+            "none",
+            "-device",
+            "e1000,mac=00:00:ab:0e:0f:00"
+        ]
 
 
+@pytest.mark.skipif(sys.platform.startswith("win"), reason="Not supported on Windows")
 def test_build_command_without_display(vm, loop, fake_qemu_binary):
 
     os.environ["DISPLAY"] = ""
@@ -282,7 +288,9 @@ def test_build_command_without_display(vm, loop, fake_qemu_binary):
         assert "-nographic" in cmd
 
 
-def test_build_command_witht_invalid_options(vm, loop, fake_qemu_binary):
+# Windows accept this kind of mistake
+@pytest.mark.skipif(sys.platform.startswith("win"), reason="Not supported on Windows")
+def test_build_command_with_invalid_options(vm, loop, fake_qemu_binary):
 
     vm.options = "'test"
     with pytest.raises(QemuError):
@@ -306,6 +314,7 @@ def test_hdb_disk_image(vm, tmpdir):
         vm.hdb_disk_image = "test"
         assert vm.hdb_disk_image == str(tmpdir / "QEMU" / "test")
 
+
 def test_hdc_disk_image(vm, tmpdir):
 
     with patch("gns3server.config.Config.get_section_config", return_value={"images_path": str(tmpdir)}):
@@ -314,6 +323,7 @@ def test_hdc_disk_image(vm, tmpdir):
         vm.hdc_disk_image = "test"
         assert vm.hdc_disk_image == str(tmpdir / "QEMU" / "test")
 
+
 def test_hdd_disk_image(vm, tmpdir):
 
     with patch("gns3server.config.Config.get_section_config", return_value={"images_path": str(tmpdir)}):

tests/modules/test_manager.py

@@ -22,6 +22,15 @@ from unittest.mock import patch
 
 
 from gns3server.modules.vpcs import VPCS
+from gns3server.modules.qemu import Qemu
+
+
+@pytest.fixture(scope="function")
+def qemu(port_manager):
+    Qemu._instance = None
+    qemu = Qemu.instance()
+    qemu.port_manager = port_manager
+    return qemu
 
 
 def test_create_vm_new_topology(loop, project, port_manager):
@@ -82,3 +91,40 @@ def test_create_vm_old_topology(loop, project, tmpdir, port_manager):
         vm_dir = os.path.join(project_dir, "project-files", "vpcs", vm.id)
         with open(os.path.join(vm_dir, "startup.vpc")) as f:
             assert f.read() == "1"
+
+
+def test_get_abs_image_path(qemu, tmpdir):
+    os.makedirs(str(tmpdir / "QEMU"))
+    path1 = str(tmpdir / "test1.bin")
+    open(path1, 'w+').close()
+
+    path2 = str(tmpdir / "QEMU" / "test2.bin")
+    open(path2, 'w+').close()
+
+    with patch("gns3server.config.Config.get_section_config", return_value={"images_path": str(tmpdir)}):
+        assert qemu.get_abs_image_path(path1) == path1
+        assert qemu.get_abs_image_path("test1.bin") == path1
+        assert qemu.get_abs_image_path(path2) == path2
+        assert qemu.get_abs_image_path("test2.bin") == path2
+        assert qemu.get_abs_image_path("../test1.bin") == path1
+
+        # We look at first in new location
+        path2 = str(tmpdir / "QEMU" / "test1.bin")
+        open(path2, 'w+').close()
+        assert qemu.get_abs_image_path("test1.bin") == path2
+
+
+def test_get_relative_image_path(qemu, tmpdir):
+    os.makedirs(str(tmpdir / "QEMU"))
+    path1 = str(tmpdir / "test1.bin")
+    open(path1, 'w+').close()
+
+    path2 = str(tmpdir / "QEMU" / "test2.bin")
+    open(path2, 'w+').close()
+
+    with patch("gns3server.config.Config.get_section_config", return_value={"images_path": str(tmpdir)}):
+        assert qemu.get_relative_image_path(path1) == path1
+        assert qemu.get_relative_image_path("test1.bin") == path1
+        assert qemu.get_relative_image_path(path2) == "test2.bin"
+        assert qemu.get_relative_image_path("test2.bin") == "test2.bin"
+        assert qemu.get_relative_image_path("../test1.bin") == path1

tests/modules/test_port_manager.py

@@ -26,34 +26,46 @@ from gns3server.modules.project import Project
 def test_reserve_tcp_port():
     pm = PortManager()
     project = Project()
-    pm.reserve_tcp_port(4242, project)
+    pm.reserve_tcp_port(2001, project)
     with pytest.raises(aiohttp.web.HTTPConflict):
-        pm.reserve_tcp_port(4242, project)
+        pm.reserve_tcp_port(2001, project)
+
+
+def test_reserve_tcp_port_outside_range():
+    pm = PortManager()
+    project = Project()
+    with pytest.raises(aiohttp.web.HTTPConflict):
+        pm.reserve_tcp_port(80, project)
 
 
 def test_reserve_udp_port():
     pm = PortManager()
     project = Project()
-    pm.reserve_udp_port(4242, project)
+    pm.reserve_udp_port(10000, project)
     with pytest.raises(aiohttp.web.HTTPConflict):
-        pm.reserve_udp_port(4242, project)
+        pm.reserve_udp_port(10000, project)
+
+
+def test_reserve_udp_port_outside_range():
+    pm = PortManager()
+    project = Project()
+    with pytest.raises(aiohttp.web.HTTPConflict):
+        pm.reserve_udp_port(80, project)
 
 
-@pytest.mark.skipif(sys.platform == 'darwin', reason="not working on darwin")
 def test_release_udp_port():
     pm = PortManager()
     project = Project()
-    pm.reserve_udp_port(4242, project)
-    pm.release_udp_port(4242, project)
-    pm.reserve_udp_port(4242, project)
+    pm.reserve_udp_port(10000, project)
+    pm.release_udp_port(10000, project)
+    pm.reserve_udp_port(10000, project)
 
 
-@pytest.mark.skipif(sys.platform != 'darwin', reason="requires darwin")
-def test_release_darwin_udp_port():
-    """Due to dynamips / darwin bug we didn't free the port"""
-    pm = PortManager()
-    project = Project()
-    pm.reserve_udp_port(4242, project)
-    pm.release_udp_port(4242, project)
+def test_find_unused_port():
+    p = PortManager().find_unused_port(1000, 10000)
+    assert p is not None
+
+
+def test_find_unused_port_invalid_range():
     with pytest.raises(aiohttp.web.HTTPConflict):
-        pm.reserve_udp_port(4242, project)
+        p = PortManager().find_unused_port(10000, 1000)

tests/modules/test_project.py

@@ -69,15 +69,10 @@ def test_changing_path_temporary_flag(tmpdir):
     with patch("gns3server.modules.project.Project.is_local", return_value=True):
         p = Project(temporary=True)
         assert os.path.exists(p.path)
+        original_path = p.path
         assert os.path.exists(os.path.join(p.path, ".gns3_temporary"))
-        p.temporary = False
-        assert not os.path.exists(os.path.join(p.path, ".gns3_temporary"))
-
-        with open(str(tmpdir / ".gns3_temporary"), "w+") as f:
-            f.write("1")
 
         p.path = str(tmpdir)
-        assert not os.path.exists(os.path.join(str(tmpdir), ".gns3_temporary"))
 
 
 def test_temporary_path():

tests/modules/virtualbox/test_virtualbox_manager.py

@@ -20,10 +20,14 @@ import pytest
 import tempfile
 import os
 import stat
+import asyncio
+
+
+from unittest.mock import patch
 
 from gns3server.modules.virtualbox import VirtualBox
 from gns3server.modules.virtualbox.virtualbox_error import VirtualBoxError
-from unittest.mock import patch
+from tests.utils import asyncio_patch
 
 
 @pytest.fixture(scope="module")
@@ -65,3 +69,31 @@ def test_vboxmanage_path(manager, tmpdir):
     tmpfile = tempfile.NamedTemporaryFile()
     with patch("gns3server.config.Config.get_section_config", return_value={"vboxmanage_path": path}):
         assert manager.find_vboxmanage() == path
+
+
+def test_get_list(manager, loop):
+    vm_list = ['"Windows 8.1" {27b4d095-ff5f-4ac4-bb9d-5f2c7861c1f1}',
+               '"Carriage',
+               'Return" {27b4d095-ff5f-4ac4-bb9d-5f2c7861c1f1}',
+               '',
+               '"<inaccessible>" {42b4d095-ff5f-4ac4-bb9d-5f2c7861c1f1}',
+               '"Linux Microcore 4.7.1" {ccd8c50b-c172-457d-99fa-dd69371ede0e}']
+
+    @asyncio.coroutine
+    def execute_mock(cmd, args):
+        if cmd == "list":
+            return vm_list
+        else:
+            if args[0] == "Windows 8.1":
+                return ["memory=512"]
+            elif args[0] == "Linux Microcore 4.7.1":
+                return ["memory=256"]
+        assert False, "Unknow {} {}".format(cmd, args)
+
+    with asyncio_patch("gns3server.modules.virtualbox.VirtualBox.execute") as mock:
+        mock.side_effect = execute_mock
+        vms = loop.run_until_complete(asyncio.async(manager.get_list()))
+    assert vms == [
+        {"vmname": "Windows 8.1", "ram": 512},
+        {"vmname": "Linux Microcore 4.7.1", "ram": 256}
+    ]

tests/modules/virtualbox/test_virtualbox_vm.py

@@ -54,3 +54,9 @@ def test_vm_invalid_virtualbox_api_version(loop, project, manager):
         with pytest.raises(VirtualBoxError):
             vm = VirtualBoxVM("test", "00010203-0405-0607-0809-0a0b0c0d0e0f", project, manager, "test", False)
             loop.run_until_complete(asyncio.async(vm.create()))
+
+
+def test_vm_adapter_add_nio_binding_adapter_not_exist(loop, vm, manager, free_console_port):
+    nio = manager.create_nio(manager.vboxmanage_path, {"type": "nio_udp", "lport": free_console_port, "rport": free_console_port, "rhost": "192.168.1.2"})
+    with pytest.raises(VirtualBoxError):
+        loop.run_until_complete(asyncio.async(vm.adapter_add_nio_binding(15, nio)))

tests/modules/vpcs/test_vpcs_vm.py

@@ -19,6 +19,7 @@ import pytest
 import aiohttp
 import asyncio
 import os
+import sys
 from tests.utils import asyncio_patch
 
 
@@ -97,9 +98,14 @@ def test_stop(loop, vm):
             loop.run_until_complete(asyncio.async(vm.start()))
             assert vm.is_running()
 
-            loop.run_until_complete(asyncio.async(vm.stop()))
+            with asyncio_patch("gns3server.utils.asyncio.wait_for_process_termination"):
+                loop.run_until_complete(asyncio.async(vm.stop()))
             assert vm.is_running() is False
-            process.terminate.assert_called_with()
+
+            if sys.platform.startswith("win"):
+                process.send_signal.assert_called_with(1)
+            else:
+                process.terminate.assert_called_with()
 
 
 def test_reload(loop, vm):
@@ -117,9 +123,15 @@ def test_reload(loop, vm):
             vm.port_add_nio_binding(0, nio)
             loop.run_until_complete(asyncio.async(vm.start()))
             assert vm.is_running()
-            loop.run_until_complete(asyncio.async(vm.reload()))
+
+            with asyncio_patch("gns3server.utils.asyncio.wait_for_process_termination"):
+                loop.run_until_complete(asyncio.async(vm.reload()))
             assert vm.is_running() is True
-            process.terminate.assert_called_with()
+
+            if sys.platform.startswith("win"):
+                process.send_signal.assert_called_with(1)
+            else:
+                process.terminate.assert_called_with()
 
 
 def test_add_nio_binding_udp(vm):
@@ -135,13 +147,13 @@ def test_add_nio_binding_tap(vm):
         assert nio.tap_device == "test"
 
 
-def test_add_nio_binding_tap_no_privileged_access(vm):
-    with patch("gns3server.modules.base_manager.BaseManager._has_privileged_access", return_value=False):
-        with pytest.raises(aiohttp.web.HTTPForbidden):
-            nio = VPCS.instance().create_nio(vm.vpcs_path, {"type": "nio_tap", "tap_device": "test"})
-            vm.port_add_nio_binding(0, nio)
-    assert vm._ethernet_adapter.ports[0] is None
-
+# def test_add_nio_binding_tap_no_privileged_access(vm):
+#     with patch("gns3server.modules.base_manager.BaseManager._has_privileged_access", return_value=False):
+#         with pytest.raises(aiohttp.web.HTTPForbidden):
+#             nio = VPCS.instance().create_nio(vm.vpcs_path, {"type": "nio_tap", "tap_device": "test"})
+#             vm.port_add_nio_binding(0, nio)
+#     assert vm._ethernet_adapter.ports[0] is None
+#
 
 def test_port_remove_nio_binding(vm):
     nio = VPCS.instance().create_nio(vm.vpcs_path, {"type": "nio_udp", "lport": 4242, "rport": 4243, "rhost": "127.0.0.1"})
@@ -159,15 +171,6 @@ def test_update_startup_script(vm):
         assert f.read() == content
 
 
-def test_update_startup_script(vm):
-    content = "echo GNS3 VPCS\nip 192.168.1.2\n"
-    vm.startup_script = content
-    filepath = os.path.join(vm.working_dir, 'startup.vpc')
-    assert os.path.exists(filepath)
-    with open(filepath) as f:
-        assert f.read() == content
-
-
 def test_update_startup_script_h(vm):
     content = "setname %h\n"
     vm.name = "pc1"
@@ -178,9 +181,9 @@ def test_update_startup_script_h(vm):
 
 
 def test_get_startup_script(vm):
-    content = "echo GNS3 VPCS\nip 192.168.1.2\n"
+    content = "echo GNS3 VPCS\nip 192.168.1.2"
     vm.startup_script = content
-    assert vm.startup_script == content
+    assert vm.startup_script == os.linesep.join(["echo GNS3 VPCS","ip 192.168.1.2"])
 
 
 def test_get_startup_script_using_default_script(vm):
@@ -190,8 +193,8 @@ def test_get_startup_script_using_default_script(vm):
     vm._script_file = None
 
     filepath = os.path.join(vm.working_dir, 'startup.vpc')
-    with open(filepath, 'w+') as f:
-        assert f.write(content)
+    with open(filepath, 'wb+') as f:
+        assert f.write(content.encode("utf-8"))
 
     assert vm.startup_script == content
     assert vm.script_file == filepath

tests/utils/test_asyncio.py

@@ -18,6 +18,7 @@
 
 import asyncio
 import pytest
+import sys
 from unittest.mock import MagicMock
 
 from gns3server.utils.asyncio import wait_run_in_executor, subprocess_check_output, wait_for_process_termination
@@ -43,6 +44,7 @@ def test_exception_wait_run_in_executor(loop):
         result = loop.run_until_complete(asyncio.async(exec))
 
 
+@pytest.mark.skipif(sys.platform.startswith("win"), reason="Not supported on Windows")
 def test_subprocess_check_output(loop, tmpdir, restore_original_path):
 
     path = str(tmpdir / "test")

tests/utils/test_interfaces.py

@@ -0,0 +1,24 @@
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2014 GNS3 Technologies Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+from gns3server.utils.interfaces import interfaces
+
+
+def test_interfaces():
+    # This test should pass on all platforms without crash
+    assert isinstance(interfaces(), list)