Update issue templates

Update README and add mkdocs.yml

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,23 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

.gitignore

@@ -40,4 +40,6 @@ docker/tmp/*
 GPATH
 GRTAGS
 GTAGS
-.python-version
+.python-version
+.DS_Store
+.#*

.travis.yml

@@ -0,0 +1,36 @@
+services:
+- docker
+before_script:
+- sudo apt update
+- sudo apt install qemu-user-static
+- "./create-binary-docker backend"
+- "./create-binary-docker backend -a arm"
+- "./create-binary-docker gateway"
+- "./create-binary-docker gateway -a arm"
+- "./create-binary-docker orioncontextbroker"
+- "./create-binary-docker orioncontextbroker -a arm"
+- "./create-binary-docker influxdbapp"
+- "./create-binary-docker influxdbapp -a arm"
+- "./create-binary-docker cul868ipe"
+- "./create-binary-docker cul868ipe -a arm"
+script:
+- docker tag openmtc/orioncontextbroker-amd64 openmtc/orion-context-broker-app-amd64
+- docker tag openmtc/orioncontextbroker-arm openmtc/orion-context-broker-app-arm
+- docker tag openmtc/cul868ipe-amd64 openmtc/cul868-ipe-amd64
+- docker tag openmtc/cul868ipe-arm openmtc/cul868-ipe-arm
+- docker tag openmtc/influxdbapp-amd64 openmtc/influxdb-app-amd64
+- docker tag openmtc/influxdbapp-arm openmtc/influxdb-app-arm
+- docker login -u "$DOCKER_USERNAME" -p "$DOCKER_PASSWORD";
+- docker push openmtc/backend-amd64
+- docker push openmtc/backend-arm
+- docker push openmtc/gateway-amd64
+- docker push openmtc/gateway-arm
+- docker push openmtc/orion-context-broker-app-amd64
+- docker push openmtc/orion-context-broker-app-arm
+- docker push openmtc/cul868-ipe-amd64
+- docker push openmtc/cul868-ipe-arm
+- docker push openmtc/influxdb-app-amd64
+- docker push openmtc/influxdb-app-arm
+env:
+  matrix:
+    secure: oQe/MxxFrPPArxb6OFzOUwG2ZlA5GYekMR6qn0Y3101v82MdrIVgDILHR41iwykTtk1XpJPV9uabsRsvY4hBmbjYzihYZFOAzFf+/KU7wROtKum0fFfLjCPb8uGV41k1JTquB94FibXboVAP7rNL0Vrpl+FGUvMGatQBtwZnGvha6Ha07qTao+X9+0dJ4YvFlvcba/jfzBZBwHw7KHpRoiyKU0dPmXpHcqnZBFcNqRXhzdVrgx0auP/tghoshW5LQWkpTV11uSx/kYuby4oo8r8nB6L0rW1jYSXs9DHiTQCfCy24xlb9YJjCD9aFcjH0lIkVIqQwJYA67MzKYMK4XV684J/Jr3+jfVOoUt0bpZaTnk+r/uiFCtEsN7q0KUlvHLUAi1YNJhKs6CRrAH3GK25QByh+suzPzZoHP42F2LYP9URlzbDH+/v7CwNuw+9pHSmxEhs18LVmggwkMos3o3ArvzMLRNJ3QG1fdoxL/Ubqxhmvhy4rZi6vGQsiF64oX8PN0sRgXekTU/ma+6CTM6qhgkocMUYzT2r/6qd/9R3jmtxSZnikjkR6Iu9NLzAUY+cqnvvjohAvqYyj+tGlQfOPxAU9H4wXnbQWLBM8yn6yt2Ki+3+Jx4Owdv0oXVJpN0GWjMzIJ1nEus5odkWW+c4wcZb6m3Ak0PrzYPp2BgE=

README.md

@@ -4,44 +4,102 @@
   </a>
 </p>
 
-The OpenMTC SDK aims to provide developers with a convenient yet flexible tool to write oneM2M compliant applications. This includes network applications (NAs), gateway application (GAs), device applications (DAs), as well as interworking proxy entities (IPEs).
+[![FIWARE IoT Agents](https://nexus.lab.fiware.org/static/badges/chapters/iot-agents.svg)](https://www.fiware.org/developers/catalogue/)
+[![License: Eclipse](https://img.shields.io/github/license/OpenMTC/OpenMTC.svg)](https://www.eclipse.org/legal/epl-v10.html)
+[![Support badge](https://nexus.lab.fiware.org/repository/raw/public/badges/stackoverflow/iot-agents.svg)](https://stackoverflow.com/questions/tagged/fiware+iot)
+<br/>
+[![Documentation badge](https://img.shields.io/readthedocs/fiware-openmtc.svg)](http://fiware-openmtc.readthedocs.org/en/latest/?badge=latest)
+![Status](https://nexus.lab.fiware.org/static/badges/statuses/iot-openmtc.svg)
+[![](https://img.shields.io/docker/pulls/openmtc/gateway-amd64.svg)](https://hub.docker.com/u/openmtc)
+[![Build Status](https://travis-ci.org/OpenMTC/OpenMTC.svg?branch=master)](https://travis-ci.org/OpenMTC/OpenMTC)
+[![Known Vulnerabilities](https://snyk.io/test/github/OpenMTC/OpenMTC/badge.svg?targetFile=openmtc-gevent%2Frequirements.txt)](https://snyk.io/test/github/OpenMTC/OpenMTC?targetFile=openmtc-gevent%2Frequirements.txt)
+[![Coverage Status](https://coveralls.io/repos/github/OpenMTC/OpenMTC/badge.svg?branch=master)](https://coveralls.io/github/OpenMTC/OpenMTC?branch=master)
 
-# Table of Content
+The OpenMTC SDK aims to provide developers with a convenient yet flexible tool
+to write oneM2M compliant applications. This includes network applications
+(NAs), gateway application (GAs), device applications (DAs), as well as
+interworking proxy entities (IPEs).
 
-- [Quick Start](doc/openmtc-get-started.md)
-- [Introduction](doc/introduction.md)
-- [Deployment](doc/deployment-guide.md)
-- [The MQTT Client](doc/onem2m-client-mqtt.md)
-- [Authentication Guide](doc/authentication.md)
-- [Installation of the OpenMTC SDK](doc/install-sdk.md)
-- [Overview REST API](doc/overview-rest-api.md)
-- [Write your first OpenMTC applications](doc/training/training-index.md)
-- [SDK - Using the Application Framework](doc/sdk-framework.md)
-- [SDK - The low-level CSE Client](doc/sdk-client.md)
-- [SDK - The Data Model](doc/sdk-datamodel.md)
-- Examples
-    - [IoT Data Visualization](doc/example-apps/IoT-data-visualization.py)
-    - [Data Aggregation](doc/example-apps/data-aggregation.py)
-    - [Simple Decision](doc/example-apps/simple-decision.py)
-    - [Simple Decision 2](doc/example-apps/simple-decision-2.py)
-- Scripts
-    - [Create App Structure Script](doc/create-app-structure.md)
-    - [Create binary docker images Script](doc/create-binary-docker.md)
-- [Code Repository Structure](doc/repository-structure.md)
-- [Developer FAQ](doc/developer-faq.md)
+This project is part of [FIWARE](https://www.fiware.org/). For more information
+check the FIWARE Catalogue entry for the
+[IoT Agents](https://github.com/Fiware/catalogue/tree/master/iot-agents).
+
+# Content
+
+-   [Install](#install)
+-   [Usage](#usage)
+-   [API](#api)
+-   [Quality Assurance](#quality-assurance)
+-   [Contributing](#contributing)
+-   [License](#license)
 
 
-# Python
+## Install
 
-The OpenMTC SDK is written in and for the Python programming language. Users should therefore have at least a certain knowledge of Python and its paradigms. For this matter, the following material is recommended:
+Information about how to install the JSON IoTAgent can be found at the
+corresponding section of the
+[Installation & Administration Guide](https://fiware-openmtc.readthedocs.io/en/latest/deployment-guide).
 
-- [The Python Homepage](http://www.python.org)
-- [Expert Python Programming by Tarek Ziadé](http://www.e-reading.by/bookreader.php/138816/Ziade_-_Expert_Python_programming.pdf)
-- [Code Like a Pythonista: Idiomatic Python by David Goodger](http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html)
+## Usage
 
-# Feedback
+Information about how to use the IoT Agent can be found in the
+[User & Programmers Manual](https://fiware-openmtc.readthedocs.io/en/latest/introduction).
 
-Please create issues for any problems and direct any comments or feedback you are having to <a href="mailto:support@openmtc.org">support@openmtc.org</a>
+## API
 
-Please let us know what you think. We are also very interested in any use case you are *not* able to implement with the SDK or if you find it difficult to do so.
+Information about the REST API can be found in the
+[API Section](https://fiware-openmtc.readthedocs.io/en/latest/overview-rest-api)
+
+
+## Python
+
+The OpenMTC SDK is written in and for the Python programming language. Users
+should therefore have at least a certain knowledge of Python and its paradigms.
+For this matter, the following material is recommended:
+
+-   [The Python Homepage](http://www.python.org)
+-   [Expert Python Programming by Tarek Ziadé](http://www.e-reading.by/bookreader.php/138816/Ziade_-_Expert_Python_programming.pdf)
+-   [Code Like a Pythonista: Idiomatic Python](http://www.omahapython.org/IdiomaticPython.html)
+
+## Feedback
+
+Please create issues for any problems and direct any comments or feedback you
+are having to <a href="mailto:support@openmtc.org">support@openmtc.org</a>
+
+Please let us know what you think. We are also very interested in any use case
+you are _not_ able to implement with the SDK or if you find it difficult to do
+so.
+
+
+## Quality Assurance
+
+This project is part of [FIWARE](https://fiware.org/) and has been rated as
+follows:
+
+-   **Version Tested:**
+    ![ ](https://img.shields.io/badge/dynamic/json.svg?label=Version&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.version&colorB=blue)
+-   **Documentation:**
+    ![ ](https://img.shields.io/badge/dynamic/json.svg?label=Completeness&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.docCompleteness&colorB=blue)
+    ![ ](https://img.shields.io/badge/dynamic/json.svg?label=Usability&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.docSoundness&colorB=blue)
+-   **Responsiveness:**
+    ![ ](https://img.shields.io/badge/dynamic/json.svg?label=Time%20to%20Respond&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.timeToCharge&colorB=blue)
+    ![ ](https://img.shields.io/badge/dynamic/json.svg?label=Time%20to%20Fix&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.timeToFix&colorB=blue)
+-   **FIWARE Testing:**
+    ![ ](https://img.shields.io/badge/dynamic/json.svg?label=Tests%20Passed&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.failureRate&colorB=blue)
+    ![ ](https://img.shields.io/badge/dynamic/json.svg?label=Scalability&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.scalability&colorB=blue)
+    ![ ](https://img.shields.io/badge/dynamic/json.svg?label=Performance&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.performance&colorB=blue)
+    ![ ](https://img.shields.io/badge/dynamic/json.svg?label=Stability&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.stability&colorB=blue)
+
+---
+
+## Contributing
+
+Contribution guidelines are detailed in the [CONTRIBUTIONS](https://github.com/OpenMTC/OpenMTC/blob/master/CONTRIBUTIONS.md) file.
+
+## License
+
+The OpenMTC SDK is licensed under the Eclipse Public License (EPL)
+version 1.
+
+© 2018 OpenMTC
 

apps/InfluxDB/bin/openmtc-influx-db

@@ -1,3 +0,0 @@
-#!/usr/bin/env bash
-
-exec python -m influxdb $@

apps/InfluxdbApp/README.md

@@ -7,7 +7,7 @@ This App will subscribe to OpenMTC data and tranfer it to an instance of the Inf
 You need a running instance of InfluxDB and configure the following parameters according to your setup.
 
 ```
-apps/influx-db \
+apps/influxdb-app \
     --ep "http://127.0.0.1:8000" \
     --influx_host "127.0.0.1" \
     --influx-port "8086" \

apps/InfluxdbApp/bin/openmtc-influxdb-app

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+
+exec python -m influxdbapp $@

apps/InfluxdbApp/docker/configure-influxdbapp-and-start

@@ -1,8 +1,8 @@
 #!/usr/bin/env bash
 
-CONFIG_FILE="/etc/openmtc/influxdb/config.json"
+CONFIG_FILE="/etc/openmtc/influxdbapp/config.json"
 
-NAME=${NAME-"InfluxDB"}
+NAME=${NAME-"InfluxdbApp"}
 EP=${EP-"http://localhost:8000"}
 CSE_BASE=${CSE_BASE-"onem2m"}
 POAS=${POAS-'["http://auto:23706"]'}
@@ -20,7 +20,7 @@ DBUSER=${DBUSER-"test"},
 DBUSER_PW=${DBUSER_PW-"test"}
 
 # defaults logging
-LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/influxdb.log"}
+LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/influxdbapp.log"}
 LOGGING_LEVEL=${LOGGING_LEVEL-"ERROR"}
 
 # ensure correct level
@@ -40,7 +40,7 @@ HOST_NAME=${EXTERNAL_IP-${LOCAL_IP}}
 
 # Configuration of the service.
 CONFIG_TEMP=${CONFIG_FILE}".tmp"
-echo -n "Configuring M2M influxdb..."
+echo -n "Configuring M2M influxdbapp..."
 JQ_STRING='.'
 
 # basics
@@ -70,4 +70,4 @@ mv ${CONFIG_TEMP} ${CONFIG_FILE}
 
 echo "done"
 
-exec python -m influxdb $@
+exec python -m influxdbapp $@

apps/InfluxdbApp/docker/influxdbapp-amd64

@@ -0,0 +1,30 @@
+############################################################
+# Dockerfile to run openmtc influxdbapp binary
+############################################################
+
+# Set the base image to use openmtc/sdk
+FROM openmtc/sdk-amd64:latest
+
+ENV MOD_NAME=influxdbapp
+
+# Set the file maintainer
+MAINTAINER rst
+
+# install openmtc dependencies
+COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
+RUN pip install --upgrade --requirement /tmp/requirements.txt
+
+# install openmtc-influxdbapp
+COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
+RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
+    --owner root --group root --no-same-owner --no-overwrite-dir \
+    --transform 's/json\.dist/json/' --show-transformed
+
+RUN mkdir -p /var/log/openmtc
+
+# add change config
+COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
+
+# entry point
+ENTRYPOINT ["/usr/local/bin/configure-and-start"]
+CMD [""]

apps/InfluxdbApp/docker/influxdbapp-arm

@@ -0,0 +1,30 @@
+############################################################
+# Dockerfile to run openmtc influxdbapp binary
+############################################################
+
+# Set the base image to use openmtc/sdk
+FROM openmtc/sdk-arm:latest
+
+ENV MOD_NAME=influxdbapp
+
+# Set the file maintainer
+MAINTAINER rst
+
+# install openmtc dependencies
+COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
+RUN pip install --upgrade --requirement /tmp/requirements.txt
+
+# install openmtc-influxdbapp
+COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
+RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
+    --owner root --group root --no-same-owner --no-overwrite-dir \
+    --transform 's/json\.dist/json/' --show-transformed
+
+RUN mkdir -p /var/log/openmtc
+
+# add change config
+COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
+
+# entry point
+ENTRYPOINT ["/usr/local/bin/configure-and-start"]
+CMD [""]

apps/InfluxdbApp/etc/systemd/system/openmtc-influxdbapp.service

@@ -1,10 +1,10 @@
 [Unit]
-Description=OpenMTC InfluxDB
+Description=OpenMTC InfluxdbApp
 After=network.target
 Wants=ntp.service
 
 [Service]
-ExecStart=/usr/local/bin/influx-db
+ExecStart=/usr/local/bin/influxdb-app
 
 [Install]
 WantedBy=multi-user.target

apps/InfluxdbApp/setup-influxdbapp.py

@@ -8,7 +8,7 @@ import sys
 from utils import get_packages, get_pkg_files, OpenMTCSdist, move_config_files
 
 # name and dir
-NAME = "influxdb"
+NAME = "influxdbapp"
 BASE_DIR = "."
 
 # import pkg

apps/InfluxdbApp/src/influxdbapp/__init__.py

@@ -2,8 +2,8 @@
 Transfer OpenMTC Data to an InfluxDB
 """
 
-__version__ = "0.1"
-__description__ = "InfluxDB"
+__version__ = "1.2.0"
+__description__ = "InfluxdbApp"
 __author_name__ = "Christian Klopp"
 __author_mail__ = "christian.klopp@fokus.fraunhofer.de"
 __requires__ = ['influxdb']

apps/InfluxdbApp/src/influxdbapp/__main__.py

@@ -2,17 +2,17 @@ from argparse import ArgumentDefaultsHelpFormatter, ArgumentParser
 
 from openmtc_app.util import prepare_app, get_value
 from openmtc_app.runner import AppRunner as Runner
-from .influx_db import InfluxDB
+from .influxdb_app import InfluxdbApp
 
 # defaults
-default_name = "InfluxDB"
+default_name = "InfluxdbApp"
 default_ep = "http://localhost:8000"
 default_labels = []
 
 # args parser
 parser = ArgumentParser(
-    description="An IPE called InfluxDB",
-    prog="InfluxDB",
+    description="An IPE called InfluxdbApp",
+    prog="InfluxdbApp",
     formatter_class=ArgumentDefaultsHelpFormatter)
 parser.add_argument("-n", "--name", help="Name used for the AE.")
 parser.add_argument("-s", "--ep", help="URL of the local Endpoint.")
@@ -47,7 +47,7 @@ db_user = get_value("db_user", (unicode, str), "test", args, config)
 db_pw = get_value("db_pw", (unicode, str), "test", args, config)
 
 # start
-app = InfluxDB(
+app = InfluxdbApp(
     name=nm, cse_base=cb, poas=poas,
     labels=lbl,
     originator_pre=originator_pre, 

apps/InfluxdbApp/src/influxdbapp/influxdb_app.py

@@ -2,7 +2,7 @@ from openmtc_app.onem2m import ResourceManagementXAE
 from connector import InfluxDBConnector
 
 
-class InfluxDB(ResourceManagementXAE):
+class InfluxdbApp(ResourceManagementXAE):
 
     def __init__(
             self,
@@ -17,7 +17,7 @@ class InfluxDB(ResourceManagementXAE):
             *args,
             **kw
             ):
-        super(InfluxDB, self).__init__(*args, **kw)
+        super(InfluxdbApp, self).__init__(*args, **kw)
         if isinstance(labels, basestring):
             self.labels = {labels}
         elif hasattr(labels, '__iter__') and len(labels):

apps/OrionContextBroker/README.md

@@ -1,6 +1,6 @@
 # Introduction
 
-OrionContextBroker is an OpenMTC AE to forward OpenMTC sensor data (via Subscription) to an instance of the Orion Context Broker. Additionally OpenMTC actuators are handled by forwarding changes on the OCB side via OpenMTC to the actuator.
+OrionContextBroker (OCB) is an OpenMTC AE to forward OpenMTC sensor data (via Subscription) to an instance of the Orion Context Broker. Additionally OpenMTC actuators are handled by forwarding changes on the OCB side via OpenMTC to the actuator.
 All Content Instances are expected to use the SenML format. It is possible to connect the AE either to an OpenMTC Gateway or an OpenMTC Backend.
 
 # Getting started
@@ -27,6 +27,191 @@ The most important parameters are:
 * orion_host (hostname:port of the Orion CB)
 * accumulate_address (Subscription Sink (RESTful HTTP) used for subscriptions to the OCB (actuator functionality))
 
+## Example Setup with Docker
+
+![](docker_setup.png)
+
+The illustration above shows an example setup, which can be created using docker-compose with the docker-compose.yml file listed at the end of this section.
+For this example an OpenMTC Interworking Proxy called CUL868IPE is used in simulation mode, in order to periodically generate sensor data without the need of real physical sensors. This IPE will exchange data with an OpenMTC Gateway. The Gateway itself is connected to an OpenMTC Backend. The OpenMTC OCB-App will on one hand forward sensor data to the OrionContextBroker and on the other hand will forward actuator events to the simulated actuator on the CUL868IPE.
+
+The following docker-compose.yml shows the necessary configuration:
+
+```
+version: "2"
+
+services:
+    gateway:
+        image: openmtc/gateway-amd64
+        ports:
+            - "8000:8000"
+        environment:
+            - ONEM2M_NOTIFICATION_DISABLED=false
+            - ONEM2M_CSE_ID=TESTUSER~TESTTENANT
+            - ONEM2M_REGISTRATION_DISABLED=false
+            - ONEM2M_REMOTE_CSE_POA=http://backend:18000
+            - ONEM2M_REMOTE_CSE_OWN_POA=http://gateway:8000
+        container_name: gateway
+        links:
+            - backend
+        networks:
+            - main
+
+    backend:
+        image: openmtc/backend-amd64
+        ports:
+            - "18000:18000"
+        environment:
+            - ONEM2M_NOTIFICATION_DISABLED=false
+            - ONEM2M_HTTP_TRANSPORT_DISABLED=false
+            - ONEM2M_HTTP_TRANSPORT_SSL_ENABLED=false
+        container_name: backend
+        networks:
+            - main
+
+    orioncontextbroker-app:
+        image: openmtc/orion-context-broker-app-amd64
+        container_name: orioncontextbroker-app
+        links:
+            - backend
+        ports:
+            - "8086:8086"
+            - "8080:8080"
+        environment:
+            - EP=http://backend:18000
+            - ORION_HOST=http://orion:1026
+            - ORION_API=v2
+            - ACCUMULATE_ADDRESS=http://orioncontextbroker-app:8080
+        networks:
+            - main
+
+    culipe:
+        image: openmtc/cul868-ipe-amd64
+        links:
+            - gateway
+            - orioncontextbroker-app
+
+        environment:
+            - EP=http://gateway:8000
+            - DEVICES=["fs20:16108-1"]
+            - SIM=true
+            - DEVICE_MAPPINGS={"S300TH_1":"kitchen","FS20_ST3_16108_1":"bath"}
+        container_name: culipe
+        networks:
+            - main
+
+    mongo:
+        image: mongo:3.4
+        command: --nojournal
+        container_name: mongo
+        networks:
+            - main
+
+    orion:
+        image: fiware/orion
+        ports:
+            - "1026:1026"
+        command: -dbhost mongo -logLevel debug
+        container_name: orion
+        networks:
+            - main
+
+networks:
+  main:
+    driver: bridge
+```
+
+### Tests with the Docker setup
+
+After the setup is started, the CUL868IPE will simulate sensor data. In order to get the current temperature value measured by the sensor, it is possible to send a request either to the OpenMTC Backend, OpenMTC Gateway or the OrionContextBroker.
+
+*OpenMTC Backend*
+```
+curl -X "GET" localhost:18000/~/TESTUSER~TESTTENANT/onem2m/CUL868IPE/S300TH_1/temperature/latest -s | \
+  jq -r '."m2m:cin".con' | \
+  base64 -d | jq '.'
+```
+```json
+[
+  {
+    "bn": "urn:dev:s300th:1",
+    "v": 3.5796869716517135,
+    "u": "Cel",
+    "t": "1527087517.700",
+    "n": "temperature"
+  }
+]
+```
+
+*OpenMTC Gateway*
+```
+curl -X "GET" localhost:8000/onem2m/CUL868IPE/S300TH_1/temperature/latest -s | \
+  jq -r '."m2m:cin".con' | \
+  base64 -d | jq '.'
+```
+```json
+[
+  {
+    "bn": "urn:dev:s300th:1",
+    "v": 3.5796869716517135,
+    "u": "Cel",
+    "t": "1527087517.700",
+    "n": "temperature"
+  }
+]
+```
+
+*OrionContextBroker*
+
+```
+curl localhost:1026/v2/entities/TESTTENANT-kitchen/ -s -S --header 'fiware-service: TESTUSER' | jq '."temperature"'
+```
+```json
+{
+  "type": "Float",
+  "value": 20.569353172,
+  "metadata": {
+    "bn": {
+      "type": "String",
+      "value": "urn:dev:s300th:1"
+    },
+    "timestamp": {
+      "type": "String",
+      "value": "1527093220.104"
+    },
+    "unit": {
+      "type": "String",
+      "value": "Cel"
+    }
+  }
+}
+```
+
+The following command will switch the current state of the actuator to "ON" by sending a request to the OCB:
+
+```bash
+curl localhost:1026/v2/entities/TESTTENANT-bath/attrs -s -S \
+     --header 'Content-Type: application/json' --header 'fiware-service: TESTUSER' \
+     -X PATCH -d @- <<EOF
+{
+  "cmd": {
+    "value": "ON",
+    "type": "String"
+  }
+}
+EOF
+```
+The state will be automatically forwarded to OpenMTC. Therefore you should see something like the following in the log:
+```
+culipe | 172.24.0.6 - - [2018-05-26 17:24:29] "POST / HTTP/1.1" 200 160 0.037561
+```
+To get the current state at the OpenMTC Backend use this:
+
+```
+curl -X "GET" localhost:18000/~/TESTUSER~TESTTENANT/onem2m/CUL868IPE/FS20_ST3_16108_1/switch/latest -s | jq '."m2m:cin".con'
+```
+```
+"ON"
+```
 # How the data is stored at the Orion CB
 
 The Orion CB uses the model of *entities* having *attributes*. The AE matches all Container having the label "openmtc:device" to entities. Attributes are matched to the SenML Key "n" of Content Instances. The types of values are determined by the AE to match typical Orion CB types (e.g. Int, String, Float...).
@@ -56,18 +241,18 @@ curl -X POST localhost:18000/onem2m/EXAMPLE_APP_NAME/EXAMPLE_DEVICE_NAME/ -H "Co
 Upload SenML Data to OpenMTC:
 
 ```json
-{
+[{
   "n": "temperature",
   "bn": "openmtc:zigbee:temp",
   "v": 24,
   "u": "Cel",
   "t": "2017-04-13 12:45:12.787239"
-}
+}]
 ```
-base64: eyJuIjogInRlbXBlcmF0dXJlIiwgImJuIjogIm9wZW5tdGM6emlnYmVlOnRlbXAiLCAidiI6IDI0LCAidSI6ICJDZWwiLCAidCI6ICIyMDE3LTA0LTEzIDEyOjQ1OjEyLjc4NzIzOSJ9Cg==
+translated to base64: W3sibiI6ICJ0ZW1wZXJhdHVyZSIsImJuIjogIm9wZW5tdGM6emlnYmVlOnRlbXAiLCJ2IjogMjQsInUiOiAiQ2VsIiwidCI6ICIyMDE3LTA0LTEzIDEyOjQ1OjEyLjc4NzIzOSJ9XQo=
 
 ```
-curl -X POST localhost:18000/onem2m/EXAMPLE_APP_NAME/EXAMPLE_DEVICE_NAME/EXAMPLE_MEASUREMENT_NAME/ -H "Content-Type: application/vnd.onem2m-res+json" -d '{"m2m:cin": {"con": "eyJuIjogInRlbXBlcmF0dXJlIiwgImJuIjogIm9wZW5tdGM6emlnYmVlOnRlbXAiLCAidiI6IDI0LCAidSI6ICJDZWwiLCAidCI6ICIyMDE3LTA0LTEzIDEyOjQ1OjEyLjc4NzIzOSJ9Cg==", "cnf": "application/json:1"}}'
+curl -X POST localhost:18000/onem2m/EXAMPLE_APP_NAME/EXAMPLE_DEVICE_NAME/EXAMPLE_MEASUREMENT_NAME/ -H "Content-Type: application/vnd.onem2m-res+json" -d '{"m2m:cin": {"con": "W3sibiI6ICJ0ZW1wZXJhdHVyZSIsImJuIjogIm9wZW5tdGM6emlnYmVlOnRlbXAiLCJ2IjogMjQsInUiOiAiQ2VsIiwidCI6ICIyMDE3LTA0LTEzIDEyOjQ1OjEyLjc4NzIzOSJ9XQo=", "cnf": "application/json:1"}}'
 ```
 
 ### Query Data Orion CB
@@ -79,7 +264,7 @@ curl localhost:1026/v2/entities/ | jq '.'
 ```json
 [
   {
-    "id": "EXAMPLE_DEVICE_NAME",
+    "id": "in-cse-1-EXAMPLE_DEVICE_NAME",
     "type": "openmtc",
     "temperature": {
       "type": "Int",

apps/OrionContextBroker/config.json

@@ -18,5 +18,6 @@
     "labels": ["openmtc:sensor_data:temperature", "openmtc:sensor_data:humidity"],
     "interval": 10,
     "orion_host": "http://localhost:1026",
-    "orion_api": "v2"
+    "orion_api": "v2",
+    "accumulate_address": "http://localhost:8080"
 }

apps/OrionContextBroker/docker/configure-orioncontextbroker-and-start

@@ -10,9 +10,11 @@ ORIGINATOR_PRE=${ORIGINATOR_PRE-"//openmtc.org/in-cse-1"}
 SSL_CRT=${SSL_CRT-"/etc/openmtc/certs/orioncontextbroker.cert.pem"}
 SSL_KEY=${SSL_KEY-"/etc/openmtc/certs/orioncontextbroker.key.pem"}
 SSL_CA=${SSL_CA-"/etc/openmtc/certs/ca-chain.cert.pem"}
+LABELS=${LABELS-'["openmtc:sensor_data"]'}
 ORION_HOST=${ORION_HOST-"http://localhost:1026"}
 ORION_API=${ORION_API-"v2"}
-ACCUMULATE_ADDRESS=${ACCUMULATE_ADDRESS-"http://localhost:8080"}
+ACCUMULATE_ADDRESS=${ACCUMULATE_ADDRESS}
+LABELS=${LABELS-'["openmtc:sensor_data"]'}
 
 # defaults logging
 LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/orioncontextbroker.log"}
@@ -45,9 +47,11 @@ JQ_STRING=${JQ_STRING}' |
     .cse_base = "'${CSE_BASE}'" |
     .poas = '${POAS}' |
     .originator_pre = "'${ORIGINATOR_PRE}'" |
+    .labels = '${LABELS}' |
     .orion_host = "'${ORION_HOST}'" |
     .orion_api = "'${ORION_API}'" |
     .accumulate_address = "'${ACCUMULATE_ADDRESS}'" |
+    .labels = '${LABELS}' |
     .ssl_certs.cert_file = "'${SSL_CRT}'" |
     .ssl_certs.key_file = "'${SSL_KEY}'" |
     .ssl_certs.ca_certs = "'${SSL_CA}'" |

apps/OrionContextBroker/etc/conf/config.json.dist

@@ -1,11 +1,11 @@
 {
     "name": "OrionContextBroker",
-    "ep": "http://localhost:8000",
+    "ep": "http://localhost:18000",
     "cse_base": "onem2m",
     "poas": [
         "http://auto:25396"
     ],
-    "originator_pre": "//openmtc.org/mn-cse-1",
+    "originator_pre": "//openmtc.org/in-cse-1",
     "ssl_certs": {
         "cert_file": "/etc/openmtc/certs/orioncontextbroker.cert.pem",
         "key_file": "/etc/openmtc/certs/orioncontextbroker.key.pem",
@@ -15,8 +15,9 @@
         "level": "INFO",
         "file": "/var/log/openmtc/orioncontextbroker.log"
     },
-    "labels": [],
+    "labels": ["openmtc:sensor_data"],
     "interval": 10,
     "orion_host": "http://localhost:1026",
-    "orion_api": "v2"
+    "orion_api": "v2",
+    "accumulate_address": "http://localhost:8080"
 }

apps/OrionContextBroker/src/orioncontextbroker/__init__.py

@@ -3,7 +3,7 @@ This App will forward all incoming sensor traffic to the Fiware Orion Context
 Broker
 """
 
-__version__ = "0.1"
+__version__ = "1.2.0"
 __description__ = "OrionContextBroker"
 __author_name__ = "Christian Klopp"
 __author_mail__ = "christian.klopp@fokus.fraunhofer.de"

apps/OrionContextBroker/src/orioncontextbroker/__main__.py

@@ -51,7 +51,7 @@ orion_host = get_value("orion_host", (unicode, str), default_orion_host, args,
 orion_api = get_value("orion_api", (unicode, str), default_orion_api, args,
                       config)
 accumulate_address = get_value("accumulate_address", (unicode, str),
-                              default_accumulate_address, args, config)
+                               default_accumulate_address, args, config)
 
 # start
 app = OrionContextBroker(

apps/OrionContextBroker/src/orioncontextbroker/orion_api.py

@@ -2,12 +2,16 @@ try:
     from urllib.parse import urljoin
 except ImportError:
     from urlparse import urljoin
+import logging
+from datetime import datetime
 
 import requests
-import json
 
 from futile.logging import LoggerMixin
 
+logging.getLogger("requests").setLevel(logging.WARNING)
+logging.getLogger("urllib3").setLevel(logging.WARNING)
+
 
 class OrionAPI(LoggerMixin):
     def __init__(self,
@@ -40,6 +44,13 @@ class OrionAPI(LoggerMixin):
             self.logger.error('Type of "{}" unknown'.format(element))
             return u"Unknown"
 
+    def is_host_alive(self):
+        req = self._request(
+            "{}/v2/entities".format(self.host),
+            method="get"
+        )
+        return req['status'] >= 0
+
     def create_entity(self,
                       entity_name,
                       entity_type="openmtc",
@@ -82,7 +93,8 @@ class OrionAPI(LoggerMixin):
                 "type": self._get_type(data_senml["v"]),
                 "metadata": {
                     "timestamp": {
-                        "value": data_senml["t"],
+                        "value": datetime.fromtimestamp(float(data_senml["t"])).replace(microsecond=0).isoformat()
+                        if data_senml["t"] != "none" else data_senml["t"],
                         "type": "String"
                     },
                     "bn": {
@@ -152,6 +164,12 @@ class OrionAPI(LoggerMixin):
         # return the subscriptionId
         return response["headers"]["Location"].split('/')[3]
 
+    def unsubscribe(self, subscription_id, fiware_service=""):
+        self._request(self.host + "/v2/subscriptions/" + subscription_id,
+                      method='delete',
+                      headers={"fiware-service": fiware_service},
+                      raw=True)
+
     def _request(self,
                  url,
                  method='get',

apps/OrionContextBroker/src/orioncontextbroker/orion_context_broker.py

@@ -1,6 +1,11 @@
 import re
+try:
+    from urllib.parse import urlparse
+except ImportError:
+    from urlparse import urlparse
+
 from flask import Flask, Response, request
-from gevent import wsgi
+from gevent.pywsgi import WSGIServer
 
 from openmtc_app.onem2m import ResourceManagementXAE
 from orion_api import OrionAPI
@@ -11,7 +16,7 @@ class OrionContextBroker(ResourceManagementXAE):
                  orion_host="http://localhost:1026",
                  orion_api="v2",
                  labels=None,
-                 accumulate_address="http://localhost:8080",
+                 accumulate_address=None,
                  *args,
                  **kw):
         super(OrionContextBroker, self).__init__(*args, **kw)
@@ -19,15 +24,26 @@ class OrionContextBroker(ResourceManagementXAE):
             self.labels = {labels}
         elif hasattr(labels, '__iter__'):
             self.labels = set(labels)
+        elif labels is None:
+            self.labels = ["openmtc:sensor_data"]
         else:
             self.labels = None
         self._entity_names = {}
-        self._subscriptions = {}
-        self.logger.critical(accumulate_address)
+        self._subscription_endpoints = {}
+        self._subscription_services = {}
+
+        # accumulate address
+        if not accumulate_address:
+            accumulate_address = "http://" + self._get_auto_host(orion_host) + ":8080"
+
+        # Orion API
+        self._dry_run = not orion_host
         self.orion_api = OrionAPI(
             orion_host=orion_host,
             api_version=orion_api,
             accumulate_endpoint="{}/accumulate".format(accumulate_address))
+        if not self._dry_run:
+            self._dry_run = not self.orion_api.is_host_alive()
 
         # Subscription Sink for OCB
         self.app = Flask(__name__)
@@ -36,22 +52,45 @@ class OrionContextBroker(ResourceManagementXAE):
             'process_notification',
             self.process_notification,
             methods=["POST"])
-        accumulate_ip, accumulate_port = accumulate_address.split('//')[
-            1].split(':')
-        self.server = wsgi.WSGIServer((accumulate_ip, int(accumulate_port)),
-                                      self.app)
+        accumulate_ip, accumulate_port = urlparse(accumulate_address).netloc.rsplit(':', 1)
+        self.server = WSGIServer(("0.0.0.0", int(accumulate_port)),
+                                 self.app)
         self.server.start()
 
+    @staticmethod
+    def _get_auto_host(ep):
+        try:
+            import socket
+            s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+            netloc = urlparse(ep).netloc.split(':')
+            s.connect((netloc[0], int(netloc[1])))
+            host = s.getsockname()[0]
+            s.close()
+        except:
+            host = "127.0.0.1"
+
+        return host
+
     def process_notification(self):
         self.logger.debug("Got from Subscription {}".format(request.json))
-        actuator = self.get_resource(
-            self._subscriptions[request.json["subscriptionId"]])
-        self.push_content(actuator, request.json["data"][0]["cmd"]["value"])
+        try:
+            actuator = self.get_resource(
+                self._subscription_endpoints[request.json["subscriptionId"]]
+            )
+        except KeyError:
+            # ignore not deleted old subscriptions
+            pass
+        else:
+            self.push_content(actuator, request.json["data"][0]["cmd"]["value"])
         return Response(status=200, headers={})
 
     def _on_register(self):
         self._discover_openmtc_ipe_entities()
 
+    def _on_shutdown(self):
+        for subscription_id, fiware_service in self._subscription_services.items():
+            self.orion_api.unsubscribe(subscription_id, fiware_service)
+
     def _sensor_filter(self, sensor_info):
         if self.labels:
             return len(self.labels.intersection(
@@ -59,7 +98,8 @@ class OrionContextBroker(ResourceManagementXAE):
         else:
             return True
 
-    def _get_entity_name(self, sensor_info):
+    @staticmethod
+    def _get_entity_name(sensor_info):
         device_type = "sensor" if sensor_info.get("sensor_labels",
                                                   None) else "actuator"
         try:
@@ -79,6 +119,9 @@ class OrionContextBroker(ResourceManagementXAE):
         return re.sub('[\W]', '_', f_s), '%s-%s' % (e_pre, dev_id)
 
     def _sensor_data_cb(self, sensor_info, sensor_data):
+        if self._dry_run:
+            return
+
         try:
             fiware_service, entity_name = self._entity_names[sensor_info['ID']]
         except KeyError:
@@ -101,6 +144,10 @@ class OrionContextBroker(ResourceManagementXAE):
                 'ID']]
         self.logger.info("Create new Entity {} on Fiware Service {}".format(
             entity_name, fiware_service))
+
+        if self._dry_run:
+            return
+
         self.orion_api.create_entity(
             entity_name, fiware_service=fiware_service)
         data_dummy = {
@@ -113,6 +160,7 @@ class OrionContextBroker(ResourceManagementXAE):
         self.orion_api.update_attributes(
             entity_name, data_dummy, fiware_service=fiware_service)
 
-        subscriptionId = self.orion_api.subscribe(
+        subscription_id = self.orion_api.subscribe(
             entity_name, fiware_service=fiware_service)
-        self._subscriptions[subscriptionId] = actuator_info['ID']
+        self._subscription_endpoints[subscription_id] = actuator_info['ID']
+        self._subscription_services[subscription_id] = fiware_service

apps/csv-injector

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+cd $(dirname ${0})
+
+. ./prep-env.sh
+
+cd csvInjector
+
+PYTHONPATH=${PYTHONPATH}:src exec python -m csvinjector $@

apps/csvInjector/MANIFEST.in

@@ -0,0 +1 @@
+include utils.py

apps/csvInjector/README.md

@@ -0,0 +1,53 @@
+# Introduction
+
+With this app you are able to integrate data of a csv file into OpenMTC. The csv file must have a column with a timestamp and one column with some kind of device id. You are able to configure the duration of the data injection. The timing of your csv will be scaled to that duration.
+
+# Getting started
+
+To get an overview of the available parameters, just execute:
+
+```
+./apps/csv-injector --help
+```
+
+# Configuration
+
+The most important paramters are:
+
+* csv-path: Path to your csv file
+* device-classifier: The id of the column with the device ids
+* date-classifier: The id of the column with the timestamp
+* time-format: Format of your timestamp (example:  %d/%m/%Y-%H:%M)
+* duration: time to inject the data (seconds)
+* repeat: repeat after full csv is injected (bool)
+
+# Example
+
+If you have a csv like this at "~/test.csv":
+
+```csv
+Date,ID,Temp,Hum,Battery
+23/01/2018-00:50,environment_1,13.3,41,100
+18/02/2018-14:01,environment_2,20.4,46,0,99
+02/03/2018-23:11,environment_1,19.1,14,100
+03/03/2018-00:01,environment_2,19,13,100
+08/03/2018-03:01,environment_1,19.5,26,0,99
+08/03/2018-10:11,environment_2,21.9,29,0,100
+09/03/2018-16:21,environment_1,21.4,27,0,99
+09/03/2018-16:51,environment_1,21.3,27,0,100
+16/03/2018-06:36,environment_2,15.1,36,0,100
+20/03/2018-09:27,environment_1,18.9,19,0,100
+20/03/2018-10:27,environment_1,20.3,20,0,99
+```
+
+The app could be started like this:
+
+```
+./apps/csv-injector \
+    --csv-path "~/test.csv" \
+    --device-classifier "ID" \
+    --date-classifier "Date" \
+    --time-format "%d/%m/%Y-%H:%M" \
+    --duration 300
+    --repeat 0
+```

apps/csvInjector/bin/openmtc-csv-injector

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+
+exec python -m csvinjector $@

apps/csvInjector/config.json

@@ -0,0 +1,26 @@
+{
+    "name": "csvInjector",
+    "ep": "http://localhost:8000",
+    "cse_base": "onem2m",
+    "poas": [
+        "http://auto:28300"
+    ],
+    "originator_pre": "//openmtc.org/mn-cse-1",
+    "ssl_certs": {
+        "cert_file": null,
+        "key_file": null,
+        "ca_certs": null
+    },
+    "logging": {
+        "level": "ERROR",
+        "file": null
+    },
+    "csv_path": "~/test.csv",
+    "csv_delim": ",",
+    "csv_quotechar": "|",
+    "device_classifier": "sensor_id",
+    "date_classifier": "date",
+    "time_format":"%d/%m/%Y-%H:%M",
+    "duration": 300,
+    "repeat": false
+}

apps/csvInjector/docker/configure-csvinjector-and-start

@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+
+CONFIG_FILE="/etc/openmtc/csvinjector/config.json"
+
+NAME=${NAME-"csvInjector"}
+EP=${EP-"http://localhost:8000"}
+CSE_BASE=${CSE_BASE-"onem2m"}
+POAS=${POAS-'["http://auto:28300"]'}
+ORIGINATOR_PRE=${ORIGINATOR_PRE-"//openmtc.org/mn-cse-1"}
+SSL_CRT=${SSL_CRT-"/etc/openmtc/certs/csvinjector.cert.pem"}
+SSL_KEY=${SSL_KEY-"/etc/openmtc/certs/csvinjector.key.pem"}
+SSL_CA=${SSL_CA-"/etc/openmtc/certs/ca-chain.cert.pem"}
+CSV_PATH=${CSV_PATH-"/test.csv"}
+CSV_DELIM=${CSV_DELIM-","}
+CSV_QUOTECHAR=${CSV_QUOTECHAR-"|"}
+CSV_DEVICE_CLASSIFIER=${CSV_DEVICE_CLASSIFIER-""}
+CSV_DATE_CLASSIFIER=${CSV_DATE_CLASSIFIER-""}
+CSV_TIME_FORMAT=${CSV_TIME_FORMAT-"%d/%m/%Y-%H:%M"}
+DURATION=${DURATION-300}
+REPEAT=${REPEAT-"False"}
+
+# defaults logging
+LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/csvinjector.log"}
+LOGGING_LEVEL=${LOGGING_LEVEL-"ERROR"}
+
+# ensure correct level
+case ${LOGGING_LEVEL} in
+    FATAL|ERROR|WARN|INFO|DEBUG)
+    ;;
+    *)
+    LOGGING_LEVEL="ERROR"
+    ;;
+esac
+
+# local ip
+LOCAL_IP=$(ip r get 8.8.8.8 | awk 'NR==1 {print $NF}')
+
+# set hostname
+HOST_NAME=${EXTERNAL_IP-${LOCAL_IP}}
+
+# Configuration of the service.
+CONFIG_TEMP=${CONFIG_FILE}".tmp"
+echo -n "Configuring M2M csvinjector..."
+JQ_STRING='.'
+
+# basics
+JQ_STRING=${JQ_STRING}' |
+    .name = "'${NAME}'" |
+    .ep = "'${EP}'" |
+    .cse_base = "'${CSE_BASE}'" |
+    .poas = '${POAS}' |
+    .originator_pre = "'${ORIGINATOR_PRE}'" |
+    .ssl_certs.cert_file = "'${SSL_CRT}'" |
+    .ssl_certs.key_file = "'${SSL_KEY}'" |
+    .ssl_certs.ca_certs = "'${SSL_CA}'" |
+    .logging.file = "'${LOGGING_FILE}'" |
+    .logging.level = "'${LOGGING_LEVEL}'" |
+    .csv_path = "'${CSV_PATH}'" |
+    .csv_delim = "'${CSV_DELIM}'" |
+    .csv_quotechar = "'${CSV_QUOTECHAR}'" |
+    .device_classifier = '${CSV_DEVICE_CLASSIFIER}' |
+    .date_classifier = '${CSV_DATE_CLASSIFIER}' |
+    .time_format = '${CSV_TIME_FORMAT}' |
+    .duration = '${DURATION}' |
+    .repeat = "'${REPEAT}'"
+'
+
+cat ${CONFIG_FILE} | jq -M "${JQ_STRING}"> ${CONFIG_TEMP}
+mv ${CONFIG_TEMP} ${CONFIG_FILE}
+
+echo "done"
+
+exec python -m csvinjector $@

apps/csvInjector/docker/csvinjector-amd64

@@ -0,0 +1,30 @@
+############################################################
+# Dockerfile to run openmtc csvinjector binary
+############################################################
+
+# Set the base image to use openmtc/sdk
+FROM openmtc/sdk-amd64:latest
+
+ENV MOD_NAME=csvinjector
+
+# Set the file maintainer
+MAINTAINER rst
+
+# install openmtc dependencies
+COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
+RUN pip install --upgrade --requirement /tmp/requirements.txt
+
+# install openmtc-csvinjector
+COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
+RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
+    --owner root --group root --no-same-owner --no-overwrite-dir \
+    --transform 's/json\.dist/json/' --show-transformed
+
+RUN mkdir -p /var/log/openmtc
+
+# add change config
+COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
+
+# entry point
+ENTRYPOINT ["/usr/local/bin/configure-and-start"]
+CMD [""]

apps/csvInjector/docker/csvinjector-arm

@@ -0,0 +1,30 @@
+############################################################
+# Dockerfile to run openmtc csvinjector binary
+############################################################
+
+# Set the base image to use openmtc/sdk
+FROM openmtc/sdk-arm:latest
+
+ENV MOD_NAME=csvinjector
+
+# Set the file maintainer
+MAINTAINER rst
+
+# install openmtc dependencies
+COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
+RUN pip install --upgrade --requirement /tmp/requirements.txt
+
+# install openmtc-csvinjector
+COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
+RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
+    --owner root --group root --no-same-owner --no-overwrite-dir \
+    --transform 's/json\.dist/json/' --show-transformed
+
+RUN mkdir -p /var/log/openmtc
+
+# add change config
+COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
+
+# entry point
+ENTRYPOINT ["/usr/local/bin/configure-and-start"]
+CMD [""]

apps/csvInjector/etc/conf/config.json.dist

@@ -0,0 +1,26 @@
+{
+    "name": "csvInjector",
+    "ep": "http://localhost:8000",
+    "cse_base": "onem2m",
+    "poas": [
+        "http://auto:28300"
+    ],
+    "originator_pre": "//openmtc.org/mn-cse-1",
+    "ssl_certs": {
+        "cert_file": "/etc/openmtc/certs/csvinjector.cert.pem",
+        "key_file": "/etc/openmtc/certs/csvinjector.key.pem",
+        "ca_certs": "/etc/openmtc/certs/ca-chain.cert.pem"
+    },
+    "logging": {
+        "level": "INFO",
+        "file": "/var/log/openmtc/csvinjector.log"
+    },
+    "csv_path": "~/test.csv",
+    "csv_delim": ",",
+    "csv_quotechar": "|",
+    "device_classifier": "sensor_id",
+    "date_classifier": "sensor_id",
+    "time_format":"%d/%m/%Y-%H:%M",
+    "duration": 300,
+    "repeat": false
+}

apps/csvInjector/etc/systemd/system/openmtc-csvinjector.service

@@ -0,0 +1,10 @@
+[Unit]
+Description=OpenMTC csvInjector
+After=network.target
+Wants=ntp.service
+
+[Service]
+ExecStart=/usr/local/bin/csv-injector
+
+[Install]
+WantedBy=multi-user.target

apps/csvInjector/setup-csvinjector.py

@@ -0,0 +1,82 @@
+#!/usr/bin/env python
+
+from setuptools import setup
+from distutils.core import setup
+from glob import glob
+import sys
+
+from utils import get_packages, get_pkg_files, OpenMTCSdist, move_config_files
+
+# name and dir
+NAME = "csvinjector"
+BASE_DIR = "."
+
+# import pkg
+sys.path.append(BASE_DIR + "/src")
+pkg = __import__(NAME)
+
+# setup name and version
+SETUP_NAME = "openmtc-" + NAME
+SETUP_VERSION = pkg.__version__
+SETUP_DESCRIPTION = pkg.__description__
+
+# meta
+SETUP_AUTHOR = pkg.__author_name__
+SETUP_AUTHOR_EMAIL = pkg.__author_mail__
+SETUP_URL = "http://www.openmtc.org"
+SETUP_LICENSE = "Fraunhofer FOKUS proprietary"
+
+# requirements
+SETUP_REQUIRES = pkg.__requires__
+SETUP_INSTALL_REQUIRES = pkg.__requires__
+
+# packages
+PACKAGES = [NAME]
+PACKAGE_DIR = {"": BASE_DIR + "/src"}
+all_packages = []
+for package in PACKAGES:
+    all_packages.extend(get_packages(package, PACKAGE_DIR))
+
+# scripts
+SETUP_SCRIPTS = glob(BASE_DIR + "/bin/*")
+
+# package data
+PACKAGE_DATA = {NAME: get_pkg_files(BASE_DIR, NAME)}
+
+# data files
+CONFIG_FILES = ("config.json",)
+CONFIG_DIR = "/etc/openmtc/" + NAME
+CONFIG_DIST_FILES = (BASE_DIR + "/etc/conf/config.json.dist",)
+DATA_FILES = [(CONFIG_DIR, CONFIG_DIST_FILES)]
+
+# cmd class
+CMD_CLASS = {'sdist': OpenMTCSdist}
+
+if __name__ == "__main__":
+    if 'bdist_wheel' in sys.argv:
+        raise RuntimeError("This setup.py does not support wheels")
+
+    ############################################################################
+    # setup
+    setup(name=SETUP_NAME,
+          version=SETUP_VERSION,
+          description=SETUP_DESCRIPTION,
+          author=SETUP_AUTHOR,
+          author_email=SETUP_AUTHOR_EMAIL,
+          url=SETUP_URL,
+          license=SETUP_LICENSE,
+          requires=SETUP_REQUIRES,
+          install_requires=SETUP_INSTALL_REQUIRES,
+          package_dir=PACKAGE_DIR,
+          packages=all_packages,
+          scripts=SETUP_SCRIPTS,
+          package_data=PACKAGE_DATA,
+          data_files=DATA_FILES,
+          cmdclass=CMD_CLASS
+          )
+
+    ############################################################################
+    # install
+    if "install" in sys.argv:
+        # only do this during install
+        move_config_files(CONFIG_DIR, CONFIG_FILES)

apps/csvInjector/src/csvinjector/__init__.py

@@ -0,0 +1,9 @@
+"""
+App to inject data from a csv file to OpenMTC
+"""
+
+__version__ = "1.2.0"
+__description__ = "csvInjector"
+__author_name__ = "Christian Klopp"
+__author_mail__ = "christian.klopp@fokus.fraunhofer.de"
+__requires__ = []

apps/csvInjector/src/csvinjector/__main__.py

@@ -0,0 +1,85 @@
+from argparse import ArgumentDefaultsHelpFormatter, ArgumentParser
+
+from openmtc_app.util import prepare_app, get_value
+from openmtc_app.runner import AppRunner as Runner
+from .csv_injector import csvInjector
+
+# defaults
+default_name = "csvInjector"
+default_ep = "http://localhost:8000"
+default_csv_path = "~/test.csv"
+default_csv_delim = ","
+default_csv_quotechar = "|"
+default_device_classifier = ""
+default_date_classifier = "DATE"
+default_time_format = "%d/%m/%Y-%H:%M"
+default_duration = 300
+default_repeat = False
+
+# args parser
+parser = ArgumentParser(
+    description="An IPE called csvInjector",
+    prog="csvInjector",
+    formatter_class=ArgumentDefaultsHelpFormatter)
+parser.add_argument("-n", "--name", help="Name used for the AE.")
+parser.add_argument("-s", "--ep", help="URL of the local Endpoint.")
+parser.add_argument("-f", "--csv-path", help="Path to CSV File")
+parser.add_argument("--csv-delim", help="Delimiter used for the provided csv")
+parser.add_argument(
+    "--csv-quotechar", help="Quotechar used for the provided csv")
+parser.add_argument(
+    "--device-classifier", help="Column used to specify different devices in csv")
+parser.add_argument(
+    "--date-classifier", help="Column used to specify where dates are defined in csv")
+parser.add_argument(
+    "--time-format", help="Format of the date column in csv (see https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior)")
+parser.add_argument(
+    "--duration", help="Time to inject the csv (if csv time data does not fit, it will be scaled)")
+parser.add_argument(
+    "--repeat", help="Repeat after csv is injected")
+
+# args, config and logging
+args, config = prepare_app(parser, __loader__, __name__, "config.json")
+
+# variables
+nm = get_value("name", (unicode, str), default_name, args, config)
+cb = config.get("cse_base", "onem2m")
+ep = get_value("ep", (unicode, str), default_ep, args, config)
+poas = config.get("poas", ["http://auto:28300"])
+originator_pre = config.get("originator_pre", "//openmtc.org/mn-cse-1")
+ssl_certs = config.get("ssl_certs", {})
+csv_path = get_value("csv_path", (unicode, str), default_csv_path, args,
+                     config)
+csv_delim = get_value("csv_delim", (unicode, str), default_csv_delim, args,
+                      config)
+csv_quotechar = get_value("csv_quotechar", (unicode, str),
+                          default_csv_quotechar, args, config)
+device_classifier = get_value("device_classifier", (unicode, str),
+                          default_device_classifier, args, config)
+date_classifier = get_value("date_classifier", (unicode, str, list),
+                          default_date_classifier, args, config)
+time_format = get_value("time_format", (unicode, str, list),
+                          default_time_format, args, config)
+duration = get_value("duration", (int, float),
+                          default_duration, args, config)
+repeat = get_value("repeat", (unicode, str),
+                          default_repeat, args, config)
+
+# start
+app = csvInjector(
+    name=nm,
+    cse_base=cb,
+    poas=poas,
+    originator_pre=originator_pre,
+    csv_path=csv_path,
+    csv_delim=csv_delim,
+    csv_quotechar=csv_quotechar,
+    device_classifier=device_classifier,
+    date_classifier=date_classifier,
+    time_format=time_format,
+    csv_inject_duration=duration,
+    repeat=repeat,
+    **ssl_certs)
+Runner(app).run(ep)
+
+print("Exiting....")

apps/csvInjector/src/csvinjector/csv_injector.py

@@ -0,0 +1,110 @@
+from openmtc_app.onem2m import XAE
+from openmtc_onem2m.model import Container
+from csv_process import csvProcessor
+import sched
+import time
+import datetime
+
+
+class csvInjector(XAE):
+    def __init__(self,
+                 csv_path,
+                 csv_delim,
+                 csv_quotechar,
+                 device_classifier,
+                 date_classifier,
+                 time_format,
+                 csv_inject_duration=0,
+                 repeat=False,
+                 *args,
+                 **kw):
+
+        super(csvInjector, self).__init__(*args, **kw)
+        self._recognized_sensors = {}
+        self._recognized_measurement_containers = {}
+        # csv key to differ between devices
+        self.device_classifier = device_classifier
+        self.date_classifier = date_classifier
+        self.csv_path = csv_path
+        self.csv_delim = csv_delim
+        self.csv_quotechar = csv_quotechar
+        self.time_format = time_format
+        self.csv_inject_duration = csv_inject_duration
+        self.repeat = repeat
+
+    def _on_register(self):
+        # start endless loop
+        self._init_scheduler()
+        self.scheduler.run()
+        if self.repeat:
+            while True:
+                self._init_scheduler()
+                self.scheduler.run()
+
+    def _init_scheduler(self):
+        # read csv
+        self.csv_data_list = csvProcessor(self.csv_path, self.csv_delim,
+                                          self.csv_quotechar, self.time_format,
+                                          self.csv_inject_duration,
+                                          self.date_classifier).csv_data
+        # setup scheduler
+        self.scheduler = sched.scheduler(time.time, time.sleep)
+        for event in self.csv_data_list:
+            if isinstance(self.date_classifier, list):
+                self.scheduler.enter(event["timestamp_schedule"], 1,
+                                     self.push_data, (event, ))
+            else:
+                self.scheduler.enter(event[self.date_classifier], 1,
+                                     self.push_data, (event, ))
+
+    def _create_measurement_container(self, device_name, name):
+        measurement_container = self.create_container(
+            self._recognized_sensors[device_name].path,
+            Container(resourceName=name),
+            max_nr_of_instances=0,
+            labels=[
+                'openmtc:sensor_data:{}'.format(name), 'openmtc:sensor_data'
+            ])
+        self._recognized_measurement_containers[device_name][
+            name] = measurement_container
+
+    def _create_sensor_structure(self, event):
+        device_container = self.create_container(
+            None,
+            Container(resourceName=event[self.device_classifier]),
+            labels=['openmtc:device'],
+            max_nr_of_instances=0)
+
+        self._recognized_sensors[event[
+            self.device_classifier]] = device_container
+        self._recognized_measurement_containers[event[
+            self.device_classifier]] = {}
+
+        for k in event.keys():
+            if k == "Date" or k == self.device_classifier or k in ("", None):
+                continue
+            self._create_measurement_container(event[self.device_classifier],
+                                               k)
+
+    def push_data(self, event):
+        sensor = event[self.device_classifier]
+        if not sensor in self._recognized_sensors:
+            self._create_sensor_structure(event)
+        device_container = self._recognized_sensors[sensor]
+        for k in event.keys():
+            if k == "Date" or k == self.device_classifier or event[k] in (
+                    "", None):
+                continue
+            if not k in self._recognized_measurement_containers[sensor].keys():
+                self._create_measurement_container(sensor, k)
+            timestamp = time.mktime(datetime.datetime.now().timetuple())
+            senml = {
+                "bn": "csv_extracted",
+                "n": k,
+                "u": "None",
+                "t": timestamp,
+                "v": event[k]
+            }
+            self.logger.debug("sensor {} sends data: {}".format(sensor, senml))
+            self.push_content(
+                self._recognized_measurement_containers[sensor][k], [senml])

apps/csvInjector/src/csvinjector/csv_process.py

@@ -0,0 +1,68 @@
+import csv
+from datetime import datetime
+from futile.logging import LoggerMixin
+
+
+class csvProcessor(LoggerMixin):
+    def __init__(self,
+                 path,
+                 delim=",",
+                 quotechar="|",
+                 time_format="%d/%m/%Y-%H:%M",
+                 duration=0,
+                 date_classifier="date"):
+        with open(path, 'rb') as csvfile:
+            self.csv_data = list(csv.DictReader(csvfile))
+        self.time_format = time_format
+        self.duration = duration
+        self.date_classifier = date_classifier
+
+        # Date Processing
+        if isinstance(self.date_classifier, list):
+            self._join_multiple_timestamps()
+        self._date_string_to_date_object()
+        self.csv_data = sorted(
+            self.csv_data, key=lambda k: k[self.date_classifier])
+        self._date_to_seconds_since_first()
+        self._scale_to_duration()
+
+    def _join_multiple_timestamps(self):
+        for entry in self.csv_data:
+            entry["timestamp_schedule"] = "-".join([entry[k] for k in self.date_classifier])
+            for date_c in self.date_classifier:
+                entry.pop(date_c, None)
+        self.date_classifier = "timestamp_schedule"
+        self.time_format = "-".join(self.time_format)
+
+    def _date_string_to_date_object(self):
+        for entry in self.csv_data:
+            entry[self.date_classifier] = datetime.strptime(
+                entry[self.date_classifier], self.time_format)
+
+    def _date_to_seconds_since_first(self):
+        for entry in self.csv_data:
+            if 'first' in locals():
+                entry[self.date_classifier] = (
+                    entry[self.date_classifier] - first).total_seconds()
+            else:
+                first = entry[self.date_classifier]
+                entry[self.date_classifier] = 0.0
+
+    def _scale_to_duration(self):
+        if self.duration <= 0:
+            return
+        scaling_factor = self.duration / self.csv_data[-1][
+            self.date_classifier]
+        self.logger.debug("Set scaling factor to {}".format(scaling_factor))
+        for entry in self.csv_data:
+            entry[self.date_classifier] = entry[
+                self.date_classifier] * scaling_factor
+
+    def getList(self):
+        return self.csv_data
+
+
+if __name__ == "__main__":
+    p = csvProcessor("example.csv", duration=300)
+    for e in p.csv_data:
+        print e

apps/csvInjector/utils.py

@@ -0,0 +1,148 @@
+import distutils.command.sdist
+import distutils.command.build_py
+import os
+import subprocess
+import sys
+
+
+def echo(msg, *args):
+    if args:
+        msg = msg % args
+    sys.stdout.write(msg + "\n")
+
+
+def get_packages(package, package_dir, excluded_list=None, included_list=None):
+    included_list = included_list or []
+    excluded_list = excluded_list or []
+
+    try:
+        root = package_dir[package]
+    except KeyError:
+        root = package_dir.get("", ".") + "/" + package
+
+    if not os.path.exists(root):
+        sys.stderr.write(
+            "Directory for package %s does not exist: %s\n" % (package, root))
+        sys.exit(1)
+
+    def on_error(error):
+        sys.stderr.write(
+            "Error while collecting packages for %s: %s\n" % (package, error))
+        sys.exit(1)
+
+    packages = [package]
+
+    r_prefix = len(root) + 1
+    for path, dirs, files in os.walk(root, onerror=on_error):
+        is_module = "__init__.py" in files and path != root
+        excluded = any(map(lambda x: x in path, excluded_list))
+        included = any(map(lambda x: x in path, included_list))
+        if is_module and (not excluded or included):
+            packages.append(package + "." + path[r_prefix:].replace("/", "."))
+
+    return packages
+
+
+def get_pkg_files(base_dir, name):
+    package_files = []
+    pkg_dir = os.path.join(base_dir, 'src', name)
+    pkg_data_dir = os.path.join(pkg_dir, 'static')
+    for (path, directories, filenames) in os.walk(pkg_data_dir):
+        for filename in filenames:
+            package_files.append(os.path.join(os.path.relpath(path, pkg_dir),
+                                              filename))
+    return package_files
+
+
+def enable_init_files(init_dir, init_dist_files):
+    for f in init_dist_files:
+        os.chmod(os.path.join(init_dir, os.path.basename(f)), 0755)
+
+
+def move_config_files(config_dir, config_files):
+    for f in config_files:
+        target_file = os.path.join(config_dir, f)
+        if not os.path.exists(target_file):
+            echo("Installing config file %s", target_file)
+            os.rename(target_file + ".dist", target_file)
+            # os.chmod(target_file, 0644)
+        else:
+            echo("Not overwriting config file %s", target_file)
+
+
+def create_openmtc_user(db_dir=None, log_dir=None):
+    try:
+        from pwd import getpwnam
+    except ImportError:
+        print "Could not import the 'pwd' module. Skipping user management"
+    else:
+        # assuming DB_DIR was created by setup already
+        try:
+            pw = getpwnam('openmtc')
+        except KeyError as e:
+            try:
+                # add system user openmtc:openmtc
+                # useradd --system -UM openmtc
+                useradd = "useradd --system -UM openmtc"
+                retcode = subprocess.call(useradd, shell=True)
+                if retcode:
+                    raise Exception("Failed to add user 'openmtc'")
+                pw = getpwnam('openmtc')
+            except Exception as e:
+                sys.stderr.write("Error creating user: %s\n" % (e, ))
+                sys.exit(1)
+        uid = pw.pw_uid
+        gid = pw.pw_gid
+
+        # set path permissions
+        if db_dir:
+            os.chown(db_dir, uid, gid)
+        if log_dir:
+            os.chown(log_dir, uid, gid)
+
+
+class OpenMTCSdist(distutils.command.sdist.sdist):
+    def make_release_tree(self, base_dir, files):
+        distutils.command.sdist.sdist.make_release_tree(self, base_dir, files)
+
+        script_name = os.path.basename(sys.argv[0])
+
+        if script_name != "setup.py":
+            os.rename(base_dir + "/" + script_name, base_dir + "/setup.py")
+            self.filelist.files.remove(script_name)
+            self.filelist.files.append("setup.py")
+
+
+class OpenMTCSdistBinary(OpenMTCSdist, object):
+    def make_release_tree(self, base_dir, files):
+        super(OpenMTCSdistBinary, self).make_release_tree(base_dir, files)
+
+        script_name = os.path.basename(sys.argv[0])
+
+        build_py = self.get_finalized_command('build_py')
+        build_py.compile = 1
+        build_py.optimize = 2
+        build_py.retain_init_py = 1
+        build_py.build_lib = base_dir
+        build_py.byte_compile(
+            [base_dir + "/" + f for f in self.filelist.files if
+             f != script_name and f.endswith(".py")])
+
+
+class OpenMTCBuildPy(distutils.command.build_py.build_py):
+    retain_init_py = 0
+
+    def byte_compile(self, files):
+        distutils.command.build_py.build_py.byte_compile(self, files)
+
+
+class OpenMTCBuildPyBinary(OpenMTCBuildPy, object):
+    retain_init_py = 0
+
+    def byte_compile(self, files):
+        super(OpenMTCBuildPyBinary, self).byte_compile(files)
+
+        for f in files:
+            if (f.endswith('.py') and (os.path.basename(f) != "__init__.py" or
+                                       not self.retain_init_py)):
+                os.unlink(f)

apps/influx-db

@@ -1,9 +0,0 @@
-#!/usr/bin/env bash
-
-cd $(dirname ${0})
-
-. ./prep-env.sh
-
-cd InfluxDB
-
-PYTHONPATH=${PYTHONPATH}:src exec python -m influxdb $@

apps/influxdb-app

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+cd $(dirname ${0})
+
+. ./prep-env.sh
+
+cd InfluxdbApp
+
+PYTHONPATH=${PYTHONPATH}:src exec python -m influxdbapp $@

apps/mqtt-connector

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+cd $(dirname ${0})
+
+. ./prep-env.sh
+
+cd mqttConnector
+
+PYTHONPATH=${PYTHONPATH}:src exec python -m mqttconnector $@

apps/mqttConnector/MANIFEST.in

@@ -0,0 +1 @@
+include utils.py

apps/mqttConnector/README.md

@@ -0,0 +1,184 @@
+# Introduction
+![](mqtt_connector_diagram.png)
+
+MQTTConnector is able to connect a MQTT-Broker with OpenMTC. The subscribes to specific topics on the broker in order to forward all published data to OpenMTC.
+
+The MQTTConnector will subscribe itself on the MQTT-Broker (e.g. MeshBlu) to a specific first topics level. Published data is expected to contain at least two more topic levels, with one representing the location and one the device. This information will later be used to create an entity within the OpenMTC resource tree. If data is published, the MQTTConnector app will be notified via the subscription and will forward the data to an instance of OpenMTC (gateway or backend).
+
+# Getting started
+
+In order to get an overview of the possible paramters of the app:
+
+```
+./apps/mqtt-connector -v
+```
+
+The most relevant paramters are the following:
+
+**EP** (Endpoint): URL of the OpenMTC endpoint to use (backend or gateway)
+Example: "http://localhost:18000**
+
+**BROKER_EP** (MQTT Endpoint): URL of the MQTT-Broker
+Example: "http://localhost:1883**
+
+**TOPIC_PRE**: definition of the first topic level in order to filter for relevant topics
+Example: "OfficeBuilding**
+
+**TOPIC_INDEX_LOCATION**: topic level representing the location
+Example: 1
+
+**TOPIC_INDEX_DEVICE**: topic level representing the device
+Example: 2
+
+**BROKER_USER**: username to connect to the MQTT Broker
+
+**BROKER_USER_PW**: password to connect to the MQTT Broker
+
+If used to together with the OpenMTC OrionContextBroker app, it is also possible to configure a specific fiware service.
+
+# Required Data Format
+
+In order to work data needs to be published to the MQTT-Broker in an onem2m specific format. This will look like the following:
+
+```json
+{
+    "m2m:cin": {
+        "con": "B64_ENCODED_SENML_PAYLOAD"
+    }
+}
+```
+
+where the B64_ENCODED_SENML_PAYLOAD could look like this:
+
+decoded:
+```json
+[
+    {
+        "bn": "23",
+        "v": 27,
+        "u": "Celsius",
+        "t": 1527757260000,
+        "n": "Temperatur"
+    }
+]
+```
+
+# Example
+
+![](mqtt_connector_ocb_example.png)
+
+In this example setup, we are using the MQTTConnector to forward data of a temperature sensor connected to the Mosquitto MQTT-Broker via OpenMTC to an instance of the Fiware Orion Context Broker. The following docker-compose configuration shows the setup in more detail.
+
+```
+version: "2"
+
+services:
+    backend:
+        image: openmtc/backend-amd64
+        ports:
+            - "18000:18000"
+        environment:
+            - ONEM2M_NOTIFICATION_DISABLED=false
+            - ONEM2M_HTTP_TRANSPORT_DISABLED=false
+            - ONEM2M_HTTP_TRANSPORT_SSL_ENABLED=false
+            - ONEM2M_HTTP_TRANSPORT_REQUIRE_CERT=false
+
+    mosquitto:
+        image: eclipse-mosquitto
+        ports:
+            - "1883:1883"
+
+    mqtt-ipe:
+        image: openmtc/mqtt-connector-app-amd64
+        environment:
+            - EP=http://backend:18000
+            - BROKER_EP=mosquitto:1883
+            - TOPIC_PRE=OfficeBuilding
+            - TOPIC_INDEX_LOCATION=1
+            - TOPIC_INDEX_DEVICE=2
+        links:
+            - mosquitto
+            - backend
+
+    orioncontextbroker-app:
+        image: openmtc/orion-context-broker-app-amd64
+        container_name: orioncontextbroker-app
+        links:
+            - backend
+            - orion
+        ports:
+            - "8086:8086"
+            - "8080:8080"
+        environment:
+            - EP=http://backend:18000
+            - ORION_HOST=http://orion:1026
+            - ORION_API=v2
+            - ACCUMULATE_ADDRESS=http://orioncontextbroker-app:8080
+            - LABELS=["openmtc:sensor_data"]
+
+    mongo:
+        image: mongo:3.4
+        command: --nojournal
+
+    orion:
+        image: fiware/orion
+        ports:
+            - "1026:1026"
+        command: -dbhost mongo -logLevel debug
+        links:
+            - mongo
+```
+
+The following will be published to topic OfficeBuilding/MeetingRoom/temperature:
+
+```json
+{
+    "m2m:cin": {
+        "con": "W3tibjogMjMsIHY6IDI3LCB1OiBDZWxzaXVzLCB0OiAxNTI3NzU3MjYwMDAwLCBuOiBUZW1wZXJhdHVyfV0K"
+    }
+}
+```
+
+The string is in "con" is the base64 representation of:
+
+```json
+[
+    {
+        "bn": "23",
+        "v": 27,
+        "u": "Celsius",
+        "t": 1527757260000,
+        "n": "temperature"
+    }
+]
+```
+
+If the data is successfully published to the MQTT-Broker it will be forwarded to the OCB. Therefore we are able to request the data from the OCB.
+
+```
+curl localhost:1026/v2/entities/MeetingRoom-temperature/attrs -H -s
+```
+
+```json
+{
+  "temperature": {
+    "type": "Float",
+    "value": 27,
+    "metadata": {
+      "bn": {
+        "type": "String",
+        "value": 23
+      },
+      "timestamp": {
+        "type": "String",
+        "value": 152775726000
+      },
+      "unit": {
+        "type": "String",
+        "value": "Celsius"
+      }
+    }
+  }
+}
+
+```

apps/mqttConnector/bin/openmtc-mqtt-connector

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+
+exec python -m mqttconnector $@

apps/mqttConnector/config.json

@@ -0,0 +1,29 @@
+{
+    "name": "mqttConnector",
+    "ep": "http://localhost:18000",
+    "cse_base": "onem2m",
+    "poas": [
+        "http://auto:21753"
+    ],
+    "originator_pre": "//openmtc.org/in-cse-1",
+    "ssl_certs": {
+        "cert_file": null,
+        "key_file": null,
+        "ca_certs": null
+    },
+    "broker_ep": "localhost:1883",
+    "topic_pre": "exampleTopic",
+    "topic_index_location": 1,
+    "topic_index_device": -1,
+    "fiware_service": null,
+    "broker_user": "",
+    "broker_user_pw": "",
+    "mqtts_enabled": false,
+    "mqtts_ca_certs": null,
+    "mqtts_certfile": null,
+    "mqtts_keyfile": null,
+    "logging": {
+        "level": "ERROR",
+        "file": null
+    }
+}

apps/mqttConnector/docker/configure-mqttconnector-and-start

@@ -0,0 +1,79 @@
+#!/usr/bin/env bash
+
+CONFIG_FILE="/etc/openmtc/mqttconnector/config.json"
+
+NAME=${NAME-"mqttConnector"}
+EP=${EP-"http://localhost:8000"}
+CSE_BASE=${CSE_BASE-"onem2m"}
+POAS=${POAS-'["http://auto:21753"]'}
+ORIGINATOR_PRE=${ORIGINATOR_PRE-"//openmtc.org/mn-cse-1"}
+SSL_CRT=${SSL_CRT-"/etc/openmtc/certs/mqttconnector.cert.pem"}
+SSL_KEY=${SSL_KEY-"/etc/openmtc/certs/mqttconnector.key.pem"}
+SSL_CA=${SSL_CA-"/etc/openmtc/certs/ca-chain.cert.pem"}
+BROKER_EP=${BROKER_EP-"localhost:1883"}
+TOPIC_PRE=${TOPIC_PRE-"exampleTopic"}
+TOPIC_INDEX_LOCATION=${TOPIC_INDEX_LOCATION-1}
+TOPIC_INDEX_DEVICE=${TOPIC_INDEX_DEVICE:-1}
+FIWARE_SERVICE=${FIWARE_SERVICE}
+BROKER_USER=${BROKER_USER-""}
+BROKER_USER_PW=${BROKER_USER_PW-""}
+MQTTS_ENABLED=${MQTTS_ENABLED-false}
+MQTTS_CA_CERTS=${MQTTS_CA_CERTS}
+MQTTS_CERTFILE=${MQTTS_CERTFILE}
+MQTTS_KEYFILE=${MQTTS_KEYFILE}
+
+# defaults logging
+LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/mqttconnector.log"}
+LOGGING_LEVEL=${LOGGING_LEVEL-"ERROR"}
+
+# ensure correct level
+case ${LOGGING_LEVEL} in
+    FATAL|ERROR|WARN|INFO|DEBUG)
+    ;;
+    *)
+    LOGGING_LEVEL="ERROR"
+    ;;
+esac
+
+# local ip
+LOCAL_IP=$(ip r get 8.8.8.8 | awk 'NR==1 {print $NF}')
+
+# set hostname
+HOST_NAME=${EXTERNAL_IP-${LOCAL_IP}}
+
+# Configuration of the service.
+CONFIG_TEMP=${CONFIG_FILE}".tmp"
+echo -n "Configuring M2M mqttconnector..."
+JQ_STRING='.'
+
+# basics
+JQ_STRING=${JQ_STRING}' |
+    .name = "'${NAME}'" |
+    .ep = "'${EP}'" |
+    .cse_base = "'${CSE_BASE}'" |
+    .poas = '${POAS}' |
+    .originator_pre = "'${ORIGINATOR_PRE}'" |
+    .ssl_certs.cert_file = "'${SSL_CRT}'" |
+    .ssl_certs.key_file = "'${SSL_KEY}'" |
+    .ssl_certs.ca_certs = "'${SSL_CA}'" |
+    .broker_ep = "'${BROKER_EP}'" |
+    .topic_pre = "'${TOPIC_PRE}'" |
+    .topic_index_location = '${TOPIC_INDEX_LOCATION}' |
+    .topic_index_device = '${TOPIC_INDEX_DEVICE}' |
+    .fiware_service = "'${FIWARE_SERVICE}'" |
+    .broker_user = "'${BROKER_USER}'" |
+    .broker_user_pw = "'${BROKER_USER_PW}'" |
+    .mqtts_enabled = '${MQTTS_ENABLED}' |
+    .mqtts_ca_certs = "'${MQTTS_CA_CERTS}'" |
+    .mqtts_certfile = "'${MQTTS_CERTFILE}'" |
+    .mqtts_keyfile = "'${MQTTS_KEYFILE}'" |
+    .logging.file |= "'${LOGGING_FILE}'" |
+    .logging.level |= "'${LOGGING_LEVEL}'"
+'
+
+cat ${CONFIG_FILE} | jq -M "${JQ_STRING}"> ${CONFIG_TEMP}
+mv ${CONFIG_TEMP} ${CONFIG_FILE}
+
+echo "done"
+
+exec python -m mqttconnector $@

apps/mqttConnector/docker/mqttconnector-amd64

@@ -0,0 +1,30 @@
+############################################################
+# Dockerfile to run openmtc mqttconnector binary
+############################################################
+
+# Set the base image to use openmtc/sdk
+FROM openmtc/sdk-amd64:latest
+
+ENV MOD_NAME=mqttconnector
+
+# Set the file maintainer
+MAINTAINER rst
+
+# install openmtc dependencies
+COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
+RUN pip install --upgrade --requirement /tmp/requirements.txt
+
+# install openmtc-mqttconnector
+COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
+RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
+    --owner root --group root --no-same-owner --no-overwrite-dir \
+    --transform 's/json\.dist/json/' --show-transformed
+
+RUN mkdir -p /var/log/openmtc
+
+# add change config
+COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
+
+# entry point
+ENTRYPOINT ["/usr/local/bin/configure-and-start"]
+CMD [""]

apps/mqttConnector/docker/mqttconnector-arm

@@ -0,0 +1,30 @@
+############################################################
+# Dockerfile to run openmtc mqttconnector binary
+############################################################
+
+# Set the base image to use openmtc/sdk
+FROM openmtc/sdk-arm:latest
+
+ENV MOD_NAME=mqttconnector
+
+# Set the file maintainer
+MAINTAINER rst
+
+# install openmtc dependencies
+COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
+RUN pip install --upgrade --requirement /tmp/requirements.txt
+
+# install openmtc-mqttconnector
+COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
+RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
+    --owner root --group root --no-same-owner --no-overwrite-dir \
+    --transform 's/json\.dist/json/' --show-transformed
+
+RUN mkdir -p /var/log/openmtc
+
+# add change config
+COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
+
+# entry point
+ENTRYPOINT ["/usr/local/bin/configure-and-start"]
+CMD [""]

apps/mqttConnector/etc/conf/config.json.dist

@@ -0,0 +1,29 @@
+{
+    "name": "mqttConnector",
+    "ep": "http://localhost:18000",
+    "cse_base": "onem2m",
+    "poas": [
+        "http://auto:21753"
+    ],
+    "originator_pre": "//openmtc.org/in-cse-1",
+    "ssl_certs": {
+        "cert_file": "/etc/openmtc/certs/mqttconnector.cert.pem",
+        "key_file": "/etc/openmtc/certs/mqttconnector.key.pem",
+        "ca_certs": "/etc/openmtc/certs/ca-chain.cert.pem"
+    },
+    "broker_ep": "localhost:8883",
+    "topic_pre": "exampleTopic",
+    "topic_index_location": 1,
+    "topic_index_device": -1,
+    "fiware_service": null,
+    "broker_user": "",
+    "broker_user_pw": "",
+    "mqtts_enabled": false,
+    "mqtts_ca_certs": null,
+    "mqtts_certfile": null,
+    "mqtts_keyfile": null,
+    "logging": {
+        "level": "INFO",
+        "file": "/var/log/openmtc/mqttconnector.log"
+    }
+}

apps/mqttConnector/etc/systemd/system/openmtc-mqttconnector.service

@@ -0,0 +1,10 @@
+[Unit]
+Description=OpenMTC mqttConnector
+After=network.target
+Wants=ntp.service
+
+[Service]
+ExecStart=/usr/local/bin/mqtt-connector
+
+[Install]
+WantedBy=multi-user.target

apps/mqttConnector/setup-mqttconnector.py

@@ -0,0 +1,82 @@
+#!/usr/bin/env python
+
+from setuptools import setup
+from distutils.core import setup
+from glob import glob
+import sys
+
+from utils import get_packages, get_pkg_files, OpenMTCSdist, move_config_files
+
+# name and dir
+NAME = "mqttconnector"
+BASE_DIR = "."
+
+# import pkg
+sys.path.append(BASE_DIR + "/src")
+pkg = __import__(NAME)
+
+# setup name and version
+SETUP_NAME = "openmtc-" + NAME
+SETUP_VERSION = pkg.__version__
+SETUP_DESCRIPTION = pkg.__description__
+
+# meta
+SETUP_AUTHOR = pkg.__author_name__
+SETUP_AUTHOR_EMAIL = pkg.__author_mail__
+SETUP_URL = "http://www.openmtc.org"
+SETUP_LICENSE = "Fraunhofer FOKUS proprietary"
+
+# requirements
+SETUP_REQUIRES = pkg.__requires__
+SETUP_INSTALL_REQUIRES = pkg.__requires__
+
+# packages
+PACKAGES = [NAME]
+PACKAGE_DIR = {"": BASE_DIR + "/src"}
+all_packages = []
+for package in PACKAGES:
+    all_packages.extend(get_packages(package, PACKAGE_DIR))
+
+# scripts
+SETUP_SCRIPTS = glob(BASE_DIR + "/bin/*")
+
+# package data
+PACKAGE_DATA = {NAME: get_pkg_files(BASE_DIR, NAME)}
+
+# data files
+CONFIG_FILES = ("config.json",)
+CONFIG_DIR = "/etc/openmtc/" + NAME
+CONFIG_DIST_FILES = (BASE_DIR + "/etc/conf/config.json.dist",)
+DATA_FILES = [(CONFIG_DIR, CONFIG_DIST_FILES)]
+
+# cmd class
+CMD_CLASS = {'sdist': OpenMTCSdist}
+
+if __name__ == "__main__":
+    if 'bdist_wheel' in sys.argv:
+        raise RuntimeError("This setup.py does not support wheels")
+
+    ############################################################################
+    # setup
+    setup(name=SETUP_NAME,
+          version=SETUP_VERSION,
+          description=SETUP_DESCRIPTION,
+          author=SETUP_AUTHOR,
+          author_email=SETUP_AUTHOR_EMAIL,
+          url=SETUP_URL,
+          license=SETUP_LICENSE,
+          requires=SETUP_REQUIRES,
+          install_requires=SETUP_INSTALL_REQUIRES,
+          package_dir=PACKAGE_DIR,
+          packages=all_packages,
+          scripts=SETUP_SCRIPTS,
+          package_data=PACKAGE_DATA,
+          data_files=DATA_FILES,
+          cmdclass=CMD_CLASS
+          )
+
+    ############################################################################
+    # install
+    if "install" in sys.argv:
+        # only do this during install
+        move_config_files(CONFIG_DIR, CONFIG_FILES)

apps/mqttConnector/src/mqttconnector/__init__.py

@@ -0,0 +1,9 @@
+"""
+TODO: Add description here
+"""
+
+__version__ = "1.2.0"
+__description__ = "mqttConnector"
+__author_name__ = "Ronald Steinke"
+__author_mail__ = "ronald.steinke@fokus.fraunhofer.de"
+__requires__ = ["paho_mqtt"]

apps/mqttConnector/src/mqttconnector/__main__.py

@@ -0,0 +1,93 @@
+from argparse import ArgumentDefaultsHelpFormatter, ArgumentParser
+
+from openmtc_app.util import prepare_app, get_value
+from openmtc_app.runner import AppRunner as Runner
+from .mqtt_connector import mqttConnector
+
+# defaults
+default_name = "mqttConnector"
+default_ep = "http://localhost:8000"
+default_topic_pre = "exampleTopic"
+default_topic_index_location = 1
+default_topic_index_device = -1
+default_fiware_service = None
+default_broker_user = ""
+default_broker_user_pw = ""
+default_mqtts_ca_certs = None
+default_mqtts_certfile = None
+default_mqtts_keyfile = None
+
+# args parser
+parser = ArgumentParser(
+    description="An IPE called mqttConnector",
+    prog="mqttConnector",
+    formatter_class=ArgumentDefaultsHelpFormatter)
+parser.add_argument("-n", "--name", help="Name used for the AE.")
+parser.add_argument("-s", "--ep", help="URL of the local Endpoint.")
+parser.add_argument("--topic-pre", help="Topic you want to be subscribed to")
+parser.add_argument(
+    "--topic-index-location", help="Index of location in topic string")
+parser.add_argument(
+    "--topic-index-device", help="Index of device name in topic string")
+parser.add_argument("--broker-user", help="credentials for MQTT broker")
+parser.add_argument("--broker-user-pw", help="credentials for MQTT broker")
+parser.add_argument("--mqtts-enabled", action='store_true')
+parser.add_argument(
+    "--mqtts-ca-certs", help="Path to CA certs or tuple of paths")
+parser.add_argument("--mqtts-certfile", help="Path to own mqtts cert")
+parser.add_argument("--mqtts-keyfile", help="Path to own mqtts key")
+# args, config and logging
+args, config = prepare_app(parser, __loader__, __name__, "config.json")
+
+# variables
+nm = get_value("name", (unicode, str), default_name, args, config)
+cb = config.get("cse_base", "onem2m")
+ep = get_value("ep", (unicode, str), default_ep, args, config)
+poas = config.get("poas", ["http://auto:21753"])
+originator_pre = config.get("originator_pre", "//openmtc.org/mn-cse-1")
+ssl_certs = config.get("ssl_certs", {})
+
+broker_ep = config.get("broker_ep", "localhost:8883")
+
+topic_pre = get_value("topic_pre", (unicode, str), default_topic_pre, args,
+                      config)
+topic_index_location = get_value("topic_index_location", (int),
+                                 default_topic_index_location, args, config)
+topic_index_device = get_value("topic_index_device", (int),
+                               default_topic_index_device, args, config)
+fiware_service = get_value("fiware_service", (unicode, str),
+                           default_fiware_service, args, config)
+broker_user = get_value("broker_user", (unicode, str), default_broker_user,
+                        args, config)
+broker_user_pw = get_value("broker_user_pw", (unicode, str),
+                           default_broker_user_pw, args, config)
+user_pw = get_value("broker_user_pw", (unicode, str), default_broker_user_pw,
+                    args, config)
+mqtts_enabled = get_value("mqtts_enabled", (bool), False, args, config)
+mqtts_ca_certs = get_value("mqtts_ca_certs", (unicode, str),
+                           default_mqtts_ca_certs, args, config)
+mqtts_certfile = get_value("mqtts_certfile", (unicode, str),
+                           default_mqtts_certfile, args, config)
+mqtts_keyfile = get_value("mqtts_keyfile", (unicode, str),
+                          default_mqtts_keyfile, args, config)
+# start
+app = mqttConnector(
+    broker_ep=broker_ep,
+    topic_pre=topic_pre,
+    broker_user=broker_user,
+    broker_user_pw=broker_user_pw,
+    topic_index_location=topic_index_location,
+    topic_index_device=topic_index_device,
+    fiware_service=fiware_service,
+    mqtts_enabled=mqtts_enabled,
+    mqtts_ca_certs=mqtts_ca_certs,
+    mqtts_certfile=mqtts_certfile,
+    mqtts_keyfile=mqtts_keyfile,
+    name=nm,
+    cse_base=cb,
+    poas=poas,
+    originator_pre=originator_pre,
+    **ssl_certs)
+Runner(app).run(ep)
+
+print("Exiting....")

apps/mqttConnector/src/mqttconnector/mqtt_connector.py

@@ -0,0 +1,127 @@
+import re
+from base64 import b64decode as base64decode
+from itertools import groupby
+from json import loads as json_decode
+from os import path as ospath
+
+from paho.mqtt import client as mqtt
+
+from openmtc_app.onem2m import XAE
+
+
+class mqttConnector(XAE):
+    interval = 10
+    _location_containers = {}
+    _device_containers = {}
+    _sensor_containers = {}
+    client = mqtt.Client()
+
+    def __init__(self,
+                 broker_ep,
+                 topic_pre,
+                 broker_user,
+                 broker_user_pw,
+                 topic_index_location=1,
+                 topic_index_device=-1,
+                 fiware_service=None,
+                 mqtts_enabled=False,
+                 mqtts_ca_certs=None,
+                 mqtts_certfile=None,
+                 mqtts_keyfile=None,
+                 *args,
+                 **kw):
+        host, port = broker_ep.split(":")
+        self._broker_host = host
+        self._broker_port = int(port)
+
+        self.topic_pre = topic_pre
+        self.topic_index_location = topic_index_location
+        self.topic_index_device = topic_index_device
+        self.fiware_service = fiware_service
+        self.broker_user = broker_user
+        self.broker_user_pw = broker_user_pw
+        self.mqtts_enabled = mqtts_enabled
+        self.mqtts_ca_certs = mqtts_ca_certs or None
+        self.mqtts_certfile = mqtts_certfile or None
+        self.mqtts_keyfile = mqtts_keyfile or None
+
+        super(mqttConnector, self).__init__(*args, **kw)
+
+    def _on_register(self):
+        def on_connect(*_):
+            def callback(*args):
+                (_, _, message) = args
+                self.logger.info(
+                    'Received message on topic %s' % message.topic)
+                self._handle_data(message.topic, message.payload)
+
+            topics = ['%s/#' % self.topic_pre]
+
+            for topic in topics:
+                self.client.message_callback_add(topic, callback)
+
+            self.client.subscribe([(topic, 1) for topic in topics])
+
+        self.client.on_connect = on_connect
+
+        # TODO(rst): this needs to be handled more general and from config
+        self.client.username_pw_set(self.broker_user, self.broker_user_pw)
+        if self.mqtts_enabled:
+            self.client.tls_set(
+                ca_certs=self.mqtts_ca_certs,
+                certfile=self.mqtts_certfile,
+                keyfile=self.mqtts_keyfile)
+            self.client.tls_insecure_set(True)
+        self.client.connect(self._broker_host, self._broker_port)
+        # TODO let gevent handle this
+        self.client.loop_forever()
+
+    def _on_shutdown(self):
+        self.client.disconnect()
+
+    def _get_target_container(self, location, device, sensor):
+        try:
+            return self._sensor_containers[(location, device, sensor)]
+        except KeyError:
+            try:
+                device_cnt = self._device_containers[(location, device)]
+            except KeyError:
+                try:
+                    location_cnt = self._location_containers[location]
+                except KeyError:
+                    location_cnt = self.create_container(None, location)
+                    self._location_containers[location] = location_cnt
+                device_cnt = self.create_container(
+                    location_cnt, device, labels=["openmtc:device"])
+                self._device_containers[(location, device)] = device_cnt
+            openmtc_id = "%s/%s/%s" % (
+                (self.fiware_service + '~' if self.fiware_service else '') +
+                location, device, sensor)
+            labels = ['openmtc:sensor_data', 'openmtc:id:%s' % openmtc_id]
+            sensor_cnt = self.create_container(
+                device_cnt, sensor, labels=labels)
+            self._sensor_containers[(location, device, sensor)] = sensor_cnt
+            return sensor_cnt
+
+    def _handle_data(self, topic, payload):
+        # get location and device
+        try:
+            location = topic.split('/')[self.topic_index_location]
+            device = topic.split('/')[self.topic_index_device]
+        except (AttributeError, ValueError):
+            self.logger.error("Topic '%s' not valid. Dropping." % topic)
+            return
+
+        # check payload
+        try:
+            readings = json_decode(
+                base64decode(json_decode(payload)['m2m:cin']['con']))
+        except (ValueError, KeyError, TypeError):
+            self.logger.error('Damaged payload; discarding')
+            return
+
+        # push data
+        for _, values in groupby(readings, key=lambda x: x['n']):
+            sensor_cnt = self._get_target_container(location, device, 'number')
+            for value in sorted(values, key=lambda x: x['t']):
+                self.push_content(sensor_cnt, [value])

apps/mqttConnector/utils.py

@@ -0,0 +1,148 @@
+import distutils.command.sdist
+import distutils.command.build_py
+import os
+import subprocess
+import sys
+
+
+def echo(msg, *args):
+    if args:
+        msg = msg % args
+    sys.stdout.write(msg + "\n")
+
+
+def get_packages(package, package_dir, excluded_list=None, included_list=None):
+    included_list = included_list or []
+    excluded_list = excluded_list or []
+
+    try:
+        root = package_dir[package]
+    except KeyError:
+        root = package_dir.get("", ".") + "/" + package
+
+    if not os.path.exists(root):
+        sys.stderr.write(
+            "Directory for package %s does not exist: %s\n" % (package, root))
+        sys.exit(1)
+
+    def on_error(error):
+        sys.stderr.write(
+            "Error while collecting packages for %s: %s\n" % (package, error))
+        sys.exit(1)
+
+    packages = [package]
+
+    r_prefix = len(root) + 1
+    for path, dirs, files in os.walk(root, onerror=on_error):
+        is_module = "__init__.py" in files and path != root
+        excluded = any(map(lambda x: x in path, excluded_list))
+        included = any(map(lambda x: x in path, included_list))
+        if is_module and (not excluded or included):
+            packages.append(package + "." + path[r_prefix:].replace("/", "."))
+
+    return packages
+
+
+def get_pkg_files(base_dir, name):
+    package_files = []
+    pkg_dir = os.path.join(base_dir, 'src', name)
+    pkg_data_dir = os.path.join(pkg_dir, 'static')
+    for (path, directories, filenames) in os.walk(pkg_data_dir):
+        for filename in filenames:
+            package_files.append(os.path.join(os.path.relpath(path, pkg_dir),
+                                              filename))
+    return package_files
+
+
+def enable_init_files(init_dir, init_dist_files):
+    for f in init_dist_files:
+        os.chmod(os.path.join(init_dir, os.path.basename(f)), 0755)
+
+
+def move_config_files(config_dir, config_files):
+    for f in config_files:
+        target_file = os.path.join(config_dir, f)
+        if not os.path.exists(target_file):
+            echo("Installing config file %s", target_file)
+            os.rename(target_file + ".dist", target_file)
+            # os.chmod(target_file, 0644)
+        else:
+            echo("Not overwriting config file %s", target_file)
+
+
+def create_openmtc_user(db_dir=None, log_dir=None):
+    try:
+        from pwd import getpwnam
+    except ImportError:
+        print "Could not import the 'pwd' module. Skipping user management"
+    else:
+        # assuming DB_DIR was created by setup already
+        try:
+            pw = getpwnam('openmtc')
+        except KeyError as e:
+            try:
+                # add system user openmtc:openmtc
+                # useradd --system -UM openmtc
+                useradd = "useradd --system -UM openmtc"
+                retcode = subprocess.call(useradd, shell=True)
+                if retcode:
+                    raise Exception("Failed to add user 'openmtc'")
+                pw = getpwnam('openmtc')
+            except Exception as e:
+                sys.stderr.write("Error creating user: %s\n" % (e, ))
+                sys.exit(1)
+        uid = pw.pw_uid
+        gid = pw.pw_gid
+
+        # set path permissions
+        if db_dir:
+            os.chown(db_dir, uid, gid)
+        if log_dir:
+            os.chown(log_dir, uid, gid)
+
+
+class OpenMTCSdist(distutils.command.sdist.sdist):
+    def make_release_tree(self, base_dir, files):
+        distutils.command.sdist.sdist.make_release_tree(self, base_dir, files)
+
+        script_name = os.path.basename(sys.argv[0])
+
+        if script_name != "setup.py":
+            os.rename(base_dir + "/" + script_name, base_dir + "/setup.py")
+            self.filelist.files.remove(script_name)
+            self.filelist.files.append("setup.py")
+
+
+class OpenMTCSdistBinary(OpenMTCSdist, object):
+    def make_release_tree(self, base_dir, files):
+        super(OpenMTCSdistBinary, self).make_release_tree(base_dir, files)
+
+        script_name = os.path.basename(sys.argv[0])
+
+        build_py = self.get_finalized_command('build_py')
+        build_py.compile = 1
+        build_py.optimize = 2
+        build_py.retain_init_py = 1
+        build_py.build_lib = base_dir
+        build_py.byte_compile(
+            [base_dir + "/" + f for f in self.filelist.files if
+             f != script_name and f.endswith(".py")])
+
+
+class OpenMTCBuildPy(distutils.command.build_py.build_py):
+    retain_init_py = 0
+
+    def byte_compile(self, files):
+        distutils.command.build_py.build_py.byte_compile(self, files)
+
+
+class OpenMTCBuildPyBinary(OpenMTCBuildPy, object):
+    retain_init_py = 0
+
+    def byte_compile(self, files):
+        super(OpenMTCBuildPyBinary, self).byte_compile(files)
+
+        for f in files:
+            if (f.endswith('.py') and (os.path.basename(f) != "__init__.py" or
+                                       not self.retain_init_py)):
+                os.unlink(f)

common/openmtc-onem2m/src/openmtc_onem2m/client/http.py

@@ -48,7 +48,7 @@ _method_map_to_http = {
 
 _clients = LRUCache(threadsafe=False)
 
-_query_params = frozenset(['rt', 'rp', 'rcn', 'da', 'drt'])
+_query_params = frozenset(['rt', 'rp', 'rcn', 'da', 'drt', 'rids', 'tids', 'ltids', 'tqi'])
 
 _header_to_field_map = {
     'X-M2M-ORIGIN': 'originator',
@@ -59,6 +59,8 @@ _header_to_field_map = {
     'X-M2M-RET':    'rqet',
     'X-M2M-OET':    'oet',
     'X-M2M-EC':     'ec',
+    'X-M2M-RVI':    'rvi',
+    'X-M2M-VSI':    'vsi',
 }
 
 
@@ -105,10 +107,12 @@ class OneM2MHTTPClient(OneM2MClient):
         else:
             ssl_options = None
 
-        client = HTTPClient(host, port, connection_timeout=120.0,
-                            concurrency=50, ssl=is_https,
-                            ssl_options=ssl_options, insecure=insecure)
-        self.request = client.request
+        def get_http_client():
+            return HTTPClient(host, port, connection_timeout=120.0,
+                              concurrency=50, ssl=is_https,
+                              ssl_options=ssl_options, insecure=insecure)
+
+        self._get_client = get_http_client
 
         self.content_type = 'application/' + ('xml' if use_xml else 'json')
 
@@ -160,11 +164,14 @@ class OneM2MHTTPClient(OneM2MClient):
             content_type += '; ty=' + str(ResourceTypeE[onem2m_request.resource_type.typename])
 
         headers = {
-            header: getattr(onem2m_request, field) for header, field in _header_to_field_map.iteritems()
+            header: getattr(onem2m_request, field)
+            for header, field in _header_to_field_map.iteritems()
             if getattr(onem2m_request, field) is not None
         }
         headers['content-type'] = content_type
 
+        headers['accept'] = self.content_type
+
         self.logger.debug("Added request params: %s", params)
 
         return {
@@ -201,9 +208,10 @@ class OneM2MHTTPClient(OneM2MClient):
         with Promise() as p:
             http_request = self.map_onem2m_request_to_http_request(onem2m_request)
             t = time()
+            client = self._get_client()
 
             try:
-                response = self.request(**http_request)
+                response = client.request(**http_request)
             except (socket_error, gaierror) as exc:
                 self._handle_network_error(exc, p, http_request, t, ConnectionFailed)
             except Exception as exc:
@@ -218,5 +226,7 @@ class OneM2MHTTPClient(OneM2MClient):
                         p.fulfill(onem2m_response)
                 finally:
                     response.release()
+            finally:
+                client.close()
 
         return p

common/openmtc-onem2m/src/openmtc_onem2m/client/mqtt.py

@@ -83,11 +83,12 @@ class OneM2MMQTTClient(OneM2MClient):
 
     __request_fields = frozenset([
         'op',
+        'to',
         'fr',
         'rqi',
         'ty',
         'pc',
-        'rol',
+        'rids',
         'ot',
         'rqet',
         'rset',
@@ -98,17 +99,28 @@ class OneM2MMQTTClient(OneM2MClient):
         'ec',
         'da',
         'gid',
-        'drt',
-        'to',
         'fc',
+        'drt',
+        'tids',
+        'ltids',
+        'tqi',
+        'rvi',
+        'vsi',
     ])
 
     __response_fields = frozenset([
         'rsc',
         'rqi',
         'pc',
-        'fr',
         'to',
+        'fr',
+        'ot',
+        'rset',
+        'ec',
+        'cts',
+        'cto',
+        'rvi',
+        'vsi',
     ])
 
     @staticmethod
@@ -370,7 +382,8 @@ class OneM2MMQTTClient(OneM2MClient):
                 sp_id, cse_id, _ = split_onem2m_address(response.to)
                 response.content = self._decode(
                     encode_onem2m_content(response.content, 'application/json',
-                                          path=sp_id + cse_id)[1]
+                                          path=sp_id + cse_id,
+                                          fields=response.fields)[1]
                 )
 
             self._publish_message(

common/openmtc-onem2m/src/openmtc_onem2m/exc.py

@@ -13,6 +13,10 @@ STATUS_ACCEPTED = STATUS(
     1000, "ACCEPTED", 202)
 STATUS_OK = STATUS(
     2000, "OK", 200)
+STATUS_UPDATED = STATUS(
+    2004, "UPDATED", 200)
+STATUS_DELETED = STATUS(
+    2002, "DELETED", 200)
 STATUS_CREATED = STATUS(
     2001, "CREATED", 201)
 STATUS_BAD_REQUEST = STATUS(

common/openmtc-onem2m/src/openmtc_onem2m/mapper/__init__.py

@@ -49,7 +49,8 @@ class OneM2MMapper(BasicMapper):
             path,
             self.originator,
             ty=type(instance),
-            pc=instance
+            pc=instance,
+            rvi='2a'
         )).get()
 
         try:
@@ -83,7 +84,8 @@ class OneM2MMapper(BasicMapper):
             OneM2MOperation.update,
             instance.path,
             self.originator,
-            pc=instance
+            pc=instance,
+            rvi='2a'
         )).get()
 
         try:
@@ -105,6 +107,8 @@ class OneM2MMapper(BasicMapper):
             path,
             self.originator,
             filter_criteria=fc,
+            rvi='2a',
+            rcn=5,
             **request_options
         )).get()
 
@@ -112,7 +116,8 @@ class OneM2MMapper(BasicMapper):
         self._send_request(OneM2MRequest(
             OneM2MOperation.delete,
             getattr(instance, "path", instance),
-            self.originator
+            self.originator,
+            rvi='2a'
         ))
 
     # TODO(rst): check if this can be removed in parent class

common/openmtc-onem2m/src/openmtc_onem2m/model.py

@@ -6,7 +6,7 @@ from openmtc.model import (Resource as Res, UnicodeAttribute, DatetimeAttribute,
 from openmtc.model.exc import ModelTypeError
 from futile import issubclass
 
-LATEST_VERSION = "1.6"
+LATEST_VERSION = "2a"
 
 
 class OneM2MIntEnum(IntEnum):
@@ -24,7 +24,7 @@ class OneM2MContentResource(ContentResource, OneM2MEntity):
 
 class OneM2MResource(Res, OneM2MEntity):
     __model_name__ = "onem2m"
-    __model_version__ = "1.6"
+    __model_version__ = "2a"
 
 
 ################################################################################
@@ -41,8 +41,8 @@ class ResourceTypeE(OneM2MIntEnum):
     eventConfig = 7
     execInstance = 8
     group = 9
-    localPolicy = 10
-    m2mServiceSubscriptionProfile = 11
+    locationPolicy = 10
+    m2mServiceSubscription = 11
     mgmtCmd = 12
     mgmtObj = 13
     node = 14
@@ -56,6 +56,16 @@ class ResourceTypeE(OneM2MIntEnum):
     statsConfig = 22
     subscription = 23
     semanticDescriptor = 24
+    notificationTargetMgmtPolicyRef = 25
+    notificationTargetPolicy = 26
+    policyDeletionRules = 27
+    flexContainer = 28
+    timeSeries = 29
+    timeSeriesInstance = 30
+    role = 31
+    token = 32
+    trafficPattern = 33
+    dynamicAuthorizationConsultation = 34
     accessControlPolicyAnnc = 10001
     AEAnnc = 10002
     containerAnnc = 10003
@@ -66,6 +76,12 @@ class ResourceTypeE(OneM2MIntEnum):
     nodeAnnc = 10014
     remoteCSEAnnc = 10016
     scheduleAnnc = 10018
+    semanticDescriptorAnnc = 10024
+    flexContainerAnnc = 10028
+    timeSeriesAnnc = 10029
+    timeSeriesInstanceAnnc = 10030
+    trafficPatternAnnc = 10033
+    dynamicAuthorizationConsultationAnnc = 10034
 
 
 @unique
@@ -106,15 +122,17 @@ class ResponseType(OneM2MIntEnum):
 
 
 # @unique
-# class ResultConentE(OneM2MIntEnum):
-#    nothing = 0
-#    attributes = 1
-#    hierarchical_address = 2
-#    hierarchical_address_and_attributes = 3
-#    attributes_and_child_resources = 4
-#    attributes_and_child_resource_references = 6
-#    child_resource_references = 6
-#    original_resource = 7
+class ResultContentE(OneM2MIntEnum):
+    nothing = 0
+    attributes = 1
+    hierarchical_address = 2
+    hierarchical_address_and_attributes = 3
+    attributes_and_child_resources = 4
+    attributes_and_child_resource_references = 5
+    child_resource_references = 6
+    original_resource = 7
+    child_resources = 8
+    modified_attributes = 9
 
 
 @unique
@@ -136,6 +154,7 @@ class RequestStatusE(OneM2MIntEnum):
 
 @unique
 class MemberTypeE(OneM2MIntEnum):
+    mixed = 0
     accessControlPolicy = 1
     AE = 2
     container = 3
@@ -159,7 +178,16 @@ class MemberTypeE(OneM2MIntEnum):
     statsCollect = 21
     statsConfig = 22
     subscription = 23
+    semanticDescriptor = 24
+    notificationTargetMgmtPolicyRef = 25
+    notificationTargetPolicy = 26
+    policyDeletionRules = 27
+    flexContainer = 28
+    timeSeries = 29
+    timeSeriesInstance = 30
+    role = 31
     token = 32
+    trafficPattern = 33
     dynamicAuthorizationConsultation = 34
     accessControlPolicyAnnc = 10001
     AEAnnc = 10002
@@ -170,11 +198,15 @@ class MemberTypeE(OneM2MIntEnum):
     mgmtObjAnnc = 10013
     nodeAnnc = 10014
     remoteCSEAnnc = 10016
-    scheduleAnnc = 10019
+    scheduleAnnc = 10018
+    semanticDescriptorAnnc = 10024
+    flexContainerAnnc = 10028
+    timeSeriesAnnc = 10029
+    timeSeriesInstanceAnnc = 10030
+    trafficPatternAnnc = 10033
     dynamicAuthorizationConsultationAnnc = 10034
-    mixed = 24
-    # Mixed is a mixture of the resource types from 1 to 23, 10001 to 10004, 10009 to  10010,
-    # 10013 to 10014 and 10016 to 10018 as listed above.
+    oldest = 20001
+    latest = 20002
 
 
 @unique
@@ -952,7 +984,7 @@ class ContentInstance(AnnounceableSubordinateResourceC,
                       SubscribableResource):
     """See TS-0001 section 9.6.7"""
 
-    stateTag = UnicodeAttribute(accesstype=Attribute.RO)
+    stateTag = Attribute(int, accesstype=Attribute.RO)
     creator = UnicodeAttribute()        # m2m:ID
     # contentInfo = typeOfContent(:EncodingType)
     # typeOfContent => Media Types
@@ -969,7 +1001,7 @@ class ContentInstance(AnnounceableSubordinateResourceC,
 
 
 class ContentInstanceAnnc(AnnouncedSubordinateResourceC):
-    stateTag = UnicodeAttribute(accesstype=Attribute.RO)
+    stateTag = Attribute(int, accesstype=Attribute.RO)
     contentInfo = UnicodeAttribute(EncodingTypeE)    # m2m:contentInfo
     contentSize = Attribute(int, accesstype=Attribute.WO)
     ontologyRef = UnicodeAttribute(accesstype=Attribute.WO)
@@ -983,7 +1015,7 @@ class ContentInstanceAnnc(AnnouncedSubordinateResourceC):
 class Container(AnnounceableResourceC, SubscribableResource):
     """See TS-0001 section 9.6.6"""
 
-    stateTag = UnicodeAttribute(accesstype=Attribute.RO)
+    stateTag = Attribute(int, accesstype=Attribute.RO)
     creator = UnicodeAttribute()
     maxNrOfInstances = Attribute(int)
     maxByteSize = Attribute(int)
@@ -1012,7 +1044,7 @@ Container.__child_types__ = (
 
 class ContainerAnnc(AnnouncedResourceC, SubscribableResource):
 
-    stateTag = UnicodeAttribute(accesstype=Attribute.RO)
+    stateTag = Attribute(int, accesstype=Attribute.RO)
     maxNrOfInstances = Attribute(int)
     maxByteSize = Attribute(int)
     maxInstanceAge = UnicodeAttribute(mandatory=False)  # todo

common/openmtc-onem2m/src/openmtc_onem2m/serializer/base.py

@@ -49,7 +49,7 @@ class OneM2MSerializer(LoggerMixin):
             except (TypeError, AttributeError, KeyError, ValueError):
                 raise CSEValueError("Invalid entry in child resources: %s",
                                     child_resource)
-        if resource_type is Notification and "notificationEvent" in data:
+        if resource_type is Notification and data.get("notificationEvent"):
             representation = data["notificationEvent"]["representation"]
             representation = self.decode(self.dumps(representation))
             data["notificationEvent"]["representation"] = representation
@@ -62,7 +62,10 @@ class OneM2MSerializer(LoggerMixin):
 class OneM2MDictSerializer(OneM2MSerializer):
     def encode_resource(self, resource, pretty=False, path=None, encoding="utf-8", fields=None,
                         encapsulated=False):
-        representation = resource.values
+        if fields and isinstance(resource, OneM2MResource):
+            representation = {k: v for k, v in resource.values.items() if fields and k in fields}
+        else:
+            representation = resource.values
 
         self.logger.debug("Encoding representation: %s", representation)
 
@@ -93,7 +96,7 @@ class OneM2MDictSerializer(OneM2MSerializer):
                 return resource_id
             return val_path + resource_id
 
-        if isinstance(resource, OneM2MResource):
+        if isinstance(resource, OneM2MResource) and "childResource" in representation:
 
             def get_child_rep(c):
                 return {
@@ -112,53 +115,12 @@ class OneM2MDictSerializer(OneM2MSerializer):
             if isinstance(resource.oldest, ContentInstance):
                 representation['oldest'] = resource.oldest.resourceID
 
-        # cleans representation
-        def clean_representation(o):
-            try:
-                # removes empty attributes
-                empty_keys = []
-                for k, v in o.items():
-                    if v is None:
-                        empty_keys.append(k)
-                    elif isinstance(v, OneM2MEntity):
-                        o[k] = self.encode_resource(v, pretty, path, encoding, fields)
-                    elif isinstance(v, list):
-
-                        def encode_list_item(item):
-                            if isinstance(item, OneM2MEntity):
-                                return self.encode_resource(item, pretty, path, encoding, fields)
-                            return item
-                        if len(v):
-                            o[k] = map(encode_list_item, v)
-                        else:
-                            empty_keys.append(k)
-                    else:
-                        try:
-                            if len(v) == 0:
-                                empty_keys.append(k)
-                        except TypeError:
-                            pass
-
-                for k in empty_keys:
-                    del o[k]
-
-                for k, v in o.items():
-                    if not isinstance(v, (unicode, str, bool, datetime,
-                                          OneM2MIntEnum)):
-                        clean_representation(v)
-            except AttributeError:
-                if isinstance(o, list):
-                    for p in o:
-                        clean_representation(p)
-
         if not isinstance(resource, OneM2MContentResource):
             representation = {
                 get_short_resource_name(k) or get_short_attribute_name(k) or
                 get_short_member_name(k): v for
                 k, v in representation.items()}
 
-            clean_representation(representation)
-
         if not isinstance(resource, (OneM2MResource, Notification,
                                      SecurityInfo, OneM2MContentResource)):
             return representation

common/openmtc-onem2m/src/openmtc_onem2m/serializer/json/__init__.py

@@ -30,12 +30,7 @@ del logger
 
 def _default(x):
     if isinstance(x, datetime):
-        try:
-            isoformat = x.isoformat
-        except AttributeError:
-            raise TypeError("%s (%s)" % (x, type(x)))
-
-        return isoformat()
+        return x.strftime("%Y%m%dT%H%M%S")
     elif isinstance(x, ContentInstance):
         return x.resourceID
     else:

common/openmtc-onem2m/src/openmtc_onem2m/serializer/util.py

@@ -27,9 +27,6 @@ def encode_onem2m_content(content, content_type, pretty=False, path=None,
     if content is None:
         return None, None
 
-    fields = fields  # TODO(rst): maybe necessary
-    # fields = ["resourceID"]
-
     serializer = get_onem2m_encoder(content_type)
 
     data = serializer.encode_resource(content, pretty=pretty, path=path,

common/openmtc-onem2m/src/openmtc_onem2m/transport.py

@@ -201,10 +201,11 @@ class OneM2MRequest(object):
 
     """Class representing a OneM2M request"""
 
-    def __init__(self, op, to, fr=None, rqi=None, ty=None, pc=None, rol=None,
+    def __init__(self, op, to, fr=None, rqi=None, ty=None, pc=None, rids=None,
                  ot=None, rqet=None, rset=None, oet=None, rt=None, rp=None,
                  rcn=None, ec=None, da=None, gid=None, filter_criteria=None,
-                 fc=None, drt=None):
+                 fc=None, drt=None, tids=None, ltids=None, tqi=None, rvi=None,
+                 vsi=None):
         # Operation
         self.operation = op
         # Target uri
@@ -216,7 +217,7 @@ class OneM2MRequest(object):
         self.resource_type = ty
         # Resource content to be transferred.
         self.content = pc
-        self.role = rol
+        self.role_ids = rids
         self.originating_timestamp = ot
         self.request_expiration_timestamp = rqet
         self.result_expiration_timestamp = rset
@@ -230,6 +231,11 @@ class OneM2MRequest(object):
         self.filter_criteria = filter_criteria or fc
         # Optional Discovery result type
         self.discovery_result_type = drt
+        self.token_ids = tids
+        self.local_token_ids = ltids
+        self.token_request_identifier = tqi
+        self.release_version_indicator = rvi
+        self.vendor_information = vsi
 
     @property
     def op(self):
@@ -272,12 +278,12 @@ class OneM2MRequest(object):
         self.content = pc
 
     @property
-    def rol(self):
-        return self.role
+    def rids(self):
+        return self.role_ids
 
-    @rol.setter
-    def rol(self, rol):
-        self.role = rol
+    @rids.setter
+    def rids(self, rids):
+        self.role_ids = rids
 
     @property
     def ot(self):
@@ -375,6 +381,46 @@ class OneM2MRequest(object):
     def drt(self, drt):
         self.discovery_result_type = drt
 
+    @property
+    def tids(self):
+        return self.token_ids
+
+    @tids.setter
+    def tids(self, tids):
+        self.token_ids = tids
+
+    @property
+    def ltids(self):
+        return self.local_token_ids
+
+    @ltids.setter
+    def ltids(self, rvi):
+        self.local_token_ids = ltids
+
+    @property
+    def tqi(self):
+        return self.token_request_identifier
+
+    @tqi.setter
+    def tqi(self, tqi):
+        self.token_request_identifier = tqi
+
+    @property
+    def rvi(self):
+        return self.release_version_indicator
+
+    @rvi.setter
+    def rvi(self, rvi):
+        self.release_version_indicator = rvi
+
+    @property
+    def vsi(self):
+        return self.vendor_information
+
+    @vsi.setter
+    def vsi(self, vsi):
+        self.vendor_information = vsi
+
     def __str__(self):
         return '%s: %s' % (self.__class__.__name__, ' | '.join([
             '%s: %s' % (str(k), str(v)) for k, v in self.__dict__.iteritems()
@@ -385,7 +431,8 @@ class OneM2MResponse(object):
     """Class representing a OneM2M response"""
 
     def __init__(self, status_code, request=None, rqi=None, pc=None, to=None,
-                 fr=None, rsc=None):
+                 fr=None, rsc=None, ot=None, rset=None, ec=None, cts=None,
+                 cto=None, rvi=None, vsi=None, fields=None):
         # Operation result
         if isinstance(status_code, STATUS):
             self.response_status_code = status_code
@@ -397,14 +444,27 @@ class OneM2MResponse(object):
             self.to = request.to
             # Originator ID
             self.originator = request.fr
+            self.originating_timestamp = request.ot
+            self.result_expiration_timestamp = request.rset
+            self.event_category = request.ec
+            self.release_version_indicator = request.rvi
+            self.vendor_information = request.vsi
         else:
             self.request_identifier = rqi
             # Target uri
             self.to = to
             # Originator ID
             self.originator = fr
+            self.originating_timestamp = ot
+            self.release_version_indicator = rvi
+            self.vendor_information = vsi
+        self.result_expiration_timestamp = rset
+        self.event_category = ec
         # Resource content to be transferred.
         self.content = pc
+        self.content_status = cts
+        self.content_offset = cto
+        self.fields = fields
 
     @property
     def status_code(self):
@@ -438,6 +498,62 @@ class OneM2MResponse(object):
     def fr(self, fr):
         self.originator = fr
 
+    @property
+    def ot(self):
+        return self.originating_timestamp
+
+    @ot.setter
+    def ot(self, ot):
+        self.originating_timestamp = ot
+
+    @property
+    def rset(self):
+        return self.result_expiration_timestamp
+
+    @rset.setter
+    def rset(self, rset):
+        self.result_expiration_timestamp = rset
+
+    @property
+    def ec(self):
+        return self.event_category
+
+    @ec.setter
+    def ec(self, ec):
+        self.event_category = ec
+
+    @property
+    def cts(self):
+        return self.content_status
+
+    @cts.setter
+    def cts(self, cts):
+        self.content_status = cts
+
+    @property
+    def cto(self):
+        return self.content_offset
+
+    @cto.setter
+    def cto(self, cto):
+        self.content_offset = cto
+
+    @property
+    def rvi(self):
+        return self.release_version_indicator
+
+    @rvi.setter
+    def rvi(self, rvi):
+        self.release_version_indicator = rvi
+
+    @property
+    def vsi(self):
+        return self.vendor_information
+
+    @vsi.setter
+    def vsi(self, vsi):
+        self.vendor_information = vsi
+
     def __str__(self):
         return '%s: %s' % (self.__class__.__name__, ' | '.join([
             '%s: %s' % (str(k), str(v)) for k, v in self.__dict__.iteritems()

create-binary-docker

@@ -260,6 +260,14 @@ rm -f "${target_docker_binary}"
 rm -f "${docker_tmp}/${name}-dependencies.txt"
 ${docker_cmd} rm -f ${build_container_name} &> /dev/null
 printf "done\n"
+
+# remove dangling images
+separator_line
+printf "### Removing dangled images..."
+for image in $(${docker_cmd} images -qa -f "dangling=true"); do
+    ${docker_cmd} rmi -f ${image} > /dev/null
+done
+printf "done\n"
 }
 
 trap cleanup SIGINT SIGTERM
@@ -391,7 +399,7 @@ printf "%s\n" $(get_requirements_from_setup_file) | tr " " "\n" > \
 separator_line
 printf "### Building %s-%s container...\n" ${name} ${machine}
 
-${docker_cmd} build -t ${target_docker_name} \
+${docker_cmd} build --force-rm -t ${target_docker_name} \
     -f ${target_docker_file} ${docker_path}
 if [ $? -gt 0 ]; then
     printf "### Building %s-%s container failed. Exiting now.\n" \
@@ -405,15 +413,6 @@ printf "### Base %s-%s container built successfully.\n" ${name} ${machine}
 # cleanup
 cleanup
 
-##############################################################################
-# remove dangling images
-separator_line
-printf "### Removing dangled images..."
-for image in $(${docker_cmd} images -qa -f "dangling=true"); do
-    ${docker_cmd} rmi -f ${image} > /dev/null
-done
-printf "done\n"
-
 ##############################################################################
 # example to run the docker file
 #${docker_cmd} run --name test -d \

doc/fiware_integration.md

@@ -0,0 +1,5 @@
+![](pics/fiware_connector.png)
+
+The FIWARE Connector provided by OpenMTC enables the exchange of sensor and actuator data between an OpenMTC Setup and FIWARE Setup with a running Orion Context Broker instance. This way it is possible to access oneM2M sensor data and control oneM2M actuators via FIWARE enabled applications.
+
+OpenMTC together with the FIWARE Connector is available as FIWARE Generic Enabler in the FIWARE Catalogue (Link).

doc/index.md

@@ -0,0 +1,9 @@
+# Open MTC IoT Agent
+
+[![FIWARE IoT Agents](https://nexus.lab.fiware.org/repository/raw/public/badges/chapters/iot-agents.svg)](https://www.fiware.org/developers/catalogue/)
+[![](https://nexus.lab.fiware.org/repository/raw/public/badges/stackoverflow/iot-agents.svg)](https://stackoverflow.com/questions/tagged/fiware+iot)
+
+The OpenMTC SDK aims to provide developers with a convenient yet flexible tool
+to write oneM2M compliant applications. This includes network applications
+(NAs), gateway application (GAs), device applications (DAs), as well as
+interworking proxy entities (IPEs).

doc/subscription-notification.md

@@ -0,0 +1,190 @@
+# Subscriptions and Notifications
+
+## General Description of Subscription and Notification
+
+The subscription and notification mechanism can be used to track changes on a resource. Therefore, a subscription is created by the resource subscriber containing subscription information. An AE or CSE can be the resource subscriber. When the resource is modified, the hosting CSE of the resource is sending a notification for the subscriber to the address(es) specified in the subscription information.
+
+## The Subscription Resource
+
+A subscription is represented by a `<subscription>` resource as child resource of the subscribed-to resource. The subscription contains subscription information, most importantly:
+
+- the list `notificationURI`
+- and the attribute `subscriberURI`.
+
+The `notificationURI` list can contain one or more addresses where the resource subscriber wants to receive notifications. The attribute `subscriberURI` contains the address of the subscriber that is used by the CSE to send a notification when the `<subscription>` resource itself is deleted.
+
+The `<subscription>` resource is supported by resource types like:
+
+- CSEBase
+- remoteCSE
+- AE
+- container
+- contentInstance
+- accessControlPolicy
+- mgmtObj
+
+**Note:** The `<contentInstance>` resource does not include the `<subscription>` resource as a child resource. To keep track of changes of `<contentInstance>` resources a subscription in the corresponding parent `container` resource needs to be created.
+
+
+## Subscription and Notification using the Application Framework
+
+The OpenMTC Application Framework provides the following methods to create subscriptions and handle notifications.
+
+### The `add_container_subscription` method
+
+The Application Framework provides the `add_container_subscription` method to create a subscription to the contentInstances of a given container and specify a handler function that is called when a notification is send. Optionally, a delete handler and filter criteria can be specified.
+
+The method with its parameters is listed below:
+
+`add_container_subscription(self, container, handler, delete_handler=None, filter_criteria=None)`
+
+- `container`: container object or container path string
+- `handler`: reference of the notification handling function
+- `delete_handler`: (optional) reference to delete handling function
+- `filter_criteria`: (optional) filterCriteria for the subscription
+
+The parameters given to the `handler` function are:
+
+- the path of the subscribed-to container
+- the content data of the contentInstance which was created
+
+The parameter given to the `delete_handler` function is:
+
+- the path of the subscription
+
+
+### Example: Using the `add_container_subscription` method
+
+This example shows how the `add_container-subscription` method is used. The following code is part of the **training app** `onem2m-ipe-sensors-actuators-final` that can be found [here](./training/apps/onem2m/ipe/sensors-actuators/onem2m-ipe-sensors-actuators-final.py).
+```python
+# add commands_container of current actuator to self._command_containers
+self._command_containers[actuator] = commands_container
+# subscribe to command container of each actuator to the handler command
+self.add_container_subscription(
+    commands_container.path,    # the Container or it's path to be subscribed
+    self.handle_command         # reference of the notification handling function
+)
+```
+
+This code creates a subscription to the path of the `commands` container and registers the `handle_command` function as the notification handling function.
+
+
+The `handle_command` function of the example application looks like this.
+```python
+def handle_command(self, container, value):
+    print('handle_command...')
+    print('container: %s' % container)
+    print('value: %s' % value)
+    print('')
+```
+
+In this example the handling function simply prints the path of the container and the JSON data to the terminal.
+
+### Curl request for testing the `handle_command` notification handling function
+
+In the following the REST API is used to demonstrate the `handle_command` handling function of the example from above. Therefore, an HTTP POST request is send to the `commands` subcontainer of the `Switch-AirCon` container.  The request contains some JSON data that is pushed as a contentInstance to the container. The creation of the contentInstance triggers the notification to be send to the IPE, because of the subscription that was created with the `add_container_subscription` method. The `handle_command` function handles the notification.
+
+For this example, the following example data is send:
+```json
+{
+	"switch_state": "on"
+}
+```
+
+The data is encoded to its base64 string:
+```
+eyAic3dpdGNoX3N0YXRlIjogIm9uIiB9
+```
+
+To start the  `onem2m-ipe-sensors-actuators-final` training application, go to the training folder  `doc/training`:
+```bash
+$ cd doc/training
+```
+
+Run the **start-app** script to start one of the training applications:
+```bash
+$ `./start-app.sh`
+```
+
+Choose number `2` to start the `onem2m-ipe-sensors-actuators-final.py`
+
+
+Send the following curl request to create the contentInstance and trigger the notification to be send:
+```shell
+curl -X POST localhost:8000/onem2m/TestIPE/devices/Switch-AirCon/commands -H 'Content-Type: application/vnd.onem2m-res+json' -d '{"m2m:cin": {"con": "eyAic3dpdGNoX3N0YXRlIjogIm9uIiB9", "cnf":"application/json:1"}}'
+```
+
+When sending the request, the notification is triggered and handled by the `handle_command` handling function that prints the following output to the terminal:
+```
+handle_command_cnt...
+container: onem2m/TestIPE/devices/Switch-AirCon/commands
+value: {u'switch_state': u'on'}
+```
+
+### The `add_subscription` method
+
+This method creates a subscription resource at the given path and registers an handler to receive notification data. In comparison to the `add_container_subscription` method, the notification does not include the content data of a contentInstance, but information about the resource and subscription that triggered the notification.
+
+The method with its parameters is listed below:
+
+`add_subscription(self, path, handler, delete_handler=None)`
+
+- `path`: path to subscribe to
+- `handler`: notification handler
+- `delete_handler`: reference to delete handling function
+
+
+The notification that is send to the handling function includes the following three parameters:
+
+- the path of the subscription
+- the notification event type
+- the resource representation
+
+### Example: Using the `add_subscription` method
+
+This example again uses the **training app** `onem2m-ipe-sensors-actuators-final`, but with some modified code using the `add_subscription` method that creates a subscription to the `Switch-AirCon`container.
+```python
+self.add_subscription(
+    "onem2m/TestIPE/devices/Switch-AirCon",
+    self.handle_command
+)
+```
+
+The handle method is modified like this:
+
+```python
+def handle_command(self, sub, net, rep):
+    print('handle_command_general...')
+    print('subscription path: %s' % sub)
+    print('notification event type: %s' % net)
+    print('representation: %s' % rep)
+    print('')
+```
+
+The terminal output of the handle method is:
+```
+handle_command...
+subscription path: onem2m/TestIPE/devices/Switch-AirCon/subscription-J3VOG9vvCfg2ifiR
+notification event type: None
+representation: Container(path='None', id='cnt1')
+```
+As a result, the information of the resource representation from the notification can be used to perform further requests on the container resource.
+
+
+## Curl request to create a Subscription using the REST API
+
+A subscription can also be created using the REST API. Therefore, a POST request is send to the path where the subscription should be created.
+
+In the following this example shows a curl request that creates a subscription in the `Switch-AirCon`container of the `onem2m-ipe-sensors-actuators-final` training app from above.
+
+```shell
+curl -X POST localhost:8000/onem2m/TestIPE/devices/Switch-AirCon/commands -H 'Content-Type: application/vnd.onem2m-res+json;ty=23' -d '{  "m2m:sub": {  "nu": "TestIPE",  "rn":"sub101"  }}'
+```
+
+This POST request creates a subscription in the `commands` container at the URI `localhost:8000/onem2m/TestIPE/devices/Switch-AirCon/commands`. The `resource type (ty)` number `23` included in the Content-Type header field represents the resource type of a subscription resource.
+The subscription created by this curl request includes the `resource name (rn)` of the subscription which is `sub101` and the `notification URI (nu)` which is `TestIPE`, the AE-ID of the IPE. When the AE-ID is used, the CSE automatically replaces it with the appropriate absolute path of the application resource, which in this example is `["//openmtc.org/mn-cse-1/CTestIPE"]`.
+
+Likewise, the curl request can include a list of notification URIs:
+```shell
+curl -X POST localhost:8000/onem2m/TestIPE/devices/Switch-AirCon/commands -H 'Content-Type: application/vnd.onem2m-res+json;ty=23' -d '{  "m2m:sub": {  "nu":["//openmtc.org/mn-cse-1/CTestIPE"],  "rn":"sub101"  }}'
+```

doc/training/training-index.md

@@ -1,23 +1,23 @@
-# Write your first OpenMTC applications 
+# Write your first OpenMTC applications
 
 
 ## Introduction
-OpenMTC is delivered with some incremental demo applications which can be consulted for further understanding or as template for other applications. The oneM2M demo applications can be found in [/doc/training/apps/onem2m/](/doc/training/apps/onem2m/).
+OpenMTC is delivered with some incremental demo applications which can be consulted for further understanding or as template for other applications. The oneM2M demo applications can be found in the [OpenMTC Repository on Github](https://github.com/OpenMTC/OpenMTC/tree/master/doc/training/apps/onem2m).
 
 The training is subdivided into examples for GUI-applications and IPE-applications. Both of these provide incremental demo applications for either sensor-actuator-applications or only sensor-applications. These are explained in the following sections.
 
 
 ## The start-app Script
 
-The [training folder](/doc/training) contains the [start-app.sh](/doc/training/start-app.sh) script. This script allows to run one of the four complete demo applications:
+The [training folder](https://github.com/OpenMTC/OpenMTC/tree/master/doc/training) contains the [start-app.sh](https://github.com/OpenMTC/OpenMTC/blob/master/doc/training/start-app.sh) script. This script allows to run one of the four complete demo applications:
 
 ```sh
-user@host:/git$ ./openmtc-open-source/doc/training/start-app.sh 
+user@host:/git$ ./openmtc-open-source/doc/training/start-app.sh
 [1] onem2m-ipe-sensors-final.py
 [2] onem2m-ipe-sensors-actuators-final.py
 [3] onem2m-gui-sensors-final.py
 [4] onem2m-gui-sensors-actuators-final.py
-Choose the app to start: 
+Choose the app to start:
 ```
 
 ## Getting started
@@ -26,23 +26,23 @@ First of all, OpenMTC needs to be install on your system. If you have not yet in
 
 ### Sensor only demo applications
 
-To run the sensor only demo application, you will need four consoles. Proceed in the following order, since both the IPE and the GUI require a running CSE.  
+To run the sensor only demo application, you will need four consoles. Proceed in the following order, since both the IPE and the GUI require a running CSE.
 
 **Console 1:** Backend
 * start the Backend by executing the following:
 * `./openmtc-open-source/openmtc-gevent/run_backend`
-* After you started all four consoles, you should get something like [this](console-outputs/training-sensor-backend.md).  
+* After you started all four consoles, you should get something like [this](console-outputs/training-sensor-backend.md).
 
 **Console 2:** Gateway
 * start the Gateway by executing the following:
 * `./openmtc-open-source/openmtc-gevent/run_gateway`
-* After you started all four consoles, you should get something like [this](console-outputs/training-sensor-gateway.md).  
+* After you started all four consoles, you should get something like [this](console-outputs/training-sensor-gateway.md).
 
 **Console 3:** IPE
 * start the IPE by executing the following:
 * `./openmtc-open-source/doc/training/start-app.sh`
 * type `1`
-* you should get something like [this](console-outputs/training-sensor-ipe.md).  
+* you should get something like [this](console-outputs/training-sensor-ipe.md).
 
 **Console 4:** GUI
 * start the GUI by executing the following:
@@ -82,7 +82,7 @@ To run the sensor-actuator demo application, you will also need four consoles. P
 
 IPE stands for Interworking Proxy Application Entity. The IPE demo applications attaches (virtual) sensors and (virtual) actuators to the CSE by sending (simulated) sensors readings and receiving and processing commands meant for (virtual) actuators attached to the hardware this IPE demo app is running on.
 
-The [IPE demo applications](/doc/training/apps/onem2m/ipe/) are available as sensor-actuator-applications or only as sensor-applications.  
+The [IPE demo applications](https://github.com/OpenMTC/OpenMTC/tree/master/doc/training/apps/onem2m/ipe) are available as sensor-actuator-applications or only as sensor-applications.
 
 **Incremental IPE demo applications**
 * [ipe-sensors](training-ipe-sensors.md) provides virtual sensors
@@ -93,9 +93,8 @@ The [IPE demo applications](/doc/training/apps/onem2m/ipe/) are available as sen
 
 GUI stands for Graphical User Interface. This is somewhat misleading, as these demo apps do not provide a real GUI. These GUI demo applications rather provide a textual user interface which receives (and eventually displays to the user) the (virtual) sensor data provided by the ipe-demo-apps. Further, they send commands to the (virtual) actuators attached to the ipe-demo-apps.
 
-The [gui demo applications](/doc/training/apps/onem2m/gui/) are available as sensor-actuator-applications or only as sensor-applications.  
+The [GUI demo applications](https://github.com/OpenMTC/OpenMTC/tree/master/doc/training/apps/onem2m/gui) are available as sensor-actuator-applications or only as sensor-applications.
 
 **Incremental GUI demo applications**
-* [GUI-sensors](training-gui-sensors.md) receives and displays sensor data 
+* [GUI-sensors](training-gui-sensors.md) receives and displays sensor data
 * [GUI-sensors-actuators](training-gui-sensors-actuators.md) receives and displays sensor data and issues commands to actuators
-

docker/configure-backend-and-start

@@ -26,6 +26,10 @@ ONEM2M_HTTP_TRANSPORT_PORT=${ONEM2M_HTTP_TRANSPORT_PORT-18000}
 ONEM2M_HTTP_TRANSPORT_SSL_ENABLED=${ONEM2M_HTTP_TRANSPORT_SSL_ENABLED-false}
 ONEM2M_HTTP_TRANSPORT_REQUIRE_CERT=${ONEM2M_HTTP_TRANSPORT_REQUIRE_CERT-true}
 
+ONEM2M_MQTT_TRANSPORT_DISABLED=${ONEM2M_MQTT_TRANSPORT_DISABLED-true}
+ONEM2M_MQTT_TRANSPORT_PORT=${ONEM2M_MQTT_TRANSPORT_PORT-1883}
+ONEM2M_MQTT_TRANSPORT_INTERFACE=${ONEM2M_MQTT_TRANSPORT_INTERFACE-"localhost"}
+
 ONEM2M_NOTIFICATION_DISABLED=${ONEM2M_NOTIFICATION_DISABLED-true}
 
 # ensure correct level
@@ -74,7 +78,10 @@ JQ_STRING=${JQ_STRING}' |
     (.plugins.openmtc_cse[] | select(.name == "HTTPTransportPlugin") | .config.port) |= '${ONEM2M_HTTP_TRANSPORT_PORT}' |
     (.plugins.openmtc_cse[] | select(.name == "HTTPTransportPlugin") | .config.enable_https) |= '${ONEM2M_HTTP_TRANSPORT_SSL_ENABLED}' |
     (.plugins.openmtc_cse[] | select(.name == "HTTPTransportPlugin") | .config.require_cert) |= '${ONEM2M_HTTP_TRANSPORT_REQUIRE_CERT}' |
-    (.plugins.openmtc_cse[] | select(.name == "NotificationHandler") | .disabled) |= '${ONEM2M_NOTIFICATION_DISABLED}'
+    (.plugins.openmtc_cse[] | select(.name == "NotificationHandler") | .disabled) |= '${ONEM2M_NOTIFICATION_DISABLED}' |
+    (.plugins.openmtc_cse[] | select(.name == "MQTTTransportPlugin") | .disabled) |= '${ONEM2M_MQTT_TRANSPORT_DISABLED}' |
+    (.plugins.openmtc_cse[] | select(.name == "MQTTTransportPlugin") | .config.port) |= '${ONEM2M_MQTT_TRANSPORT_PORT}' |
+    (.plugins.openmtc_cse[] | select(.name == "MQTTTransportPlugin") | .config.interface) |= "'${ONEM2M_MQTT_TRANSPORT_INTERFACE}'"
 '
 
 cat ${CONFIG_FILE} | jq -M "${JQ_STRING}"> ${CONFIG_TEMP}

docker/configure-gateway-and-start

@@ -32,13 +32,19 @@ ONEM2M_HTTP_TRANSPORT_PORT=${ONEM2M_HTTP_TRANSPORT_PORT-8000}
 ONEM2M_HTTP_TRANSPORT_SSL_ENABLED=${ONEM2M_HTTP_TRANSPORT_SSL_ENABLED-false}
 ONEM2M_HTTP_TRANSPORT_REQUIRE_CERT=${ONEM2M_HTTP_TRANSPORT_REQUIRE_CERT-true}
 
+ONEM2M_MQTT_TRANSPORT_DISABLED=${ONEM2M_MQTT_TRANSPORT_DISABLED-true}
+ONEM2M_MQTT_TRANSPORT_PORT=${ONEM2M_MQTT_TRANSPORT_PORT-1883}
+ONEM2M_MQTT_TRANSPORT_INTERFACE=${ONEM2M_MQTT_TRANSPORT_INTERFACE-"localhost"}
+
 ONEM2M_NOTIFICATION_DISABLED=${ONEM2M_NOTIFICATION_DISABLED-true}
 
 ONEM2M_REGISTRATION_DISABLED=${ONEM2M_REGISTRATION_DISABLED-true}
 ONEM2M_REMOTE_CSE_ID=${ONEM2M_REMOTE_CSE_ID-"in-cse-1"}
 ONEM2M_REMOTE_CSE_POA=${ONEM2M_REMOTE_CSE_POA-"http://localhost:18000"}
 ${ONEM2M_HTTP_TRANSPORT_SSL_ENABLED} && SCHEME="https" || SCHEME="http"
-ONEM2M_REMOTE_CSE_OWN_POA="${SCHEME}://${HOST_NAME}:${ONEM2M_HTTP_TRANSPORT_PORT}"
+own_poa="${SCHEME}://${HOST_NAME}:${ONEM2M_HTTP_TRANSPORT_PORT}"
+ONEM2M_REMOTE_CSE_OWN_POA=${ONEM2M_REMOTE_CSE_OWN_POA-"${own_poa}"}
+unset own_poa
 ONEM2M_REMOTE_CSE_BASE=${ONEM2M_REMOTE_CSE_BASE-"onem2m"}
 ONEM2M_REMOTE_CSE_TYPE=${ONEM2M_REMOTE_CSE_TYPE-"IN-CSE"}
 
@@ -82,6 +88,9 @@ JQ_STRING=${JQ_STRING}' |
     (.plugins.openmtc_cse[] | select(.name == "HTTPTransportPlugin") | .config.port) |= '${ONEM2M_HTTP_TRANSPORT_PORT}' |
     (.plugins.openmtc_cse[] | select(.name == "HTTPTransportPlugin") | .config.enable_https) |= '${ONEM2M_HTTP_TRANSPORT_SSL_ENABLED}' |
     (.plugins.openmtc_cse[] | select(.name == "HTTPTransportPlugin") | .config.require_cert) |= '${ONEM2M_HTTP_TRANSPORT_REQUIRE_CERT}' |
+    (.plugins.openmtc_cse[] | select(.name == "MQTTTransportPlugin") | .disabled) |= '${ONEM2M_MQTT_TRANSPORT_DISABLED}' |
+    (.plugins.openmtc_cse[] | select(.name == "MQTTTransportPlugin") | .config.port) |= '${ONEM2M_MQTT_TRANSPORT_PORT}' |
+    (.plugins.openmtc_cse[] | select(.name == "MQTTTransportPlugin") | .config.interface) |= "'${ONEM2M_MQTT_TRANSPORT_INTERFACE}'" |
     (.plugins.openmtc_cse[] | select(.name == "NotificationHandler") | .disabled) |= '${ONEM2M_NOTIFICATION_DISABLED}' |
     (.plugins.openmtc_cse[] | select(.name == "RegistrationHandler") | .disabled) |= '${ONEM2M_REGISTRATION_DISABLED}' |
     (.plugins.openmtc_cse[] | select(.name == "RegistrationHandler") | .config.remote_cses) |= [{cse_id: "'${ONEM2M_REMOTE_CSE_ID}'", poa: ["'${ONEM2M_REMOTE_CSE_POA}'"], own_poa: ["'${ONEM2M_REMOTE_CSE_OWN_POA}'"], cse_base: "'${ONEM2M_REMOTE_CSE_BASE}'", cse_type: "'${ONEM2M_REMOTE_CSE_TYPE}'"}]

docker/sdk-amd64

@@ -10,6 +10,9 @@ ENV MOD_NAME=sdk
 # Set the file maintainer
 MAINTAINER rst/tgu
 
+# update pip to latest version
+RUN pip install --upgrade pip
+
 # install openmtc dependencies
 COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
 RUN pip install --upgrade --requirement /tmp/requirements.txt

docker/sdk-arm

@@ -10,6 +10,9 @@ ENV MOD_NAME=sdk
 # Set the file maintainer
 MAINTAINER rst/tgu
 
+# update pip to latest version
+RUN pip install --upgrade pip setuptools
+
 # install openmtc dependencies
 COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
 RUN pip install --upgrade --requirement /tmp/requirements.txt

ipes/CUL868IPE/src/cul868ipe/__init__.py

@@ -2,7 +2,7 @@
 Interworking Proxy for Cul868 devices.
 """
 
-__version__ = "4.9.9"
+__version__ = "1.2.0"
 __description__ = "The OpenMTC Cul868IPE"
 __author_name__ = "Ronny Kreuch"
 __author_mail__ = "ronny.kreuch@fokus.fraunhofer.de"

ipes/CUL868IPE/src/cul868ipe/cul_868_coordinator.py

@@ -25,7 +25,7 @@ class CUL868Coordinator(LoggerMixin):
     PROTOCOL_FS20 = "F"
     PROTOCOL_HMS = "H"
 
-    def __init__(self, device="/dev/ttyACM1"):
+    def __init__(self, device="/dev/ttyACM1", sim=True):
         super(CUL868Coordinator, self).__init__()
         self.running = False
         self.device = device
@@ -37,6 +37,7 @@ class CUL868Coordinator(LoggerMixin):
             "H": HMSParser()
         }
 
+        self.sim = sim
         self.sim_parsers = {
             "K": SIMParser(),
             # "E": SIMParser(),
@@ -140,10 +141,16 @@ class CUL868Coordinator(LoggerMixin):
         self.logger.debug("Command sent")
 
     def switch_on(self, house_code, device_code):
-        self._send_fs20(house_code, device_code, self.COMMAND_ON)
+        self.logger.info("Switch on %s-%s" % (house_code, device_code))
+        if not self.sim:
+            self._send_fs20(house_code, device_code, self.COMMAND_ON)
 
     def switch_off(self, house_code, device_code):
-        self._send_fs20(house_code, device_code, self.COMMAND_OFF)
+        self.logger.info("Switch off %s-%s" % (house_code, device_code))
+        if not self.sim:
+            self._send_fs20(house_code, device_code, self.COMMAND_OFF)
 
     def toggle(self, house_code, device_code):
-        self._send_fs20(house_code, device_code, self.COMMAND_TOGGLE)
+        self.logger.info("Toggle %s-%s" % (house_code, device_code))
+        if not self.sim:
+            self._send_fs20(house_code, device_code, self.COMMAND_TOGGLE)

ipes/CUL868IPE/src/cul868ipe/cul_868_ipe.py

@@ -42,7 +42,7 @@ class CUL868IPE(XAE):
 
         self._old_fs20_values = {}
 
-        self.cul = CUL868Coordinator(device=device)
+        self.cul = CUL868Coordinator(device=device, sim=sim)
 
         for d in map(lambda s: CULDevice(*s.split(":")[:2]), cul_devices):
             if d.type == "fs20":

mkdocs.yml

@@ -0,0 +1,52 @@
+site_name: Open MTC
+site_url: https://github.com/OpenMTC/OpenMTC
+repo_url: https://github.com/OpenMTC/OpenMTC.git
+site_description: FIWARE Open MTC IoT Agent document site
+docs_dir: doc/
+site_dir: html
+markdown_extensions: [toc,tables, fenced_code]
+use_directory_urls: false
+theme: readthedocs
+extra_css: ["https://fiware.org/style/fiware_readthedocs.css", "https://fiware.org/style/fiware_readthedocs_iot.css"]
+pages:
+    - Home: index.md
+    - 'User & Programmers Manual':
+      -   Introduction: introduction.md
+      -   Quick Start: openmtc-get-started.md
+      -   The MQTT Client: onem2m-client-mqtt.md
+      -   Authentication Guide: authentication.md
+      -   Overview REST API: overview-rest-api.md
+      -   Write your first OpenMTC applications: training/training-index.md 
+      -   Console Outputs:
+          -   Sensor Backend: training/console-outputs/training-sensor-backend.md
+          -   Sensor Gateway: training/console-outputs/training-sensor-gateway.md
+          -   Sensor IPE: training/console-outputs/training-sensor-ipe.md
+          -   Sensor GUI: training/console-outputs/training-sensor-gui.md
+          -   Sensor Actuator Backend: training/console-outputs/training-sensor-actuator-backend.md
+          -   Sensor Actuator Gateway: training/console-outputs/training-sensor-actuator-gateway.md
+          -   Sensor Actuator IPE: training/console-outputs/training-sensor-actuator-ipe.md
+          -   Sensor Actuator GUI: training/console-outputs/training-sensor-actuator-gui.md
+      -   Training:
+          -   IPE Sensors: training/training-ipe-sensors.md 
+          -   IPE Sensors Actuators: training/training-ipe-sensors-actuators.md 
+          -   GUI Sensors: training/training-gui-sensors.md 
+          -   GUI Sensors Actuators: training/training-gui-sensors-actuators.md
+      -   SDK - Using the Application Framework: sdk-framework.md
+      -   SDK - The low-level CSE Client: sdk-client.md
+      -   SDK - The Data Model: sdk-datamodel.md
+      -   Examples: 
+          -   IoT Data Visualization: example-apps/IoT-data-visualization.py
+          -   Data Aggregation: example-apps/data-aggregation.py
+          -   Simple Decision: example-apps/simple-decision.py
+          -   Simple Decision 2: example-apps/simple-decision-2.py
+      -   Scripts:
+          -   Create App Structure Script: create-app-structure.md
+          -   Create binary docker images Script: create-binary-docker.md
+      -   Code Repository Structure: repository-structure.md
+      -   Developer FAQ: developer-faq.md
+    - 'Installation & Administration Manual': 
+       -   Deployment: deployment-guide.md
+       -   Installation of the OpenMTC SDK: install-sdk.md
+    - Appendix : various.md
+    - Resource Names & Types : reference-doc/resource-names-and-types.md
+    - Gateway & Backend Configuration : reference-doc/gateway-and-backend-configuration.md

openmtc-app/src/openmtc_app/notification/__init__.py

@@ -205,8 +205,9 @@ class MqttNotificationHandler(BaseNotificationHandler):
         from openmtc_onem2m.exc import get_response_status
 
         def wrapper(request):
-            notification = self._unpack_notification(request.content)
-            self._callback(request.originator, **notification)
+            if not request.content.verificationRequest:
+                notification = self._unpack_notification(request.content)
+                self._callback(request.originator, **notification)
             return OneM2MResponse(status_code=get_response_status(2000), request=request)
 
         self._client = get_client(self._endpoint.geturl(), handle_request_func=wrapper)
@@ -255,9 +256,17 @@ class HttpNotificationHandler(BaseNotificationHandler):
             assert 'x-m2m-ri' in request.headers, 'Missing request id'
             assert 'content-type' in request.headers, 'Unspecified content type'
 
+            if not request.data:
+                if request.environ.get('HTTP_TRANSFER_ENCODING') == 'chunked':
+                    request.data = request.environ['wsgi.input'].read()
+                else:
+                    cl = int(request.environ.get('CONTENT_LENGTH'), 0)
+                    request.data = request.environ['wsgi.input'].read(cl)
+
             notification = get_onem2m_decoder(request.content_type).decode(request.data)
-            notification = self._unpack_notification(notification)
-            self._callback(request.headers['x-m2m-origin'], **notification)
+            if not notification.verificationRequest:
+                notification = self._unpack_notification(notification)
+                self._callback(request.headers['x-m2m-origin'], **notification)
 
             return Response(
                 headers={

openmtc-app/src/openmtc_app/onem2m.py

@@ -547,7 +547,7 @@ class XAE(LoggerMixin):
         """
         return self._get_content_from_cin(
             self.mapper.get(
-                getattr(container, 'path', container) + '/latest'
+                getattr(container, 'path', container) + '/la'
             )
         )
 
@@ -765,6 +765,7 @@ class ResourceManagementXAE(XAE):
     def _discover_openmtc_ipe_entities(self):
         # connected to backend or gateway?
         cse_base = self.get_resource(self.cse_base)
+        self._cse_id = cse_base.CSE_ID
         self.logger.debug("CSE_BASE: %s", cse_base)
 
         if cse_base.cseType in (CSETypeIDE.MN_CSE, CSETypeIDE.AEN_CSE):
@@ -832,24 +833,25 @@ class ResourceManagementXAE(XAE):
             except IndexError:
                 continue
             sensor = self.get_resource(sensor_path)
-            sensor_info = self._discovered_sensors[sensor_path] = {
-                'ID': sensor_path,
-                'dev_name': dev_path.split('/')[-1],
-                'cse_id': sensor_path.split('/')[1],
-                'dev_labels': self._discovered_devices[dev_path].labels,
-                'sensor_labels': sensor.labels,
-                'type': 'sensor',
-                'n': None,
-                'u': None,
-                'blacklisted': False
-            }
-            if self._sensor_filter(sensor_info):
-                self._handle_new_sensor(sensor_path)
-            else:
-                self._discovered_sensors[sensor_path]['blacklisted'] = True
+            if sensor:
+                sensor_info = self._discovered_sensors[sensor_path] = {
+                    'ID': sensor_path,
+                    'dev_name': dev_path.split('/')[-1],
+                    'cse_id': sensor_path.split('/')[1],
+                    'dev_labels': self._discovered_devices[dev_path].labels,
+                    'sensor_labels': sensor.labels,
+                    'type': 'sensor',
+                    'n': None,
+                    'u': None,
+                    'blacklisted': False
+                }
+                if self._sensor_filter(sensor_info):
+                    self._handle_new_sensor(sensor_path)
+                else:
+                    self._discovered_sensors[sensor_path]['blacklisted'] = True
 
     def _handle_new_sensor(self, sensor_path):
-        latest = self.get_resource(sensor_path + '/latest')
+        latest = self.get_resource(sensor_path + '/la')
         if latest:
             spawn(self._handle_sensor_data, sensor_path, self._get_content_from_cin(latest))
 
@@ -859,6 +861,8 @@ class ResourceManagementXAE(XAE):
         self._discovered_sensors[sensor_path]['sub_ref'] = sub_ref
 
     def _handle_delete(self, sub_ref):
+        if sub_ref[0] != '/':
+            sub_ref = self._cse_id + '/' + sub_ref
         self._discovered_sensors = {k: v for k, v in self._discovered_sensors.items()
                                     if v['sub_ref'] != sub_ref}
         self._discovered_devices = {k: v for k, v in self._discovered_devices.items()
@@ -890,15 +894,16 @@ class ResourceManagementXAE(XAE):
             except IndexError:
                 continue
             actuator = self.get_resource(actuator_path)
-            actuator_info = self._discovered_actuators[actuator_path] = {
-                'ID': actuator_path,
-                'dev_name': dev_path.split('/')[-1],
-                'cse_id': actuator_path.split('/')[1],
-                'dev_labels': self._discovered_devices[dev_path].labels,
-                'actuator_labels': actuator.labels,
-                'type': 'actuator'
-            }
-            self._new_actuator(actuator_info)
+            if actuator:
+                actuator_info = self._discovered_actuators[actuator_path] = {
+                    'ID': actuator_path,
+                    'dev_name': dev_path.split('/')[-1],
+                    'cse_id': actuator_path.split('/')[1],
+                    'dev_labels': self._discovered_devices[dev_path].labels,
+                    'actuator_labels': actuator.labels,
+                    'type': 'actuator'
+                }
+                self._new_actuator(actuator_info)
 
     @abstractmethod
     def _sensor_data_cb(self, sensor_info, sensor_data):

openmtc-gevent/dependencies.txt

@@ -4,7 +4,7 @@
 # or   $ sudo pip install --requirement dependencies.txt
 
 urllib3
-gevent>=1.0
+gevent>=1.1
 iso8601>=0.1.5
 werkzeug>=0.9
 funcy

openmtc-gevent/etc/conf/config-backend.json.dist

@@ -53,6 +53,15 @@
                     "require_cert": true
                 }
             },
+            {
+                "name": "MQTTTransportPlugin",
+                "package": "openmtc_cse.plugins.transport_gevent_mqtt",
+                "disabled": true,
+                "config": {
+                    "interface": "localhost",
+                    "port": 1883
+                }
+            },
             {
                 "name": "NotificationHandler",
                 "package": "openmtc_cse.plugins.notification_handler",

openmtc-gevent/etc/conf/config-gateway.json.dist

@@ -53,6 +53,15 @@
                     "require_cert": true
                 }
             },
+            {
+                "name": "MQTTTransportPlugin",
+                "package": "openmtc_cse.plugins.transport_gevent_mqtt",
+                "disabled": true,
+                "config": {
+                    "interface": "localhost",
+                    "port": 1883
+                }
+            },
             {
                 "name": "NotificationHandler",
                 "package": "openmtc_cse.plugins.notification_handler",

openmtc-gevent/requirements.txt

@@ -0,0 +1 @@
+dependencies.txt

server/openmtc-cse/src/openmtc_cse/methoddomain/__init__.py

@@ -13,6 +13,12 @@ from openmtc_server.exc import ConfigurationError
 from openmtc_server.util import log_error
 
 
+container_virtual_mapping = {
+    'la': 'latest',
+    'ol': 'oldest'
+}
+
+
 class OneM2MMethodDomain(Component):
     def __init__(self, config, *args, **kw):
         super(OneM2MMethodDomain, self).__init__(*args, **kw)
@@ -209,11 +215,11 @@ class OneM2MMethodDomain(Component):
         # oldest, latest -> Container
         # TODO(rst): fanOutPoint -> group
         # TODO(rst): pollingChannelURI -> pollingChannel
-        if path.endswith(('latest', 'oldest')):
+        if path.endswith(tuple(container_virtual_mapping.keys())):
             parent_path, virtual = path.rsplit('/', 1)
             parent = get_resource(parent_path)
             if isinstance(parent, model.Container):
-                resource = getattr(parent, virtual)
+                resource = getattr(parent, container_virtual_mapping[virtual])
                 if resource is None:
                     raise CSENotFound()
                 return resource

server/openmtc-cse/src/openmtc_cse/methoddomain/controller/__init__.py

@@ -22,14 +22,16 @@ from openmtc_onem2m.exc import (CSEOperationNotAllowed, STATUS_OK, CSETypeError,
                                 CSEMissingValue, CSEValueError, STATUS_CREATED,
                                 CSEError, CSESyntaxError, CSEBadRequest,
                                 CSEPermissionDenied, STATUS_NOT_FOUND, CSEConflict,
-                                CSEContentsUnacceptable, CSETargetNotReachable)
+                                CSEContentsUnacceptable, CSETargetNotReachable,
+                                STATUS_UPDATED, STATUS_DELETED)
 from openmtc_onem2m.model import (ExpiringResource, Notification,
                                   AccessControlOperationE, ResourceTypeE,
                                   NotificationContentTypeE, FilterUsageE,
                                   get_short_resource_name, URIList,
                                   DiscResTypeE, Container, AccessControlPolicy,
                                   AccessControlPolicyIDHolder, AccessControlRuleC,
-                                  DynAuthDasRequestC, SecurityInfo, SecurityInfoTypeE, AE)
+                                  DynAuthDasRequestC, SecurityInfo, SecurityInfoTypeE,
+                                  AE, ResultContentE, ContentInstance)
 from openmtc_onem2m.transport import (OneM2MResponse, OneM2MRequest,
                                       OneM2MOperation, OneM2MErrorResponse)
 from openmtc_onem2m.util import split_onem2m_address
@@ -51,9 +53,22 @@ class OneM2MDefaultController(LoggerMixin):
     def __init__(self, db_session, resource_type, handle_onem2m_request):
         super(OneM2MDefaultController, self).__init__()
         self.resource_type = resource_type
-        self.db_session = db_session
         self.handle_onem2m_request = handle_onem2m_request
 
+        # DB wrapper
+
+        def _update(resource, fields=None):
+            if isinstance(resource, (Container, ContentInstance)):
+                resource.stateTag += 1
+            return db_session.update(resource, fields)
+        self._create = db_session.store
+        self._get = db_session.get
+        self._update = _update
+        self._delete = db_session.delete
+        self._get_collection = db_session.get_collection
+        self._get_latest_content_instance = db_session.get_latest_content_instance
+        self._get_oldest_content_instance = db_session.get_oldest_content_instance
+
     def __call__(self, request, target_resource):
         self.logger.debug("%s servicing request", type(self).__name__)
 
@@ -91,6 +106,10 @@ class OneM2MDefaultController(LoggerMixin):
         self._dynamic_authorization_supported = dynamic_authorization.get('enabled', False)
         self._dynamic_authorization_poa = dynamic_authorization.get('poa', [])
 
+        # release version indicator
+        if not self.request.rvi:
+            self.request.rvi = '2a'
+
         return self._handle_request()
 
     def _handle_request(self):
@@ -200,7 +219,7 @@ class OneM2MDefaultController(LoggerMixin):
         self._check_privileges(parent)
 
     def _get_parent_of_resource(self, resource):
-        return self.db_session.get(resource.parentID)
+        return self._get(resource.parentID)
 
     def _check_privileges(self, resource):
         # get all ACPs
@@ -374,7 +393,7 @@ class OneM2MDefaultController(LoggerMixin):
         try:
             for dac_id in resource.dynamicAuthorizationConsultationIDs:
                 try:
-                    dac = self.db_session.get(dac_id)
+                    dac = self._get(dac_id)
                     if dac.dynamicAuthorizationEnabled:
                         return dac
                 except DBNotFound:
@@ -384,7 +403,7 @@ class OneM2MDefaultController(LoggerMixin):
         except AttributeError:
             return None
         else:
-            return self._get_dynamic_authorization_consultation(self.db_session.get(pid))
+            return self._get_dynamic_authorization_consultation(self._get(pid))
 
     def _create_dynamic_policy(self, resource, acp_info):
         acp = AccessControlPolicy(
@@ -407,7 +426,7 @@ class OneM2MDefaultController(LoggerMixin):
         resp = self.api.handle_onem2m_request(req).get()
         dyn_acp_id = resp.content.resourceID
         resource.accessControlPolicyIDs.append(dyn_acp_id)
-        self.db_session.update(resource, ['accessControlPolicyIDs'])
+        self._update(resource, ['accessControlPolicyIDs'])
 
     def _perform_evaluation(self, policies, privilege_type):
 
@@ -725,7 +744,7 @@ class OneM2MDefaultController(LoggerMixin):
 
         self.resource = resource
 
-        return self.db_session.store(resource)
+        return self._create(resource)
 
     def _set_resource_id(self, values):
         short_name = get_short_resource_name(self.resource_type.typename)
@@ -789,6 +808,12 @@ class OneM2MDefaultController(LoggerMixin):
         self.fields = self.request.content and self.request.content.values
 
     def _handle_retrieve(self):
+        try:
+            self.request.rcn = ResultContentE(int(self.request.rcn))
+        except ValueError:
+            raise CSEBadRequest()
+        except TypeError:
+            self.request.rcn = ResultContentE.attributes
         try:
             fu = self.request.filter_criteria.filterUsage
         except AttributeError:
@@ -857,7 +882,7 @@ class OneM2MDefaultController(LoggerMixin):
                 self.logger.debug("is resource '%s' virtual? -> %s", s.name,
                                   s.virtual)
                 if not s.virtual:
-                    sub_node = self.db_session.get(s.path)
+                    sub_node = self._get(s.path)
                     self._do_discovery(sub_node)
 
     def _prepare_resource(self):
@@ -872,17 +897,22 @@ class OneM2MDefaultController(LoggerMixin):
         if self.fields and isinstance(self.fields, list):
             res.set_values({k: v if k in self.fields else None for
                             k, v in res.get_values().items()})
-        return self._retrieve_children()
+
+        if self.request.rcn == ResultContentE.attributes_and_child_resource_references:
+            self._retrieve_children()
 
     def _send_retrieve_response(self):
-        return OneM2MResponse(STATUS_OK, pc=self.result, request=self.request)
+        fields = self.resource.values.keys()
+        if self.request.rcn != ResultContentE.attributes_and_child_resource_references:
+            fields = [k for k in fields if k != 'childResource']
+        return OneM2MResponse(STATUS_OK, pc=self.result, request=self.request, fields=fields)
 
     def _retrieve_children(self):
         return self._retrieve_children_for_resource(self.resource)
 
     def _retrieve_children_for_resource(self, resource):
         self.logger.debug("getting children of: %s", resource)
-        children = self.db_session.get_collection(None, resource)
+        children = self._get_collection(None, resource)
         resource.childResource = children
 
     # UPDATE
@@ -988,8 +1018,8 @@ class OneM2MDefaultController(LoggerMixin):
 
         # self.resource = resource
 
-        return self.db_session.update(self.resource)
-        # return self.db_session.update(resource, values.keys())
+        return self._update(self.resource)
+        # return self._update(resource, values.keys())
 
     def _set_mandatory_update_attributes(self, values):
         values["lastModifiedTime"] = self.now
@@ -1000,7 +1030,7 @@ class OneM2MDefaultController(LoggerMixin):
                                      self.request)
 
     def _send_update_response(self):
-        return OneM2MResponse(STATUS_OK, pc=self.resource,
+        return OneM2MResponse(STATUS_UPDATED, pc=self.resource,
                               request=self.request)
 
     # DELETE
@@ -1014,14 +1044,14 @@ class OneM2MDefaultController(LoggerMixin):
         return self._send_delete_response()
 
     def _get_parent(self):
-        self.parent = self.db_session.get(self.resource.parent_path)
+        self.parent = self._get(self.resource.parent_path)
 
     def _delete_resource(self):
         self._delete_children()
         self._do_delete_resource()
 
     def _do_delete_resource(self):
-        return self.db_session.delete(self.resource)
+        return self._delete(self.resource)
 
     def _delete_children(self):
         self._retrieve_children()
@@ -1045,7 +1075,7 @@ class OneM2MDefaultController(LoggerMixin):
         self.events.resource_deleted.fire(self.resource, self.request)
 
     def _send_delete_response(self):
-        return OneM2MResponse(STATUS_OK, request=self.request)
+        return OneM2MResponse(STATUS_DELETED, request=self.request)
 
 
 # see TS-0004 7.4.4
@@ -1115,7 +1145,7 @@ class AEController(OneM2MDefaultController):
             ae_id = get_generic_ae_id()
 
         try:
-            self.db_session.get(ae_id)
+            self._get(ae_id)
         except DBNotFound:
             pass
         else:
@@ -1283,17 +1313,14 @@ class ContentInstanceController(OneM2MDefaultController):
     def _create_resource(self):
         super(ContentInstanceController, self)._create_resource()
 
-        # handle_old_instances
-        max_nr_of_instances = self.parent.maxNrOfInstances
-        current_nr_of_instances = self.parent.currentNrOfInstances
-        if 0 < max_nr_of_instances <= current_nr_of_instances:
+        def remove_oldest_child():
             self.parent.currentNrOfInstances -= 1
             self.parent.currentByteSize -= self.parent.oldest.contentSize
 
-            self.db_session.delete(self.parent.oldest)
+            self._delete(self.parent.oldest)
 
             if self.parent.currentNrOfInstances >= 1:
-                oldest = self.db_session.get_oldest_content_instance(
+                oldest = self._get_oldest_content_instance(
                     self.parent)
                 self.logger.debug("Setting new oldest: %s", oldest)
                 self.parent.oldest = oldest
@@ -1301,6 +1328,14 @@ class ContentInstanceController(OneM2MDefaultController):
                 self.logger.debug("Setting oldest to None")
                 self.parent.oldest = None
 
+        # handle_old_instances
+        if 0 < self.parent.maxNrOfInstances <= self.parent.currentNrOfInstances:
+            remove_oldest_child()
+
+        while (0 < self.parent.maxByteSize <
+               self.parent.currentByteSize + self.resource.contentSize):
+            remove_oldest_child()
+
         # handle_new_instance
         self.parent.currentNrOfInstances += 1
         self.parent.currentByteSize += self.resource.contentSize
@@ -1309,7 +1344,13 @@ class ContentInstanceController(OneM2MDefaultController):
                               self.resource)
             self.parent.oldest = self.resource
         self.parent.latest = self.resource
-        self.db_session.update(self.parent)
+        self._update(self.parent)
+
+    def _check_create_representation(self):
+        super(ContentInstanceController, self)._check_create_representation()
+
+        if self.request.content.resourceName in ('la', 'ol'):
+            raise CSEConflict()
 
     def _set_mandatory_create_attributes(self, vals):
         self.request.name = None
@@ -1323,21 +1364,23 @@ class ContentInstanceController(OneM2MDefaultController):
     def _delete_resource(self):
         super(ContentInstanceController, self)._delete_resource()
 
-        cnt = self.db_session.get(self.resource.parentID)
+        cnt = self._get(self.resource.parentID)
         # TODO(rst): handle byte size
         try:
-            ci_l = self.db_session.get_latest_content_instance(cnt)
-            ci_o = self.db_session.get_oldest_content_instance(cnt)
+            ci_l = self._get_latest_content_instance(cnt)
+            ci_o = self._get_oldest_content_instance(cnt)
         except (DBError, KeyError):
             cnt.latest = None
             cnt.oldest = None
             cnt.currentNrOfInstances = 0
+            cnt.currentByteSize = 0
         else:
             cnt.latest = ci_l
             cnt.oldest = ci_o
             cnt.currentNrOfInstances -= 1
+            cnt.currentByteSize -= self.resource.contentSize
 
-        return self.db_session.update(cnt)
+        return self._update(cnt)
 
 
 class AccessControlPolicyController(OneM2MDefaultController):

server/openmtc-cse/src/openmtc_cse/plugins/transport_gevent_http/wsgi.py

@@ -6,7 +6,7 @@ from operator import itemgetter
 from socket import AF_INET, AF_INET6, getaddrinfo, SOCK_STREAM, inet_pton
 
 from funcy import pluck
-from gevent.wsgi import WSGIHandler, WSGIServer
+from gevent.pywsgi import WSGIHandler, WSGIServer
 from werkzeug.wrappers import (BaseRequest, CommonRequestDescriptorsMixin,
                                UserAgentMixin, AcceptMixin, Response)
 
@@ -249,11 +249,21 @@ class OpenMTCWSGIApplication(LoggerMixin):
         #  request and response primitives, and vice versa, if applicable.
         ec = get_header("x-m2m-ec")
 
+        # The X-M2M-CTS header shall be mapped to the Content Status parameter
+        # of response primitives and vice versa, if applicable.
+        rvi = get_header("x-m2m-rvi")
+
+        # The X-M2M-CTS header shall be mapped to the Content Status parameter
+        # of response primitives and vice versa, if applicable.
+        vsi = get_header("x-m2m-vsi")
+
         onem2m_request = OneM2MRequest(op=op, to=to, fr=fr, rqi=rqi, ty=ty,
                                        pc=pc, ot=ot, rqet=rqet, rset=rset,
-                                       oet=oet, rt=rt, ec=ec, gid=gid)
+                                       oet=oet, rt=rt, ec=ec, gid=gid, rvi=rvi,
+                                       vsi=vsi)
 
-        not_filter_params = ('rt', 'rp', 'rcn', 'da', 'drt')
+        not_filter_params = ('rt', 'rp', 'rcn', 'da', 'drt', 'rids', 'tids',
+                             'ltids', 'tqi')
         multiple_params = ('lbl', 'ty', 'cty', 'atr')
 
         if http_request.query_string:
@@ -315,9 +325,18 @@ class OpenMTCWSGIApplication(LoggerMixin):
 
         headers = {
             "x-m2m-ri": str(response.rqi),
-            "x-m2m-rsc": str(response.rsc)
+            "x-m2m-rsc": str(response.rsc),
+            'x-m2m-rvi': str(response.rvi)
         }
 
+        if response.fr:
+            headers['x-m2m-origin'] = str(response.fr)
+
+        response_fields = ['ot', 'rset', 'ec', 'cts', 'cto', 'vsi']
+        for f in response_fields:
+            if getattr(response, f):
+                headers['x-m2m-%s' % f] = str(getattr(response, f))
+
         try:
             headers['Content-Location'] = (resource_id_pre + response.content.resourceID)
         except (AttributeError, TypeError):
@@ -339,7 +358,8 @@ class OpenMTCWSGIApplication(LoggerMixin):
         try:
             content_type, payload = encode_onem2m_content(response.content,
                                                           accept, pretty,
-                                                          path=resource_id_pre)
+                                                          path=resource_id_pre,
+                                                          fields=response.fields)
         except CSEContentsUnacceptable as e:
             status_code = e.status_code
             content_type = "text/plain"
@@ -369,7 +389,8 @@ class OpenMTCWSGIApplication(LoggerMixin):
 
             headers = {
                 "x-m2m-ri": str(response.rqi),
-                "x-m2m-rsc": str(response.rsc)
+                "x-m2m-rsc": str(response.rsc),
+                'x-m2m-rvi': str(response.rvi)
             }
         except AttributeError:
             status_code = STATUS_INTERNAL_SERVER_ERROR.http_status_code

setup-gevent-all.py

@@ -13,7 +13,7 @@ from utils import (get_packages, OpenMTCSdist, OpenMTCBuildPy,
 
 # name and version
 SETUP_NAME = "openmtc-all"
-SETUP_VERSION = "4.99.0"
+SETUP_VERSION = "1.2.0"
 SETUP_DESCRIPTION = "The OpenMTC Backend and Gateway (GEvent version)"
 
 # meta
@@ -29,7 +29,7 @@ SETUP_REQUIRES = [
     "flask", "pyxb (==1.2.3)", "enum34", "dtls", "geventhttpclient",
     # server only
     "funcy", "netifaces", "decorator", "mimeparse", "coapthon", "rdflib",
-    "fyzz", "yapps"
+    "fyzz", "yapps", "paho_mqtt"
 ]
 SETUP_INSTALL_REQUIRES = [
     "urllib3", "gevent >= 1.0", "iso8601 >= 0.1.5", "werkzeug >= 0.9",
@@ -37,7 +37,7 @@ SETUP_INSTALL_REQUIRES = [
     "flask", "pyxb == 1.2.3", "enum34", "dtls", "geventhttpclient",
     # server only
     "funcy", "netifaces", "decorator", "mimeparse", "coapthon", "rdflib",
-    "fyzz", "yapps"
+    "fyzz", "yapps", "paho_mqtt"
 ]
 
 # packages