Compare commits

...

78 Commits

Author SHA1 Message Date
Jason Fox
db06cd73bd Fix Link Box + Standardize Badge order (#33)
* Fix Link Box + Standardize Badge order

* Correct link
2019-07-16 17:25:33 +02:00
aor-fokus
e1afaf9c3c added training section and changed name of contributions to credits 2019-04-08 15:36:51 +02:00
aor-fokus
94682098ea
Update issue templates (#31) 2019-02-14 15:55:20 +01:00
aor-fokus
1052fd4a08
Release 1.3 (#30)
* Test debian stretch+python3 (#18)

* changes starting with python3 explicit

* removes python modules which are not available for python3

* exchanges fyzz query parsing with rdflib functionality

* fixes interop tests

* replaces reduce with for loop in nodb driver

* simple python2 -> python3 conversions

* adds changes for handling different string handling in python3

* test stretch building with travis

* installing python-setuptools in docker

* installing python-setuptools in docker

* changing python2 to python3 in docker makefiles

* changing python2 to python3 and some other test changes

* push docker only in master branche

* running version of openmtc

* fix some port problems

* porting path library completly now

* restoring travis.yml

* testing new travis.yml

* add sudo

* updating travis OS from trusty to xenial

* upgrade pip before

* show running docker logs

* show more logs

* for debugging

* showlogs of docker after failure

* testing new travis.yml

* finish travis.yml

* Adding roadmap (#26)

* adding roadmap

* adding a nicer view for some documents

* creating contributions.md (#27)

* travis only building on master branch (#25)

* deleting some typo

* another typo

* adding a contributer

* bump version to 1.3.0

* better link for contributions

* Port and fix simple apps

* add version tag
2019-02-14 15:41:53 +01:00
aor-fokus
c3ac4f9dfb fixing type of timestamp 2019-02-05 16:23:17 +01:00
aor-fokus
7781b15c4e
Adding Contributions to readme 2019-02-05 15:51:35 +01:00
aor-fokus
b9034a916f
Coverage badge 2019-02-05 12:30:43 +01:00
aor-fokus
5257a11cf0
Adding vulnerabilities (snyk) badge to README 2019-01-25 17:51:30 +01:00
aor-fokus
461698b32a Better Workaround for Snyk to find requirements.txt 2019-01-25 17:48:05 +01:00
aor-fokus
734a51fb73 Workaround for Snyk to find requirements.txt 2019-01-25 17:43:58 +01:00
Ronald Steinke
8836af62a6 Merge remote-tracking branch 'origin/master' 2019-01-11 17:58:38 +01:00
Ronald Steinke
ea4745296e bump version to 1.2.0 2019-01-11 17:55:26 +01:00
Ronald Steinke
b75be75384 fixes typo 2019-01-11 17:52:58 +01:00
aor-fokus
9b73c6d337
Added docker pulls badge 2019-01-09 16:09:47 +01:00
aor-fokus
c737effe0e New Chapter for the configuration of gateway&backend 2019-01-09 16:06:33 +01:00
aor-fokus
58560b7573
Added Travis badge to readme 2019-01-09 15:49:58 +01:00
Ronald Steinke
fbd7d987e8
Merge pull request #14 from jason-fox/feature/readme+docs
Update README and add mkdocs.yml
2018-12-19 15:24:41 +01:00
Jason Fox
3fc8bdb14e Update to standard-readme format
* Moved badges to top - two rows
* Formatted Markdown file
* Added lede at the top of file so a description paragraph will display on mobile
* Standardized some Section names (e.g. Install)
* Added standard QA section - TSC **must** requirement
* Expanded License to include copyrights
* Updated catalogue links to new GitHub location

[standard-readme](https://github.com/RichardLitt/standard-readme) is a simple standard naming convention for README files which makes reading and navigation of the documentation easier - this is a TSC **should** requirement

Created Readthedocs index file.
2018-10-30 16:18:37 +01:00
Ronald Steinke
61e688f921 adds basic interoperability testing 2018-10-05 17:18:06 +02:00
Ronald Steinke
06957bb967 updates SDK to use la instead of latest 2018-10-05 15:38:27 +02:00
Ronald Steinke
6cb4ee3d41 handles release version indicator and updates request/response attributes 2018-10-05 15:33:20 +02:00
Ronald Steinke
8cf2ba09ae adds correct resultContent handling 2018-10-05 15:31:05 +02:00
Ronald Steinke
1647abd1c1 updates resourceTypeE and memberTypeE in model 2018-10-05 15:08:35 +02:00
Ronald Steinke
2bfad1e360 updates to correct time format 2018-10-05 11:45:37 +02:00
Ronald Steinke
f624ac1fe5 fixes chunked data receiving in notification http server 2018-10-05 11:44:56 +02:00
Ronald Steinke
5c83578524 handles verification request correctly 2018-10-05 11:44:13 +02:00
Ronald Steinke
193778dec0 fixes contentInstance handling 2018-10-05 11:43:04 +02:00
Ronald Steinke
15f5aac79a handles stateTag attribute correctly 2018-10-05 11:42:03 +02:00
Ronald Steinke
dfbae4880b changes stateTag to be integer 2018-10-05 11:30:42 +02:00
Ronald Steinke
bc819069f2 fixes virtual container parts addressing 2018-10-05 10:48:50 +02:00
Ronald Steinke
6f923ec2cd fixes headers and return codes 2018-10-05 10:44:46 +02:00
Ronald Steinke
7f2889ed5f small fixes for orion app when run without OCB 2018-10-05 10:40:33 +02:00
Ronald Steinke
4e29b4a04d updates version to 1.1.0 2018-10-02 17:44:55 +02:00
Christian Klopp
d26053456c ocb doc: fix broken curl commands 2018-07-03 14:03:09 +02:00
Christian Klopp
294d5469c8 fix travis build 2018-06-20 17:02:10 +02:00
Christian Klopp
482a2a0f3c rename influxdb application to influxdbApp in order to avoid confilicts on imports 2018-06-20 12:45:31 +02:00
Christian Klopp
99e895fe39 fix ocb app doc 2018-06-20 12:42:57 +02:00
Ronald Steinke
0597cc409b fix broken links 2018-06-18 14:57:40 +02:00
Ronald Steinke
71e6b5c175 fixes mqtt docker start procedure 2018-06-18 14:51:34 +02:00
Ronald Steinke
3c11330c22 adds more apps to travis 2018-06-18 14:49:44 +02:00
Ronald Steinke
1d4098f7a6 addition to Orion AE 2018-06-18 14:48:50 +02:00
Ronald Steinke
669566a119 aligns docker files 2018-06-18 14:48:01 +02:00
Christian Klopp
20eff3a239 mqtt connector doc: minor enhancements 2018-06-12 16:08:00 +02:00
Christian Klopp
1e2491c7e5 doc: fix broken link 2018-06-12 16:05:53 +02:00
Christian Klopp
a8e017f218 Merge branch 'master' of https://github.com/OpenMTC/OpenMTC 2018-06-12 15:01:08 +02:00
Christian Klopp
8552786c71 add mqtt connector app to travis 2018-06-12 14:53:54 +02:00
Christian Klopp
d0dfabf61f adds documentation for subscription and notification 2018-06-12 14:33:35 +02:00
Christian Klopp
abc8b618db add new mqtt connector app 2018-06-12 14:32:51 +02:00
Christian Klopp
7dc51ee940 changes http client to open a new connection for every request 2018-06-12 14:32:19 +02:00
Christian Klopp
808027db5e openmtc-app/src/openmtc_app/onem2m.py 2018-06-12 14:31:27 +02:00
Christian Klopp
c8ef11a306 adds ability to set own_poa explicitly in gateway docker containers 2018-06-12 14:30:29 +02:00
Christian Klopp
531befbaf4 enhances cleanup for building docker containers in error cases 2018-06-12 14:29:42 +02:00
Christian Klopp
c2b47944b2 fix gevent issue 2018-06-12 14:28:51 +02:00
Christian Klopp
c0cd609d73 updates pip to latest version in sdk docker image 2018-06-12 14:27:44 +02:00
Christian Klopp
54593be26c adds logging and correct handling of actuators in sim mode for cul868ipe 2018-06-12 14:25:56 +02:00
Christian Klopp
e71dfd15e8 Enhance subscription handling in orion ae + adds possibility to guess accumulate address in orion ae 2018-06-12 14:25:12 +02:00
Christian Klopp
81ba52b953 update ocb app doc 2018-06-12 14:20:41 +02:00
ckl-fokus
59e046b4d3
minor fix for docker builds on arm 2018-05-25 20:41:46 +02:00
ckl-fokus
3e20dde68d
Fix travis.yml 2018-05-25 18:37:19 +02:00
Ronald Steinke
ca667df371 updates for the FIWARE integration setup 2018-05-24 13:35:06 +02:00
Ronald Steinke
3a3a147853 updates Orion AE documentation 2018-05-24 13:34:00 +02:00
Ronald Steinke
c7db9bd467 enables to set own POA explicitly for docker containers 2018-05-24 13:33:12 +02:00
Ronald Steinke
aecf7c0c02 fixes docker container building issues 2018-05-22 13:20:42 +02:00
Ronald Steinke
692f55f6af adds csvinjector to travis tests 2018-05-22 13:19:32 +02:00
Ronald Steinke
414559d7e4 fixes config of orion ae 2018-04-19 16:28:43 +02:00
Ronald Steinke
d0eeb89fb0 fixes handling of deleted subscriptions for local resource in orion ae 2018-04-19 16:28:17 +02:00
Christian Klopp
c9a8c0970a fix scheme in docker config gateway 2018-04-07 00:37:04 +02:00
Christian Klopp
3ad445b56f another minor fix at backend docker script 2018-04-05 15:44:35 +02:00
Christian Klopp
5dfb8cd684 minor fix at backend docker script 2018-04-05 15:23:07 +02:00
Christian Klopp
2b07763a51 add doc for csvInjector 2018-04-05 15:08:27 +02:00
Christian Klopp
3bdf30c8f7 add csvinjector to travis 2018-04-05 14:23:04 +02:00
Christian Klopp
a09f454ae0 add fiware integration pic 2018-04-05 14:08:54 +02:00
Christian Klopp
955903a464 add csv injector app 2018-04-05 14:08:27 +02:00
Christian Klopp
72eeb68563 add fiware integration doc 2018-04-05 14:08:00 +02:00
Christian Klopp
c27f2934ef add mqtt config vars 2018-04-05 14:07:35 +02:00
Christian Klopp
262d6b65fb better defaults, fix wrong wsgi interface 2018-04-05 14:05:57 +02:00
Christian Klopp
900541b3b2 add DS_Store 2018-04-05 14:04:52 +02:00
Christian Klopp
f48e8e8d68 travis support 2018-03-12 18:06:36 +01:00
179 changed files with 3840 additions and 743 deletions

23
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@ -0,0 +1,23 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
**Additional context**
Add any other context about the problem here.

View File

@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

4
.gitignore vendored
View File

@ -40,4 +40,6 @@ docker/tmp/*
GPATH
GRTAGS
GTAGS
.python-version
.python-version
.DS_Store
.#*

55
.travis.yml Normal file
View File

@ -0,0 +1,55 @@
services:
- docker
dist: xenial
before_script:
- sudo apt update
- sudo apt install qemu-user-static python3 python3-pip python3-setuptools
- sudo pip3 install --upgrade pip
- sudo python3 -m pip install --upgrade setuptools
- sudo python3 -m pip install --upgrade pyresttest
- "./create-binary-docker backend"
- "./create-binary-docker backend -a arm"
- "./create-binary-docker gateway"
- "./create-binary-docker gateway -a arm"
- "./create-binary-docker orioncontextbroker"
- "./create-binary-docker orioncontextbroker -a arm"
- "./create-binary-docker influxdbapp"
- "./create-binary-docker influxdbapp -a arm"
- "./create-binary-docker cul868ipe"
- "./create-binary-docker cul868ipe -a arm"
- docker run -d --name backend -p 0.0.0.0:18000:18000 -e "ONEM2M_CSE_ID=backend" -e "ONEM2M_NOTIFICATION_DISABLED=false" openmtc/backend-amd64 -v
- docker logs backend
- docker run -d --name gateway -p 0.0.0.0:8000:8000 -e "ONEM2M_HTTP_TRANSPORT_PORT=8000" -e "ONEM2M_CSE_ID=gateway" -e "ONEM2M_REMOTE_CSE_POA=<POA>" -e "ONEM2M_REMOTE_CSE_ID=backend" -e "ONEM2M_NOTIFICATION_DISABLED=false" -e "ONEM2M_REGISTRATION_DISABLED=false" openmtc/gateway-amd64 -v
- docker logs gateway
script:
- docker tag openmtc/orioncontextbroker-amd64 openmtc/orion-context-broker-app-amd64
- docker tag openmtc/orioncontextbroker-arm openmtc/orion-context-broker-app-arm
- docker tag openmtc/cul868ipe-amd64 openmtc/cul868-ipe-amd64
- docker tag openmtc/cul868ipe-arm openmtc/cul868-ipe-arm
- docker tag openmtc/influxdbapp-amd64 openmtc/influxdb-app-amd64
- docker tag openmtc/influxdbapp-arm openmtc/influxdb-app-arm
- docker logs gateway
- docker logs backend
after_script:
- pyresttest http://localhost:8000 tests/interoperability/basic.yaml
env:
matrix:
secure: oQe/MxxFrPPArxb6OFzOUwG2ZlA5GYekMR6qn0Y3101v82MdrIVgDILHR41iwykTtk1XpJPV9uabsRsvY4hBmbjYzihYZFOAzFf+/KU7wROtKum0fFfLjCPb8uGV41k1JTquB94FibXboVAP7rNL0Vrpl+FGUvMGatQBtwZnGvha6Ha07qTao+X9+0dJ4YvFlvcba/jfzBZBwHw7KHpRoiyKU0dPmXpHcqnZBFcNqRXhzdVrgx0auP/tghoshW5LQWkpTV11uSx/kYuby4oo8r8nB6L0rW1jYSXs9DHiTQCfCy24xlb9YJjCD9aFcjH0lIkVIqQwJYA67MzKYMK4XV684J/Jr3+jfVOoUt0bpZaTnk+r/uiFCtEsN7q0KUlvHLUAi1YNJhKs6CRrAH3GK25QByh+suzPzZoHP42F2LYP9URlzbDH+/v7CwNuw+9pHSmxEhs18LVmggwkMos3o3ArvzMLRNJ3QG1fdoxL/Ubqxhmvhy4rZi6vGQsiF64oX8PN0sRgXekTU/ma+6CTM6qhgkocMUYzT2r/6qd/9R3jmtxSZnikjkR6Iu9NLzAUY+cqnvvjohAvqYyj+tGlQfOPxAU9H4wXnbQWLBM8yn6yt2Ki+3+Jx4Owdv0oXVJpN0GWjMzIJ1nEus5odkWW+c4wcZb6m3Ak0PrzYPp2BgE=
after_success:
- if [ "$TRAVIS_BRANCH" == "master" ]; then
docker login -u "$DOCKER_USERNAME" -p "$DOCKER_PASSWORD";
docker push openmtc/backend-amd64;
docker push openmtc/backend-arm;
docker push openmtc/gateway-amd64;
docker push openmtc/gateway-arm;
docker push openmtc/orion-context-broker-app-amd64;
docker push openmtc/orion-context-broker-app-arm;
docker push openmtc/cul868-ipe-amd64;
docker push openmtc/cul868-ipe-arm;
docker push openmtc/influxdb-app-amd64;
docker push openmtc/influxdb-app-arm;
fi
after_failure:
- docker logs backend
- docker logs gateway
- coveralls

6
CREDITS.md Normal file
View File

@ -0,0 +1,6 @@
# Contributors ordered by number of commits
Ronald Steinke <rst-fokus>
Christian Klopp <ckl-fokus>
Alexander Ortlieb <aor-fokus>
Jason Fox <jason-fox>

129
README.md
View File

@ -4,44 +4,111 @@
</a>
</p>
The OpenMTC SDK aims to provide developers with a convenient yet flexible tool to write oneM2M compliant applications. This includes network applications (NAs), gateway application (GAs), device applications (DAs), as well as interworking proxy entities (IPEs).
[![FIWARE IoT Agents](https://nexus.lab.fiware.org/static/badges/chapters/iot-agents.svg)](https://www.fiware.org/developers/catalogue/)
[![License: Eclipse](https://img.shields.io/github/license/OpenMTC/OpenMTC.svg)](https://www.eclipse.org/legal/epl-v10.html)
[![](https://img.shields.io/docker/pulls/openmtc/gateway-amd64.svg)](https://hub.docker.com/u/openmtc)
[![Support badge](https://nexus.lab.fiware.org/repository/raw/public/badges/stackoverflow/iot-agents.svg)](https://stackoverflow.com/questions/tagged/fiware+iot)
<br/>
[![Documentation badge](https://img.shields.io/readthedocs/fiware-openmtc.svg)](http://fiware-openmtc.readthedocs.org/en/latest/?badge=latest)
[![Build Status](https://travis-ci.org/OpenMTC/OpenMTC.svg?branch=master)](https://travis-ci.org/OpenMTC/OpenMTC)
[![Coverage Status](https://coveralls.io/repos/github/OpenMTC/OpenMTC/badge.svg?branch=master)](https://coveralls.io/github/OpenMTC/OpenMTC?branch=master)
[![Known Vulnerabilities](https://snyk.io/test/github/OpenMTC/OpenMTC/badge.svg?targetFile=openmtc-gevent%2Frequirements.txt)](https://snyk.io/test/github/OpenMTC/OpenMTC?targetFile=openmtc-gevent%2Frequirements.txt)
![Status](https://nexus.lab.fiware.org/static/badges/statuses/iot-openmtc.svg)
# Table of Content
The OpenMTC SDK aims to provide developers with a convenient yet flexible tool
to write oneM2M compliant applications. This includes network applications
(NAs), gateway application (GAs), device applications (DAs), as well as
interworking proxy entities (IPEs).
- [Quick Start](doc/openmtc-get-started.md)
- [Introduction](doc/introduction.md)
- [Deployment](doc/deployment-guide.md)
- [The MQTT Client](doc/onem2m-client-mqtt.md)
- [Authentication Guide](doc/authentication.md)
- [Installation of the OpenMTC SDK](doc/install-sdk.md)
- [Overview REST API](doc/overview-rest-api.md)
- [Write your first OpenMTC applications](doc/training/training-index.md)
- [SDK - Using the Application Framework](doc/sdk-framework.md)
- [SDK - The low-level CSE Client](doc/sdk-client.md)
- [SDK - The Data Model](doc/sdk-datamodel.md)
- Examples
- [IoT Data Visualization](doc/example-apps/IoT-data-visualization.py)
- [Data Aggregation](doc/example-apps/data-aggregation.py)
- [Simple Decision](doc/example-apps/simple-decision.py)
- [Simple Decision 2](doc/example-apps/simple-decision-2.py)
- Scripts
- [Create App Structure Script](doc/create-app-structure.md)
- [Create binary docker images Script](doc/create-binary-docker.md)
- [Code Repository Structure](doc/repository-structure.md)
- [Developer FAQ](doc/developer-faq.md)
This project is part of [FIWARE](https://www.fiware.org/). For more information
check the FIWARE Catalogue entry for the
[IoT Agents](https://github.com/Fiware/catalogue/tree/master/iot-agents).
| :octocat: [Git Repository](https://github.com/OpenMTC/OpenMTC) | :whale: [Docker Hub](https://hub.docker.com/u/openmtc) | :books: [Documentation](https://fiware-openmtc.readthedocs.io/) | :page_facing_up: [Site](https://www.openmtc.org/) | :mortar_board: [Academy](https://fiware-academy.readthedocs.io/en/latest/iot-agents/open-mtc) |
| -------------------------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------- |
# Content
- [Install](#install)
- [Usage](#usage)
- [API](#api)
- [Training](#training)
- [Quality Assurance](#quality-assurance)
- [Contributing](#contributing)
- [License](#license)
# Python
## Install
The OpenMTC SDK is written in and for the Python programming language. Users should therefore have at least a certain knowledge of Python and its paradigms. For this matter, the following material is recommended:
Information about how to install the JSON IoTAgent can be found at the
corresponding section of the
[Installation & Administration Guide](https://fiware-openmtc.readthedocs.io/en/latest/deployment-guide).
- [The Python Homepage](http://www.python.org)
- [Expert Python Programming by Tarek Ziadé](http://www.e-reading.by/bookreader.php/138816/Ziade_-_Expert_Python_programming.pdf)
- [Code Like a Pythonista: Idiomatic Python by David Goodger](http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html)
## Usage
# Feedback
Information about how to use the IoT Agent can be found in the
[User & Programmers Manual](https://fiware-openmtc.readthedocs.io/en/latest/introduction).
Please create issues for any problems and direct any comments or feedback you are having to <a href="mailto:support@openmtc.org">support@openmtc.org</a>
## Training
Please let us know what you think. We are also very interested in any use case you are *not* able to implement with the SDK or if you find it difficult to do so.
Information about our Training can be found in the
[Training Section](https://fiware-openmtc.readthedocs.io/en/latest/training/training-ipe-sensors/index.html).
## API
Information about the REST API can be found in the
[API Section](https://fiware-openmtc.readthedocs.io/en/latest/overview-rest-api)
## Python
The OpenMTC SDK is written in and for the Python programming language. Users
should therefore have at least a certain knowledge of Python and its paradigms.
For this matter, the following material is recommended:
- [The Python Homepage](http://www.python.org)
- [Expert Python Programming by Tarek Ziadé](http://www.e-reading.by/bookreader.php/138816/Ziade_-_Expert_Python_programming.pdf)
- [Code Like a Pythonista: Idiomatic Python](http://www.omahapython.org/IdiomaticPython.html)
## Feedback
Please create issues for any problems and direct any comments or feedback you
are having to <a href="mailto:support@openmtc.org">support@openmtc.org</a>
Please let us know what you think. We are also very interested in any use case
you are _not_ able to implement with the SDK or if you find it difficult to do
so.
## Quality Assurance
This project is part of [FIWARE](https://fiware.org/) and has been rated as
follows:
- **Version Tested:**
![ ](https://img.shields.io/badge/dynamic/json.svg?label=Version&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.version&colorB=blue)
- **Documentation:**
![ ](https://img.shields.io/badge/dynamic/json.svg?label=Completeness&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.docCompleteness&colorB=blue)
![ ](https://img.shields.io/badge/dynamic/json.svg?label=Usability&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.docSoundness&colorB=blue)
- **Responsiveness:**
![ ](https://img.shields.io/badge/dynamic/json.svg?label=Time%20to%20Respond&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.timeToCharge&colorB=blue)
![ ](https://img.shields.io/badge/dynamic/json.svg?label=Time%20to%20Fix&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.timeToFix&colorB=blue)
- **FIWARE Testing:**
![ ](https://img.shields.io/badge/dynamic/json.svg?label=Tests%20Passed&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.failureRate&colorB=blue)
![ ](https://img.shields.io/badge/dynamic/json.svg?label=Scalability&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.scalability&colorB=blue)
![ ](https://img.shields.io/badge/dynamic/json.svg?label=Performance&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.performance&colorB=blue)
![ ](https://img.shields.io/badge/dynamic/json.svg?label=Stability&url=https://fiware.github.io/catalogue/json/open_mtc.json&query=$.stability&colorB=blue)
---
## Contributing
Contribution guidelines are detailed in the [CREDITS](CREDITS.md) file.
## License
The OpenMTC SDK is licensed under the Eclipse Public License (EPL)
version 1.
© 2018-2019 OpenMTC

View File

@ -1,3 +0,0 @@
#!/usr/bin/env bash
exec python -m influxdb $@

View File

@ -7,7 +7,7 @@ This App will subscribe to OpenMTC data and tranfer it to an instance of the Inf
You need a running instance of InfluxDB and configure the following parameters according to your setup.
```
apps/influx-db \
apps/influxdb-app \
--ep "http://127.0.0.1:8000" \
--influx_host "127.0.0.1" \
--influx-port "8086" \

View File

@ -0,0 +1,3 @@
#!/usr/bin/env bash
exec python3 -m influxdbapp $@

View File

@ -1,8 +1,8 @@
#!/usr/bin/env bash
CONFIG_FILE="/etc/openmtc/influxdb/config.json"
CONFIG_FILE="/etc/openmtc/influxdbapp/config.json"
NAME=${NAME-"InfluxDB"}
NAME=${NAME-"InfluxdbApp"}
EP=${EP-"http://localhost:8000"}
CSE_BASE=${CSE_BASE-"onem2m"}
POAS=${POAS-'["http://auto:23706"]'}
@ -20,7 +20,7 @@ DBUSER=${DBUSER-"test"},
DBUSER_PW=${DBUSER_PW-"test"}
# defaults logging
LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/influxdb.log"}
LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/influxdbapp.log"}
LOGGING_LEVEL=${LOGGING_LEVEL-"ERROR"}
# ensure correct level
@ -40,7 +40,7 @@ HOST_NAME=${EXTERNAL_IP-${LOCAL_IP}}
# Configuration of the service.
CONFIG_TEMP=${CONFIG_FILE}".tmp"
echo -n "Configuring M2M influxdb..."
echo -n "Configuring M2M influxdbapp..."
JQ_STRING='.'
# basics
@ -70,4 +70,4 @@ mv ${CONFIG_TEMP} ${CONFIG_FILE}
echo "done"
exec python -m influxdb $@
exec python3 -m influxdbapp $@

View File

@ -0,0 +1,30 @@
############################################################
# Dockerfile to run openmtc influxdbapp binary
############################################################
# Set the base image to use openmtc/sdk
FROM openmtc/sdk-amd64:latest
ENV MOD_NAME=influxdbapp
# Set the file maintainer
MAINTAINER rst
# install openmtc dependencies
COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
RUN pip install --upgrade --requirement /tmp/requirements.txt
# install openmtc-influxdbapp
COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
--owner root --group root --no-same-owner --no-overwrite-dir \
--transform 's/json\.dist/json/' --show-transformed
RUN mkdir -p /var/log/openmtc
# add change config
COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
# entry point
ENTRYPOINT ["/usr/local/bin/configure-and-start"]
CMD [""]

View File

@ -0,0 +1,30 @@
############################################################
# Dockerfile to run openmtc influxdbapp binary
############################################################
# Set the base image to use openmtc/sdk
FROM openmtc/sdk-arm:latest
ENV MOD_NAME=influxdbapp
# Set the file maintainer
MAINTAINER rst
# install openmtc dependencies
COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
RUN pip install --upgrade --requirement /tmp/requirements.txt
# install openmtc-influxdbapp
COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
--owner root --group root --no-same-owner --no-overwrite-dir \
--transform 's/json\.dist/json/' --show-transformed
RUN mkdir -p /var/log/openmtc
# add change config
COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
# entry point
ENTRYPOINT ["/usr/local/bin/configure-and-start"]
CMD [""]

View File

@ -1,10 +1,10 @@
[Unit]
Description=OpenMTC InfluxDB
Description=OpenMTC InfluxdbApp
After=network.target
Wants=ntp.service
[Service]
ExecStart=/usr/local/bin/influx-db
ExecStart=/usr/local/bin/influxdb-app
[Install]
WantedBy=multi-user.target

View File

@ -1,4 +1,4 @@
#!/usr/bin/env python
#!/usr/bin/env python3
from setuptools import setup
from distutils.core import setup
@ -8,7 +8,7 @@ import sys
from utils import get_packages, get_pkg_files, OpenMTCSdist, move_config_files
# name and dir
NAME = "influxdb"
NAME = "influxdbapp"
BASE_DIR = "."
# import pkg

View File

@ -2,8 +2,8 @@
Transfer OpenMTC Data to an InfluxDB
"""
__version__ = "0.1"
__description__ = "InfluxDB"
__version__ = "1.3.0"
__description__ = "InfluxdbApp"
__author_name__ = "Christian Klopp"
__author_mail__ = "christian.klopp@fokus.fraunhofer.de"
__requires__ = ['influxdb']

View File

@ -2,17 +2,17 @@ from argparse import ArgumentDefaultsHelpFormatter, ArgumentParser
from openmtc_app.util import prepare_app, get_value
from openmtc_app.runner import AppRunner as Runner
from .influx_db import InfluxDB
from .influxdb_app import InfluxdbApp
# defaults
default_name = "InfluxDB"
default_name = "InfluxdbApp"
default_ep = "http://localhost:8000"
default_labels = []
# args parser
parser = ArgumentParser(
description="An IPE called InfluxDB",
prog="InfluxDB",
description="An IPE called InfluxdbApp",
prog="InfluxdbApp",
formatter_class=ArgumentDefaultsHelpFormatter)
parser.add_argument("-n", "--name", help="Name used for the AE.")
parser.add_argument("-s", "--ep", help="URL of the local Endpoint.")
@ -30,24 +30,24 @@ parser.add_argument("--db-pw", help="InfluxDB User password")
args, config = prepare_app(parser, __loader__, __name__, "config.json")
# variables
nm = get_value("name", (unicode, str), default_name, args, config)
nm = get_value("name", str, default_name, args, config)
cb = config.get("cse_base", "onem2m")
ep = get_value("ep", (unicode, str), default_ep, args, config)
ep = get_value("ep", str, default_ep, args, config)
poas = config.get("poas", ["http://auto:23706"])
originator_pre = config.get("originator_pre", "//openmtc.org/mn-cse-1")
ssl_certs = config.get("ssl_certs", {})
lbl = get_value("labels", list, default_labels, args, config)
influx_host = get_value("influx_host", (unicode, str), "localhost", args, config)
influx_port = get_value("influx_port", (unicode, str), "8086", args, config)
influx_user = get_value("influx_user", (unicode, str), "root", args, config)
influx_password = get_value("influx_password", (unicode, str), "root", args, config)
db_name = get_value("db_name", (unicode, str), "example", args, config)
db_user = get_value("db_user", (unicode, str), "test", args, config)
db_pw = get_value("db_pw", (unicode, str), "test", args, config)
influx_host = get_value("influx_host", str, "localhost", args, config)
influx_port = get_value("influx_port", str, "8086", args, config)
influx_user = get_value("influx_user", str, "root", args, config)
influx_password = get_value("influx_password", str, "root", args, config)
db_name = get_value("db_name", str, "example", args, config)
db_user = get_value("db_user", str, "test", args, config)
db_pw = get_value("db_pw", str, "test", args, config)
# start
app = InfluxDB(
app = InfluxdbApp(
name=nm, cse_base=cb, poas=poas,
labels=lbl,
originator_pre=originator_pre,

View File

@ -1,8 +1,8 @@
from openmtc_app.onem2m import ResourceManagementXAE
from connector import InfluxDBConnector
from .connector import InfluxDBConnector
class InfluxDB(ResourceManagementXAE):
class InfluxdbApp(ResourceManagementXAE):
def __init__(
self,
@ -17,8 +17,8 @@ class InfluxDB(ResourceManagementXAE):
*args,
**kw
):
super(InfluxDB, self).__init__(*args, **kw)
if isinstance(labels, basestring):
super(InfluxdbApp, self).__init__(*args, **kw)
if isinstance(labels, str):
self.labels = {labels}
elif hasattr(labels, '__iter__') and len(labels):
self.labels = set(labels)

View File

@ -35,8 +35,8 @@ def get_packages(package, package_dir, excluded_list=None, included_list=None):
r_prefix = len(root) + 1
for path, dirs, files in os.walk(root, onerror=on_error):
is_module = "__init__.py" in files and path != root
excluded = any(map(lambda x: x in path, excluded_list))
included = any(map(lambda x: x in path, included_list))
excluded = any([x in path for x in excluded_list])
included = any([x in path for x in included_list])
if is_module and (not excluded or included):
packages.append(package + "." + path[r_prefix:].replace("/", "."))
@ -56,7 +56,7 @@ def get_pkg_files(base_dir, name):
def enable_init_files(init_dir, init_dist_files):
for f in init_dist_files:
os.chmod(os.path.join(init_dir, os.path.basename(f)), 0755)
os.chmod(os.path.join(init_dir, os.path.basename(f)), 0o755)
def move_config_files(config_dir, config_files):
@ -74,7 +74,7 @@ def create_openmtc_user(db_dir=None, log_dir=None):
try:
from pwd import getpwnam
except ImportError:
print "Could not import the 'pwd' module. Skipping user management"
print("Could not import the 'pwd' module. Skipping user management")
else:
# assuming DB_DIR was created by setup already
try:

View File

@ -1,6 +1,6 @@
# Introduction
OrionContextBroker is an OpenMTC AE to forward OpenMTC sensor data (via Subscription) to an instance of the Orion Context Broker. Additionally OpenMTC actuators are handled by forwarding changes on the OCB side via OpenMTC to the actuator.
OrionContextBroker (OCB) is an OpenMTC AE to forward OpenMTC sensor data (via Subscription) to an instance of the Orion Context Broker. Additionally OpenMTC actuators are handled by forwarding changes on the OCB side via OpenMTC to the actuator.
All Content Instances are expected to use the SenML format. It is possible to connect the AE either to an OpenMTC Gateway or an OpenMTC Backend.
# Getting started
@ -27,6 +27,191 @@ The most important parameters are:
* orion_host (hostname:port of the Orion CB)
* accumulate_address (Subscription Sink (RESTful HTTP) used for subscriptions to the OCB (actuator functionality))
## Example Setup with Docker
![](docker_setup.png)
The illustration above shows an example setup, which can be created using docker-compose with the docker-compose.yml file listed at the end of this section.
For this example an OpenMTC Interworking Proxy called CUL868IPE is used in simulation mode, in order to periodically generate sensor data without the need of real physical sensors. This IPE will exchange data with an OpenMTC Gateway. The Gateway itself is connected to an OpenMTC Backend. The OpenMTC OCB-App will on one hand forward sensor data to the OrionContextBroker and on the other hand will forward actuator events to the simulated actuator on the CUL868IPE.
The following docker-compose.yml shows the necessary configuration:
```
version: "2"
services:
gateway:
image: openmtc/gateway-amd64
ports:
- "8000:8000"
environment:
- ONEM2M_NOTIFICATION_DISABLED=false
- ONEM2M_CSE_ID=TESTUSER~TESTTENANT
- ONEM2M_REGISTRATION_DISABLED=false
- ONEM2M_REMOTE_CSE_POA=http://backend:18000
- ONEM2M_REMOTE_CSE_OWN_POA=http://gateway:8000
container_name: gateway
links:
- backend
networks:
- main
backend:
image: openmtc/backend-amd64
ports:
- "18000:18000"
environment:
- ONEM2M_NOTIFICATION_DISABLED=false
- ONEM2M_HTTP_TRANSPORT_DISABLED=false
- ONEM2M_HTTP_TRANSPORT_SSL_ENABLED=false
container_name: backend
networks:
- main
orioncontextbroker-app:
image: openmtc/orion-context-broker-app-amd64
container_name: orioncontextbroker-app
links:
- backend
ports:
- "8086:8086"
- "8080:8080"
environment:
- EP=http://backend:18000
- ORION_HOST=http://orion:1026
- ORION_API=v2
- ACCUMULATE_ADDRESS=http://orioncontextbroker-app:8080
networks:
- main
culipe:
image: openmtc/cul868-ipe-amd64
links:
- gateway
- orioncontextbroker-app
environment:
- EP=http://gateway:8000
- DEVICES=["fs20:16108-1"]
- SIM=true
- DEVICE_MAPPINGS={"S300TH_1":"kitchen","FS20_ST3_16108_1":"bath"}
container_name: culipe
networks:
- main
mongo:
image: mongo:3.4
command: --nojournal
container_name: mongo
networks:
- main
orion:
image: fiware/orion
ports:
- "1026:1026"
command: -dbhost mongo -logLevel debug
container_name: orion
networks:
- main
networks:
main:
driver: bridge
```
### Tests with the Docker setup
After the setup is started, the CUL868IPE will simulate sensor data. In order to get the current temperature value measured by the sensor, it is possible to send a request either to the OpenMTC Backend, OpenMTC Gateway or the OrionContextBroker.
*OpenMTC Backend*
```
curl -X "GET" localhost:18000/~/TESTUSER~TESTTENANT/onem2m/CUL868IPE/S300TH_1/temperature/latest -s | \
jq -r '."m2m:cin".con' | \
base64 -d | jq '.'
```
```json
[
{
"bn": "urn:dev:s300th:1",
"v": 3.5796869716517135,
"u": "Cel",
"t": "1527087517.700",
"n": "temperature"
}
]
```
*OpenMTC Gateway*
```
curl -X "GET" localhost:8000/onem2m/CUL868IPE/S300TH_1/temperature/latest -s | \
jq -r '."m2m:cin".con' | \
base64 -d | jq '.'
```
```json
[
{
"bn": "urn:dev:s300th:1",
"v": 3.5796869716517135,
"u": "Cel",
"t": "1527087517.700",
"n": "temperature"
}
]
```
*OrionContextBroker*
```
curl localhost:1026/v2/entities/TESTTENANT-kitchen/ -s -S --header 'fiware-service: TESTUSER' | jq '."temperature"'
```
```json
{
"type": "Float",
"value": 20.569353172,
"metadata": {
"bn": {
"type": "String",
"value": "urn:dev:s300th:1"
},
"timestamp": {
"type": "String",
"value": "1527093220.104"
},
"unit": {
"type": "String",
"value": "Cel"
}
}
}
```
The following command will switch the current state of the actuator to "ON" by sending a request to the OCB:
```bash
curl localhost:1026/v2/entities/TESTTENANT-bath/attrs -s -S \
--header 'Content-Type: application/json' --header 'fiware-service: TESTUSER' \
-X PATCH -d @- <<EOF
{
"cmd": {
"value": "ON",
"type": "String"
}
}
EOF
```
The state will be automatically forwarded to OpenMTC. Therefore you should see something like the following in the log:
```
culipe | 172.24.0.6 - - [2018-05-26 17:24:29] "POST / HTTP/1.1" 200 160 0.037561
```
To get the current state at the OpenMTC Backend use this:
```
curl -X "GET" localhost:18000/~/TESTUSER~TESTTENANT/onem2m/CUL868IPE/FS20_ST3_16108_1/switch/latest -s | jq '."m2m:cin".con'
```
```
"ON"
```
# How the data is stored at the Orion CB
The Orion CB uses the model of *entities* having *attributes*. The AE matches all Container having the label "openmtc:device" to entities. Attributes are matched to the SenML Key "n" of Content Instances. The types of values are determined by the AE to match typical Orion CB types (e.g. Int, String, Float...).
@ -56,18 +241,18 @@ curl -X POST localhost:18000/onem2m/EXAMPLE_APP_NAME/EXAMPLE_DEVICE_NAME/ -H "Co
Upload SenML Data to OpenMTC:
```json
{
[{
"n": "temperature",
"bn": "openmtc:zigbee:temp",
"v": 24,
"u": "Cel",
"t": "2017-04-13 12:45:12.787239"
}
}]
```
base64: eyJuIjogInRlbXBlcmF0dXJlIiwgImJuIjogIm9wZW5tdGM6emlnYmVlOnRlbXAiLCAidiI6IDI0LCAidSI6ICJDZWwiLCAidCI6ICIyMDE3LTA0LTEzIDEyOjQ1OjEyLjc4NzIzOSJ9Cg==
translated to base64: W3sibiI6ICJ0ZW1wZXJhdHVyZSIsImJuIjogIm9wZW5tdGM6emlnYmVlOnRlbXAiLCJ2IjogMjQsInUiOiAiQ2VsIiwidCI6ICIyMDE3LTA0LTEzIDEyOjQ1OjEyLjc4NzIzOSJ9XQo=
```
curl -X POST localhost:18000/onem2m/EXAMPLE_APP_NAME/EXAMPLE_DEVICE_NAME/EXAMPLE_MEASUREMENT_NAME/ -H "Content-Type: application/vnd.onem2m-res+json" -d '{"m2m:cin": {"con": "eyJuIjogInRlbXBlcmF0dXJlIiwgImJuIjogIm9wZW5tdGM6emlnYmVlOnRlbXAiLCAidiI6IDI0LCAidSI6ICJDZWwiLCAidCI6ICIyMDE3LTA0LTEzIDEyOjQ1OjEyLjc4NzIzOSJ9Cg==", "cnf": "application/json:1"}}'
curl -X POST localhost:18000/onem2m/EXAMPLE_APP_NAME/EXAMPLE_DEVICE_NAME/EXAMPLE_MEASUREMENT_NAME/ -H "Content-Type: application/vnd.onem2m-res+json" -d '{"m2m:cin": {"con": "W3sibiI6ICJ0ZW1wZXJhdHVyZSIsImJuIjogIm9wZW5tdGM6emlnYmVlOnRlbXAiLCJ2IjogMjQsInUiOiAiQ2VsIiwidCI6ICIyMDE3LTA0LTEzIDEyOjQ1OjEyLjc4NzIzOSJ9XQo=", "cnf": "application/json:1"}}'
```
### Query Data Orion CB
@ -79,7 +264,7 @@ curl localhost:1026/v2/entities/ | jq '.'
```json
[
{
"id": "EXAMPLE_DEVICE_NAME",
"id": "in-cse-1-EXAMPLE_DEVICE_NAME",
"type": "openmtc",
"temperature": {
"type": "Int",

View File

@ -1,3 +1,3 @@
#!/usr/bin/env bash
exec python -m orioncontextbroker $@
exec python3 -m orioncontextbroker $@

View File

@ -18,5 +18,6 @@
"labels": ["openmtc:sensor_data:temperature", "openmtc:sensor_data:humidity"],
"interval": 10,
"orion_host": "http://localhost:1026",
"orion_api": "v2"
"orion_api": "v2",
"accumulate_address": "http://localhost:8080"
}

View File

@ -10,9 +10,11 @@ ORIGINATOR_PRE=${ORIGINATOR_PRE-"//openmtc.org/in-cse-1"}
SSL_CRT=${SSL_CRT-"/etc/openmtc/certs/orioncontextbroker.cert.pem"}
SSL_KEY=${SSL_KEY-"/etc/openmtc/certs/orioncontextbroker.key.pem"}
SSL_CA=${SSL_CA-"/etc/openmtc/certs/ca-chain.cert.pem"}
LABELS=${LABELS-'["openmtc:sensor_data"]'}
ORION_HOST=${ORION_HOST-"http://localhost:1026"}
ORION_API=${ORION_API-"v2"}
ACCUMULATE_ADDRESS=${ACCUMULATE_ADDRESS-"http://localhost:8080"}
ACCUMULATE_ADDRESS=${ACCUMULATE_ADDRESS}
LABELS=${LABELS-'["openmtc:sensor_data"]'}
# defaults logging
LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/orioncontextbroker.log"}
@ -45,9 +47,11 @@ JQ_STRING=${JQ_STRING}' |
.cse_base = "'${CSE_BASE}'" |
.poas = '${POAS}' |
.originator_pre = "'${ORIGINATOR_PRE}'" |
.labels = '${LABELS}' |
.orion_host = "'${ORION_HOST}'" |
.orion_api = "'${ORION_API}'" |
.accumulate_address = "'${ACCUMULATE_ADDRESS}'" |
.labels = '${LABELS}' |
.ssl_certs.cert_file = "'${SSL_CRT}'" |
.ssl_certs.key_file = "'${SSL_KEY}'" |
.ssl_certs.ca_certs = "'${SSL_CA}'" |
@ -60,4 +64,4 @@ mv ${CONFIG_TEMP} ${CONFIG_FILE}
echo "done"
exec python -m orioncontextbroker $@
exec python3 -m orioncontextbroker $@

Binary file not shown.

After

Width:  |  Height:  |  Size: 132 KiB

View File

@ -1,11 +1,11 @@
{
"name": "OrionContextBroker",
"ep": "http://localhost:8000",
"ep": "http://localhost:18000",
"cse_base": "onem2m",
"poas": [
"http://auto:25396"
],
"originator_pre": "//openmtc.org/mn-cse-1",
"originator_pre": "//openmtc.org/in-cse-1",
"ssl_certs": {
"cert_file": "/etc/openmtc/certs/orioncontextbroker.cert.pem",
"key_file": "/etc/openmtc/certs/orioncontextbroker.key.pem",
@ -15,8 +15,9 @@
"level": "INFO",
"file": "/var/log/openmtc/orioncontextbroker.log"
},
"labels": [],
"labels": ["openmtc:sensor_data"],
"interval": 10,
"orion_host": "http://localhost:1026",
"orion_api": "v2"
"orion_api": "v2",
"accumulate_address": "http://localhost:8080"
}

View File

@ -1,4 +1,4 @@
#!/usr/bin/env python
#!/usr/bin/env python3
from setuptools import setup
from distutils.core import setup

View File

@ -3,7 +3,7 @@ This App will forward all incoming sensor traffic to the Fiware Orion Context
Broker
"""
__version__ = "0.1"
__version__ = "1.3.0"
__description__ = "OrionContextBroker"
__author_name__ = "Christian Klopp"
__author_mail__ = "christian.klopp@fokus.fraunhofer.de"

View File

@ -38,20 +38,17 @@ parser.add_argument(
args, config = prepare_app(parser, __loader__, __name__, "config.json")
# variables
nm = get_value("name", (unicode, str), default_name, args, config)
nm = get_value("name", str, default_name, args, config)
cb = config.get("cse_base", "onem2m")
ep = get_value("ep", (unicode, str), default_ep, args, config)
ep = get_value("ep", str, default_ep, args, config)
poas = config.get("poas", ["http://auto:25396"])
originator_pre = config.get("originator_pre", "//openmtc.org/mn-cse-1")
ssl_certs = config.get("ssl_certs", {})
interval = get_value("interval", int, default_ep, args, config)
lbl = get_value("labels", list, default_labels, args, config)
orion_host = get_value("orion_host", (unicode, str), default_orion_host, args,
config)
orion_api = get_value("orion_api", (unicode, str), default_orion_api, args,
config)
accumulate_address = get_value("accumulate_address", (unicode, str),
default_accumulate_address, args, config)
orion_host = get_value("orion_host", str, default_orion_host, args, config)
orion_api = get_value("orion_api", str, default_orion_api, args, config)
accumulate_address = get_value("accumulate_address", str, default_accumulate_address, args, config)
# start
app = OrionContextBroker(

View File

@ -1,13 +1,14 @@
try:
from urllib.parse import urljoin
except ImportError:
from urlparse import urljoin
from urllib.parse import urljoin
import logging
from datetime import datetime
import requests
import json
from futile.logging import LoggerMixin
logging.getLogger("requests").setLevel(logging.WARNING)
logging.getLogger("urllib3").setLevel(logging.WARNING)
class OrionAPI(LoggerMixin):
def __init__(self,
@ -29,16 +30,23 @@ class OrionAPI(LoggerMixin):
def _get_type(self, element):
if isinstance(element, int):
return u"Int"
return "Int"
elif isinstance(element, float):
return u"Float"
return "Float"
elif isinstance(element, bool):
return u"Boolean"
elif isinstance(element, (str, unicode)):
return u"String"
return "Boolean"
elif isinstance(element, str):
return "String"
else:
self.logger.error('Type of "{}" unknown'.format(element))
return u"Unknown"
return "Unknown"
def is_host_alive(self):
req = self._request(
"{}/v2/entities".format(self.host),
method="get"
)
return req['status'] >= 0
def create_entity(self,
entity_name,
@ -82,7 +90,8 @@ class OrionAPI(LoggerMixin):
"type": self._get_type(data_senml["v"]),
"metadata": {
"timestamp": {
"value": data_senml["t"],
"value": datetime.fromtimestamp(float(data_senml["t"])).replace(microsecond=0).isoformat()
if data_senml["t"] != "none" else data_senml["t"],
"type": "String"
},
"bn": {
@ -152,6 +161,12 @@ class OrionAPI(LoggerMixin):
# return the subscriptionId
return response["headers"]["Location"].split('/')[3]
def unsubscribe(self, subscription_id, fiware_service=""):
self._request(self.host + "/v2/subscriptions/" + subscription_id,
method='delete',
headers={"fiware-service": fiware_service},
raw=True)
def _request(self,
url,
method='get',

View File

@ -1,9 +1,11 @@
import re
from urllib.parse import urlparse
from flask import Flask, Response, request
from gevent import wsgi
from gevent.pywsgi import WSGIServer
from openmtc_app.onem2m import ResourceManagementXAE
from orion_api import OrionAPI
from .orion_api import OrionAPI
class OrionContextBroker(ResourceManagementXAE):
@ -11,23 +13,34 @@ class OrionContextBroker(ResourceManagementXAE):
orion_host="http://localhost:1026",
orion_api="v2",
labels=None,
accumulate_address="http://localhost:8080",
accumulate_address=None,
*args,
**kw):
super(OrionContextBroker, self).__init__(*args, **kw)
if isinstance(labels, basestring):
if isinstance(labels, str):
self.labels = {labels}
elif hasattr(labels, '__iter__'):
self.labels = set(labels)
elif labels is None:
self.labels = ["openmtc:sensor_data"]
else:
self.labels = None
self._entity_names = {}
self._subscriptions = {}
self.logger.critical(accumulate_address)
self._subscription_endpoints = {}
self._subscription_services = {}
# accumulate address
if not accumulate_address:
accumulate_address = "http://" + self._get_auto_host(orion_host) + ":8080"
# Orion API
self._dry_run = not orion_host
self.orion_api = OrionAPI(
orion_host=orion_host,
api_version=orion_api,
accumulate_endpoint="{}/accumulate".format(accumulate_address))
if not self._dry_run:
self._dry_run = not self.orion_api.is_host_alive()
# Subscription Sink for OCB
self.app = Flask(__name__)
@ -36,22 +49,45 @@ class OrionContextBroker(ResourceManagementXAE):
'process_notification',
self.process_notification,
methods=["POST"])
accumulate_ip, accumulate_port = accumulate_address.split('//')[
1].split(':')
self.server = wsgi.WSGIServer((accumulate_ip, int(accumulate_port)),
self.app)
accumulate_ip, accumulate_port = urlparse(accumulate_address).netloc.rsplit(':', 1)
self.server = WSGIServer(("0.0.0.0", int(accumulate_port)),
self.app)
self.server.start()
@staticmethod
def _get_auto_host(ep):
try:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
netloc = urlparse(ep).netloc.split(':')
s.connect((netloc[0], int(netloc[1])))
host = s.getsockname()[0]
s.close()
except:
host = "127.0.0.1"
return host
def process_notification(self):
self.logger.debug("Got from Subscription {}".format(request.json))
actuator = self.get_resource(
self._subscriptions[request.json["subscriptionId"]])
self.push_content(actuator, request.json["data"][0]["cmd"]["value"])
try:
actuator = self.get_resource(
self._subscription_endpoints[request.json["subscriptionId"]]
)
except KeyError:
# ignore not deleted old subscriptions
pass
else:
self.push_content(actuator, request.json["data"][0]["cmd"]["value"])
return Response(status=200, headers={})
def _on_register(self):
self._discover_openmtc_ipe_entities()
def _on_shutdown(self):
for subscription_id, fiware_service in self._subscription_services.items():
self.orion_api.unsubscribe(subscription_id, fiware_service)
def _sensor_filter(self, sensor_info):
if self.labels:
return len(self.labels.intersection(
@ -59,13 +95,13 @@ class OrionContextBroker(ResourceManagementXAE):
else:
return True
def _get_entity_name(self, sensor_info):
@staticmethod
def _get_entity_name(sensor_info):
device_type = "sensor" if sensor_info.get("sensor_labels",
None) else "actuator"
try:
id_label = filter(
lambda x: (x.startswith('openmtc:id:')),
sensor_info['{}_labels'.format(device_type)]).pop()
id_label = [x for x in sensor_info['{}_labels'.format(device_type)]
if x.startswith('openmtc:id:')].pop()
cse_id, dev_id = re.sub('^openmtc:id:', '',
id_label).split('/')[:2]
except (IndexError, ValueError):
@ -79,6 +115,9 @@ class OrionContextBroker(ResourceManagementXAE):
return re.sub('[\W]', '_', f_s), '%s-%s' % (e_pre, dev_id)
def _sensor_data_cb(self, sensor_info, sensor_data):
if self._dry_run:
return
try:
fiware_service, entity_name = self._entity_names[sensor_info['ID']]
except KeyError:
@ -101,6 +140,10 @@ class OrionContextBroker(ResourceManagementXAE):
'ID']]
self.logger.info("Create new Entity {} on Fiware Service {}".format(
entity_name, fiware_service))
if self._dry_run:
return
self.orion_api.create_entity(
entity_name, fiware_service=fiware_service)
data_dummy = {
@ -113,6 +156,7 @@ class OrionContextBroker(ResourceManagementXAE):
self.orion_api.update_attributes(
entity_name, data_dummy, fiware_service=fiware_service)
subscriptionId = self.orion_api.subscribe(
subscription_id = self.orion_api.subscribe(
entity_name, fiware_service=fiware_service)
self._subscriptions[subscriptionId] = actuator_info['ID']
self._subscription_endpoints[subscription_id] = actuator_info['ID']
self._subscription_services[subscription_id] = fiware_service

View File

@ -35,8 +35,8 @@ def get_packages(package, package_dir, excluded_list=None, included_list=None):
r_prefix = len(root) + 1
for path, dirs, files in os.walk(root, onerror=on_error):
is_module = "__init__.py" in files and path != root
excluded = any(map(lambda x: x in path, excluded_list))
included = any(map(lambda x: x in path, included_list))
excluded = any([x in path for x in excluded_list])
included = any([x in path for x in included_list])
if is_module and (not excluded or included):
packages.append(package + "." + path[r_prefix:].replace("/", "."))
@ -56,7 +56,7 @@ def get_pkg_files(base_dir, name):
def enable_init_files(init_dir, init_dist_files):
for f in init_dist_files:
os.chmod(os.path.join(init_dir, os.path.basename(f)), 0755)
os.chmod(os.path.join(init_dir, os.path.basename(f)), 0o755)
def move_config_files(config_dir, config_files):
@ -74,7 +74,7 @@ def create_openmtc_user(db_dir=None, log_dir=None):
try:
from pwd import getpwnam
except ImportError:
print "Could not import the 'pwd' module. Skipping user management"
print("Could not import the 'pwd' module. Skipping user management")
else:
# assuming DB_DIR was created by setup already
try:

9
apps/csv-injector Executable file
View File

@ -0,0 +1,9 @@
#!/usr/bin/env bash
cd $(dirname ${0})
. ./prep-env.sh
cd csvInjector
PYTHONPATH=${PYTHONPATH}:src exec python3 -m csvinjector $@

View File

@ -0,0 +1 @@
include utils.py

View File

@ -0,0 +1,53 @@
# Introduction
With this app you are able to integrate data of a csv file into OpenMTC. The csv file must have a column with a timestamp and one column with some kind of device id. You are able to configure the duration of the data injection. The timing of your csv will be scaled to that duration.
# Getting started
To get an overview of the available parameters, just execute:
```
./apps/csv-injector --help
```
# Configuration
The most important paramters are:
* csv-path: Path to your csv file
* device-classifier: The id of the column with the device ids
* date-classifier: The id of the column with the timestamp
* time-format: Format of your timestamp (example: %d/%m/%Y-%H:%M)
* duration: time to inject the data (seconds)
* repeat: repeat after full csv is injected (bool)
# Example
If you have a csv like this at "~/test.csv":
```csv
Date,ID,Temp,Hum,Battery
23/01/2018-00:50,environment_1,13.3,41,100
18/02/2018-14:01,environment_2,20.4,46,0,99
02/03/2018-23:11,environment_1,19.1,14,100
03/03/2018-00:01,environment_2,19,13,100
08/03/2018-03:01,environment_1,19.5,26,0,99
08/03/2018-10:11,environment_2,21.9,29,0,100
09/03/2018-16:21,environment_1,21.4,27,0,99
09/03/2018-16:51,environment_1,21.3,27,0,100
16/03/2018-06:36,environment_2,15.1,36,0,100
20/03/2018-09:27,environment_1,18.9,19,0,100
20/03/2018-10:27,environment_1,20.3,20,0,99
```
The app could be started like this:
```
./apps/csv-injector \
--csv-path "~/test.csv" \
--device-classifier "ID" \
--date-classifier "Date" \
--time-format "%d/%m/%Y-%H:%M" \
--duration 300
--repeat 0
```

View File

@ -0,0 +1,3 @@
#!/usr/bin/env bash
exec python3 -m csvinjector $@

View File

@ -0,0 +1,26 @@
{
"name": "csvInjector",
"ep": "http://localhost:8000",
"cse_base": "onem2m",
"poas": [
"http://auto:28300"
],
"originator_pre": "//openmtc.org/mn-cse-1",
"ssl_certs": {
"cert_file": null,
"key_file": null,
"ca_certs": null
},
"logging": {
"level": "ERROR",
"file": null
},
"csv_path": "~/test.csv",
"csv_delim": ",",
"csv_quotechar": "|",
"device_classifier": "sensor_id",
"date_classifier": "date",
"time_format":"%d/%m/%Y-%H:%M",
"duration": 300,
"repeat": false
}

View File

@ -0,0 +1,73 @@
#!/usr/bin/env bash
CONFIG_FILE="/etc/openmtc/csvinjector/config.json"
NAME=${NAME-"csvInjector"}
EP=${EP-"http://localhost:8000"}
CSE_BASE=${CSE_BASE-"onem2m"}
POAS=${POAS-'["http://auto:28300"]'}
ORIGINATOR_PRE=${ORIGINATOR_PRE-"//openmtc.org/mn-cse-1"}
SSL_CRT=${SSL_CRT-"/etc/openmtc/certs/csvinjector.cert.pem"}
SSL_KEY=${SSL_KEY-"/etc/openmtc/certs/csvinjector.key.pem"}
SSL_CA=${SSL_CA-"/etc/openmtc/certs/ca-chain.cert.pem"}
CSV_PATH=${CSV_PATH-"/test.csv"}
CSV_DELIM=${CSV_DELIM-","}
CSV_QUOTECHAR=${CSV_QUOTECHAR-"|"}
CSV_DEVICE_CLASSIFIER=${CSV_DEVICE_CLASSIFIER-""}
CSV_DATE_CLASSIFIER=${CSV_DATE_CLASSIFIER-""}
CSV_TIME_FORMAT=${CSV_TIME_FORMAT-"%d/%m/%Y-%H:%M"}
DURATION=${DURATION-300}
REPEAT=${REPEAT-"False"}
# defaults logging
LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/csvinjector.log"}
LOGGING_LEVEL=${LOGGING_LEVEL-"ERROR"}
# ensure correct level
case ${LOGGING_LEVEL} in
FATAL|ERROR|WARN|INFO|DEBUG)
;;
*)
LOGGING_LEVEL="ERROR"
;;
esac
# local ip
LOCAL_IP=$(ip r get 8.8.8.8 | awk 'NR==1 {print $NF}')
# set hostname
HOST_NAME=${EXTERNAL_IP-${LOCAL_IP}}
# Configuration of the service.
CONFIG_TEMP=${CONFIG_FILE}".tmp"
echo -n "Configuring M2M csvinjector..."
JQ_STRING='.'
# basics
JQ_STRING=${JQ_STRING}' |
.name = "'${NAME}'" |
.ep = "'${EP}'" |
.cse_base = "'${CSE_BASE}'" |
.poas = '${POAS}' |
.originator_pre = "'${ORIGINATOR_PRE}'" |
.ssl_certs.cert_file = "'${SSL_CRT}'" |
.ssl_certs.key_file = "'${SSL_KEY}'" |
.ssl_certs.ca_certs = "'${SSL_CA}'" |
.logging.file = "'${LOGGING_FILE}'" |
.logging.level = "'${LOGGING_LEVEL}'" |
.csv_path = "'${CSV_PATH}'" |
.csv_delim = "'${CSV_DELIM}'" |
.csv_quotechar = "'${CSV_QUOTECHAR}'" |
.device_classifier = '${CSV_DEVICE_CLASSIFIER}' |
.date_classifier = '${CSV_DATE_CLASSIFIER}' |
.time_format = '${CSV_TIME_FORMAT}' |
.duration = '${DURATION}' |
.repeat = "'${REPEAT}'"
'
cat ${CONFIG_FILE} | jq -M "${JQ_STRING}"> ${CONFIG_TEMP}
mv ${CONFIG_TEMP} ${CONFIG_FILE}
echo "done"
exec python3 -m csvinjector $@

View File

@ -0,0 +1,30 @@
############################################################
# Dockerfile to run openmtc csvinjector binary
############################################################
# Set the base image to use openmtc/sdk
FROM openmtc/sdk-amd64:latest
ENV MOD_NAME=csvinjector
# Set the file maintainer
MAINTAINER rst
# install openmtc dependencies
COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
RUN pip install --upgrade --requirement /tmp/requirements.txt
# install openmtc-csvinjector
COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
--owner root --group root --no-same-owner --no-overwrite-dir \
--transform 's/json\.dist/json/' --show-transformed
RUN mkdir -p /var/log/openmtc
# add change config
COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
# entry point
ENTRYPOINT ["/usr/local/bin/configure-and-start"]
CMD [""]

View File

@ -0,0 +1,30 @@
############################################################
# Dockerfile to run openmtc csvinjector binary
############################################################
# Set the base image to use openmtc/sdk
FROM openmtc/sdk-arm:latest
ENV MOD_NAME=csvinjector
# Set the file maintainer
MAINTAINER rst
# install openmtc dependencies
COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
RUN pip install --upgrade --requirement /tmp/requirements.txt
# install openmtc-csvinjector
COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
--owner root --group root --no-same-owner --no-overwrite-dir \
--transform 's/json\.dist/json/' --show-transformed
RUN mkdir -p /var/log/openmtc
# add change config
COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
# entry point
ENTRYPOINT ["/usr/local/bin/configure-and-start"]
CMD [""]

View File

@ -0,0 +1,26 @@
{
"name": "csvInjector",
"ep": "http://localhost:8000",
"cse_base": "onem2m",
"poas": [
"http://auto:28300"
],
"originator_pre": "//openmtc.org/mn-cse-1",
"ssl_certs": {
"cert_file": "/etc/openmtc/certs/csvinjector.cert.pem",
"key_file": "/etc/openmtc/certs/csvinjector.key.pem",
"ca_certs": "/etc/openmtc/certs/ca-chain.cert.pem"
},
"logging": {
"level": "INFO",
"file": "/var/log/openmtc/csvinjector.log"
},
"csv_path": "~/test.csv",
"csv_delim": ",",
"csv_quotechar": "|",
"device_classifier": "sensor_id",
"date_classifier": "sensor_id",
"time_format":"%d/%m/%Y-%H:%M",
"duration": 300,
"repeat": false
}

View File

@ -0,0 +1,10 @@
[Unit]
Description=OpenMTC csvInjector
After=network.target
Wants=ntp.service
[Service]
ExecStart=/usr/local/bin/csv-injector
[Install]
WantedBy=multi-user.target

View File

@ -0,0 +1,82 @@
#!/usr/bin/env python3
from setuptools import setup
from distutils.core import setup
from glob import glob
import sys
from utils import get_packages, get_pkg_files, OpenMTCSdist, move_config_files
# name and dir
NAME = "csvinjector"
BASE_DIR = "."
# import pkg
sys.path.append(BASE_DIR + "/src")
pkg = __import__(NAME)
# setup name and version
SETUP_NAME = "openmtc-" + NAME
SETUP_VERSION = pkg.__version__
SETUP_DESCRIPTION = pkg.__description__
# meta
SETUP_AUTHOR = pkg.__author_name__
SETUP_AUTHOR_EMAIL = pkg.__author_mail__
SETUP_URL = "http://www.openmtc.org"
SETUP_LICENSE = "Fraunhofer FOKUS proprietary"
# requirements
SETUP_REQUIRES = pkg.__requires__
SETUP_INSTALL_REQUIRES = pkg.__requires__
# packages
PACKAGES = [NAME]
PACKAGE_DIR = {"": BASE_DIR + "/src"}
all_packages = []
for package in PACKAGES:
all_packages.extend(get_packages(package, PACKAGE_DIR))
# scripts
SETUP_SCRIPTS = glob(BASE_DIR + "/bin/*")
# package data
PACKAGE_DATA = {NAME: get_pkg_files(BASE_DIR, NAME)}
# data files
CONFIG_FILES = ("config.json",)
CONFIG_DIR = "/etc/openmtc/" + NAME
CONFIG_DIST_FILES = (BASE_DIR + "/etc/conf/config.json.dist",)
DATA_FILES = [(CONFIG_DIR, CONFIG_DIST_FILES)]
# cmd class
CMD_CLASS = {'sdist': OpenMTCSdist}
if __name__ == "__main__":
if 'bdist_wheel' in sys.argv:
raise RuntimeError("This setup.py does not support wheels")
############################################################################
# setup
setup(name=SETUP_NAME,
version=SETUP_VERSION,
description=SETUP_DESCRIPTION,
author=SETUP_AUTHOR,
author_email=SETUP_AUTHOR_EMAIL,
url=SETUP_URL,
license=SETUP_LICENSE,
requires=SETUP_REQUIRES,
install_requires=SETUP_INSTALL_REQUIRES,
package_dir=PACKAGE_DIR,
packages=all_packages,
scripts=SETUP_SCRIPTS,
package_data=PACKAGE_DATA,
data_files=DATA_FILES,
cmdclass=CMD_CLASS
)
############################################################################
# install
if "install" in sys.argv:
# only do this during install
move_config_files(CONFIG_DIR, CONFIG_FILES)

View File

@ -0,0 +1,9 @@
"""
App to inject data from a csv file to OpenMTC
"""
__version__ = "1.3.0"
__description__ = "csvInjector"
__author_name__ = "Christian Klopp"
__author_mail__ = "christian.klopp@fokus.fraunhofer.de"
__requires__ = []

View File

@ -0,0 +1,77 @@
from argparse import ArgumentDefaultsHelpFormatter, ArgumentParser
from openmtc_app.util import prepare_app, get_value
from openmtc_app.runner import AppRunner as Runner
from .csv_injector import csvInjector
# defaults
default_name = "csvInjector"
default_ep = "http://localhost:8000"
default_csv_path = "~/test.csv"
default_csv_delim = ","
default_csv_quotechar = "|"
default_device_classifier = ""
default_date_classifier = "DATE"
default_time_format = "%d/%m/%Y-%H:%M"
default_duration = 300
default_repeat = False
# args parser
parser = ArgumentParser(
description="An IPE called csvInjector",
prog="csvInjector",
formatter_class=ArgumentDefaultsHelpFormatter)
parser.add_argument("-n", "--name", help="Name used for the AE.")
parser.add_argument("-s", "--ep", help="URL of the local Endpoint.")
parser.add_argument("-f", "--csv-path", help="Path to CSV File")
parser.add_argument("--csv-delim", help="Delimiter used for the provided csv")
parser.add_argument(
"--csv-quotechar", help="Quotechar used for the provided csv")
parser.add_argument(
"--device-classifier", help="Column used to specify different devices in csv")
parser.add_argument(
"--date-classifier", help="Column used to specify where dates are defined in csv")
parser.add_argument(
"--time-format", help="Format of the date column in csv (see https://docs.python.org/2/library/datetime.html#strftime-strptime-behavior)")
parser.add_argument(
"--duration", help="Time to inject the csv (if csv time data does not fit, it will be scaled)")
parser.add_argument(
"--repeat", help="Repeat after csv is injected")
# args, config and logging
args, config = prepare_app(parser, __loader__, __name__, "config.json")
# variables
nm = get_value("name", str, default_name, args, config)
cb = config.get("cse_base", "onem2m")
ep = get_value("ep", str, default_ep, args, config)
poas = config.get("poas", ["http://auto:28300"])
originator_pre = config.get("originator_pre", "//openmtc.org/mn-cse-1")
ssl_certs = config.get("ssl_certs", {})
csv_path = get_value("csv_path", str, default_csv_path, args, config)
csv_delim = get_value("csv_delim", str, default_csv_delim, args, config)
csv_quotechar = get_value("csv_quotechar", str, default_csv_quotechar, args, config)
device_classifier = get_value("device_classifier", str, default_device_classifier, args, config)
date_classifier = get_value("date_classifier", (str, list), default_date_classifier, args, config)
time_format = get_value("time_format", (str, list), default_time_format, args, config)
duration = get_value("duration", (int, float), default_duration, args, config)
repeat = get_value("repeat", str, default_repeat, args, config)
# start
app = csvInjector(
name=nm,
cse_base=cb,
poas=poas,
originator_pre=originator_pre,
csv_path=csv_path,
csv_delim=csv_delim,
csv_quotechar=csv_quotechar,
device_classifier=device_classifier,
date_classifier=date_classifier,
time_format=time_format,
csv_inject_duration=duration,
repeat=repeat,
**ssl_certs)
Runner(app).run(ep)
print("Exiting....")

View File

@ -0,0 +1,110 @@
from openmtc_app.onem2m import XAE
from openmtc_onem2m.model import Container
from .csv_process import csvProcessor
import sched
import time
import datetime
class csvInjector(XAE):
def __init__(self,
csv_path,
csv_delim,
csv_quotechar,
device_classifier,
date_classifier,
time_format,
csv_inject_duration=0,
repeat=False,
*args,
**kw):
super(csvInjector, self).__init__(*args, **kw)
self._recognized_sensors = {}
self._recognized_measurement_containers = {}
# csv key to differ between devices
self.device_classifier = device_classifier
self.date_classifier = date_classifier
self.csv_path = csv_path
self.csv_delim = csv_delim
self.csv_quotechar = csv_quotechar
self.time_format = time_format
self.csv_inject_duration = csv_inject_duration
self.repeat = repeat
def _on_register(self):
# start endless loop
self._init_scheduler()
self.scheduler.run()
if self.repeat:
while True:
self._init_scheduler()
self.scheduler.run()
def _init_scheduler(self):
# read csv
self.csv_data_list = csvProcessor(self.csv_path, self.csv_delim,
self.csv_quotechar, self.time_format,
self.csv_inject_duration,
self.date_classifier).csv_data
# setup scheduler
self.scheduler = sched.scheduler(time.time, time.sleep)
for event in self.csv_data_list:
if isinstance(self.date_classifier, list):
self.scheduler.enter(event["timestamp_schedule"], 1,
self.push_data, (event, ))
else:
self.scheduler.enter(event[self.date_classifier], 1,
self.push_data, (event, ))
def _create_measurement_container(self, device_name, name):
measurement_container = self.create_container(
self._recognized_sensors[device_name].path,
Container(resourceName=name),
max_nr_of_instances=0,
labels=[
'openmtc:sensor_data:{}'.format(name), 'openmtc:sensor_data'
])
self._recognized_measurement_containers[device_name][
name] = measurement_container
def _create_sensor_structure(self, event):
device_container = self.create_container(
None,
Container(resourceName=event[self.device_classifier]),
labels=['openmtc:device'],
max_nr_of_instances=0)
self._recognized_sensors[event[
self.device_classifier]] = device_container
self._recognized_measurement_containers[event[
self.device_classifier]] = {}
for k in event.keys():
if k == "Date" or k == self.device_classifier or k in ("", None):
continue
self._create_measurement_container(event[self.device_classifier],
k)
def push_data(self, event):
sensor = event[self.device_classifier]
if not sensor in self._recognized_sensors:
self._create_sensor_structure(event)
device_container = self._recognized_sensors[sensor]
for k in event.keys():
if k == "Date" or k == self.device_classifier or event[k] in (
"", None):
continue
if k not in self._recognized_measurement_containers[sensor].keys():
self._create_measurement_container(sensor, k)
timestamp = time.mktime(datetime.datetime.now().timetuple())
senml = {
"bn": "csv_extracted",
"n": k,
"u": "None",
"t": timestamp,
"v": event[k]
}
self.logger.debug("sensor {} sends data: {}".format(sensor, senml))
self.push_content(
self._recognized_measurement_containers[sensor][k], [senml])

View File

@ -0,0 +1,68 @@
import csv
from datetime import datetime
from futile.logging import LoggerMixin
class csvProcessor(LoggerMixin):
def __init__(self,
path,
delim=",",
quotechar="|",
time_format="%d/%m/%Y-%H:%M",
duration=0,
date_classifier="date"):
with open(path, 'rb') as csvfile:
self.csv_data = list(csv.DictReader(csvfile))
self.time_format = time_format
self.duration = duration
self.date_classifier = date_classifier
# Date Processing
if isinstance(self.date_classifier, list):
self._join_multiple_timestamps()
self._date_string_to_date_object()
self.csv_data = sorted(
self.csv_data, key=lambda k: k[self.date_classifier])
self._date_to_seconds_since_first()
self._scale_to_duration()
def _join_multiple_timestamps(self):
for entry in self.csv_data:
entry["timestamp_schedule"] = "-".join([entry[k] for k in self.date_classifier])
for date_c in self.date_classifier:
entry.pop(date_c, None)
self.date_classifier = "timestamp_schedule"
self.time_format = "-".join(self.time_format)
def _date_string_to_date_object(self):
for entry in self.csv_data:
entry[self.date_classifier] = datetime.strptime(
entry[self.date_classifier], self.time_format)
def _date_to_seconds_since_first(self):
for entry in self.csv_data:
if 'first' in locals():
entry[self.date_classifier] = (
entry[self.date_classifier] - first).total_seconds()
else:
first = entry[self.date_classifier]
entry[self.date_classifier] = 0.0
def _scale_to_duration(self):
if self.duration <= 0:
return
scaling_factor = self.duration / self.csv_data[-1][
self.date_classifier]
self.logger.debug("Set scaling factor to {}".format(scaling_factor))
for entry in self.csv_data:
entry[self.date_classifier] = entry[
self.date_classifier] * scaling_factor
def getList(self):
return self.csv_data
if __name__ == "__main__":
p = csvProcessor("example.csv", duration=300)
for e in p.csv_data:
print(e)

148
apps/csvInjector/utils.py Normal file
View File

@ -0,0 +1,148 @@
import distutils.command.sdist
import distutils.command.build_py
import os
import subprocess
import sys
def echo(msg, *args):
if args:
msg = msg % args
sys.stdout.write(msg + "\n")
def get_packages(package, package_dir, excluded_list=None, included_list=None):
included_list = included_list or []
excluded_list = excluded_list or []
try:
root = package_dir[package]
except KeyError:
root = package_dir.get("", ".") + "/" + package
if not os.path.exists(root):
sys.stderr.write(
"Directory for package %s does not exist: %s\n" % (package, root))
sys.exit(1)
def on_error(error):
sys.stderr.write(
"Error while collecting packages for %s: %s\n" % (package, error))
sys.exit(1)
packages = [package]
r_prefix = len(root) + 1
for path, dirs, files in os.walk(root, onerror=on_error):
is_module = "__init__.py" in files and path != root
excluded = any([x in path for x in excluded_list])
included = any([x in path for x in included_list])
if is_module and (not excluded or included):
packages.append(package + "." + path[r_prefix:].replace("/", "."))
return packages
def get_pkg_files(base_dir, name):
package_files = []
pkg_dir = os.path.join(base_dir, 'src', name)
pkg_data_dir = os.path.join(pkg_dir, 'static')
for (path, directories, filenames) in os.walk(pkg_data_dir):
for filename in filenames:
package_files.append(os.path.join(os.path.relpath(path, pkg_dir),
filename))
return package_files
def enable_init_files(init_dir, init_dist_files):
for f in init_dist_files:
os.chmod(os.path.join(init_dir, os.path.basename(f)), 0o755)
def move_config_files(config_dir, config_files):
for f in config_files:
target_file = os.path.join(config_dir, f)
if not os.path.exists(target_file):
echo("Installing config file %s", target_file)
os.rename(target_file + ".dist", target_file)
# os.chmod(target_file, 0644)
else:
echo("Not overwriting config file %s", target_file)
def create_openmtc_user(db_dir=None, log_dir=None):
try:
from pwd import getpwnam
except ImportError:
print("Could not import the 'pwd' module. Skipping user management")
else:
# assuming DB_DIR was created by setup already
try:
pw = getpwnam('openmtc')
except KeyError as e:
try:
# add system user openmtc:openmtc
# useradd --system -UM openmtc
useradd = "useradd --system -UM openmtc"
retcode = subprocess.call(useradd, shell=True)
if retcode:
raise Exception("Failed to add user 'openmtc'")
pw = getpwnam('openmtc')
except Exception as e:
sys.stderr.write("Error creating user: %s\n" % (e, ))
sys.exit(1)
uid = pw.pw_uid
gid = pw.pw_gid
# set path permissions
if db_dir:
os.chown(db_dir, uid, gid)
if log_dir:
os.chown(log_dir, uid, gid)
class OpenMTCSdist(distutils.command.sdist.sdist):
def make_release_tree(self, base_dir, files):
distutils.command.sdist.sdist.make_release_tree(self, base_dir, files)
script_name = os.path.basename(sys.argv[0])
if script_name != "setup.py":
os.rename(base_dir + "/" + script_name, base_dir + "/setup.py")
self.filelist.files.remove(script_name)
self.filelist.files.append("setup.py")
class OpenMTCSdistBinary(OpenMTCSdist, object):
def make_release_tree(self, base_dir, files):
super(OpenMTCSdistBinary, self).make_release_tree(base_dir, files)
script_name = os.path.basename(sys.argv[0])
build_py = self.get_finalized_command('build_py')
build_py.compile = 1
build_py.optimize = 2
build_py.retain_init_py = 1
build_py.build_lib = base_dir
build_py.byte_compile(
[base_dir + "/" + f for f in self.filelist.files if
f != script_name and f.endswith(".py")])
class OpenMTCBuildPy(distutils.command.build_py.build_py):
retain_init_py = 0
def byte_compile(self, files):
distutils.command.build_py.build_py.byte_compile(self, files)
class OpenMTCBuildPyBinary(OpenMTCBuildPy, object):
retain_init_py = 0
def byte_compile(self, files):
super(OpenMTCBuildPyBinary, self).byte_compile(files)
for f in files:
if (f.endswith('.py') and (os.path.basename(f) != "__init__.py" or
not self.retain_init_py)):
os.unlink(f)

View File

@ -1,9 +0,0 @@
#!/usr/bin/env bash
cd $(dirname ${0})
. ./prep-env.sh
cd InfluxDB
PYTHONPATH=${PYTHONPATH}:src exec python -m influxdb $@

9
apps/influxdb-app Executable file
View File

@ -0,0 +1,9 @@
#!/usr/bin/env bash
cd $(dirname ${0})
. ./prep-env.sh
cd InfluxdbApp
PYTHONPATH=${PYTHONPATH}:src exec python3 -m influxdbapp $@

9
apps/mqtt-connector Executable file
View File

@ -0,0 +1,9 @@
#!/usr/bin/env bash
cd $(dirname ${0})
. ./prep-env.sh
cd mqttConnector
PYTHONPATH=${PYTHONPATH}:src exec python3 -m mqttconnector $@

View File

@ -0,0 +1 @@
include utils.py

View File

@ -0,0 +1,184 @@
# Introduction
![](mqtt_connector_diagram.png)
MQTTConnector is able to connect a MQTT-Broker with OpenMTC. The subscribes to specific topics on the broker in order to forward all published data to OpenMTC.
The MQTTConnector will subscribe itself on the MQTT-Broker (e.g. MeshBlu) to a specific first topics level. Published data is expected to contain at least two more topic levels, with one representing the location and one the device. This information will later be used to create an entity within the OpenMTC resource tree. If data is published, the MQTTConnector app will be notified via the subscription and will forward the data to an instance of OpenMTC (gateway or backend).
# Getting started
In order to get an overview of the possible paramters of the app:
```
./apps/mqtt-connector -v
```
The most relevant paramters are the following:
**EP** (Endpoint): URL of the OpenMTC endpoint to use (backend or gateway)
Example: "http://localhost:18000**
**BROKER_EP** (MQTT Endpoint): URL of the MQTT-Broker
Example: "http://localhost:1883**
**TOPIC_PRE**: definition of the first topic level in order to filter for relevant topics
Example: "OfficeBuilding**
**TOPIC_INDEX_LOCATION**: topic level representing the location
Example: 1
**TOPIC_INDEX_DEVICE**: topic level representing the device
Example: 2
**BROKER_USER**: username to connect to the MQTT Broker
**BROKER_USER_PW**: password to connect to the MQTT Broker
If used to together with the OpenMTC OrionContextBroker app, it is also possible to configure a specific fiware service.
# Required Data Format
In order to work data needs to be published to the MQTT-Broker in an onem2m specific format. This will look like the following:
```json
{
"m2m:cin": {
"con": "B64_ENCODED_SENML_PAYLOAD"
}
}
```
where the B64_ENCODED_SENML_PAYLOAD could look like this:
decoded:
```json
[
{
"bn": "23",
"v": 27,
"u": "Celsius",
"t": 1527757260000,
"n": "Temperatur"
}
]
```
# Example
![](mqtt_connector_ocb_example.png)
In this example setup, we are using the MQTTConnector to forward data of a temperature sensor connected to the Mosquitto MQTT-Broker via OpenMTC to an instance of the Fiware Orion Context Broker. The following docker-compose configuration shows the setup in more detail.
```
version: "2"
services:
backend:
image: openmtc/backend-amd64
ports:
- "18000:18000"
environment:
- ONEM2M_NOTIFICATION_DISABLED=false
- ONEM2M_HTTP_TRANSPORT_DISABLED=false
- ONEM2M_HTTP_TRANSPORT_SSL_ENABLED=false
- ONEM2M_HTTP_TRANSPORT_REQUIRE_CERT=false
mosquitto:
image: eclipse-mosquitto
ports:
- "1883:1883"
mqtt-ipe:
image: openmtc/mqtt-connector-app-amd64
environment:
- EP=http://backend:18000
- BROKER_EP=mosquitto:1883
- TOPIC_PRE=OfficeBuilding
- TOPIC_INDEX_LOCATION=1
- TOPIC_INDEX_DEVICE=2
links:
- mosquitto
- backend
orioncontextbroker-app:
image: openmtc/orion-context-broker-app-amd64
container_name: orioncontextbroker-app
links:
- backend
- orion
ports:
- "8086:8086"
- "8080:8080"
environment:
- EP=http://backend:18000
- ORION_HOST=http://orion:1026
- ORION_API=v2
- ACCUMULATE_ADDRESS=http://orioncontextbroker-app:8080
- LABELS=["openmtc:sensor_data"]
mongo:
image: mongo:3.4
command: --nojournal
orion:
image: fiware/orion
ports:
- "1026:1026"
command: -dbhost mongo -logLevel debug
links:
- mongo
```
The following will be published to topic OfficeBuilding/MeetingRoom/temperature:
```json
{
"m2m:cin": {
"con": "W3tibjogMjMsIHY6IDI3LCB1OiBDZWxzaXVzLCB0OiAxNTI3NzU3MjYwMDAwLCBuOiBUZW1wZXJhdHVyfV0K"
}
}
```
The string is in "con" is the base64 representation of:
```json
[
{
"bn": "23",
"v": 27,
"u": "Celsius",
"t": 1527757260000,
"n": "temperature"
}
]
```
If the data is successfully published to the MQTT-Broker it will be forwarded to the OCB. Therefore we are able to request the data from the OCB.
```
curl localhost:1026/v2/entities/MeetingRoom-temperature/attrs -H -s
```
```json
{
"temperature": {
"type": "Float",
"value": 27,
"metadata": {
"bn": {
"type": "String",
"value": 23
},
"timestamp": {
"type": "String",
"value": 152775726000
},
"unit": {
"type": "String",
"value": "Celsius"
}
}
}
}
```

View File

@ -0,0 +1,3 @@
#!/usr/bin/env bash
exec python3 -m mqttconnector $@

View File

@ -0,0 +1,29 @@
{
"name": "mqttConnector",
"ep": "http://localhost:18000",
"cse_base": "onem2m",
"poas": [
"http://auto:21753"
],
"originator_pre": "//openmtc.org/in-cse-1",
"ssl_certs": {
"cert_file": null,
"key_file": null,
"ca_certs": null
},
"broker_ep": "localhost:1883",
"topic_pre": "exampleTopic",
"topic_index_location": 1,
"topic_index_device": -1,
"fiware_service": null,
"broker_user": "",
"broker_user_pw": "",
"mqtts_enabled": false,
"mqtts_ca_certs": null,
"mqtts_certfile": null,
"mqtts_keyfile": null,
"logging": {
"level": "ERROR",
"file": null
}
}

View File

@ -0,0 +1,79 @@
#!/usr/bin/env bash
CONFIG_FILE="/etc/openmtc/mqttconnector/config.json"
NAME=${NAME-"mqttConnector"}
EP=${EP-"http://localhost:8000"}
CSE_BASE=${CSE_BASE-"onem2m"}
POAS=${POAS-'["http://auto:21753"]'}
ORIGINATOR_PRE=${ORIGINATOR_PRE-"//openmtc.org/mn-cse-1"}
SSL_CRT=${SSL_CRT-"/etc/openmtc/certs/mqttconnector.cert.pem"}
SSL_KEY=${SSL_KEY-"/etc/openmtc/certs/mqttconnector.key.pem"}
SSL_CA=${SSL_CA-"/etc/openmtc/certs/ca-chain.cert.pem"}
BROKER_EP=${BROKER_EP-"localhost:1883"}
TOPIC_PRE=${TOPIC_PRE-"exampleTopic"}
TOPIC_INDEX_LOCATION=${TOPIC_INDEX_LOCATION-1}
TOPIC_INDEX_DEVICE=${TOPIC_INDEX_DEVICE:-1}
FIWARE_SERVICE=${FIWARE_SERVICE}
BROKER_USER=${BROKER_USER-""}
BROKER_USER_PW=${BROKER_USER_PW-""}
MQTTS_ENABLED=${MQTTS_ENABLED-false}
MQTTS_CA_CERTS=${MQTTS_CA_CERTS}
MQTTS_CERTFILE=${MQTTS_CERTFILE}
MQTTS_KEYFILE=${MQTTS_KEYFILE}
# defaults logging
LOGGING_FILE=${LOGGING_FILE-"/var/log/openmtc/mqttconnector.log"}
LOGGING_LEVEL=${LOGGING_LEVEL-"ERROR"}
# ensure correct level
case ${LOGGING_LEVEL} in
FATAL|ERROR|WARN|INFO|DEBUG)
;;
*)
LOGGING_LEVEL="ERROR"
;;
esac
# local ip
LOCAL_IP=$(ip r get 8.8.8.8 | awk 'NR==1 {print $NF}')
# set hostname
HOST_NAME=${EXTERNAL_IP-${LOCAL_IP}}
# Configuration of the service.
CONFIG_TEMP=${CONFIG_FILE}".tmp"
echo -n "Configuring M2M mqttconnector..."
JQ_STRING='.'
# basics
JQ_STRING=${JQ_STRING}' |
.name = "'${NAME}'" |
.ep = "'${EP}'" |
.cse_base = "'${CSE_BASE}'" |
.poas = '${POAS}' |
.originator_pre = "'${ORIGINATOR_PRE}'" |
.ssl_certs.cert_file = "'${SSL_CRT}'" |
.ssl_certs.key_file = "'${SSL_KEY}'" |
.ssl_certs.ca_certs = "'${SSL_CA}'" |
.broker_ep = "'${BROKER_EP}'" |
.topic_pre = "'${TOPIC_PRE}'" |
.topic_index_location = '${TOPIC_INDEX_LOCATION}' |
.topic_index_device = '${TOPIC_INDEX_DEVICE}' |
.fiware_service = "'${FIWARE_SERVICE}'" |
.broker_user = "'${BROKER_USER}'" |
.broker_user_pw = "'${BROKER_USER_PW}'" |
.mqtts_enabled = '${MQTTS_ENABLED}' |
.mqtts_ca_certs = "'${MQTTS_CA_CERTS}'" |
.mqtts_certfile = "'${MQTTS_CERTFILE}'" |
.mqtts_keyfile = "'${MQTTS_KEYFILE}'" |
.logging.file |= "'${LOGGING_FILE}'" |
.logging.level |= "'${LOGGING_LEVEL}'"
'
cat ${CONFIG_FILE} | jq -M "${JQ_STRING}"> ${CONFIG_TEMP}
mv ${CONFIG_TEMP} ${CONFIG_FILE}
echo "done"
exec python3 -m mqttconnector $@

View File

@ -0,0 +1,30 @@
############################################################
# Dockerfile to run openmtc mqttconnector binary
############################################################
# Set the base image to use openmtc/sdk
FROM openmtc/sdk-amd64:latest
ENV MOD_NAME=mqttconnector
# Set the file maintainer
MAINTAINER rst
# install openmtc dependencies
COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
RUN pip install --upgrade --requirement /tmp/requirements.txt
# install openmtc-mqttconnector
COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
--owner root --group root --no-same-owner --no-overwrite-dir \
--transform 's/json\.dist/json/' --show-transformed
RUN mkdir -p /var/log/openmtc
# add change config
COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
# entry point
ENTRYPOINT ["/usr/local/bin/configure-and-start"]
CMD [""]

View File

@ -0,0 +1,30 @@
############################################################
# Dockerfile to run openmtc mqttconnector binary
############################################################
# Set the base image to use openmtc/sdk
FROM openmtc/sdk-arm:latest
ENV MOD_NAME=mqttconnector
# Set the file maintainer
MAINTAINER rst
# install openmtc dependencies
COPY tmp/$MOD_NAME-dependencies.txt /tmp/requirements.txt
RUN pip install --upgrade --requirement /tmp/requirements.txt
# install openmtc-mqttconnector
COPY tmp/openmtc-$MOD_NAME.tar.gz /tmp/openmtc-$MOD_NAME.tar.gz
RUN tar xzf /tmp/openmtc-$MOD_NAME.tar.gz -C / \
--owner root --group root --no-same-owner --no-overwrite-dir \
--transform 's/json\.dist/json/' --show-transformed
RUN mkdir -p /var/log/openmtc
# add change config
COPY configure-$MOD_NAME-and-start /usr/local/bin/configure-and-start
# entry point
ENTRYPOINT ["/usr/local/bin/configure-and-start"]
CMD [""]

View File

@ -0,0 +1,29 @@
{
"name": "mqttConnector",
"ep": "http://localhost:18000",
"cse_base": "onem2m",
"poas": [
"http://auto:21753"
],
"originator_pre": "//openmtc.org/in-cse-1",
"ssl_certs": {
"cert_file": "/etc/openmtc/certs/mqttconnector.cert.pem",
"key_file": "/etc/openmtc/certs/mqttconnector.key.pem",
"ca_certs": "/etc/openmtc/certs/ca-chain.cert.pem"
},
"broker_ep": "localhost:8883",
"topic_pre": "exampleTopic",
"topic_index_location": 1,
"topic_index_device": -1,
"fiware_service": null,
"broker_user": "",
"broker_user_pw": "",
"mqtts_enabled": false,
"mqtts_ca_certs": null,
"mqtts_certfile": null,
"mqtts_keyfile": null,
"logging": {
"level": "INFO",
"file": "/var/log/openmtc/mqttconnector.log"
}
}

View File

@ -0,0 +1,10 @@
[Unit]
Description=OpenMTC mqttConnector
After=network.target
Wants=ntp.service
[Service]
ExecStart=/usr/local/bin/mqtt-connector
[Install]
WantedBy=multi-user.target

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

View File

@ -0,0 +1,82 @@
#!/usr/bin/env python3
from setuptools import setup
from distutils.core import setup
from glob import glob
import sys
from utils import get_packages, get_pkg_files, OpenMTCSdist, move_config_files
# name and dir
NAME = "mqttconnector"
BASE_DIR = "."
# import pkg
sys.path.append(BASE_DIR + "/src")
pkg = __import__(NAME)
# setup name and version
SETUP_NAME = "openmtc-" + NAME
SETUP_VERSION = pkg.__version__
SETUP_DESCRIPTION = pkg.__description__
# meta
SETUP_AUTHOR = pkg.__author_name__
SETUP_AUTHOR_EMAIL = pkg.__author_mail__
SETUP_URL = "http://www.openmtc.org"
SETUP_LICENSE = "Fraunhofer FOKUS proprietary"
# requirements
SETUP_REQUIRES = pkg.__requires__
SETUP_INSTALL_REQUIRES = pkg.__requires__
# packages
PACKAGES = [NAME]
PACKAGE_DIR = {"": BASE_DIR + "/src"}
all_packages = []
for package in PACKAGES:
all_packages.extend(get_packages(package, PACKAGE_DIR))
# scripts
SETUP_SCRIPTS = glob(BASE_DIR + "/bin/*")
# package data
PACKAGE_DATA = {NAME: get_pkg_files(BASE_DIR, NAME)}
# data files
CONFIG_FILES = ("config.json",)
CONFIG_DIR = "/etc/openmtc/" + NAME
CONFIG_DIST_FILES = (BASE_DIR + "/etc/conf/config.json.dist",)
DATA_FILES = [(CONFIG_DIR, CONFIG_DIST_FILES)]
# cmd class
CMD_CLASS = {'sdist': OpenMTCSdist}
if __name__ == "__main__":
if 'bdist_wheel' in sys.argv:
raise RuntimeError("This setup.py does not support wheels")
############################################################################
# setup
setup(name=SETUP_NAME,
version=SETUP_VERSION,
description=SETUP_DESCRIPTION,
author=SETUP_AUTHOR,
author_email=SETUP_AUTHOR_EMAIL,
url=SETUP_URL,
license=SETUP_LICENSE,
requires=SETUP_REQUIRES,
install_requires=SETUP_INSTALL_REQUIRES,
package_dir=PACKAGE_DIR,
packages=all_packages,
scripts=SETUP_SCRIPTS,
package_data=PACKAGE_DATA,
data_files=DATA_FILES,
cmdclass=CMD_CLASS
)
############################################################################
# install
if "install" in sys.argv:
# only do this during install
move_config_files(CONFIG_DIR, CONFIG_FILES)

View File

@ -0,0 +1,9 @@
"""
TODO: Add description here
"""
__version__ = "1.3.0"
__description__ = "mqttConnector"
__author_name__ = "Ronald Steinke"
__author_mail__ = "ronald.steinke@fokus.fraunhofer.de"
__requires__ = ["paho_mqtt"]

View File

@ -0,0 +1,84 @@
from argparse import ArgumentDefaultsHelpFormatter, ArgumentParser
from openmtc_app.util import prepare_app, get_value
from openmtc_app.runner import AppRunner as Runner
from .mqtt_connector import mqttConnector
# defaults
default_name = "mqttConnector"
default_ep = "http://localhost:8000"
default_topic_pre = "exampleTopic"
default_topic_index_location = 1
default_topic_index_device = -1
default_fiware_service = None
default_broker_user = ""
default_broker_user_pw = ""
default_mqtts_ca_certs = None
default_mqtts_certfile = None
default_mqtts_keyfile = None
# args parser
parser = ArgumentParser(
description="An IPE called mqttConnector",
prog="mqttConnector",
formatter_class=ArgumentDefaultsHelpFormatter)
parser.add_argument("-n", "--name", help="Name used for the AE.")
parser.add_argument("-s", "--ep", help="URL of the local Endpoint.")
parser.add_argument("--topic-pre", help="Topic you want to be subscribed to")
parser.add_argument(
"--topic-index-location", help="Index of location in topic string")
parser.add_argument(
"--topic-index-device", help="Index of device name in topic string")
parser.add_argument("--broker-user", help="credentials for MQTT broker")
parser.add_argument("--broker-user-pw", help="credentials for MQTT broker")
parser.add_argument("--mqtts-enabled", action='store_true')
parser.add_argument(
"--mqtts-ca-certs", help="Path to CA certs or tuple of paths")
parser.add_argument("--mqtts-certfile", help="Path to own mqtts cert")
parser.add_argument("--mqtts-keyfile", help="Path to own mqtts key")
# args, config and logging
args, config = prepare_app(parser, __loader__, __name__, "config.json")
# variables
nm = get_value("name", str, default_name, args, config)
cb = config.get("cse_base", "onem2m")
ep = get_value("ep", str, default_ep, args, config)
poas = config.get("poas", ["http://auto:21753"])
originator_pre = config.get("originator_pre", "//openmtc.org/mn-cse-1")
ssl_certs = config.get("ssl_certs", {})
broker_ep = config.get("broker_ep", "localhost:8883")
topic_pre = get_value("topic_pre", str, default_topic_pre, args, config)
topic_index_location = get_value("topic_index_location", int,
default_topic_index_location, args, config)
topic_index_device = get_value("topic_index_device", int,
default_topic_index_device, args, config)
fiware_service = get_value("fiware_service", str, default_fiware_service, args, config)
broker_user = get_value("broker_user", str, default_broker_user, args, config)
broker_user_pw = get_value("broker_user_pw", str, default_broker_user_pw, args, config)
user_pw = get_value("broker_user_pw", str, default_broker_user_pw, args, config)
mqtts_enabled = get_value("mqtts_enabled", (bool), False, args, config)
mqtts_ca_certs = get_value("mqtts_ca_certs", str, default_mqtts_ca_certs, args, config)
mqtts_certfile = get_value("mqtts_certfile", str, default_mqtts_certfile, args, config)
mqtts_keyfile = get_value("mqtts_keyfile", str, default_mqtts_keyfile, args, config)
# start
app = mqttConnector(
broker_ep=broker_ep,
topic_pre=topic_pre,
broker_user=broker_user,
broker_user_pw=broker_user_pw,
topic_index_location=topic_index_location,
topic_index_device=topic_index_device,
fiware_service=fiware_service,
mqtts_enabled=mqtts_enabled,
mqtts_ca_certs=mqtts_ca_certs,
mqtts_certfile=mqtts_certfile,
mqtts_keyfile=mqtts_keyfile,
name=nm,
cse_base=cb,
poas=poas,
originator_pre=originator_pre,
**ssl_certs)
Runner(app).run(ep)
print("Exiting....")

View File

@ -0,0 +1,127 @@
import re
from base64 import b64decode as base64decode
from itertools import groupby
from json import loads as json_decode
from os import path as ospath
from paho.mqtt import client as mqtt
from openmtc_app.onem2m import XAE
class mqttConnector(XAE):
interval = 10
_location_containers = {}
_device_containers = {}
_sensor_containers = {}
client = mqtt.Client()
def __init__(self,
broker_ep,
topic_pre,
broker_user,
broker_user_pw,
topic_index_location=1,
topic_index_device=-1,
fiware_service=None,
mqtts_enabled=False,
mqtts_ca_certs=None,
mqtts_certfile=None,
mqtts_keyfile=None,
*args,
**kw):
host, port = broker_ep.split(":")
self._broker_host = host
self._broker_port = int(port)
self.topic_pre = topic_pre
self.topic_index_location = topic_index_location
self.topic_index_device = topic_index_device
self.fiware_service = fiware_service
self.broker_user = broker_user
self.broker_user_pw = broker_user_pw
self.mqtts_enabled = mqtts_enabled
self.mqtts_ca_certs = mqtts_ca_certs or None
self.mqtts_certfile = mqtts_certfile or None
self.mqtts_keyfile = mqtts_keyfile or None
super(mqttConnector, self).__init__(*args, **kw)
def _on_register(self):
def on_connect(*_):
def callback(*args):
(_, _, message) = args
self.logger.info(
'Received message on topic %s' % message.topic)
self._handle_data(message.topic, message.payload)
topics = ['%s/#' % self.topic_pre]
for topic in topics:
self.client.message_callback_add(topic, callback)
self.client.subscribe([(topic, 1) for topic in topics])
self.client.on_connect = on_connect
# TODO(rst): this needs to be handled more general and from config
self.client.username_pw_set(self.broker_user, self.broker_user_pw)
if self.mqtts_enabled:
self.client.tls_set(
ca_certs=self.mqtts_ca_certs,
certfile=self.mqtts_certfile,
keyfile=self.mqtts_keyfile)
self.client.tls_insecure_set(True)
self.client.connect(self._broker_host, self._broker_port)
# TODO let gevent handle this
self.client.loop_forever()
def _on_shutdown(self):
self.client.disconnect()
def _get_target_container(self, location, device, sensor):
try:
return self._sensor_containers[(location, device, sensor)]
except KeyError:
try:
device_cnt = self._device_containers[(location, device)]
except KeyError:
try:
location_cnt = self._location_containers[location]
except KeyError:
location_cnt = self.create_container(None, location)
self._location_containers[location] = location_cnt
device_cnt = self.create_container(
location_cnt, device, labels=["openmtc:device"])
self._device_containers[(location, device)] = device_cnt
openmtc_id = "%s/%s/%s" % (
(self.fiware_service + '~' if self.fiware_service else '') +
location, device, sensor)
labels = ['openmtc:sensor_data', 'openmtc:id:%s' % openmtc_id]
sensor_cnt = self.create_container(
device_cnt, sensor, labels=labels)
self._sensor_containers[(location, device, sensor)] = sensor_cnt
return sensor_cnt
def _handle_data(self, topic, payload):
# get location and device
try:
location = topic.split('/')[self.topic_index_location]
device = topic.split('/')[self.topic_index_device]
except (AttributeError, ValueError):
self.logger.error("Topic '%s' not valid. Dropping." % topic)
return
# check payload
try:
readings = json_decode(
base64decode(json_decode(payload)['m2m:cin']['con']).decode('utf-8'))
except (ValueError, KeyError, TypeError):
self.logger.error('Damaged payload; discarding')
return
# push data
for _, values in groupby(readings, key=lambda x: x['n']):
sensor_cnt = self._get_target_container(location, device, 'number')
for value in sorted(values, key=lambda x: x['t']):
self.push_content(sensor_cnt, [value])

148
apps/mqttConnector/utils.py Normal file
View File

@ -0,0 +1,148 @@
import distutils.command.sdist
import distutils.command.build_py
import os
import subprocess
import sys
def echo(msg, *args):
if args:
msg = msg % args
sys.stdout.write(msg + "\n")
def get_packages(package, package_dir, excluded_list=None, included_list=None):
included_list = included_list or []
excluded_list = excluded_list or []
try:
root = package_dir[package]
except KeyError:
root = package_dir.get("", ".") + "/" + package
if not os.path.exists(root):
sys.stderr.write(
"Directory for package %s does not exist: %s\n" % (package, root))
sys.exit(1)
def on_error(error):
sys.stderr.write(
"Error while collecting packages for %s: %s\n" % (package, error))
sys.exit(1)
packages = [package]
r_prefix = len(root) + 1
for path, dirs, files in os.walk(root, onerror=on_error):
is_module = "__init__.py" in files and path != root
excluded = any([x in path for x in excluded_list])
included = any([x in path for x in included_list])
if is_module and (not excluded or included):
packages.append(package + "." + path[r_prefix:].replace("/", "."))
return packages
def get_pkg_files(base_dir, name):
package_files = []
pkg_dir = os.path.join(base_dir, 'src', name)
pkg_data_dir = os.path.join(pkg_dir, 'static')
for (path, directories, filenames) in os.walk(pkg_data_dir):
for filename in filenames:
package_files.append(os.path.join(os.path.relpath(path, pkg_dir),
filename))
return package_files
def enable_init_files(init_dir, init_dist_files):
for f in init_dist_files:
os.chmod(os.path.join(init_dir, os.path.basename(f)), 0o755)
def move_config_files(config_dir, config_files):
for f in config_files:
target_file = os.path.join(config_dir, f)
if not os.path.exists(target_file):
echo("Installing config file %s", target_file)
os.rename(target_file + ".dist", target_file)
# os.chmod(target_file, 0644)
else:
echo("Not overwriting config file %s", target_file)
def create_openmtc_user(db_dir=None, log_dir=None):
try:
from pwd import getpwnam
except ImportError:
print("Could not import the 'pwd' module. Skipping user management")
else:
# assuming DB_DIR was created by setup already
try:
pw = getpwnam('openmtc')
except KeyError as e:
try:
# add system user openmtc:openmtc
# useradd --system -UM openmtc
useradd = "useradd --system -UM openmtc"
retcode = subprocess.call(useradd, shell=True)
if retcode:
raise Exception("Failed to add user 'openmtc'")
pw = getpwnam('openmtc')
except Exception as e:
sys.stderr.write("Error creating user: %s\n" % (e, ))
sys.exit(1)
uid = pw.pw_uid
gid = pw.pw_gid
# set path permissions
if db_dir:
os.chown(db_dir, uid, gid)
if log_dir:
os.chown(log_dir, uid, gid)
class OpenMTCSdist(distutils.command.sdist.sdist):
def make_release_tree(self, base_dir, files):
distutils.command.sdist.sdist.make_release_tree(self, base_dir, files)
script_name = os.path.basename(sys.argv[0])
if script_name != "setup.py":
os.rename(base_dir + "/" + script_name, base_dir + "/setup.py")
self.filelist.files.remove(script_name)
self.filelist.files.append("setup.py")
class OpenMTCSdistBinary(OpenMTCSdist, object):
def make_release_tree(self, base_dir, files):
super(OpenMTCSdistBinary, self).make_release_tree(base_dir, files)
script_name = os.path.basename(sys.argv[0])
build_py = self.get_finalized_command('build_py')
build_py.compile = 1
build_py.optimize = 2
build_py.retain_init_py = 1
build_py.build_lib = base_dir
build_py.byte_compile(
[base_dir + "/" + f for f in self.filelist.files if
f != script_name and f.endswith(".py")])
class OpenMTCBuildPy(distutils.command.build_py.build_py):
retain_init_py = 0
def byte_compile(self, files):
distutils.command.build_py.build_py.byte_compile(self, files)
class OpenMTCBuildPyBinary(OpenMTCBuildPy, object):
retain_init_py = 0
def byte_compile(self, files):
super(OpenMTCBuildPyBinary, self).byte_compile(files)
for f in files:
if (f.endswith('.py') and (os.path.basename(f) != "__init__.py" or
not self.retain_init_py)):
os.unlink(f)

View File

@ -6,4 +6,4 @@ cd $(dirname ${0})
cd OrionContextBroker
PYTHONPATH=${PYTHONPATH}:src exec python -m orioncontextbroker $@
PYTHONPATH=${PYTHONPATH}:src exec python3 -m orioncontextbroker $@

View File

@ -1,11 +1,13 @@
import urllib
import urllib.request
import urllib.parse
import urllib.error
import ssl
from socket import (
gaierror,
error as socket_error,
)
from time import time
from urlparse import urlparse
from urllib.parse import urlparse
from aplus import Promise
from futile.caching import LRUCache
from geventhttpclient.client import HTTPClient
@ -48,7 +50,7 @@ _method_map_to_http = {
_clients = LRUCache(threadsafe=False)
_query_params = frozenset(['rt', 'rp', 'rcn', 'da', 'drt'])
_query_params = frozenset(['rt', 'rp', 'rcn', 'da', 'drt', 'rids', 'tids', 'ltids', 'tqi'])
_header_to_field_map = {
'X-M2M-ORIGIN': 'originator',
@ -59,6 +61,8 @@ _header_to_field_map = {
'X-M2M-RET': 'rqet',
'X-M2M-OET': 'oet',
'X-M2M-EC': 'ec',
'X-M2M-RVI': 'rvi',
'X-M2M-VSI': 'vsi',
}
@ -105,10 +109,12 @@ class OneM2MHTTPClient(OneM2MClient):
else:
ssl_options = None
client = HTTPClient(host, port, connection_timeout=120.0,
concurrency=50, ssl=is_https,
ssl_options=ssl_options, insecure=insecure)
self.request = client.request
def get_http_client():
return HTTPClient(host, port, connection_timeout=120.0,
concurrency=50, ssl=is_https,
ssl_options=ssl_options, insecure=insecure)
self._get_client = get_http_client
self.content_type = 'application/' + ('xml' if use_xml else 'json')
@ -141,7 +147,7 @@ class OneM2MHTTPClient(OneM2MClient):
filter_criteria = onem2m_request.fc
params.update({
(get_short_attribute_name(name) or get_short_member_name(name)): val
for name, val in filter_criteria.get_values(True).iteritems()
for name, val in filter_criteria.get_values(True).items()
})
if onem2m_request.ae_notifying:
@ -150,7 +156,7 @@ class OneM2MHTTPClient(OneM2MClient):
path = normalize_path(onem2m_request.to)
if params:
path += '?' + urllib.urlencode(params, True)
path += '?' + urllib.parse.urlencode(params, True)
content_type, data = encode_onem2m_content(onem2m_request.content, self.content_type, path=path)
@ -160,11 +166,14 @@ class OneM2MHTTPClient(OneM2MClient):
content_type += '; ty=' + str(ResourceTypeE[onem2m_request.resource_type.typename])
headers = {
header: getattr(onem2m_request, field) for header, field in _header_to_field_map.iteritems()
header: getattr(onem2m_request, field)
for header, field in _header_to_field_map.items()
if getattr(onem2m_request, field) is not None
}
headers['content-type'] = content_type
headers['accept'] = self.content_type
self.logger.debug("Added request params: %s", params)
return {
@ -194,16 +203,17 @@ class OneM2MHTTPClient(OneM2MClient):
get_response_status(rsc),
request=onem2m_request,
rsc=rsc,
pc=decode_onem2m_content(response.read(), response.get("content-type"))
pc=decode_onem2m_content(response.read().decode("utf-8"), response.get("content-type"))
)
def send_onem2m_request(self, onem2m_request):
with Promise() as p:
http_request = self.map_onem2m_request_to_http_request(onem2m_request)
t = time()
client = self._get_client()
try:
response = self.request(**http_request)
response = client.request(**http_request)
except (socket_error, gaierror) as exc:
self._handle_network_error(exc, p, http_request, t, ConnectionFailed)
except Exception as exc:
@ -218,5 +228,7 @@ class OneM2MHTTPClient(OneM2MClient):
p.fulfill(onem2m_response)
finally:
response.release()
finally:
client.close()
return p

View File

@ -32,7 +32,7 @@ from simplejson import (
JSONDecodeError,
)
from socket import error as SocketError
from urlparse import urlparse
from urllib.parse import urlparse
from openmtc_onem2m.util import split_onem2m_address
#: Dictionary mapping supported schemes to port numbers
@ -83,11 +83,12 @@ class OneM2MMQTTClient(OneM2MClient):
__request_fields = frozenset([
'op',
'to',
'fr',
'rqi',
'ty',
'pc',
'rol',
'rids',
'ot',
'rqet',
'rset',
@ -98,17 +99,28 @@ class OneM2MMQTTClient(OneM2MClient):
'ec',
'da',
'gid',
'drt',
'to',
'fc',
'drt',
'tids',
'ltids',
'tqi',
'rvi',
'vsi',
])
__response_fields = frozenset([
'rsc',
'rqi',
'pc',
'fr',
'to',
'fr',
'ot',
'rset',
'ec',
'cts',
'cto',
'rvi',
'vsi',
])
@staticmethod
@ -341,7 +353,7 @@ class OneM2MMQTTClient(OneM2MClient):
self.logger.debug('Decoded JSON request: %s' % (request, ))
op = OneM2MOperation._member_map_.values()[request['op'] - 1]
op = list(OneM2MOperation._member_map_.values())[request['op'] - 1]
to = request['to']
del request['op'], request['to']
@ -370,7 +382,8 @@ class OneM2MMQTTClient(OneM2MClient):
sp_id, cse_id, _ = split_onem2m_address(response.to)
response.content = self._decode(
encode_onem2m_content(response.content, 'application/json',
path=sp_id + cse_id)[1]
path=sp_id + cse_id,
fields=response.fields)[1]
)
self._publish_message(
@ -472,7 +485,8 @@ class OneM2MMQTTClient(OneM2MClient):
else:
request.ty = None
request.op = 1 + OneM2MOperation._member_map_.keys().index(OneM2MOperation[request.op].name)
request.op = 1 + list(OneM2MOperation._member_map_.keys()).index(
OneM2MOperation[request.op].name)
if request.pc:
request.pc = self._decode(
encode_onem2m_content(request.pc, 'application/json', path=request.to)[1]

View File

@ -13,6 +13,10 @@ STATUS_ACCEPTED = STATUS(
1000, "ACCEPTED", 202)
STATUS_OK = STATUS(
2000, "OK", 200)
STATUS_UPDATED = STATUS(
2004, "UPDATED", 200)
STATUS_DELETED = STATUS(
2002, "DELETED", 200)
STATUS_CREATED = STATUS(
2001, "CREATED", 201)
STATUS_BAD_REQUEST = STATUS(

View File

@ -1,7 +1,4 @@
try:
from urllib.parse import urlparse
except ImportError:
from urlparse import urlparse
from urllib.parse import urlparse
from openmtc.mapper import BasicMapper, MapperError
from openmtc_onem2m import OneM2MRequest
@ -49,7 +46,8 @@ class OneM2MMapper(BasicMapper):
path,
self.originator,
ty=type(instance),
pc=instance
pc=instance,
rvi='2a'
)).get()
try:
@ -83,7 +81,8 @@ class OneM2MMapper(BasicMapper):
OneM2MOperation.update,
instance.path,
self.originator,
pc=instance
pc=instance,
rvi='2a'
)).get()
try:
@ -105,6 +104,8 @@ class OneM2MMapper(BasicMapper):
path,
self.originator,
filter_criteria=fc,
rvi='2a',
rcn=5,
**request_options
)).get()
@ -112,7 +113,8 @@ class OneM2MMapper(BasicMapper):
self._send_request(OneM2MRequest(
OneM2MOperation.delete,
getattr(instance, "path", instance),
self.originator
self.originator,
rvi='2a'
))
# TODO(rst): check if this can be removed in parent class

View File

@ -2,11 +2,12 @@ from enum import IntEnum, unique
from openmtc.model import (Resource as Res, UnicodeAttribute, DatetimeAttribute,
Attribute, ListAttribute, Entity, EntityAttribute,
AnyURI, StringListAttribute, ContentResource)
AnyURI, StringListAttribute, ContentResource,
BytesAttribute)
from openmtc.model.exc import ModelTypeError
from futile import issubclass
LATEST_VERSION = "1.6"
LATEST_VERSION = "2a"
class OneM2MIntEnum(IntEnum):
@ -24,7 +25,7 @@ class OneM2MContentResource(ContentResource, OneM2MEntity):
class OneM2MResource(Res, OneM2MEntity):
__model_name__ = "onem2m"
__model_version__ = "1.6"
__model_version__ = "2a"
################################################################################
@ -41,8 +42,8 @@ class ResourceTypeE(OneM2MIntEnum):
eventConfig = 7
execInstance = 8
group = 9
localPolicy = 10
m2mServiceSubscriptionProfile = 11
locationPolicy = 10
m2mServiceSubscription = 11
mgmtCmd = 12
mgmtObj = 13
node = 14
@ -56,6 +57,16 @@ class ResourceTypeE(OneM2MIntEnum):
statsConfig = 22
subscription = 23
semanticDescriptor = 24
notificationTargetMgmtPolicyRef = 25
notificationTargetPolicy = 26
policyDeletionRules = 27
flexContainer = 28
timeSeries = 29
timeSeriesInstance = 30
role = 31
token = 32
trafficPattern = 33
dynamicAuthorizationConsultation = 34
accessControlPolicyAnnc = 10001
AEAnnc = 10002
containerAnnc = 10003
@ -66,6 +77,12 @@ class ResourceTypeE(OneM2MIntEnum):
nodeAnnc = 10014
remoteCSEAnnc = 10016
scheduleAnnc = 10018
semanticDescriptorAnnc = 10024
flexContainerAnnc = 10028
timeSeriesAnnc = 10029
timeSeriesInstanceAnnc = 10030
trafficPatternAnnc = 10033
dynamicAuthorizationConsultationAnnc = 10034
@unique
@ -106,15 +123,17 @@ class ResponseType(OneM2MIntEnum):
# @unique
# class ResultConentE(OneM2MIntEnum):
# nothing = 0
# attributes = 1
# hierarchical_address = 2
# hierarchical_address_and_attributes = 3
# attributes_and_child_resources = 4
# attributes_and_child_resource_references = 6
# child_resource_references = 6
# original_resource = 7
class ResultContentE(OneM2MIntEnum):
nothing = 0
attributes = 1
hierarchical_address = 2
hierarchical_address_and_attributes = 3
attributes_and_child_resources = 4
attributes_and_child_resource_references = 5
child_resource_references = 6
original_resource = 7
child_resources = 8
modified_attributes = 9
@unique
@ -136,6 +155,7 @@ class RequestStatusE(OneM2MIntEnum):
@unique
class MemberTypeE(OneM2MIntEnum):
mixed = 0
accessControlPolicy = 1
AE = 2
container = 3
@ -159,7 +179,16 @@ class MemberTypeE(OneM2MIntEnum):
statsCollect = 21
statsConfig = 22
subscription = 23
semanticDescriptor = 24
notificationTargetMgmtPolicyRef = 25
notificationTargetPolicy = 26
policyDeletionRules = 27
flexContainer = 28
timeSeries = 29
timeSeriesInstance = 30
role = 31
token = 32
trafficPattern = 33
dynamicAuthorizationConsultation = 34
accessControlPolicyAnnc = 10001
AEAnnc = 10002
@ -170,11 +199,15 @@ class MemberTypeE(OneM2MIntEnum):
mgmtObjAnnc = 10013
nodeAnnc = 10014
remoteCSEAnnc = 10016
scheduleAnnc = 10019
scheduleAnnc = 10018
semanticDescriptorAnnc = 10024
flexContainerAnnc = 10028
timeSeriesAnnc = 10029
timeSeriesInstanceAnnc = 10030
trafficPatternAnnc = 10033
dynamicAuthorizationConsultationAnnc = 10034
mixed = 24
# Mixed is a mixture of the resource types from 1 to 23, 10001 to 10004, 10009 to 10010,
# 10013 to 10014 and 10016 to 10018 as listed above.
oldest = 20001
latest = 20002
@unique
@ -375,7 +408,7 @@ class FilterUsageE(OneM2MIntEnum):
@unique
class CountryCodeE(OneM2MIntEnum):
india = 91
usa = 01
usa = 1
@unique
@ -568,7 +601,7 @@ class ResourceC(LabeledResource):
typename = None
resourceName = UnicodeAttribute(accesstype=Attribute.WO)
resourceName = UnicodeAttribute(accesstype=Attribute.WO, mandatory=False)
resourceType = EntityAttribute(ResourceTypeE, accesstype=Attribute.RO)
resourceID = IDS(accesstype=Attribute.RO)
@ -757,8 +790,7 @@ class Subscription(RegularResourceC):
notificationForwardingURI = Attribute(AnyURI)
batchNotify = EntityAttribute(BatchNotify)
rateLimit = EntityAttribute(RateLimit)
preSubscriptionNotify = Attribute(int, accesstype=Attribute.WO,
mandatory=False)
preSubscriptionNotify = Attribute(int, accesstype=Attribute.WO, mandatory=False)
pendingNotification = Attribute(PendingNotificationE)
notificationStoragePriority = Attribute(int)
latestNotify = Attribute(bool)
@ -952,15 +984,15 @@ class ContentInstance(AnnounceableSubordinateResourceC,
SubscribableResource):
"""See TS-0001 section 9.6.7"""
stateTag = UnicodeAttribute(accesstype=Attribute.RO)
stateTag = Attribute(int, accesstype=Attribute.RO)
creator = UnicodeAttribute() # m2m:ID
# contentInfo = typeOfContent(:EncodingType)
# typeOfContent => Media Types
# ex: application/json:1
contentInfo = UnicodeAttribute() # m2m:contentInfo
contentSize = Attribute(int, accesstype=Attribute.RO)
ontologyRef = UnicodeAttribute(accesstype=Attribute.WO)
content = Attribute(bytes, accesstype=Attribute.WO, mandatory=True)
ontologyRef = UnicodeAttribute(accesstype=Attribute.WO, mandatory=False)
content = BytesAttribute(accesstype=Attribute.WO, mandatory=True)
__child_types__ = (
Subscription,
@ -969,11 +1001,11 @@ class ContentInstance(AnnounceableSubordinateResourceC,
class ContentInstanceAnnc(AnnouncedSubordinateResourceC):
stateTag = UnicodeAttribute(accesstype=Attribute.RO)
stateTag = Attribute(int, accesstype=Attribute.RO)
contentInfo = UnicodeAttribute(EncodingTypeE) # m2m:contentInfo
contentSize = Attribute(int, accesstype=Attribute.WO)
ontologyRef = UnicodeAttribute(accesstype=Attribute.WO)
content = Attribute(bytes, accesstype=Attribute.WO, mandatory=True)
ontologyRef = UnicodeAttribute(accesstype=Attribute.WO, mandatory=False)
content = BytesAttribute(accesstype=Attribute.WO, mandatory=True)
################################################################################
@ -983,7 +1015,7 @@ class ContentInstanceAnnc(AnnouncedSubordinateResourceC):
class Container(AnnounceableResourceC, SubscribableResource):
"""See TS-0001 section 9.6.6"""
stateTag = UnicodeAttribute(accesstype=Attribute.RO)
stateTag = Attribute(int, accesstype=Attribute.RO)
creator = UnicodeAttribute()
maxNrOfInstances = Attribute(int)
maxByteSize = Attribute(int)
@ -1012,7 +1044,7 @@ Container.__child_types__ = (
class ContainerAnnc(AnnouncedResourceC, SubscribableResource):
stateTag = UnicodeAttribute(accesstype=Attribute.RO)
stateTag = Attribute(int, accesstype=Attribute.RO)
maxNrOfInstances = Attribute(int)
maxByteSize = Attribute(int)
maxInstanceAge = UnicodeAttribute(mandatory=False) # todo
@ -1075,7 +1107,7 @@ class AEAnnc(AnnouncedResourceC, SubscribableResource):
typename = "AEAnnc"
appName = UnicodeAttribute(accesstype=Attribute.WO)
appName = UnicodeAttribute(accesstype=Attribute.WO, mandatory=False)
App_ID = UnicodeAttribute()
AE_ID = UnicodeAttribute()
pointOfAccess = StringListAttribute()
@ -1344,8 +1376,7 @@ long_to_short_attribute_mapping = {
"relatedSemantics": "rels",
}
short_to_long_attribute_mapping = {v: k for k, v in
long_to_short_attribute_mapping.items()}
short_to_long_attribute_mapping = {v: k for k, v in long_to_short_attribute_mapping.items()}
def get_long_attribute_name(n):
@ -1427,8 +1458,7 @@ long_to_short_resource_mapping = {
"dynamicAuthorizationConsultation": "dac"
}
short_to_long_resource_mapping = {v: k for k, v in
long_to_short_resource_mapping.items()}
short_to_long_resource_mapping = {v: k for k, v in long_to_short_resource_mapping.items()}
def get_long_resource_name(n):
@ -1552,8 +1582,7 @@ long_to_short_member_mapping = {
"escertkeMessage": "eckm"
}
short_to_long_member_mapping = {v: k for k, v in
long_to_short_member_mapping.items()}
short_to_long_member_mapping = {v: k for k, v in long_to_short_member_mapping.items()}
def get_long_member_name(n):
@ -1569,8 +1598,7 @@ long_to_short_root_mapping = {
"responsePrimitive": "rsp"
}
short_to_long_root_mapping = {v: k for k, v in
long_to_short_root_mapping.items()}
short_to_long_root_mapping = {v: k for k, v in long_to_short_root_mapping.items()}
def get_long_root_name(n):
@ -1604,8 +1632,7 @@ long_to_short_parameter_mapping = {
"responseStatusCode": "rsc"
}
short_to_long_parameter_mapping = {v: k for k, v in
long_to_short_parameter_mapping.items()}
short_to_long_parameter_mapping = {v: k for k, v in long_to_short_parameter_mapping.items()}
def get_long_parameter_name(n):
@ -1616,13 +1643,13 @@ def get_short_parameter_name(n):
return long_to_short_parameter_mapping.get(n)
_all_types = {k: v for k, v in globals().iteritems()
_all_types = {k: v for k, v in globals().items()
if issubclass(v, OneM2MEntity) and not v.__subclasses__()}
_all_types_short = {}
_all_types_long = {}
for k, v in _all_types.iteritems():
for k, v in _all_types.items():
if get_short_resource_name(k):
long_name = k
short_name = get_short_resource_name(k)
@ -1653,13 +1680,13 @@ for k, v in _all_types.iteritems():
_all_types_long[long_name] = v
_resource_types = {k: v for k, v in _all_types.iteritems()
_resource_types = {k: v for k, v in _all_types.items()
if issubclass(v, ResourceC)}
_resource_types_short = {}
_resource_types_long = {}
for k, v in _resource_types.iteritems():
for k, v in _resource_types.items():
if get_short_resource_name(k):
long_name = k
short_name = get_short_resource_name(k)
@ -1693,8 +1720,8 @@ def get_onem2m_resource_type(typename):
def get_onem2m_types():
return _all_types.values()
return list(_all_types.values())
def get_onem2m_resource_types():
return _resource_types.values()
return list(_resource_types.values())

View File

@ -22,7 +22,7 @@ def create_onem2m_serializer(content_type):
def get_onem2m_supported_content_types():
return _factories.keys()
return list(_factories.keys())
def get_onem2m_decoder(content_type):
@ -37,6 +37,8 @@ def get_onem2m_decoder(content_type):
serializer = create_onem2m_serializer(content_type)
_serializers[content_type] = serializer
return serializer
get_serializer = get_onem2m_decoder

View File

@ -22,9 +22,7 @@ def get_typename(tn):
return _typename_matcher.findall(tn).pop()
class OneM2MSerializer(LoggerMixin):
__metaclass__ = ABCMeta
class OneM2MSerializer(LoggerMixin, metaclass=ABCMeta):
@abstractmethod
def encode_resource(self, resource, response, pretty=False,
encoding="utf-8", fields=None):
@ -45,11 +43,11 @@ class OneM2MSerializer(LoggerMixin):
res_type = ResourceTypeE(v["type"])
res_cls = get_onem2m_resource_type(res_type.name)
return res_cls(v["name"], resourceID=v["value"], resourceType=res_type)
child_resource = map(map_child_resource, child_resource)
child_resource = list(map(map_child_resource, child_resource))
except (TypeError, AttributeError, KeyError, ValueError):
raise CSEValueError("Invalid entry in child resources: %s",
child_resource)
if resource_type is Notification and "notificationEvent" in data:
if resource_type is Notification and data.get("notificationEvent"):
representation = data["notificationEvent"]["representation"]
representation = self.decode(self.dumps(representation))
data["notificationEvent"]["representation"] = representation
@ -62,7 +60,10 @@ class OneM2MSerializer(LoggerMixin):
class OneM2MDictSerializer(OneM2MSerializer):
def encode_resource(self, resource, pretty=False, path=None, encoding="utf-8", fields=None,
encapsulated=False):
representation = resource.values
if fields and isinstance(resource, OneM2MResource):
representation = {k: v for k, v in resource.values.items() if fields and k in fields}
else:
representation = resource.values
self.logger.debug("Encoding representation: %s", representation)
@ -77,7 +78,7 @@ class OneM2MDictSerializer(OneM2MSerializer):
)
representation["notificationEvent"] = {
get_short_attribute_name(k) or get_short_member_name(k): v
for k, v in e.iteritems()
for k, v in e.items()
}
except (AttributeError, KeyError):
self.logger.exception("failed to encode notify")
@ -93,7 +94,7 @@ class OneM2MDictSerializer(OneM2MSerializer):
return resource_id
return val_path + resource_id
if isinstance(resource, OneM2MResource):
if isinstance(resource, OneM2MResource) and "childResource" in representation:
def get_child_rep(c):
return {
@ -101,7 +102,7 @@ class OneM2MDictSerializer(OneM2MSerializer):
"nm": c.basename,
"typ": c.resourceType
}
representation["childResource"] = map(get_child_rep, representation["childResource"])
representation["childResource"] = list(map(get_child_rep, representation["childResource"]))
if isinstance(resource, URIList):
representation = [make_val(path, x) for x in representation]
@ -112,53 +113,12 @@ class OneM2MDictSerializer(OneM2MSerializer):
if isinstance(resource.oldest, ContentInstance):
representation['oldest'] = resource.oldest.resourceID
# cleans representation
def clean_representation(o):
try:
# removes empty attributes
empty_keys = []
for k, v in o.items():
if v is None:
empty_keys.append(k)
elif isinstance(v, OneM2MEntity):
o[k] = self.encode_resource(v, pretty, path, encoding, fields)
elif isinstance(v, list):
def encode_list_item(item):
if isinstance(item, OneM2MEntity):
return self.encode_resource(item, pretty, path, encoding, fields)
return item
if len(v):
o[k] = map(encode_list_item, v)
else:
empty_keys.append(k)
else:
try:
if len(v) == 0:
empty_keys.append(k)
except TypeError:
pass
for k in empty_keys:
del o[k]
for k, v in o.items():
if not isinstance(v, (unicode, str, bool, datetime,
OneM2MIntEnum)):
clean_representation(v)
except AttributeError:
if isinstance(o, list):
for p in o:
clean_representation(p)
if not isinstance(resource, OneM2MContentResource):
representation = {
get_short_resource_name(k) or get_short_attribute_name(k) or
get_short_member_name(k): v for
k, v in representation.items()}
clean_representation(representation)
if not isinstance(resource, (OneM2MResource, Notification,
SecurityInfo, OneM2MContentResource)):
return representation
@ -175,7 +135,7 @@ class OneM2MDictSerializer(OneM2MSerializer):
return self.dumps({typename: representation})
def _handle_partial_addressing(self, resource, pretty):
for k, v in resource.iteritems():
for k, v in resource.items():
if k in ('latest', 'oldest') and isinstance(v, ContentInstance):
resource[k] = v.resourceID
if pretty:
@ -186,7 +146,7 @@ class OneM2MDictSerializer(OneM2MSerializer):
def convert_to_long_keys(d):
return {get_long_resource_name(k) or get_long_attribute_name(k) or
get_long_member_name(k) or k: v for k, v in d.iteritems()}
get_long_member_name(k) or k: v for k, v in d.items()}
try:
if hasattr(s, "read"):
@ -199,7 +159,7 @@ class OneM2MDictSerializer(OneM2MSerializer):
self.logger.debug("Read data: %s", data)
try:
typename, data = data.items()[0]
typename, data = list(data.items())[0]
return get_onem2m_type(get_typename(typename)), data
except (AttributeError, IndexError, TypeError):
raise CSESyntaxError("Not a valid resource representation")

View File

@ -30,14 +30,11 @@ del logger
def _default(x):
if isinstance(x, datetime):
try:
isoformat = x.isoformat
except AttributeError:
raise TypeError("%s (%s)" % (x, type(x)))
return isoformat()
return x.strftime("%Y%m%dT%H%M%S")
elif isinstance(x, ContentInstance):
return x.resourceID
elif isinstance(x, bytes):
return x.decode('utf-8')
else:
try: # handle model classes
return x.values

View File

@ -6,7 +6,7 @@ logger = get_logger(__name__)
def decode_onem2m_content(content, content_type):
if content == "":
if not content:
content = None
if content_type and content is not None:
serializer = get_onem2m_decoder(content_type)
@ -27,9 +27,6 @@ def encode_onem2m_content(content, content_type, pretty=False, path=None,
if content is None:
return None, None
fields = fields # TODO(rst): maybe necessary
# fields = ["resourceID"]
serializer = get_onem2m_encoder(content_type)
data = serializer.encode_resource(content, pretty=pretty, path=path,

View File

@ -201,22 +201,25 @@ class OneM2MRequest(object):
"""Class representing a OneM2M request"""
def __init__(self, op, to, fr=None, rqi=None, ty=None, pc=None, rol=None,
def __init__(self, op, to, fr=None, rqi=None, ty=None, pc=None, rids=None,
ot=None, rqet=None, rset=None, oet=None, rt=None, rp=None,
rcn=None, ec=None, da=None, gid=None, filter_criteria=None,
fc=None, drt=None):
fc=None, drt=None, tids=None, ltids=None, tqi=None, rvi=None,
vsi=None):
# Operation
self.operation = op
# Target uri
self.to = to
# Originator ID
self.originator = fr # original long name is from
self.request_identifier = rqi or ''.join(random.sample(string.letters + string.digits, 16))
self.request_identifier = rqi or ''.join(
random.sample(string.ascii_letters + string.digits, 16)
)
# Type of a created resource
self.resource_type = ty
# Resource content to be transferred.
self.content = pc
self.role = rol
self.role_ids = rids
self.originating_timestamp = ot
self.request_expiration_timestamp = rqet
self.result_expiration_timestamp = rset
@ -230,6 +233,11 @@ class OneM2MRequest(object):
self.filter_criteria = filter_criteria or fc
# Optional Discovery result type
self.discovery_result_type = drt
self.token_ids = tids
self.local_token_ids = ltids
self.token_request_identifier = tqi
self.release_version_indicator = rvi
self.vendor_information = vsi
@property
def op(self):
@ -272,12 +280,12 @@ class OneM2MRequest(object):
self.content = pc
@property
def rol(self):
return self.role
def rids(self):
return self.role_ids
@rol.setter
def rol(self, rol):
self.role = rol
@rids.setter
def rids(self, rids):
self.role_ids = rids
@property
def ot(self):
@ -375,9 +383,49 @@ class OneM2MRequest(object):
def drt(self, drt):
self.discovery_result_type = drt
@property
def tids(self):
return self.token_ids
@tids.setter
def tids(self, tids):
self.token_ids = tids
@property
def ltids(self):
return self.local_token_ids
@ltids.setter
def ltids(self, rvi):
self.local_token_ids = ltids
@property
def tqi(self):
return self.token_request_identifier
@tqi.setter
def tqi(self, tqi):
self.token_request_identifier = tqi
@property
def rvi(self):
return self.release_version_indicator
@rvi.setter
def rvi(self, rvi):
self.release_version_indicator = rvi
@property
def vsi(self):
return self.vendor_information
@vsi.setter
def vsi(self, vsi):
self.vendor_information = vsi
def __str__(self):
return '%s: %s' % (self.__class__.__name__, ' | '.join([
'%s: %s' % (str(k), str(v)) for k, v in self.__dict__.iteritems()
'%s: %s' % (str(k), str(v)) for k, v in self.__dict__.items()
]))
@ -385,7 +433,8 @@ class OneM2MResponse(object):
"""Class representing a OneM2M response"""
def __init__(self, status_code, request=None, rqi=None, pc=None, to=None,
fr=None, rsc=None):
fr=None, rsc=None, ot=None, rset=None, ec=None, cts=None,
cto=None, rvi=None, vsi=None, fields=None):
# Operation result
if isinstance(status_code, STATUS):
self.response_status_code = status_code
@ -397,14 +446,27 @@ class OneM2MResponse(object):
self.to = request.to
# Originator ID
self.originator = request.fr
self.originating_timestamp = request.ot
self.result_expiration_timestamp = request.rset
self.event_category = request.ec
self.release_version_indicator = request.rvi
self.vendor_information = request.vsi
else:
self.request_identifier = rqi
# Target uri
self.to = to
# Originator ID
self.originator = fr
self.originating_timestamp = ot
self.release_version_indicator = rvi
self.vendor_information = vsi
self.result_expiration_timestamp = rset
self.event_category = ec
# Resource content to be transferred.
self.content = pc
self.content_status = cts
self.content_offset = cto
self.fields = fields
@property
def status_code(self):
@ -438,9 +500,65 @@ class OneM2MResponse(object):
def fr(self, fr):
self.originator = fr
@property
def ot(self):
return self.originating_timestamp
@ot.setter
def ot(self, ot):
self.originating_timestamp = ot
@property
def rset(self):
return self.result_expiration_timestamp
@rset.setter
def rset(self, rset):
self.result_expiration_timestamp = rset
@property
def ec(self):
return self.event_category
@ec.setter
def ec(self, ec):
self.event_category = ec
@property
def cts(self):
return self.content_status
@cts.setter
def cts(self, cts):
self.content_status = cts
@property
def cto(self):
return self.content_offset
@cto.setter
def cto(self, cto):
self.content_offset = cto
@property
def rvi(self):
return self.release_version_indicator
@rvi.setter
def rvi(self, rvi):
self.release_version_indicator = rvi
@property
def vsi(self):
return self.vendor_information
@vsi.setter
def vsi(self, vsi):
self.vendor_information = vsi
def __str__(self):
return '%s: %s' % (self.__class__.__name__, ' | '.join([
'%s: %s' % (str(k), str(v)) for k, v in self.__dict__.iteritems()
'%s: %s' % (str(k), str(v)) for k, v in self.__dict__.items()
]))

View File

@ -1,3 +1,4 @@
import platform
import sys
from logging import DEBUG
from threading import Thread
@ -6,8 +7,8 @@ from traceback import print_stack
from futile.logging import LoggerMixin
from openmtc.exc import OpenMTCError
if sys.subversion[0] != "CPython":
from inspect import ismethod, getargspec
if platform.python_implementation != "CPython":
from inspect import ismethod, getfullargspec
# TODO: kca: can't pass in values for then/error currently
@ -160,11 +161,11 @@ class Promise(LoggerMixin):
"""
self._errbacks.append(f)
if sys.subversion[0] != "CPython":
if platform.python_implementation != "CPython":
def _invoke(self, func, value):
try:
if value is None:
args, _, _, _ = getargspec(func)
args = getfullargspec(func).args
arglen = len(args)
if not arglen or (arglen == 1 and ismethod(func)):
return func()
@ -181,11 +182,11 @@ class Promise(LoggerMixin):
try:
if value is None:
try:
target = func.im_func
target = func.__func__
except AttributeError:
argcount = func.func_code.co_argcount
argcount = func.__code__.co_argcount
else:
argcount = target.func_code.co_argcount - 1
argcount = target.__code__.co_argcount - 1
if argcount == 0:
return func()
@ -365,7 +366,7 @@ def listPromise(*args):
if not arg.isFulfilled():
return
value = map(lambda p: p.value, args)
value = [p.value for p in args]
ret._fulfill(value)
for arg in args:

View File

@ -31,9 +31,7 @@ class ExtraOptionsStrategy(Enum):
fatal = "fatal"
class ConfigurationOption(LoggerMixin):
__metaclass__ = ABCMeta
class ConfigurationOption(LoggerMixin, metaclass=ABCMeta):
def __init__(self, type, default=NOT_SET, converter=identity,
*args, **kw):
super(ConfigurationOption, self).__init__(*args, **kw)
@ -76,7 +74,7 @@ class ListOption(SimpleOption):
def _convert(self, v):
v = super(ListOption, self)._convert(v)
return map(self._convert_content, v)
return list(map(self._convert_content, v))
def _convert_content(self, v):
if not isinstance(v, self.content_type):
@ -92,7 +90,7 @@ class BooleanOption(ConfigurationOption):
def _convert(self, v):
if isinstance(v, (bool, int)):
return bool(v)
if isinstance(v, basestring):
if isinstance(v, str):
return v and v.lower() not in ("0", "no", "n", "f", "false")
raise ConfigurationValueError("Illegal value for boolean: %s" % (v, ))

View File

@ -5,7 +5,7 @@ from enum import Enum
from iso8601 import parse_date, ParseError
from operator import attrgetter
from futile import basestring, issubclass, NOT_SET
from futile import issubclass, NOT_SET
from futile.logging import LoggerMixin
from openmtc.model.exc import ModelError, ModelTypeError
@ -26,7 +26,7 @@ class Collection(Sequence, Mapping):
def __getitem__(self, index):
if isinstance(index, (int, slice)):
return self._map.values()[index]
return list(self._map.values())[index]
return self._map[index]
def __contains__(self, v):
@ -47,7 +47,7 @@ class Collection(Sequence, Mapping):
return self._map.get(k, default)
def __iter__(self):
return self._map.itervalues()
return iter(self._map.values())
def __len__(self):
return len(self._map)
@ -76,7 +76,7 @@ class Collection(Sequence, Mapping):
class Member(LoggerMixin):
def __init__(self, type=unicode, version="1.0", *args, **kw):
def __init__(self, type=str, version="1.0", *args, **kw):
super(Member, self).__init__(*args, **kw)
self.type = type
self.version = version
@ -109,7 +109,7 @@ class Attribute(Member):
RO = "RO"
WO = "WO"
def __init__(self, type=unicode, default=None,
def __init__(self, type=str, default=None,
accesstype=None, mandatory=None,
update_mandatory=None,
id_attribute=None, path_attribute=None,
@ -157,24 +157,22 @@ class Attribute(Member):
return self.default
try:
unicode
class BytesAttribute(Attribute):
def __init__(self, default=None, accesstype=None,
mandatory=None, *args, **kw):
super(BytesAttribute, self).__init__(type=bytes,
default=default,
accesstype=accesstype,
mandatory=mandatory, *args,
**kw)
class UnicodeAttribute(Attribute):
def __init__(self, default=None, accesstype=None,
mandatory=False, *args, **kw):
super(UnicodeAttribute, self).__init__(type=unicode,
default=default,
accesstype=accesstype,
mandatory=mandatory, *args,
**kw)
def convert(self, value, instance):
if isinstance(value, str):
return bytes(value, "utf-8")
return super(BytesAttribute, self).convert(value, instance)
def convert(self, value, instance):
if isinstance(value, str):
return value.decode("utf-8")
return super(UnicodeAttribute, self).convert(value, instance)
except NameError:
UnicodeAttribute = Attribute
UnicodeAttribute = Attribute
class DatetimeAttribute(Attribute):
@ -187,7 +185,7 @@ class DatetimeAttribute(Attribute):
**kw)
def convert(self, value, instance):
if isinstance(value, basestring):
if isinstance(value, str):
try:
return parse_date(value)
except ParseError as e:
@ -196,7 +194,7 @@ class DatetimeAttribute(Attribute):
class ListAttribute(Attribute):
def __init__(self, content_type=unicode, type=list,
def __init__(self, content_type=str, type=list,
default=NOT_SET, *args, **kw):
super(ListAttribute, self).__init__(type=type,
default=default, *args, **kw)
@ -239,7 +237,7 @@ class ListAttribute(Attribute):
class StringListAttribute(Attribute):
def __init__(self, content_type=unicode, type=list,
def __init__(self, content_type=str, type=list,
default=NOT_SET, *args, **kw):
super(StringListAttribute, self).__init__(type=type, default=default,
*args, **kw)
@ -427,24 +425,22 @@ class ResourceType(ABCMeta):
# TODO: caching
@property
def attribute_names(self):
return map(attrgetter("name"), self.attributes)
return list(map(attrgetter("name"), self.attributes))
@property
def collection_names(self):
return map(attrgetter("name"), self.collections)
return list(map(attrgetter("name"), self.collections))
@property
def subresource_names(self):
return map(attrgetter("name"), self.subresources)
return list(map(attrgetter("name"), self.subresources))
@property
def member_names(self):
return map(attrgetter("name"), self.__members__)
return list(map(attrgetter("name"), self.__members__))
class Entity(LoggerMixin):
__metaclass__ = ResourceType
class Entity(LoggerMixin, metaclass=ResourceType):
def __init__(self, *args, **kw):
self.set_values(kw)
@ -467,7 +463,7 @@ class Entity(LoggerMixin):
# TODO: proper solution?
if (v is not None and isinstance(member, ListAttribute) and
not isinstance(v, (list, tuple, set))):
v = v.values()[0]
v = list(v.values())[0]
setattr(self, member.name, v)
except KeyError:
pass
@ -487,7 +483,7 @@ class Entity(LoggerMixin):
"""
if values:
raise ModelTypeError("%s resource has no attribute %s" %
(self.typename, values.keys()[0]))
(self.typename, list(values.keys())[0]))
@classmethod
def get_typename(cls):
@ -569,7 +565,7 @@ class Resource(Entity):
__model_version__ = None
def __init__(self, path=None, parent=None, *args, **kw):
if path is not None and not isinstance(path, basestring):
if path is not None and not isinstance(path, str):
raise TypeError(path)
self.__path = path
self.parent = parent
@ -629,7 +625,7 @@ class Resource(Entity):
# FIXME: move into de-serializer and handle dicts
if (v is not None and isinstance(member, ListAttribute) and
not isinstance(v, (list, tuple, set))):
v = v.values()[0]
v = list(v.values())[0]
setattr(self, member.name, v)
except KeyError:
try:
@ -637,7 +633,7 @@ class Resource(Entity):
# TODO: proper solution?
if (v is not None and isinstance(member, ListAttribute) and
not isinstance(v, (list, tuple, set))):
v = v.values()[0]
v = list(v.values())[0]
setattr(self, member.name, v)
except KeyError:
pass

View File

@ -289,7 +289,7 @@ cd \$(dirname \${0})
cd ${APP_NAME}
PYTHONPATH=\${PYTHONPATH}:src exec python -m ${PKG_NAME} \$@
PYTHONPATH=\${PYTHONPATH}:src exec python3 -m ${PKG_NAME} \$@
EOF
chmod +x "${START_SCRIPT}"
@ -301,7 +301,7 @@ BIN_SCRIPT="${APP_FOLDER}/bin/openmtc-${APP_SCRIPT}"
cat > "${BIN_SCRIPT}" << EOF
#!/usr/bin/env bash
exec python -m ${PKG_NAME} \$@
exec python3 -m ${PKG_NAME} \$@
EOF
chmod +x "${BIN_SCRIPT}"
@ -333,7 +333,7 @@ fi
# create setup file
SETUP_FILE="${APP_FOLDER}/setup-${PKG_NAME}.py"
cat > "${SETUP_FILE}" << EOF
#!/usr/bin/env python
#!/usr/bin/env python3
from setuptools import setup
from distutils.core import setup
@ -489,7 +489,7 @@ mv \${CONFIG_TEMP} \${CONFIG_FILE}
echo "done"
exec python -m ${PKG_NAME} \$@
exec python3 -m ${PKG_NAME} \$@
EOF
if [ ${WEB_APP} == "false" ]; then

View File

@ -47,7 +47,7 @@ if [ ${#find_result[*]} -eq 0 ]; then
fi
if [ ${#find_result[*]} -gt 1 ]; then
echo "Too many setup files matching the name. Exiting Now!."
echo "Too many setup files matching the name. Exiting Now!. "
exit 1
fi
@ -68,7 +68,7 @@ get_target_from_setup_file ()
local module_name=${setup_file%.py}
cd ${working_dir}
python - << END_OF_PYTHON
python3 - << END_OF_PYTHON
from importlib import import_module
setup = import_module('${module_name}', '${module_name}')
print("%s-%s" % (setup.SETUP_NAME, setup.SETUP_VERSION))
@ -92,7 +92,7 @@ rm -rf ${working_dir}/build
# build
cd ${working_dir}
python ${setup_file} bdist --plat-name docker >/dev/null 2>${log_file}
python3 ${setup_file} bdist --plat-name docker >/dev/null 2>${log_file}
# clean up after
rm -rf ${working_dir}/build
@ -113,14 +113,14 @@ rm ${log_file}
################################################################################
# clean binary_package
binary_archive="${working_dir}/dist/${binary_prefix}.docker.tar.gz"
printf "### Stripping .py files..."
#printf "### Stripping .py files..."
cp ${target_file} ${binary_archive}
gzip -d ${binary_archive}
tar --wildcards --delete -f ${binary_archive%".gz"} "*.py"
gzip ${binary_archive%".gz"}
printf "done\n"
#gzip -d ${binary_archive}
#tar --wildcards --delete -f ${binary_archive%".gz"} "*.py"
#gzip ${binary_archive%".gz"}
#printf "done\n"
rm ${target_file}
printf "### Created binary archive at %s.\n" ${binary_archive}
#printf "### Created binary archive at %s.\n" ${binary_archive}
################################################################################
# set correct permissions

View File

@ -260,6 +260,14 @@ rm -f "${target_docker_binary}"
rm -f "${docker_tmp}/${name}-dependencies.txt"
${docker_cmd} rm -f ${build_container_name} &> /dev/null
printf "done\n"
# remove dangling images
separator_line
printf "### Removing dangled images..."
for image in $(${docker_cmd} images -qa -f "dangling=true"); do
${docker_cmd} rmi -f ${image} > /dev/null
done
printf "done\n"
}
trap cleanup SIGINT SIGTERM
@ -391,7 +399,7 @@ printf "%s\n" $(get_requirements_from_setup_file) | tr " " "\n" > \
separator_line
printf "### Building %s-%s container...\n" ${name} ${machine}
${docker_cmd} build -t ${target_docker_name} \
${docker_cmd} build --force-rm -t ${target_docker_name} \
-f ${target_docker_file} ${docker_path}
if [ $? -gt 0 ]; then
printf "### Building %s-%s container failed. Exiting now.\n" \
@ -405,15 +413,6 @@ printf "### Base %s-%s container built successfully.\n" ${name} ${machine}
# cleanup
cleanup
##############################################################################
# remove dangling images
separator_line
printf "### Removing dangled images..."
for image in $(${docker_cmd} images -qa -f "dangling=true"); do
${docker_cmd} rmi -f ${image} > /dev/null
done
printf "done\n"
##############################################################################
# example to run the docker file
#${docker_cmd} run --name test -d \

View File

@ -1,5 +1,7 @@
import urllib
import urllib.request
import urllib.parse
import urllib.error
from openmtc_app.onem2m import XAE
import uuid
@ -15,9 +17,9 @@ class DataVisualization(XAE):
self.sensor_register = []
self.sensor_values = []
self.name = uuid.uuid1()
self.things_name = urllib.urlopen("https://dweet.io/follow/%s" % self.name)
print "Thing name :", self.name
print "link for the current data type and values :", self.things_name.geturl()
self.things_name = urllib.request.urlopen("https://dweet.io/follow/%s" % self.name)
print("Thing name :", self.name)
print("link for the current data type and values :", self.things_name.geturl())
# start endless loop
self.periodic_discover(self.remote_cse,
{'labels': ["openmtc:sensor_data"]},
@ -28,19 +30,19 @@ class DataVisualization(XAE):
self.add_container_subscription(uri, self.handle_sensor_data)
def handle_sensor_data(self, container, content):
data ={}
data = {}
self.sensor_register.append(content[0]['n'])
self.sensor_values.append(content[0]['v'])
for i, k in zip(self.sensor_register , self.sensor_values):
for i, k in zip(self.sensor_register, self.sensor_values):
data.update({i: k})
params = urllib.urlencode(data)
urllib.urlopen("https://dweet.io/dweet/for/%s?%s" % (self.name, params))
params = urllib.parse.urlencode(data)
urllib.request.urlopen("https://dweet.io/dweet/for/%s?%s" % (self.name, params))
if __name__ == "__main__":
from openmtc_app.flask_runner import SimpleFlaskRunner as Runner
ep = "http://localhost:8000"
Runner(DataVisualization(), port=6050, host='auto').run(ep)
Runner(DataVisualization(poas=['http://localhost:21345'])).run(ep)

View File

@ -91,4 +91,4 @@ if __name__ == "__main__":
from openmtc_app.flask_runner import SimpleFlaskRunner as Runner
ep = "http://localhost:8000"
Runner(DataAggregation(), port=6050, host='auto').run(ep)
Runner(DataAggregation(poas=['http://localhost:21346'])).run(ep)

View File

@ -19,4 +19,4 @@ if __name__ == "__main__":
from openmtc_app.flask_runner import SimpleFlaskRunner as Runner
ep = "http://localhost:8000"
Runner(SimpleDecision2(), port=6050, host='auto').run(ep)
Runner(SimpleDecision2(poas=['http://localhost:21387'])).run(ep)

View File

@ -47,4 +47,4 @@ if __name__ == "__main__":
from openmtc_app.flask_runner import SimpleFlaskRunner as Runner
ep = "http://localhost:8000"
Runner(SimpleDecision(), port=6050, host='auto').run(ep)
Runner(SimpleDecision(poas=['http://localhost:22245'])).run(ep)

View File

@ -31,5 +31,6 @@ app_file=${app_array[$[${choice}-1]]}
################################################################################
# run app_file
cd ${base_path}
. ../../common/prep-env.sh
python ${app_file}
cd ..
. ../common/prep-env.sh
python3 ${app_file}

View File

@ -0,0 +1,5 @@
![](pics/fiware_connector.png)
The FIWARE Connector provided by OpenMTC enables the exchange of sensor and actuator data between an OpenMTC Setup and FIWARE Setup with a running Orion Context Broker instance. This way it is possible to access oneM2M sensor data and control oneM2M actuators via FIWARE enabled applications.
OpenMTC together with the FIWARE Connector is available as FIWARE Generic Enabler in the FIWARE Catalogue (Link).

9
doc/index.md Normal file
View File

@ -0,0 +1,9 @@
# Open MTC IoT Agent
[![FIWARE IoT Agents](https://nexus.lab.fiware.org/repository/raw/public/badges/chapters/iot-agents.svg)](https://www.fiware.org/developers/catalogue/)
[![](https://nexus.lab.fiware.org/repository/raw/public/badges/stackoverflow/iot-agents.svg)](https://stackoverflow.com/questions/tagged/fiware+iot)
The OpenMTC SDK aims to provide developers with a convenient yet flexible tool
to write oneM2M compliant applications. This includes network applications
(NAs), gateway application (GAs), device applications (DAs), as well as
interworking proxy entities (IPEs).

Binary file not shown.

After

Width:  |  Height:  |  Size: 165 KiB

View File

@ -0,0 +1,190 @@
# Subscriptions and Notifications
## General Description of Subscription and Notification
The subscription and notification mechanism can be used to track changes on a resource. Therefore, a subscription is created by the resource subscriber containing subscription information. An AE or CSE can be the resource subscriber. When the resource is modified, the hosting CSE of the resource is sending a notification for the subscriber to the address(es) specified in the subscription information.
## The Subscription Resource
A subscription is represented by a `<subscription>` resource as child resource of the subscribed-to resource. The subscription contains subscription information, most importantly:
- the list `notificationURI`
- and the attribute `subscriberURI`.
The `notificationURI` list can contain one or more addresses where the resource subscriber wants to receive notifications. The attribute `subscriberURI` contains the address of the subscriber that is used by the CSE to send a notification when the `<subscription>` resource itself is deleted.
The `<subscription>` resource is supported by resource types like:
- CSEBase
- remoteCSE
- AE
- container
- contentInstance
- accessControlPolicy
- mgmtObj
**Note:** The `<contentInstance>` resource does not include the `<subscription>` resource as a child resource. To keep track of changes of `<contentInstance>` resources a subscription in the corresponding parent `container` resource needs to be created.
## Subscription and Notification using the Application Framework
The OpenMTC Application Framework provides the following methods to create subscriptions and handle notifications.
### The `add_container_subscription` method
The Application Framework provides the `add_container_subscription` method to create a subscription to the contentInstances of a given container and specify a handler function that is called when a notification is send. Optionally, a delete handler and filter criteria can be specified.
The method with its parameters is listed below:
`add_container_subscription(self, container, handler, delete_handler=None, filter_criteria=None)`
- `container`: container object or container path string
- `handler`: reference of the notification handling function
- `delete_handler`: (optional) reference to delete handling function
- `filter_criteria`: (optional) filterCriteria for the subscription
The parameters given to the `handler` function are:
- the path of the subscribed-to container
- the content data of the contentInstance which was created
The parameter given to the `delete_handler` function is:
- the path of the subscription
### Example: Using the `add_container_subscription` method
This example shows how the `add_container-subscription` method is used. The following code is part of the **training app** `onem2m-ipe-sensors-actuators-final` that can be found [here](./training/apps/onem2m/ipe/sensors-actuators/onem2m-ipe-sensors-actuators-final.py).
```python
# add commands_container of current actuator to self._command_containers
self._command_containers[actuator] = commands_container
# subscribe to command container of each actuator to the handler command
self.add_container_subscription(
commands_container.path, # the Container or it's path to be subscribed
self.handle_command # reference of the notification handling function
)
```
This code creates a subscription to the path of the `commands` container and registers the `handle_command` function as the notification handling function.
The `handle_command` function of the example application looks like this.
```python
def handle_command(self, container, value):
print('handle_command...')
print('container: %s' % container)
print('value: %s' % value)
print('')
```
In this example the handling function simply prints the path of the container and the JSON data to the terminal.
### Curl request for testing the `handle_command` notification handling function
In the following the REST API is used to demonstrate the `handle_command` handling function of the example from above. Therefore, an HTTP POST request is send to the `commands` subcontainer of the `Switch-AirCon` container. The request contains some JSON data that is pushed as a contentInstance to the container. The creation of the contentInstance triggers the notification to be send to the IPE, because of the subscription that was created with the `add_container_subscription` method. The `handle_command` function handles the notification.
For this example, the following example data is send:
```json
{
"switch_state": "on"
}
```
The data is encoded to its base64 string:
```
eyAic3dpdGNoX3N0YXRlIjogIm9uIiB9
```
To start the `onem2m-ipe-sensors-actuators-final` training application, go to the training folder `doc/training`:
```bash
$ cd doc/training
```
Run the **start-app** script to start one of the training applications:
```bash
$ `./start-app.sh`
```
Choose number `2` to start the `onem2m-ipe-sensors-actuators-final.py`
Send the following curl request to create the contentInstance and trigger the notification to be send:
```shell
curl -X POST localhost:8000/onem2m/TestIPE/devices/Switch-AirCon/commands -H 'Content-Type: application/vnd.onem2m-res+json' -d '{"m2m:cin": {"con": "eyAic3dpdGNoX3N0YXRlIjogIm9uIiB9", "cnf":"application/json:1"}}'
```
When sending the request, the notification is triggered and handled by the `handle_command` handling function that prints the following output to the terminal:
```
handle_command_cnt...
container: onem2m/TestIPE/devices/Switch-AirCon/commands
value: {u'switch_state': u'on'}
```
### The `add_subscription` method
This method creates a subscription resource at the given path and registers an handler to receive notification data. In comparison to the `add_container_subscription` method, the notification does not include the content data of a contentInstance, but information about the resource and subscription that triggered the notification.
The method with its parameters is listed below:
`add_subscription(self, path, handler, delete_handler=None)`
- `path`: path to subscribe to
- `handler`: notification handler
- `delete_handler`: reference to delete handling function
The notification that is send to the handling function includes the following three parameters:
- the path of the subscription
- the notification event type
- the resource representation
### Example: Using the `add_subscription` method
This example again uses the **training app** `onem2m-ipe-sensors-actuators-final`, but with some modified code using the `add_subscription` method that creates a subscription to the `Switch-AirCon`container.
```python
self.add_subscription(
"onem2m/TestIPE/devices/Switch-AirCon",
self.handle_command
)
```
The handle method is modified like this:
```python
def handle_command(self, sub, net, rep):
print('handle_command_general...')
print('subscription path: %s' % sub)
print('notification event type: %s' % net)
print('representation: %s' % rep)
print('')
```
The terminal output of the handle method is:
```
handle_command...
subscription path: onem2m/TestIPE/devices/Switch-AirCon/subscription-J3VOG9vvCfg2ifiR
notification event type: None
representation: Container(path='None', id='cnt1')
```
As a result, the information of the resource representation from the notification can be used to perform further requests on the container resource.
## Curl request to create a Subscription using the REST API
A subscription can also be created using the REST API. Therefore, a POST request is send to the path where the subscription should be created.
In the following this example shows a curl request that creates a subscription in the `Switch-AirCon`container of the `onem2m-ipe-sensors-actuators-final` training app from above.
```shell
curl -X POST localhost:8000/onem2m/TestIPE/devices/Switch-AirCon/commands -H 'Content-Type: application/vnd.onem2m-res+json;ty=23' -d '{ "m2m:sub": { "nu": "TestIPE", "rn":"sub101" }}'
```
This POST request creates a subscription in the `commands` container at the URI `localhost:8000/onem2m/TestIPE/devices/Switch-AirCon/commands`. The `resource type (ty)` number `23` included in the Content-Type header field represents the resource type of a subscription resource.
The subscription created by this curl request includes the `resource name (rn)` of the subscription which is `sub101` and the `notification URI (nu)` which is `TestIPE`, the AE-ID of the IPE. When the AE-ID is used, the CSE automatically replaces it with the appropriate absolute path of the application resource, which in this example is `["//openmtc.org/mn-cse-1/CTestIPE"]`.
Likewise, the curl request can include a list of notification URIs:
```shell
curl -X POST localhost:8000/onem2m/TestIPE/devices/Switch-AirCon/commands -H 'Content-Type: application/vnd.onem2m-res+json;ty=23' -d '{ "m2m:sub": { "nu":["//openmtc.org/mn-cse-1/CTestIPE"], "rn":"sub101" }}'
```

View File

@ -4,13 +4,13 @@ from openmtc_onem2m.model import AE
my_app = AE()
print my_app.path
print(my_app.path)
#>>> None
print my_app.App_ID
print(my_app.App_ID)
#>>> None
print my_app.parent_path
print(my_app.parent_path)
#>>> None
print my_app.labels
print(my_app.labels)
#>>> None
print my_app.attributes
print(my_app.attributes)
#>>> [UnicodeAttribute(name="AE-ID", type=unicode), UnicodeAttribute(name="App-ID", type=unicode), ListAttribute(name="accessControlPolicyIDs", type=list), ListAttribute(name="announceTo", type=list), UnicodeAttribute(name="announcedAttribute", type=unicode), ListAttribute(name="childResources", type=list), DatetimeAttribute(name="creationTime", type=datetime), DatetimeAttribute(name="expirationTime", type=datetime), UnicodeAttribute(name="labels", type=unicode), DatetimeAttribute(name="lastModifiedTime", type=datetime), UnicodeAttribute(name="name", type=unicode), UnicodeAttribute(name="nodeLink", type=unicode), UnicodeAttribute(name="ontologyRef", type=unicode), ListAttribute(name="pointOfAccess", type=list)]

View File

@ -26,13 +26,13 @@ promise = client.send_onem2m_request(onem2m_request)
# reteive the OneM2MResponse from the returned promise
onem2m_response = promise.get()
print onem2m_response.to
print(onem2m_response.to)
#>>> onem2m
print onem2m_response.response_status_code
print(onem2m_response.response_status_code)
#>>> STATUS(numeric_code=2001, description='CREATED', http_status_code=201)
print onem2m_response.content
print(onem2m_response.content)
#>>> AE(path='None', id='ae0')
print onem2m_response.content.App_ID
print(onem2m_response.content.App_ID)
#>>> myApp
print onem2m_response.content.labels
print(onem2m_response.content.labels)
#>>> [u'keyword1', u'keyword2']

View File

@ -17,12 +17,12 @@ promise = client.send_onem2m_request(onem2m_request)
onem2m_response = promise.get()
print onem2m_response.response_status_code
print(onem2m_response.response_status_code)
#>>> STATUS(numeric_code=2001, description='CREATED', http_status_code=201)
# Build path to retieve from
path = "onem2m/" + onem2m_response.content.resourceName
print path
print(path)
#>>> onem2m/MYAPP
# Retrieve the AE from the CSE
@ -30,18 +30,18 @@ onem2m_request = OneM2MRequest("retrieve", to=path)
promise = client.send_onem2m_request(onem2m_request)
onem2m_response = promise.get()
print onem2m_response.response_status_code
print(onem2m_response.response_status_code)
#>>> STATUS(numeric_code=2000, description='OK', http_status_code=200)
print onem2m_response.content
print(onem2m_response.content)
#>>> AE(path='None', id='ae0')
# Set the local AE to the retrieved content
my_app = None
my_app = onem2m_response.content
print my_app.App_ID
print(my_app.App_ID)
#>>> myApp
print my_app.resourceName
print(my_app.resourceName)
#>>> MYAPP
print my_app.labels
print(my_app.labels)
#>>> [u'keyword1', u'keyword2']

Some files were not shown because too many files have changed in this diff Show More