ExternalVendorCode/open-balena
1
0
Fork 0
mirror of https://github.com/balena-io/open-balena.git synced 2024-12-20 22:23:10 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge branch 'landr-test' of https://github.com/balena-io/open-balena into landr-test

Browse Source
This commit is contained in:
andrew 2022-04-25 15:28:56 -07:00
commit ea605d1444
12 changed files with 777 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.gitignore vendored
View File

@ -2,6 +2,10 @@
.project
.vagrant/
meta.json
landr-dist
*.traineddata
/config
/docker-compose.yml
/package-lock.json

48
README.md
View File

@ -1,27 +1,8 @@
![logo](./docs/assets/openbalena-logo.svg)
---
**openbalena is an open source platform that helps you deploy and manage connected devices. Devices run
[balenaOS][balena-os-website], a host operating system designed for running
containers on IoT devices, and are managed via the [balena CLI][balena-cli],
which you can use to configure your application containers, push updates, check
status, view logs, and more.**
**openbalena is an open source platform to manage IoT and edge device fleets at scale.**
## Highlights
* **Slim, container-based OS:** Bring the benefits of balenaOS, our container-optimized OS, to your entire fleet of edge devices.
* **Manage your fleet from the command line:** Provision and deploy code quickly to devices from the command line, update your entire fleet with a single command.
* **Extensible with powerful API & SDK:** Customize openbalena to fit your operational needs at any scale.
## Field-ready fleet management
openbalena’s backend services, composed of battle-tested components that we’ve run in production on [balenaCloud][balena-cloud-website]
for years, can store device information securely and reliably, allow remote
management via a built-in VPN service, and efficiently distribute container
images to your devices.
## Features
- **Simple provisioning**: Adding devices to your fleet is a breeze
- **Easy updates**: Remotely update the software on your devices with a single command
- **Container-based**: Benefit from the power of virtualization, optimized for the edge
@ -29,13 +10,17 @@ images to your devices.
- **Powerful API & SDK**: Extend openBalena to fit your needs
- **Built-in VPN**: Access your devices regardless of their network environment
To learn more about openBalena, visit [balena.io/open][open-balena-website].
## Motivation
## Getting Started
openbalena is a platform that helps you deploy and manage connected devices. Devices run [balenaOS][balena-os-website], a host operating system designed for running containers on IoT devices, and are managed via the [balena CLI][balena-cli], which you can use to configure your application containers, push updates, check status, view logs, and more.
Our [Getting Started guide][getting-started] is the most direct path to getting
an openBalena installation up and running and successfully deploying your
application to your device(s).
openbalena’s backend services, composed of battle-tested components that we’ve run in production on [balenaCloud][balena-cloud-website] for years, can store device information securely and reliably, allow remote management via a built-in VPN service, and efficiently distribute container images to your devices.
To learn more about openbalena, visit [balena.io/open][open-balena-website].
## Setup and Configuration
Our [Getting Started guide][getting-started] is the most direct path to getting an openbalena installation up and running and successfully deploying your application to your device(s).
## Compatibility
@ -48,7 +33,6 @@ If you are updating from previous openBalena versions, ensure you update the bal
CLI and reprovision any devices to at least the minimum required versions in order
for them to be fully compatible with this release, as some features may not work.
## Documentation
While we're still working on the project documentation, please refer to the
@ -105,8 +89,12 @@ improvements and new functionality is planned:
- Remote host OS updates
- Support for custom device types
## License
## Differences between openBalena and balenaCloud
OpenBalena is licensed under the terms of AGPL v3. See [LICENSE](LICENSE) for details.
### Differences between openBalena and balenaCloud
| openBalena | balenaCloud |
| ----- | ---- |
@ -124,12 +112,6 @@ improvements and new functionality is planned:
Additionally, refer back to the [roadmap](#roadmap) above for planned but not yet implemented features.
## License
OpenBalena is licensed under the terms of AGPL v3. See [LICENSE](LICENSE) for details.
[balena-cli]: https://github.com/balena-io/balena-cli
[balena-cloud-website]: https://balena.io/cloud
[balena-engine]: https://github.com/balena-os/balena-engine

14
docs/01-introduction.md Normal file
View File

@ -0,0 +1,14 @@
# Introduction
To understand how openbalena works, let's start with learning about balenaOS.
## What is balenaOS?
balenaOS has been designed to include the minimal set of required components to reliably support operation of the Docker engine in embedded scenarios. It uses the Yocto framework as a foundation, systemd as the init system.
The networking stack consists of Network Manager, DNSmasq and Modem Manager. We have found these components to be a robust stack for dealing with the diversity of hardware and unpredictability of configuration of networks in which a device may be booted.
In addition, we include Avahi, OpenSSH, and OpenVPN, which add support for mDNS, SSH, and VPN connections respectively.
This foundation is uniquely suited to running arbitrary containers on a wide range of embedded devices which balenaOS supports. Balena has also made available a wide selection of base images for containers which are optimised for the same scenario and allow developers to create applications based on the Debian, Alpine, or Fedora distributions. That is not to say, of course, that any other container base image may not be used, but that the images by balena have been verified to work well with balenaOS, implementing patterns which are particularly suitable for embedded devices, like balenaOS itself.
![balenaOS components](/docs/assets/balenaOS-components.png)

366
docs/02-getting-started.md Normal file
View File

@ -0,0 +1,366 @@
# Getting started guide
This guide will walk you through the steps of deploying an openBalena server,
that together with the balena CLI, will enable you to create and manage a fleet
of devices running on your own infrastructure, on premises or in the cloud. The
openBalena servers must be reachable by the devices, which is easiest to achieve
with cloud providers like AWS, Google Cloud, Digital Ocean and others.
This guide assumes a setup with two separate machines:
- The openBalena _server_, running Linux. These instructions were tested with an
Ubuntu 18.04 x64 server.
- The _local machine_, running Linux, Windows or macOS where the balena CLI runs
(as a client to the openBalena server). The local machine should also have a
working installation of [Docker](https://docs.docker.com/get-docker/) so that
application images can be built and deployed to your devices, although it is
also possible to use balenaEngine on a balenaOS device instead of Docker.
### Preparing a server for openBalena
Login to the server via SSH and run the following commands.
1. First, install or update essential software:
```bash
apt-get update && apt-get install -y build-essential git docker.io libssl-dev nodejs npm
```
2. Install docker-compose:
```bash
curl -L https://github.com/docker/compose/releases/download/1.27.4/docker-compose-Linux-x86_64 -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
```
Test your docker-compose installation with `$ docker-compose --version`.
3. Create a new user, assign admin permissions and add to `docker` group:
```bash
adduser balena
usermod -aG sudo balena
usermod -aG docker balena
```
#### Install openBalena on the server
1. On the server still, login as the new user and change into the home directory:
```bash
su balena
cd ~
```
2. Clone the openBalena repository and change into the new directory:
```bash
git clone https://github.com/balena-io/open-balena.git
cd open-balena/
```
3. Run the `quickstart` script as below. This will create a new `config`
directory and generate appropriate SSL certificates and configuration for the
server. The provided email and password will be used to automatically create
the user account for interacting with the server and will be needed later on
for logging in via the balena CLI. Replace the domain name for the `-d`
argument appropriately.
```bash
./scripts/quickstart -U <email@address> -P <password> -d mydomain.com
```
For more available options, see the script's help:
```bash
./scripts/quickstart -h
```
4. At this point, the openBalena server can be started with:
```bash
systemctl start docker
./scripts/compose up -d
```
The `-d` argument spawns the containers as background services.
5. Tail the logs of the containers with:
```bash
./scripts/compose exec <service-name> journalctl -fn100
```
Replace `<service-name>` with the name of any one of the services defined
in `compose/services.yml`; eg. `api` or `registry`.
6. The server can be stopped with:
```bash
./scripts/compose stop
```
When updating openBalena to a new version, the steps are:
```bash
./scripts/compose down
git pull
./scripts/compose build
./scripts/compose up -d
```
#### Domain Configuration
The following CNAME records must be configured to point to the openBalena server:
```text
api.mydomain.com
registry.mydomain.com
vpn.mydomain.com
s3.mydomain.com
tunnel.mydomain.com
```
Check with your internet domain name registrar for instructions on how to
configure CNAME records.
#### Test the openBalena server
To confirm that everything is running correctly, try a simple request from the
local machine to the server:
```bash
curl -k https://api.mydomain.com/ping
OK
```
Congratulations! The openBalena server is up and running. The next step is to
setup the local machine to use the server, provision a device and deploy a
small project.
### Install self-signed certificates on the local machine
The installation of the openBalena server produces a few self-signed certificates
that must be installed on the local machine, so that it can securely communicate
with the server.
The root certificate is found at `config/certs/root/ca.crt` on the server. Copy
it to some folder on the local machine and keep a note the path -- it will be
used later during the CLI installation. Follow the steps below for the specific
platform of the local machine.
#### Linux:
```bash
sudo cp ca.crt /usr/local/share/ca-certificates/ca.crt
sudo update-ca-certificates
sudo systemctl restart docker
```
#### macOS:
```bash
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ca.crt
osascript -e 'quit app "Docker"' && open -a Docker
```
#### Windows:
```bash
certutil -addstore -f "ROOT" ca.crt
```
The Docker daemon on the local machine must then be restarted for Docker to
pick up the new certificate.
### Install the balena CLI on the local machine
Follow the [balena CLI installation
instructions](https://github.com/balena-io/balena-cli/blob/master/INSTALL.md)
to install the balena CLI on the local machine.
By default, the CLI targets the balenaCloud servers at `balena-cloud.com`, and
needs to be configured to target the openBalena server instead. Add the following
line to the CLI's configuration file, replacing `"mydomain.com"` with the domain
name of the openBalena server:
```yaml
balenaUrl: 'mydomain.com'
```
The CLI configuration file can be found at:
- On Linux or macOS: `~/.balenarc.yml`
- On Windows: `%UserProfile%\_balenarc.yml`
If the file does not already exist, just create it.
Wrapping up the CLI installation, set an environment variable that points to the
root certificate copied previously on the local machine. This step is to ensure
the CLI can securely interact with the openBalena server.
| Shell | Command |
| ------------------ | ---------------------------------------------- |
| bash | `export NODE_EXTRA_CA_CERTS='/path/to/ca.crt'` |
| Windows cmd.exe | `set NODE_EXTRA_CA_CERTS=C:\path\to\ca.crt` |
| Windows PowerShell | `$Env:NODE_EXTRA_CA_CERTS="C:\path\to\ca.crt"` |
### Deploy an application
The commands below should be run on a terminal on the local machine (where the
balena CLI is installed). Ensure that the `NODE_EXTRA_CA_CERTS` environment
variable is set, as discussed above.
#### Login to openBalena
Run `balena login`, select `Credentials` and use the email and password
specified during quickstart to login to the openBalena server. At any time, the
`balena whoami` command may be used to check which server the CLI is logged in to.
#### Create an application
Create a new application with `balena app create myApp`. Select the application's
default device type with the interactive prompt. The examples in this guide assume
a Raspberry Pi 3.
An application contains devices that share the same architecture (such as ARM
or Intel i386), and also contains code releases that are deployed to the devices.
When a device is provisioned, it is added to an application, but can be migrated
to another application at any time. There is no limit to the number of applications
that can be created or to the number of devices that can be provisioned.
At any time, the server can be queried for all the applications it knows about
with the following command:
```bash
balena apps
ID APP NAME DEVICE TYPE ONLINE DEVICES DEVICE COUNT
1 myApp raspberrypi3
```
#### Provision a new device
Once we have an application, it’s time to start provisioning devices. To do this,
first download a balenaOS image from [balena.io](https://balena.io/os/#download).
Pick the development image that is appropriate for your device.
Unzip the downloaded image and use the balena CLI to configure it:
```bash
balena os configure ~/Downloads/balena-cloud-raspberrypi3-2.58.3+rev1-dev-v11.14.0.img --app myApp
```
Flash the configured image to an SD card using [Etcher](https://balena.io/etcher).
Insert the SD card into the device and power it on. The device will register with
the openBalena server and after about two minutes will be inspectable:
```bash
balena devices
ID UUID DEVICE NAME DEVICE TYPE APPLICATION NAME STATUS IS ONLINE SUPERVISOR VERSION OS VERSION
4 59d7700 winter-tree raspberrypi3 myApp Idle true 11.14.0 balenaOS 2.58.3+rev1
balena device 59d7700
== WINTER TREE
ID: 4
DEVICE TYPE: raspberrypi3
STATUS: online
IS ONLINE: true
IP ADDRESS: 192.168.43.247
APPLICATION NAME: myApp
UUID: 59d7700755ec5de06783eda8034c9d3d
SUPERVISOR VERSION: 11.14.0
OS VERSION: balenaOS 2.58.3+rev1
```
It's time to deploy code to the device.
#### Deploy a project
Application release images are built on the local machine using the balena CLI.
Ensure the root certificate has been correctly installed on the local machine,
as discussed above.
Let's create a trivial project that logs "Idling...". On an empty directory,
create a new file named `Dockerfile.template` with the following contents:
```dockerfile
FROM balenalib/%%BALENA_MACHINE_NAME%%-alpine
CMD [ "balena-idle" ]
```
Then build and deploy the project with:
```bash
balena deploy myApp --logs
```
The project will have been successfully built when a friendly unicorn appears in
the terminal:
```bash
[Info] Compose file detected
...
[Info] Creating release...
[Info] Pushing images to registry...
[Info] Saving release...
[Success] Deploy succeeded!
[Success] Release: f62a74c220b92949ec78761c74366046
\
\
\\
\\
>\/7
_.-(6' \
(=___._/` \
) \ |
/ / |
/ > /
j < _\
_.-' : ``.
\ r=._\ `.
<`\\_ \ .`-.
\ r-7 `-. ._ ' . `\
\`, `-.`7 7) )
\/ \| \' / `-._
|| .'
\\ (
>\ >
,.-' >.'
<.'_.''
<'
```
This command packages up the local directory, creates a new Docker image from
it and pushes it to the openBalena server. In turn, the server will deploy it to
all provisioned devices and within a couple of minutes, they will all run the
new release. Logs can be viewed with:
```bash
balena logs 59d7700 --tail
[Logs] [10/28/2020, 11:40:16 AM] Supervisor starting
[Logs] [10/28/2020, 11:40:50 AM] Creating network 'default'
[Logs] [10/28/2020, 11:42:38 AM] Creating volume 'resin-data'
[Logs] [10/28/2020, 11:42:40 AM] Downloading image …
…
[Logs] [10/28/2020, 11:44:00 AM] [main] Idling...
```
Enjoy Balenafying All the Things!
## Next steps
- Try out [local mode](https://www.balena.io/docs/learn/develop/local-mode),
which allows you to build and sync code to your device locally for rapid
development.
- Develop an application with [multiple containers](https://www.balena.io/docs/learn/develop/multicontainer)
to provide a more modular approach to application management.
- Manage your device fleet with the use of [configuration](https://www.balena.io/docs/learn/manage/configuration/)
and [environment](https://www.balena.io/docs/learn/manage/serv-vars/) variables.
- Explore our [example projects](https://balena.io/blog/tags/etcher-featured/)
to give you an idea of more things you can do with balena.
- If you find yourself stuck or confused, help is just [a click away](https://www.balena.io/support).
- Pin selected devices to selected code releases using
[sample scripts](https://github.com/balena-io-examples/staged-releases).
- To change the superuser password after setting the credentials, follow this [forum post](https://forums.balena.io/t/upate-superuser-password/4738/6).

132
docs/03-architecture-review.md Normal file
View File

@ -0,0 +1,132 @@
# Architecture overview
## Introduction
BalenaOS is an operating system optimized for running [Docker](https://www.docker.com/) containers on embedded devices, with an emphasis on reliability over long periods of operation, as well as a productive developer workflow inspired by the lessons learned while building balena.
The core insight behind balenaOS is that Linux containers offer, for the first time, a practical path to using virtualization on embedded devices. VMs and hypervisors have led to huge leaps in productivity and automation for cloud deployments, but their abstraction of hardware, as well as their resource overhead and lack of hardware support, means that they are not suitable for embedded scenarios. With OS-level virtualization, as implemented for Linux containers, both of those objections are lifted for Linux devices, of which there are many in the Internet of Things.
BalenaOS is an operating system built for easy portability to multiple device types (via the [Yocto framework](https://www.yoctoproject.org/) and optimized for Linux containers, and Docker in particular. There are many decisions, large and small, we have made to enable that vision, which are present throughout our architecture.
The first version of balenaOS was developed as part of the balena platform, and has run on thousands of embedded devices on balena, deployed in many different contexts for several years. BalenaOS v2 represents the combination of the learnings we extracted over those years, as well as our determination to make balenaOS a first-class open-source project, able to run as an independent operating system, for any context where embedded devices and containers intersect.
We look forward to working with the community to grow and mature balenaOS into an operating system with even broader device support, a broader operating envelope, and as always, taking advantage of the most modern developments in security and reliability.
## Userspace Components
The balenaOS userspace packages only provide the bare essentials for running containers, while still offering flexibility. The philosophy is that software and services always default to being in a container unless they are generically useful to all containers, or they absolutely can’t live in a container. The userspace consists of many open source components, but in this section, we will highlight some of the most important services.
![BalenaOS Components](/docs/assets/balenaOS-components.png)
### systemd
[systemd](https://www.freedesktop.org/wiki/Software/systemd/) is the init system of balenaOS, and it is responsible for launching and managing all the other services. BalenaOS leverages many of the great features of systemd, such as adjusting OOM scores for critical services and running services in separate mount namespaces. systemd also allows us to manage service dependencies easily.
### Supervisor
The Supervisor is a lightweight container that runs on devices. Its main roles are to ensure your app is running, and keep communications with the balenaCloud API server, downloading new application containers and updates to existing containers as you push them in addition to sending logs to your dashboard. It also provides an [API interface](https://www.balena.io/docs/reference/supervisor/supervisor-api/), which allows you to query the update status and perform certain actions on the device.
### BalenaEngine
[BalenaEngine](https://www.balena.io/engine/) is balena's modified Docker daemon fork that allows the management and running of application service images, containers, volumes, and networking. BalenaEngine supports container deltas for 10-70x more efficient bandwidth usage, has 3.5x smaller binaries, uses RAM and storage more conservatively, and focuses on atomicity and durability of container pulling.
### NetworkManager and ModemManager
BalenaOS uses [NetworkManager](https://wiki.gnome.org/Projects/NetworkManager) accompanied by [ModemManager](https://www.freedesktop.org/wiki/Software/ModemManager/), to deliver a stable and reliable connection to the internet, be it via ethernet, WiFi or cellular modem. Additionally, to make headless configuration of the device’s network easy, there is a `system-connections` folder in the boot partition, which is copied into `/etc/NetworkManager/system-connections`. So any valid NetworkManager connection file can just be dropped into the boot partition before device commissioning.
### Avahi
In order to improve the [development experience](https://www.balena.io/docs/learn/develop/local-mode/) of balenaOS, there is an [Avahi](https://wiki.archlinux.org/index.php/Avahi) daemon that starts advertising the device as `balena.local` or `<hostname>.local` on boot if the image is a development image.
### Dnsmasq
[Dnsmasq](https://wiki.archlinux.org/index.php/Dnsmasq) manages the nameservers that NetworkManager provides for balenaOS. NetworkManager discovers the nameservers that can be used, and a binary called `resolvconf` writes them to a tmpfs location, from where Dnsmasq will take over and manage these nameservers to give the user the fastest most responsive DNS resolution.
### chrony
[chrony](https://chrony.tuxfamily.org/) is used by balenaOS to keep the system time synchronized.
__Note__: BalenaOS versions less than v2.13.0 used systemd-timesyncd for time management.
### OpenVPN
[OpenVPN](https://community.openvpn.net/openvpn) is used as the VPN service by balenaOS, which allows a device to be connected to remotely and enabling remote SSH access.
### OpenSSH
[OpenSSH](https://www.openssh.com/) is used in balenaOS as the SSH server and client, allowing remote login using the SSH protocol.
__Note__: BalenaOS versions < v2.38.0 use [dropbear](https://matt.ucc.asn.au/dropbear/dropbear.html) as the SSH server and client
## Image Partition Layout
![Image partition layout](/docs/assets/image-partition-layout.png)
The first partition, `resin-boot`, holds important boot files according to each board (e.g.m kernel image, bootloader image). It also holds the `config.json` file, which is the central point of [configuring balenaOS](https://www.balena.io/docs/reference/OS/configuration) and defining its behavior. For example, using `config.json`, you can set your hostname, add SSH keys, allow persistent logging, or define custom DNS servers.
`resin-rootA` is the partition that holds the read-only root filesystem; it holds almost everything that balenaOS is.
`resin-rootB` is an empty partition that is only used when the rootfs is to be updated. We follow the A-B update strategy for balenaOS upgrades. Essentially, we have one active partition that is the OS’s current rootfs and one dormant one that is empty. During a balenaOS [update](https://www.balena.io/docs/reference/OS/updates/self-service/) we download the new rootfs to the dormant partition and try to switch them. If the switch is successful, the dormant partition becomes the new rootfs, if not, we roll back to the old active partition.
`resin-state` is the partition that holds persistent data, as explained in the [Stateless and Read-only rootfs](#Stateless-and-Read-Only-rootFS) section.
`resin-data` is the storage partition that contains the Supervisor and application containers and volumes.
## Stateless and Read-Only rootFS
BalenaOS comes with a read-only root filesystem, so we can ensure our host OS is stateless, but we still need some data to be persistent over system reboots. We achieve this with a very simple mechanism, i.e., bind mounts.
BalenaOS contains a partition named `resin-state` that is meant to hold all this persistent data. Inside we populate a Linux filesystem hierarchy standard with the rootfs paths that we require to be persistent. After this partition is populated, we are ready to bind mount the respective rootfs paths to this read-write location, thus allowing different components (e.g., `journald`, when persistent logging is enabled) to be able to write data to disk.
A diagram of our read-only rootfs can be seen below:
![Read only rootFS](/docs/assets/balenaOS-read-only-rootfs.png)
## Development vs. Production images
Each version of balenaOS is available in development and production variants, both built from the same source, but with slightly differing feature sets. The development images enable a number of useful features while developing, namely:
* Passwordless [SSH access](https://www.balena.io/docs/learn/manage/ssh-access/) into balenaOS on port 22222 as the root user.
* Docker socket exposed on port `2375`, which allows `balena push` / `build` / `deploy`, that enables remote Docker builds on the target device (see [Deploy to your Fleet](https://www.balena.io/docs/learn/deploy/deployment/)).
* Getty console attached to tty1 and serial.
* Capable of entering [local mode](https://www.balena.io/docs/learn/develop/local-mode/) for rapid development of application containers locally.
__Note:__ Raspberry Pi devices don’t have Getty attached to serial.
Production images disable passwordless root access, and an SSH key must be [added](https://www.balena.io/docs/reference/OS/configuration/#sshkeys) to `config.json` to access a production image.
In both development and production versions of balenaOS, logs are written to an 8 MB journald RAM buffer in order to avoid wear on the flash storage used by most of the supported boards.
To persist logs on the device, enable persistent logging by setting the `"persistentLogging": true` [key](https://www.balena.io/docs/reference/OS/configuration/#persistentlogging) in `config.json`. The logs can be accessed via the host OS at `/var/log/journal`. For versions of balenaOS < 2.45.0, persistent logs are limited to 8 MB and stored in the state partition of the device. balenaOS versions >= 2.45.0 store a maximum of 32 MB of persistent logs in the data partition of the device.
## OS Yocto composition
BalenaOS is composed of multiple [Yocto](https://www.yoctoproject.org/) layers. The Yocto Project build system uses these layers to compile balenaOS for the various [supported devices](/os/docs/supported-boards/). Below is an example from the [Raspberry Pi family](https://github.com/balena-os/balena-raspberrypi/blob/master/layers/meta-balena-raspberrypi/conf/samples/bblayers.conf.sample).
__Note:__ Instructions for building your own version of balenaOS are available [here](/os/docs/custom-build/#Bake-your-own-Image).
| Layer Name | Repository | Description |
|--------------------------------------------|--------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| poky/meta | https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta | Poky build tools and metadata. |
| poky/meta-poky | https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta-poky | |
| meta-openembedded/meta-oe | https://github.com/openembedded/meta-openembedded/tree/master/meta-oe | Base layer for OpenEmbedded build system. |
| meta-openembedded/meta-filesystems | https://github.com/openembedded/meta-openembedded/tree/master/meta-filesystems | OpenEmbedded filesystems layer. |
| meta-openembedded/meta-networking | https://github.com/openembedded/meta-openembedded/tree/master/meta-networking | OpenEmbedded networking-related packages and configuration. |
| meta-openembedded/meta-python | https://github.com/openembedded/meta-openembedded/tree/master/meta-python | Layer containing Python modules for OpenEmbedded. |
| meta-raspberrypi | https://github.com/agherzan/meta-raspberrypi | General hardware specific BSP overlay for the Raspberry Pi device family. |
| meta-balena/meta-balena-common | https://github.com/balena-os/meta-balena/tree/development/meta-balena-common | Enables building balenaOS for supported machines. |
| meta-balena/meta-balena-warrior | https://github.com/balena-os/meta-balena/tree/development/meta-balena-warrior | Enables building balenaOS for Warrior supported BSPs. |
| balena-raspberrypi/meta-balena-raspberrypi | https://github.com/balena-os/balena-raspberrypi/tree/master/layers/meta-balena-raspberrypi | Enables building balenaOS for chosen meta-raspberrypi machines. |
| meta-rust | https://github.com/meta-rust/meta-rust | OpenEmbedded/Yocto layer for Rust and Cargo. |
At the base is [Poky](https://www.yoctoproject.org/software-item/poky/), the Yocto Project's reference distribution. Poky contains the OpenEmbedded Build System (BitBake and OpenEmbedded-Core) as well as a set of metadata. On top of Poky, we add the collection of packages from meta-openembedded.
The next layer adds the Board Support Package (BSP). This layer provides board-specific configuration and packages (e.g., bootloader and kernel), thus enabling building for physical hardware (not emulators).
The core code of balenaOS resides in the meta-balena-common layer. This layer also needs a Poky version-specific layer (e.g., meta-balena-warrior) based on the requirements of the BSP layer.
Next is the board-specific meta-balena configuration layer. This layer works in conjunction with a BSP layer. For example, the Raspberry Pi family is supported by the meta-raspberrypi BSP layer and the corresponding meta-balena-raspberrypi layer configures balenaOS to the Raspberry Pi's needs
The final meta-rust layer enables support for the rust compiler and the cargo package manager.
__Note:__ Instructions for adding custom board support may be found [here](/os/docs/custom-build/#Supporting-your-Own-Board).

206
docs/04-building-your-own.md Normal file
View File

@ -0,0 +1,206 @@
# Building your own image
## How to start
In order to build your very own version of balenaOS for one of our supported boards, you will first need to make sure you have a working [Yocto environment setup](http://www.yoctoproject.org/docs/current/yocto-project-qs/yocto-project-qs.html).
Then pick the device type you want to build, in this example we will use the Raspberry Pi 3. So first we need to grab the [`balena-raspberrypi`](https://github.com/balena-os/balena-raspberrypi) and initialise all its submodules.
``` bash
git clone https://github.com/balena-os/balena-raspberrypi
cd balena-raspberrypi/
git submodule update --init --recursive
```
We can then use the helpful `BARYS` tool to setup up and start our build. To see all the functionality `BARYS` provides run `./balena-yocto-scripts/build/barys -h`
from with in the repo.
Now to actually build a development version of balenaOS for the Raspberry Pi 3, we can run the following:
``` bash
./balena-yocto-scripts/build/barys -m raspberrypi3
```
Now sit tight and maybe go and make several cups of tea, this is going to take a little while.
By default, the images are created in the `build/tmp/deploy/<yocto-machine-name>` directory with the file extension `.resinos-img` or `.balenaos-img`. Rename the file extension to `.img` to flash the image to a SD card or USB drive using [balenaEtcher](https://www.balena.io/etcher/).
## Supporting your Own Board
Pre-requisites: a [Yocto](https://www.yoctoproject.org) Board Support Package (BSP) layer for your particular board. It should be compatible to the Yocto releases balenaOS supports.
Repositories used to build balenaOS host Operating System (OS) are typically named `balena-<board-family>`. For example, consider [balena-raspberrypi](https://github.com/balena-os/balena-raspberrypi) which is used for building the OS for [Raspberryi Pi](https://raspberrypi.org), or [balena-intel](https://github.com/balena-os/balena-intel) repository which can be used to build a balena image for the Intel NUC boards.
Contributing support for a new board consist of creating a a Yocto package that includes:
* general hardware support for the specific board,
* the balenaOS-specific software features,
* deployment-specific features (i.e. settings to create SD card images or self-flashing images)
The following documentation walks you through the steps of creating such a Yocto package. Because of the substantial difference between the hardware of many boards, this document provides general directions, and often it might be helpful to see the examples of already supported boards. The list of the relevant repositories is found the end of this document.
### Board Support Repository Breakout
The `balena-<board-family>` repositories use [git submodules](https://git-scm.com/docs/git-submodule) for including required Yocto layers from the relevant sub-projects.
The root directory shall contain 2 directory entries:
* a `layers` directory
* [balena-yocto-scripts](https://github.com/balena-os/balena-yocto-scripts) git submodule.
_Note: you add submodules by `git submodule add <url> <directory>`, see the git documentation for more details._
The root directory generally also includes the following files:
* `CHANGELOG.md`
* `LICENSE`
* `README.md`
* `VERSION`
and one or more files named `<yocto-machine-name>.coffee`, one for each of the boards that the repository will add support for (eg. [`raspberry-pi3.coffee`](https://github.com/balena-os/balena-raspberrypi/blob/master/raspberrypi3.coffee) for Raspberry Pi 3 in `balena-raspberrypi`). This file contains information on the Yocto build for the specific board, in [CoffeeScript](http://coffeescript.org/) format. A minimal version of this file, using Raspberry Pi 3 as the example, would be:
``` coffeescript
module.exports =
yocto:
machine: 'raspberrypi3'
image: 'balena-image'
fstype: 'balenaos-img'
version: 'yocto-jethro'
deployArtifact: 'balena-image-raspberrypi3.balenaos-img'
compressed: true
```
The `layers` directory contains the git submodules of the yocto layers used in the build process. This normally means the following components are present:
- [poky](https://www.yoctoproject.org/tools-resources/projects/poky) at the version/revision required by the board BSP
- [meta-openembedded](https://github.com/openembedded/meta-openembedded) at the revision poky uses
- [meta-balena](https://github.com/balena-os/meta-balena) using the master branch
- [oe-meta-go](https://github.com/balena-os/oe-meta-go) using the master branch (there were no branches corresponding to the yocto releases at the time this howto was written)
- Yocto BSP layer for the board (for example, the BSP layer for Raspberry Pi is [meta-raspberrypi](https://github.com/agherzan/meta-raspberrypi))
- any additional Yocto layers required by the board BSP (check the Yocto BSP layer of the respective board for instructions on how to build the BSP and what are the Yocto dependencies of that particular BSP layer)
In addition to the above git submodules, the "layers" directory also contains a `meta-balena-<board-family>` directory (please note this directory is _not_ a git submodule, but an actual directory in the repository). This directory contains the required customization for making a board balena enabled. For example, the [balena-raspberrypi](https://github.com/balena-os/balena-raspberrypi) repository contains the directory `layers/meta-balena-raspberrypi` to supplement the BSP from `layers/meta-raspberrypi` git submodule, with any changes that might be required by balenaOS.
The layout so far looks as follows:
```
├── CHANGELOG.md
├── LICENSE
├── README.md
├── VERSION
├── layers
│ ├── meta-openembedded
│ ├── meta-<board-family>
│ ├── meta-balena
│ ├── meta-balena-<board-family>
│ ├── oe-meta-go
│ └── poky
├── <board>.coffee
└── balena-yocto-scripts
```
### meta-balena-`<board-family>` breakout
This directory contains:
* `COPYING.Apache-2.0` file with the [Apache Version 2.0 license](http://www.apache.org/licenses/LICENSE-2.0),
* `README.md` file specifying the supported boards
and a number of directories out of which the mandatory ones are:
- `conf` directory - contains the following files:
- `layer.conf`, see the [layer.conf](https://github.com/balena-os/balena-raspberrypi/blob/master/layers/meta-balena-raspberrypi/conf/layer.conf) from `meta-balena-raspberrypi` for an example, and see [Yocto documentation](http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html#bsp-filelayout-layer)
- `samples/bblayers.conf.sample` file in which all the required Yocto layers are listed, see this [bblayers.conf.sample](https://github.com/balena-os/balena-raspberrypi/blob/master/layers/meta-balena-raspberrypi/conf/samples/bblayers.conf.sample) from `meta-balena-raspberrypi` for an example, and see the [Yocto documentation](http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html#var-BBLAYERS)
- `samples/local.conf.sample` file which defines part of the build configuration (see the meta-balena [README.md](https://github.com/balena-os/meta-balena/blob/master/README.md) for an overview of some of the variables use in the `local.conf.sample` file). An existing file can be used (e.g. [local.conf.sample](https://github.com/balena-os/balena-raspberrypi/blob/master/layers/meta-balena-raspberrypi/conf/samples/local.conf.sample)) but making sure the "Supported machines" area lists the appropriate machines this repository is used for. See also the [Yocto documentation](http://www.yoctoproject.org/docs/2.0/mega-manual/mega-manual.html#structure-build-conf-local.conf).
- `recipes-containers/docker-disk` directory, which contains `docker-balena-supervisor-disk.bbappend` that shall define the following variable(s):
- `SUPERVISOR_REPOSITORY_<yocto-machine-name>`: this variable is used to specify the build of the supervisor. It can be one of (must match the architecture of the board):
* **balena/armv7hf-supervisor** (for armv7 boards),
* **balena/i386-supervisor**
(for x86 boards),
* **balena/amd64-supervisor** (for x86-64 boards),
* **balena/rpi-supervisor** (for raspberry pi 1),
* **balena/armel-supervisor** (for armv5 boards).
- `LED_FILE_<yocto-machine-name>`: this variable should point to the [Linux sysfs path of an unused LED](https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-class-led) if available for that particular board. This allows the unused LED to be flashed for quick visual device identification purposes. If no such unused LED exists, this variable shall not be used.
- `recipes-core/images` directory, which contains at least a `balena-image.bbappend` file. Depending on the type of board you are adding support for, you should have your device support either just `balena-image` or both `balena-image-flasher` and `balena-image`. Generally, `balena-image` is for boards that boot directly
from external storage (these boards do not have internal storage to install balena on). `balena-image-flasher` is used when the targeted board has internal storage so this flasher image is burned onto an SD card or USB stick that is used for the initial boot. When booted, this flasher image will automatically install balena on internal storage.
The `balena-image.bbappend` file shall define the following variables:
- `IMAGE_FSTYPES_<yocto-machine-name>`: this variable is used to declare the type of the produced image (it can be ext3, ext4, balenaos-img etc. The usual type for a board that can boot from SD card, USB, is "balenaos-img").
- `BALENA_BOOT_PARTITION_FILES_<yocto-machine-name>`: this allows adding files from the build's deploy directory into the vfat formatted resin-boot partition (can be used to add bootloader config files, first stage bootloader, initramfs or anything else needed for the booting process to take place for your particular board). If the board uses different bootloader configuration files when booting from either external media (USB thumb drive, SD card etc.) or from internal media (mSATA, eMMC etc) then you would want make use of this variable to make sure the different bootloader configuration files get copied over and further manipulated as needed (see `INTERNAL_DEVICE_BOOTLOADER_CONFIG_<yocto-machine-name>` and `INTERNAL_DEVICE_BOOTLOADER_CONFIG_PATH_<yocto-machine-name>` below). Please note that you only reference these files here, it is the responsibility of a `.bb` or `.bbappend` to provide and deploy them (for bootloader config files this is done with an append typically in `recipes-bsp/<your board's bootloader>/<your board's bootloader>.bbappend`, see [balena-intel grub bbappend](https://github.com/balena-os/balena-intel/blob/master/layers/meta-balena-genericx86/recipes-bsp/grub/grub_%25.bbappend) for an example)
It is a space separated list of items with the following format: *FilenameRelativeToDeployDir:FilenameOnTheTarget*. If *FilenameOnTheTarget* is omitted then the *FilenameRelativeToDeployDir* will be used.
For example to have the Intel NUC `bzImage-intel-corei7-64.bin` copied from deploy directory over to the boot partition, renamed to `vmlinuz`:
```bash
BALENA_BOOT_PARTITION_FILES_nuc = "bzImage-intel-corei7-64.bin:vmlinuz"
```
The `balena-image-flasher.bbappend` file shall define the following variables:
- `IMAGE_FSTYPES_<yocto-machine-name>` (see above)
- `BALENA_BOOT_PARTITION_FILES_<yocto-machine-name>` (see above). For example, if the board uses different bootloader configuration files for booting from SD/USB and internal storage (see below the use of `INTERNAL_DEVICE_BOOTLOADER_CONFIG` variable), then make sure these files end up in the boot partition (i.e. they should be listed in this `BALENA_BOOT_PARTITION_FILES_<yocto-machine-name>` variable)
- `recipes-kernel/linux directory`: shall contain a `.bbappend` to the kernel recipe used by the respective board. This kernel `.bbappend` must "inherit kernel-balena" in order to add the necessary kernel configs for using with balena.
- `recipes-support/balena-init` directory - shall contain a `balena-init-flasher.bbappend` file if you intend to install balena to internal storage and hence use the flasher image. This shall define the following variables:
- `INTERNAL_DEVICE_KERNEL_<yocto-machine-name>`: this variable is used to identify the internal storage where balena will be written to.
- `INTERNAL_DEVICE_BOOTLOADER_CONFIG_<yocto-machine-name>`: this variable is used to specify the filename of the bootloader configuration file used by your board when booting from internal media (must be the same with the *FilenameOnTheTarget* parameter of the bootloader internal config file used in the `BALENA_BOOT_PARTITION_FILES_<yocto-machine-name>` variable from `recipes-core/images/balena-image-flasher.bbappend`)
- `INTERNAL_DEVICE_BOOTLOADER_CONFIG_PATH_<yocto-machine-name>`: this variable is used to specify the relative path (including filename) to the resin-boot partition where `INTERNAL_DEVICE_BOOTLOADER_CONFIG_<yocto-machine-name>` will be copied to.
For example, setting
```bash
INTERNAL_DEVICE_BOOTLOADER_CONFIG_intel-corei7-64 = "grub.cfg_internal"
```
and
```bash
INTERNAL_DEVICE_BOOTLOADER_CONFIG_PATH_intel-corei7-64 = "/EFI/BOOT/grub.cfg"
```
will result that after flashing the file `grub.cfg`_internal is copied with the name `grub.cfg` to the /EFI/BOOT/ directory on the resin-boot partition.
The directory structure then looks similar to this:
```
├── COPYING.Apache-2.0
├── README.md
├── conf
│ ├── layer.conf
│ └── samples
│ ├── bblayers.conf.sample
│ └── local.conf.sample
├── recipes-bsp
│ └── bootfiles
├── recipes-containers
│ └── docker-disk
│ └── docker-balena-supervisor-disk.bbappend
├── recipes-core
│ ├── images
│ │ └── balena-image.bbappend
├── recipes-kernel
│ └── linux
│ ├── linux-<board-family>-<version>
│ │ └── <patch files>
│ ├── linux-<board-family>_%.bbappend
│ └── linux-<board>_<version>.bbappend
└── recipes-support
└── balena-init
├── files
│ └── balena-init-board
└── balena-init-board.bbappend
```
### Building
See the [meta-balena Readme](https://github.com/balena-os/meta-balena/blob/master/README.md) on how to build the new balenaOS image after setting up the new board package as defined above.
### Configure
The image(s) created during this build process are considered "unmanaged". These can be flashed onto devices and used for local development. If you would like to configure this image to connect to balenaCloud when provisioned, then you would need a "managed" image. A managed image is pre-configured to provision to balenaCloud when booted. Use the [balena CLI os configure command](https://www.balena.io/docs/reference/balena-cli/#os-configure-image) to configure your image for [supported device types](https://www.balena.io/docs/reference/hardware/devices).
### Troubleshooting
For specific examples on how board support is provided for existing devices, see the repositories in the [Supported Boards](/os/docs/supported-boards/) section.

40
docs/05-supported-architectures.md Normal file
View File

@ -0,0 +1,40 @@
# Supported architectures
## ARMv7
| Device | Link | BalenaOS Repo |
|-----------------------------|------------------------------------------------------------------------------|--------------------------------------------------|
| Raspberry Pi 3 Model B | https://www.raspberrypi.org/products/raspberry-pi-3-model-b/ | https://github.com/balena-os/balena-raspberrypi |
| Raspberry Pi 2 Model B | https://www.raspberrypi.org/products/raspberry-pi-2-model-b/ | https://github.com/balena-os/balena-raspberrypi |
| Beaglebone Black | http://beagleboard.org/black | https://github.com/balena-os/balena-beaglebone |
| Beaglebone Green | http://beagleboard.org/green | https://github.com/balena-os/balena-beaglebone |
| Beaglebone Green Wireless | http://beagleboard.org/green-wireless | https://github.com/balena-os/balena-beaglebone |
| Samsung Artik 5 | https://www.artik.io/modules/overview/artik-5/ | https://github.com/balena-os/balena-artik |
| Samsung Artik 10 | https://www.artik.io/modules/overview/artik-10/ | https://github.com/balena-os/balena-artik |
| Odroid C1/C1+ | https://wiki.odroid.com/odroid-c1/odroid-c1 | https://github.com/balena-os/balena-odroid |
| Odroid XU4 | https://wiki.odroid.com/odroid-xu4/odroid-xu4 | https://github.com/balena-os/balena-odroid |
| SolidRun HummingBoard-base | https://www.solid-run.com/product/hummingboard-carrier-base/ | https://github.com/balena-os/balena-fsl-arm |
| Boundary Devices Nitrogen6X | https://boundarydevices.com/product/nitrogen6x/ | https://github.com/balena-os/balena-fsl-arm |
| Parallella Board | https://www.parallella.org/board/ | https://github.com/balena-os/balena-parallella |
| VIA vab820 | http://www.viatech.com/en/boards/pico-itx/vab-820/ | https://github.com/balena-os/balena-fsl-arm |
## ARMv6
| Device | Link | BalenaOS Repo |
|-----------------------------|------------------------------------------------------------------------------|--------------------------------------------------|
| Raspberry Pi 1 Model A+ | https://www.raspberrypi.org/products/raspberry-pi-1-model-a-plus/ | https://github.com/balena-os/balena-raspberrypi |
| Raspberry Pi ZERO | https://www.raspberrypi.org/products/raspberry-pi-zero/ | https://github.com/balena-os/balena-raspberrypi |
## ARMv5
| Device | Link | BalenaOS Repo |
|-----------------------------|------------------------------------------------------------------------------|--------------------------------------------------|
| Technologic Systems TS-7700 | http://wiki.embeddedarm.com/wiki/TS-7700 | https://github.com/balena-os/balena-ts |
## Intel x86_32
| Device | Link | BalenaOS Repo |
|-----------------------------|------------------------------------------------------------------------------|--------------------------------------------------|
| Intel Edison | http://www.intel.com/content/www/us/en/do-it-yourself/edison.html | https://github.com/balena-os/balena-edison |
## Intel x86_64
| Device | Link | BalenaOS Repo |
|-----------------------------|------------------------------------------------------------------------------|--------------------------------------------------|
| Intel NUC | http://www.intel.co.uk/content/www/uk/en/nuc/overview.html | https://github.com/balena-os/balena-intel |

0
docs/06-contributing-to-openbalena.md Normal file
View File

BIN
docs/assets/balenaOS-components.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 100 KiB

BIN
docs/assets/balenaOS-read-only-rootfs.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 174 KiB

BIN
docs/assets/image-partition-layout.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 122 KiB

BIN
logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 7.9 KiB