Install Flockport

Flockport currently supports the following distributions

  • Ubuntu Xenial
  • Ubuntu Trusty
  • Debian Jessie
  • CentOS 7
  • Alpine Linux 3.5

Download the installer from the downloads page and extract it.

tar -cpzf flockport-standalone.tar.gz

cd into the flockport folder and run install.sh. Please note you need to be root to install Flockport

./install.sh

This will automatically detect the OS and install Flockport and required packages.

Once installed you can run the help command to get a summary of available commands

flockport help

Running a subcommand provides usage information for the command

flockport run

A default Alpine Linux container is included to help you get started.

You can find screencasts that go over some of the functionality here
You can quickly test drive Flockport with VM images available here
Please disable Selinux or any firewalls before configuring containers, networks, storage and cluster services. They can interfere in unpredictable ways. Once configured services are working you can add the relevant exceptions and enable them again.

Flockport basics

We will use c1 and c2 as example container names in the sections below.

Use the list command to list all containers on your system.

flockport ls

This will list all containers.

To get an overview of runtime information use the ps command.

flockport ps

This will list all running containers.

To get detailed container specific information use the info command.

flockport info c1

This will show detailed information on the c1 container.

You can start a container with the start command

flockport start c1

This will start the c1 container.

You can now run ps or info to get live stats and detailed run time information on c1

flockport info c1

You can stop c1 with the stop command.

flockport stop c1

This will stop the c1 container.

You can access a running container with the attach command.

flockport attach c1

This will drop you into a shell in the container. You can run commands in the container and the environment is identical to an OS like a VM. Use the exit commands to exit the container shell.

You can directly execute commands on the container with the exec command.

flockport exec c1 ifconfig

This will run ifconfig in c1 and list network interfaces.

Containers can be deleted with the rm or remove command

flockport rm c1

This will delete the c1 container. Incase the c1 container has snapshots the delete will fail. You can only delete containers that have snapshots once all the snapshots have been deleted

Container ports

Containers run in an internal network that can only be accessed by other containers and the host. We will cover this in more depth in the container networking section.

To make any applications running in containers available to the outside world you need to forward container ports to the host.

Now any system that can access the host can acess the container on the forwarded port.

You can use the flockport pub command to forward ports

flockport pub c1 80:80

This forwards container c1's port 80 to host port 80. Similarly

flockport pub c1 80:3000

This forward c1's port 80 to host port 3000. You can also simply run

flockport pub c1 80

This forwards c1's port 80 to host port 80

To stop port forwarding run

flockport unpub c1 80:80

This will stop the port forwarding to the host.

If you forwarded the same port

flockport unpub c1 80

You can use the ps or info command to get an overview of forwarded ports

flockport info c1

When you stop a container any forwarded ports are automaticaly stopped. To ensure ports are published on container startup use the autoports command

flockport autoports c1 80:80

This will ensure c1 port 80 is published to host port 80 on container start

Its important to note here that all containers are available by their IP on the host. So from the host you can access all containers by their IP and do not need to forward ports. Port forwarding is only required when you need to access a container from beyond the host. In this case the container port is forwarded to the host so accessing the hostip:port will access the container

Volumes and storage

Volumes are a way to share host data with the container or separate any state data from the container. This is done so the container only holds the application and the application data is kept elsewhere.

You are perfectly free to store data in the container and the data is safe untill you delete the container.

Flockport is not opinionated on how to architect or deploy your application. Statelessness is a function of application design and architecture and an application designed to be stateless will operate statelessly irrespective of the underlying system be it bare metal, VMs or containers.

Forcing it on containers is an indulgence that increases complexity. We give you the tools to build both stateless and stateful containers.

To share a host folder with a container

flockport attachvol c1 /var/www/web

This makes /var/www/web on the host available on the container

You can also use a container volume. A container volume is a container that shares a specific folder within itself.

You can create a container volume by using the createvol command. Simply specify a name for the volume and a shared folder.

flockport createvol mysql /var/lib/mysql

This creates a 'mysql' container volume on your system with a /var/lib/mysql shared folder in the container.

Now you can share this container volume with any container and they will share the /var/lib/mysql folder of the container volume.

flockport attachvol c1 mysql

This will attach the mysql container volume to c1 and the /var/www/mysql folder specified when creating the container volume will be shared with c1

You can use the listvolumes command to list all container volumes

flockport listvolumes

This will list all container volumes on the host along with their shared dirs

You can use the info command to get an overview of all attached volumes or in the case of container volumes shared directories

flockport info c1

The benefit of using container volumes as opposed to a folder on your host is portability. The main advantage of containers is portability across systems. Tying data to the host takes away one of the core benefits of containers as opposed to container volumes that can be moved across servers seamlessly.

Images

The Flockport app store provides base os images, dev runtimes and applications. To list all available images.

flockport images

Thus will list all images available in the app store. To download images use the get command.

flockport get xenial

This will download the xenial baseimage and make it available for use.

All Flockport images are signed and downloaded securely. You need a Flockport ID to download images. The same id will be used for other services for instance signing in to the Flockport forums.

You can register from the Flockport app itself.

flockport register

This will prompt you for a username, password and email and create a flockport account.

Once registered you can login in with your username and password

flockport login

This will prompt you for your username and password and generate an auth token. Once logged in you can download images

flockport get xenial

Once downloaded you can run the xenial image

flockport run xenial

This will launch a copy of the xenial image ready for use

'run' launches a copy of an image. 'start' starts the actual container image. So if you start the xenial image any changes remain in the image.

When you launch a container with 'run' you ensure the orginal image is untouched and any changes go to the copy.

Under the hood Flockport uses overlayfs to launch an snapshot of the image in a new layer. The name of the new container is autogenerated and can be specified with the -n flag

flockport run xenial -n  myxenial

This will launch a copy of xenial with the name myxenial

Please note the default Jessie kernel does not support overlayfs. Upgrading the Jessie kernel adds overlayfs support. When 'run' detects overlayfs support missing it makes a copy of the container. Similarly if the underlying filesystem is btrfs run will launch a btrfs subvolume clone

You can also download a large collection of ready to deploy applications from the App store

flockport get wordpress

This will download the wordpress application. Once downloaded you can run the wordpress image and access the application in your browser

flockport run wordpress

The will start a copy of the wordpress container under a random generated name. Applications are available on the container IP. You can point your browser to the container IP or container IP:PORT if the application is on a specific port to access the app

You can also forward the container port to the host and it will be available on the host IP. For instance

flockport pub wordpress 80

This will forward container port 80 to the host port 80. Now you can access the app on the host IP

You can also forward ports when you start or run a container

flockport run wordpress -p 80

This will forward port 80 to the wordpress container

Apps are also configured to be available at a default url. In the case of ready to use apps its myappname.app ie mywordpress.app.

To access mywordpress.app you need to associate the container IP with the url in your /etc/hosts file. You can retrieve the container IP with the ps or info command. You can edit the hosts file manual or use the addhosts command

flockport addhosts mywordpress.app [container IP]

When you need to forward multiple containers to the same port for instance host port 80 the Flockport managed web server becomes useful. This is an Nginx reverse proxy that you can use to easily serve multiple container applications.

Snapshots and clones

You can make snapshots of containers with the snap command.

flockport snap c1 c2

This makes a snapshot of c1 as c2.

When you create a snapshot or run an image overlayfs is used in the background. Overlayfs lets create a new filesystem with another filesystem as the base, basically mounting one filesystem over another, with all changes going to the new layer.

A snapshot container basically mounts a new layer over the baseimage. The base remain untouched while all changes to the snapshot.

You can further make a snapshot of c2

flockport snap c2 c3

Each snapshot is a new overlayfs layer. For instance you can make a snapshot of a base OS image and add an application and then use that as an image to run the application.

Any changes goes to the new layer while the application layer remains untouched. You can update the base layer and it will show up in the new layer. Any updates to the base layer do not require a image rebuild.

An image and container is essentially the same thing. We only use the term image to refer to a container when it is used as a base to launch a new container.

You can get details on container layers with the history command.

flockport history c3

This will list the snapshot history of the c3 container. When used on a base image, history will show the list of snapshots made from the base image

You can also use the diff command to see changes in snapshotted containers

flockport diff c3 c2

This will list all the filesystem changes in c3 from c2

You can also clone containers

Cloning containers creates a copy of the container. Flockport supports btrfs and if the underlying filesystem is btrfs a subvolume clone is created.

flockport clone c1 c2

This makes a copy of c1 to c2.

Images can be exported and imported

flockport export c1

This exports c1 as a tar archive that can be imported on any flockport system. If the underlying image has overlayfs layers it's a good idea to use the merge option to create a single image before export

flockport export c1 -m

if c1 is a snapshot of for instance an alpine35 baseimage which exists on the new system then a merge is not required. But if the base layer is not present on the new system the c1 container will fail to start as as the underlying layer is missing.

Container networking

The Flockport installer sets up a default bridge fx0 with the 10.0.4.0/24 subnet. All containers are configured to connect to this bridge for their networking.

You can change the default 10.0.4.0/24 subnet with the setsubnet command.

flockport setsubnet 10.0.10.0/24

This will change the default subnet to 10.0.10.0 and containers will get an IP in the 10.0.10.0/24 range eg 10.0.10.40

You can also add new bridges with their own subnets and connect containers to them.

flockport addbr br0 10.0.50.0/24

This will add a new network bridge br0 with subnet 10.0.50.0/24 to your host.

You can use the setnet command to connect containers to this new bridge

flockport setnet c1 eth0 br0

This will set the container network to the br0 bridge.

Containers can have multiple network interfaces. The default is eth0. You can add more interfaces. Each interface needs a corresponding bridge.

Multiple interfaces can link to the same host bridge or multiple bridges. This gives users significant flexibility in setting up their container networks

You can add a container network interface with the addnet command.

flockport addnet c1

This will add a new network interface to c1 container. Network interfaces names are named in ascending order ie eth0, eth1

You can also specify a bridge with the -l flag when adding a network interface

flockport addnet c1 -l br0

This will add a network interface to c1 linked to br0 bridge

You can list container network interfaces with the listnet command

flockport listnet c1

This lists all network interfaces on the c1 container

You can set static IPs with the setip command. Please note the IP must be in the same subnet as the container's network.

flockport setip c1 10.0.4.10

This will set c1 eth0 interface IP to 10.0.4.10. You can revert to dhcp with

flockport setip c1 dhcp

To set IP for other interfaces specify them with a colon separator

flockport setip c1 eth1:10.0.4.10

This sets c1 eth1 interface IP to 10.0.4.10. To set the interface to dhcp

flockport setip c1 eth1:dhcp

This will set c1 eth0 interface back to default dhcp

You can delete network interfaces with the delnet command

flockport delnet c1 eth1

This will delete the eth1 network interface from c1

To delete a Flockport created bridge on the host use the delbr command

flockport delbr br0

When creating a network bridge a number of defaults are used. A DHCP service and masquerading to ensure containers on the bridge have outward internet access is enabled by default.

You can customise these options with relevant flags

flockport addbr br0 10.0.50.0/24 -n off -d domain

The -s flag sets the subnet range and must be in the x.x.x.x/xx subnet format ie 10.0.10.0/24. The -n flag sets masquerading for outward internet access for the br0 network interface. The -d flag sets the domain. For instance the default flockport bridge is set to the '.app' domain. When you add a bridge you can set a specific domain for that network.

Building containers

You can use 'recipes' to build containers. Flockport recipes are a set of instructions to build containers. This makes it easy to not only create and share containers but also define your applications for repeatable builds.

The build command is used to build containers from recipes.

To build a container simply point it at a folder with a .recipe file.

flockport build wordpress

This will build a container as per the instructions in the wordpress.recipe file

Recipes should have the .recipe extension. Any supporting assets required for the container build ie configuration files, should be in the same folder

The recipe format uses a few main instructions

NAME minio50
FROM go18
ENV MINIO_HOME /opt/minio
COPY conf/json.conf /var/www/minio/json.conf
RUN apk add minio
VOLUME minio-data
PORT 9000
DB mysql minio minio

NAME defines the name of the container. If not provided a name is auto generated.

FROM defines the base image, it can be a base OS image, a language run time or any container you want to base your build on.

ENV defines any environmental variables to be set. These are available in the container at both build time and run time. They are saved. For build time env variables it's better to just use RUN. Remember to include the '=' when specifiying variable in RUN.

RUN MINIO_HOME=/opt/minio

COPY copies any assets and resources to the specified container folder. Any files to be copies must be in the recipe folder and path must be relative to the .recipe file

RUN is a set of commands to run inside the container to process the build. You can use multiple RUN commands or separate them with a && and \\. RUN follows the semantics of Bash. Flockport does not use layers to build containers so you can use as many RUN commands as required or use a single one.

VOLUME is any container volume or host folder you want to attach to the container. In the case of a host folder you can specify the folder directly. In the example below we use a container volume. Volumes can also be specified at run time when running the container

PORTS specifies any ports to be forwarded. In the example below we are forwarding port 9000 used by the Minio GUI. Ports specified in .recipe files are added as autoports so whenever the container is started the ports are automatically forwarded

DB specifies any database required for the application. This allows you to directly create a database for an application. This accepts 5 options separated by single spaces. The first specifies the db container name. The base db container must available on the system. Only Flockport mysql or postgresql instances are supported for the DB option. The other 4 options specify the dbname dbuser allowedip and password respectively. The last 2 are optional. If the allowed IP is not specified the default subnet for that host is used. If the password is not specified a password is automatically generated.

The recipe format is designed to ease transition. The Flockport autopilot module provides far more advanced capabilities for both building and deploying containers.

You can see more recipes in the flockport/build folder at online at the Flockport repo.

The build command can also consume a github url. It will automaticaly download the repo and process any .recipe file to build the container

flockport build https://www.github.com/flockport/redmine

Here is the actual recipe used to build Minio. The application is run as user minio

NAME minio-201710

FROM alpine36

ENV MINIO_VERSION RELEASE.2017-10-27T18-59-02Z

ENV MC_VERSION RELEASE.2017-10-14T00-51-16Z

ENV MINIO_PATH /home/minio

RUN set -ex && \
    addgroup -g 1100 minio && \
    adduser -D -u 1100 -G minio minio

RUN mkdir -p $MINIO_PATH/bin /var/export

RUN curl -sSL https://dl.minio.io/server/minio/release/linux-amd64/minio.$MINIO_VERSION -o minio.$MINIO_VERSION && \
    curl -sSL https://dl.minio.io/server/minio/release/linux-amd64/minio.$MINIO_VERSION.sha256sum -o minio.sha && \
    sha256sum -c minio.sha && \
    mv minio.$MINIO_VERSION $MINIO_PATH/bin/minio && \
    chmod +x $MINIO_PATH/bin/minio

RUN curl -sSL https://dl.minio.io/client/mc/release/linux-amd64/mc.$MC_VERSION -o mc.$MC_VERSION && \
    curl -sSL https://dl.minio.io/client/mc/release/linux-amd64/mc.$MC_VERSION.sha256sum -o mc.sha && \
    sha256sum -c mc.sha && \
    mv mc.$MC_VERSION $MINIO_PATH/bin/mc && \
    chmod +x $MINIO_PATH/bin/mc

RUN ln -s $MINIO_PATH/bin/minio /usr/local/bin && \
    ln -s $MINIO_PATH/bin/mc /usr/local/bin

RUN chown -R minio:minio $MINIO_PATH /var/export

COPY minio.openrc /etc/init.d/minio

RUN chmod +x /etc/init.d/minio && \
    rc-update add minio default
        

You will notice we did not specify any ports or volumes in this recipe. These can also be specified when running the container as required.

The /var/export folder in the recipe is used as the minio data store. Currently it's in the container and data can be stored in the container without issue. Separation is not enforced in Flockport.

You can also use a host volume or a data volume container and attach it to the minio container. To specify it in the recipe we could add a volume entry

VOLUME /var/export

This will share host folder /var/export with the container. We could also specify a data volume container.

VOLUME minio-data

Here minio-data is a pre-created data volume container with /var/export defined as the shared folder.

To use a data volume container you must create it first with the createvol command.

Minio uses the 9000 port and this can be forwarded at run time or specified in the recipe

PORT 9000

This will forward port 9000 to the host on container start. Multiple ports can be specified

PORT 80 3000 9000

This will forward container ports 80, 3000, and 9000 to the host

You can also specify mappings to host ports if required with colons

PORT 80:8080 3000:3100 9000:9100

This will forward PORT 80, 3000, and 9000 in the container to port 8080, 3100 and 9100 on the host respectively.

Remember you do not need to specify volumes or ports in the container. They can be specified at runtime

For instance when you run this container you can use the command

flockport run minio-201710 -v /var/export -p 80

This will attach the host /var/export to the container and forward container port 80

One more important part of container builds to cover quickly is the keyword DB

Minio does not need a DB but most apps do. So how do you use a database in a recipe

You can specify a DB with the DB keyword and the build systems will build the specified database container

DB mariadb10 wordpress wordpress

Another option is to use the FROM command to build from a database container. In this case you will have a single container with database integrated

FROM mariadb10

This will use the mariadb10 container as the base container to build from

Its generally a good idea to keep databases separate from the app container so app upgrades are simpler.

This is also used to build apps that are PHP based for instance and when building a wordpress container a php container is used as the base container

NAME wordpress483

FROM php71

This uses the php71 container as the base container for the build

For recipes that are single files and don't require any assets you can simply point to them

flockport build nginx.recipe
flockport build /var/builds/nginx.recipe

This will build the container based on the nginx.recipe file

Building from repos

You can also provide a github repo url to the build command

flockport build https://github.com/flockport/recipes/nodejs

This will clone and repo and build the container based on the .recipe file in the repo

This is useful to have version control on your recipe repositories and directy build updated container as required.

Its also a quick and useful way to share yout containers

Tags and groups

You can tag and group containers for easier management.

flockport tag c1 app db

Now c1 is tagged with app and db. Tags allow you to group containers and actions

You can use the info command to get a list of tags associated with a container

To perform groups actions simply use the -g flag

flockport -g db info

This will show you info on all containers with the db tag

flockport -d db start

This will start all container in the db group

You can also group containers.

flockport addgroup db

This creates a db group

Now you can add mutliple containers to the db group

flockport addgroup db c1 c2 c3

This adds containers c1 c2 c3 to the groub 'db'.

A group allows you to add multiple containers to a group. A tag allows you to add multiple tags to a container. All tags and groups are the same. The difference is in how you apply them

For instance to enable autostart on you can add the autostart tag

flockport tag c1 autostart

This will place c1 in the autostart group and enable container autostart on system boot. Similarly

flockport addgroup autostart c1 c2 c3

This will place c1, c2 and c3 in the autostart group and enable autostart on boot

You can use the deltag command to delete tags

flockport deltag c1 autostart

This deletes th aurostart tag from c1 container

You can also use the delgroup command

flockport delgroup autostart c1

This deletes container c1 from the autostart group

Limiting container resources

To set cpu and memory limits on containers use the addlimit and dellimit commands

flockport addlimit c1 -c 1 -m 512M

This limits container c1 to 1 cpu and 512MB RAM

You can use the addlimit command to allocate CPU cores and memory. You can see the limits in action when you run htop inside the container.

You can remove limits with the dellimit command

flockport dellimit c1 -c -m

This resets any cpu and moeory limits set

Please note if there is only one cpu on your host then a container with a limit set will not start

Local discovery

On local systems all containers can access each other on the .app domain. So if you have an Nginx container for instance it can access a wordpress backend container at wordpress.app. This is configured into the dnsmasq instance that provides DHCP services to containers.