How To Compile Linux with LXC and Create Application Container

August 1, 2014 | By
| Reply More

LXC (LinuX Containers) is an operating system–level virtualization method for running multiple isolated Linux systems (containers) on a single host.

The Linux kernel comprises cgroups (control groups) for resource isolation (CPU, memory, block I/O, network, etc.) which does not require starting any virtual machines. Cgroups also provide namespace isolation to completely isolate application's view of the operating environment, including process trees, network, user ids and mounted file systems.

LXC combines cgroups and namespace support to provide an isolated environment for applications.

Isolation with namespaces

  • pid
  • mnt
  • net
  • uts
  • ipc
  • user

Isolation with cgroups

  • memory
  • cpu
  • blkio
  • devices

1. Kernel for LXC

Pick any kernel above linux kernel version – 3.3 and ensure the following options are enabled either through ‘make menuconfig’ or kernel compilation process.

filename: linux-3.3/.config
CONFIG_USER_SCHED=n

CONFIG_CGROUPS=y

CONFIG_CGROUP_SCHED=y

CONFIG_CGROUP_DEBUG=y

CONFIG_CGROUP_NS=y

CONFIG_CGROUP_FREEZER=y
CONFIG_CGROUP_DEVICE=y

CONFIG_CPUSETS=y

CONFIG_PROC_PID_CPUSET=y

CONFIG_CGROUP_CPUACCT=y

CONFIG_RESOURCE_COUNTERS=y

CONFIG_NAMESPACES=y

CONFIG_UTS_NS=y

CONFIG_IPC_NS=y

CONFIG_USER_NS=y

CONFIG_PID_NS=y

CONFIG_NET_NS=y

CONFIG_VLAN_8021Q=y

CONFIG_VLAN_8021Q_GVRP=n

CONFIG_MACVLAN=y

CONFIG_VETH=y

CONFIG_DEVPTS_MULTIPLE_INSTANCES=y

CONFIG_SECURITY_FILE_CAPABILITIES=y

CONFIG_FAIR_GROUP_SCHED=y

CONFIG_RT_GROUP_SCHED=n

CONFIG_BLK_CGROUP=n

CONFIG_MACVTAP=n

CONFIG_CFS_BANDWIDTH=y

CONFIG_NETPRIO_CGROUP=y

2. Compile lxc

Download LXC sources from https://linuxcontainers.org/downloads/, pick any version lxc-x.x.x.tar.gz

Sample Compilation

[katla@regina]# tar xvf lxc-1.0.4.tar.gz

[katla@regina]# cd lxc-1.0.4

[katla@regina]# export CC=arm-gnu-eabi-linux-gcc; ./configure --host=arm

[katla@regina]# make

[katla@regina]# ls * | grep lxc-*

lxc-create lxc-start lxc-execute lxc-console lxc-stop lxc-freeze lxc-ps lxc-monitor lxc-unfreeze lxc-info ..

Copy these binaries to /usr/bin or /sbin or /bin directory of rootfs of target running above compiled linux kernel.

3. LXC and networking

Bridge set-up and linux host interface (eth0) configuration on target machine (ARM)

bridge-start

brctl show

brctl addbr host0

ifconfig eth0 192.168.10.2 up

route add default gw 10.140.47.254 eth0

ifconfig host0 192.168.10.1 up

LXC NET

4. Running Application Containers

container lxc

We can use the lxc-execute command to create a application container (name: guest) in which we can run a command that is effectively isolated from the rest of the system.

For example, the following command creates an application container named guest that runs sleep for 300 seconds.

[katla@regina]# lxc-execute -n guest -- sleep 300

While the container is active, we can monitor it by running commands such as lxc-ls --active and lxc-info -n guest from another window.

[katla@regina]# lxc-ls --active guest

[katla@regina]# lxc-info -n guest

state: RUNNING pid: 4021

If we need to customize an application container, we can use a configuration file.

For example, we might want to change the container's network configuration or the system directories that it mounts.

The following example shows settings from a sample configuration file where the rootfs is mostly not shared except for mount entries to ensure that lxc-init and certain library and binary directory paths are available.

lxc.utsname = guest

lxc.tty = 1

lxc.pts = 1

lxc.rootfs = /tmp/guest/rootfs

lxc.mount.entry=/lib /tmp/guest/rootfs/lib none ro,bind 0 0

lxc.mount.entry=/usr/libexec /tmp/guest/rootfs/usr/lib none ro,bind 0 0

lxc.mount.entry=/lib64 /tmp/guest/rootfs/lib64 none ro,bind 0 0

lxc.mount.entry=/usr/lib64 /tmp/guest/rootfs/usr/lib64 none ro,bind 0 0

lxc.mount.entry=/bin /tmp/guest/rootfs/bin none ro,bind 0 0

lxc.mount.entry=/usr/bin /tmp/guest/rootfs/usr/bin none ro,bind 0 0

lxc.cgroup.cpuset.cpus=1

The mount entry for /usr/libexec is required so that the container can access /usr/libexec/lxc/lxc-init on the host system.

The example configuration file mounts both /bin and /usr/bin. In practice, we should limit the host system directories that an application container mounts to only those directories that the container needs to run the application.

Note
To avoid potential conflict with system containers, do not use the /container directory for application containers.

We must also configure the required directories under the rootfs directory:

[katla@regina]# TMPDIR=/tmp/guest/rootfs

[katla@regina]# mkdir -p $TMPDIR/lib $TMPDIR/usr/lib $TMPDIR/lib32 $TMPDIR/usr/lib32 \ $TMPDIR/bin $TMPDIR/usr/bin $TMPDIR/dev/pts $TMPDIR/dev/shm $TMPDIR/proc

In this example, the directories include /dev/pts, /dev/shm, and /proc in addition to the mount point entries defined in the configuration file.

We can then use the -f option to specify the configuration file (config) to lxc-execute:

[katla@regina]# lxc-execute -n guest -f config -- ps -ef

UID PID PPID C STIME TTY TIME CMD

0 1 0 0 07:36 ? 00:00:00 /usr/lib/lxc/lxc-init -- ps -ef 0 2 1 0 08:46 ? 00:00:00 ps -ef

This example shows that the ps command runs as a child of lxc-init.

As for system containers, we can set cgroup entries in the configuration file and use the lxc-cgroup command to control the system resources to which an application container has access.

Note

lxc-execute is intended to run application containers that share the host's root file system, and not to run system containers that we create using lxc-create. Use lxc-start to run system containers.

5. Sample Configuration & logs

Filename: /etc/fstab

cgroup /cgroup cgroup defaults 0 0

Filename: /etc/lxc/guest/config

# Default pivot location

lxc.pivotdir = lxc_putold

# Default mount entries

lxc.mount.entry = proc proc proc nodev,noexec,nosuid 0 0

lxc.mount.entry = sysfs sys sysfs defaults 0 0

lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none bind,optional 0 0

# Default console settings
lxc.tty = 4 lxc.pts = 1024

# Default capabilities

lxc.cap.drop = sys_module mac_admin mac_override sys_time

# When using LXC with apparmor, the container will be confined by default.

# If you wish for it to instead run unconfined, copy the following line

# (uncommented) to the container's configuration file.

#lxc.aa_profile = unconfined

# To support container nesting on an Ubuntu host while retaining most of

# apparmor's added security, use the following two lines instead.

#lxc.aa_profile = lxc-container-default-with-nesting

#lxc.hook.mount = /usr/share/lxc/hooks/mountcgroups

# If you wish to allow mounting block filesystems, then use the following

# line instead, and make sure to grant access to the block device and/or loop

# devices below in lxc.cgroup.devices.allow.

#lxc.aa_profile = lxc-container-default-with-mounting

# Default cgroup limits lxc.cgroup.devices.deny = a

## Allow any mknod (but not using the node)

lxc.cgroup.devices.allow = c *:* m

lxc.cgroup.devices.allow = b *:* m

## /dev/null and zero

lxc.cgroup.devices.allow = c 1:3 rwm

lxc.cgroup.devices.allow = c 1:5 rwm

## consoles

lxc.cgroup.devices.allow = c 5:0 rwm

lxc.cgroup.devices.allow = c 5:1 rwm

## /dev/{,u}random lxc.cgroup.devices.allow = c 1:8 rwm

lxc.cgroup.devices.allow = c 1:9 rwm ## /dev/pts/*

lxc.cgroup.devices.allow = c 5:2 rwm

lxc.cgroup.devices.allow = c 136:* rwm

## rtc

lxc.cgroup.devices.allow = c 254:0 rm

Filed Under : LINUX HOWTO

Free Linux Ebook to Download

Leave a Reply

Commenting Policy:
Promotion of your products ? Comment gets deleted.
All comments are subject to moderation.