tag:blogger.com,1999:blog-48201524909713920372024-03-08T09:59:37.412-08:00Another Day At the Coal FacePeter Lenderyouhttp://www.blogger.com/profile/03145232715670048365noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-4820152490971392037.post-68290362887132496692016-05-13T06:12:00.000-07:002016-05-13T06:13:01.829-07:00A Lightweight multi-host cloud using LXD<h1 id="toc_0" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 28px; margin: 0px 0px 10px; padding: 0px; position: relative;">
<br /></h1>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
In this post I'm going to explain how to create a multi-host cloud using LXD, if you don't know about LXD then you can get more information from the ubuntu link -> <a href="http://www.ubuntu.com/cloud/lxd" style="-webkit-print-color-adjust: exact; color: #4183c4;">LXD</a>.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
It's lightweight in a few ways:</div>
<ul style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding-left: 30px;">
<li style="-webkit-print-color-adjust: exact; margin: 0px;">It uses LXD containers for the hypervisor so removes the Virtualisation overhead</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">It doesn't provide all the features of a normal cloud like shared storage, messaging, Telemetry, Identity etc. although it is more than capable of running most workloads.</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">It can run on bare-metal or Virtual Machines or Cloud instances</li>
</ul>
<h2 id="toc_1" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 24px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Motivation</h2>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
I've used multi-hosts lxd setups for a while but it was always a pain to get connectivity between hosts, because each host had it's own dnsmasq instance. We had to have a route defined to each lxd subnet via the host configured for the office (and the vpn) which was a pain.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
Also because each host had it's own dnsmasq for the hostnames we had to have a forwarders in our main office dns to a unique sub-domain.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
If for example we wanted to load balance between 2 servers on 2 hosts then they would have to have a different domain name, e.g. service1.leg1.lxd and service2.leg2.lxd. Migrating between hosts would change the IP address and the hostname meaning we'd have to re-configure a load-balancer.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
Multi-cast was a real pain, especially in AWS on EC2 instances.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
So to address these issues and using this <a href="https://www.flockport.com/build-layer2-and-layer-3-overlay-networks-with-peervpn/" style="-webkit-print-color-adjust: exact; color: #4183c4;">Flockport article</a> I tried to work out how to use a layer 2 network on top of the layer 3.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
As it turned out it wasn't that difficult.</div>
<h3 id="toc_2" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 18px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Advantages</h3>
<ul style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding-left: 30px;">
<li style="-webkit-print-color-adjust: exact; margin: 0px;">A shared address range across all hosts (an Alternative is the <a href="https://wiki.ubuntu.com/FanNetworking" style="-webkit-print-color-adjust: exact; color: #4183c4; margin-top: 0px;">Ubuntu Fan Network</a>)</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">Single configuration for dns resolution</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">Simplified routing for external connections</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">The ability to have a much bigger IP address range</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">A whole lot easier to set up than openstack</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">Live migration of instances between hosts (Although according to <a href="https://www.stgraber.org/2016/04/25/lxd-2-0-live-migration-912/" style="-webkit-print-color-adjust: exact; color: #4183c4; margin-top: 0px;">Stéphane Graber's website</a>, this is not ready for production<ul style="-webkit-print-color-adjust: exact; margin: 15px 0px; padding-left: 30px;">
<li style="-webkit-print-color-adjust: exact; margin: 0px;">When I tested this it seemed to work and the IP address and host name in dns was the same</li>
</ul>
</li>
</ul>
<h3 id="toc_3" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 18px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Use Cases</h3>
<ul style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding-left: 30px;">
<li style="-webkit-print-color-adjust: exact; margin: 0px;">Replicating a real cluster of servers for development</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">Increasing the density of usage on AWS/EC2, therefore reducing costs.<ul style="-webkit-print-color-adjust: exact; margin: 0px 0px 15px; padding-left: 30px;">
<li style="-webkit-print-color-adjust: exact; margin: 0px;">Most of our software is idle outside of peak usage reducing the number of EC2 instances can be a significant cost saving</li>
</ul>
</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">Getting multicast to work on EC2</li>
</ul>
<h2 id="toc_4" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 24px; margin: 20px 0px 10px; padding: 0px; position: relative;">
What does this example give us?</h2>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
The final cloud will have an Layer 2 network overlay between hosts, a single DNSMASQ providing DHCP and DNS resolution,</div>
<h2 id="toc_5" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 24px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Lets do It</h2>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
For this example I've used Vagrant and virtual box on my mac, I create 3 machines with Ubuntu Xenial on them with no extra network BUT I've port forwarded port 7000 to a local port on the mac.</div>
<ul style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding-left: 30px;">
<li style="-webkit-print-color-adjust: exact; margin: 0px;">The host has an IP address of 192.168.99.1</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">lxd1 has port 7000 to port 192.168.99.1:7001</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">lxd2 has port 7000 to port 192.168.99.1:7002</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">lxd3 has port 7000 to port 192.168.99.1:7003</li>
</ul>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
This was done to ensure that there was no direct connectivity between the VMs. In AWS for example each machine may have an IP address but there's no MULTICAST between them (most cloud environments don't support multicast).</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
<strong style="-webkit-print-color-adjust: exact;">( Our software is much easier and scalable if multicast is available )</strong></div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
The project for this file is <a href="https://github.com/plenderyou/lxd-cloud" style="-webkit-print-color-adjust: exact; color: #4183c4;">lxd-cloud</a> hosted on github</div>
<h4 id="toc_6" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 16px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Step 1 Install the software.</h4>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
On each box we're going to install:</div>
<table style="-webkit-print-color-adjust: exact; border-collapse: collapse; color: black; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding: 0px;"><thead style="-webkit-print-color-adjust: exact;">
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">Software</th><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">Reason</th></tr>
</thead><tbody style="-webkit-print-color-adjust: exact;">
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">peervpn</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">Provides the layer 3 network between the machines</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">bridge-utils</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">This is used to add the vpn tap device to the lxd bridge</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">libssl-dev</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">Used to compile peervpn</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">build-essential</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">Used to compile peervpn</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">zfsutils-linux</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">For use of zfs for lxd</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">lxd</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">Well it wouldn't work without it</td></tr>
</tbody></table>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px;">
<pre class="line-numbers language-none" style="-webkit-print-color-adjust: exact; background: rgb(76, 63, 51); border-radius: 0.5em; border: 0.3em solid rgb(122, 102, 81); box-shadow: black 1px 1px 0.5em inset; color: white; counter-reset: linenumber 0; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; font-size: 13px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em 1em 1em 3.8em; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;"><code class=" language-none" style="-webkit-print-color-adjust: exact; background: none; border-radius: 3px; border: none; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; line-height: 1.5; margin: 0px; padding: 0px; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;">sudo apt-get install bridge-utils libssl-dev build-essential zfsutils-linux lxd
wget http://www.peervpn.net/files/peervpn-$VERSION.tar.gz
tar -xvf peervpn-$VERSION.tar.gz
cd peervpn-$VERSION
make
sudo make install<span class="line-numbers-rows" style="-webkit-print-color-adjust: exact; -webkit-user-select: none; border-right-color: rgb(153, 153, 153); border-right-style: solid; border-right-width: 1px; left: -3.8em; letter-spacing: -1px; pointer-events: none; position: absolute; top: 0px; width: 3em;"><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span></span></code></pre>
</div>
<h4 id="toc_7" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 16px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Step 2 initialise lxd</h4>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
On each host we will initialise lxd, below we use an zfs loopback device</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px;">
<pre class="line-numbers language-none" style="-webkit-print-color-adjust: exact; background: rgb(76, 63, 51); border-radius: 0.5em; border: 0.3em solid rgb(122, 102, 81); box-shadow: black 1px 1px 0.5em inset; color: white; counter-reset: linenumber 0; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; font-size: 13px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em 1em 1em 3.8em; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;"><code class=" language-none" style="-webkit-print-color-adjust: exact; background: none; border-radius: 3px; border: none; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; line-height: 1.5; margin: 0px; padding: 0px; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;">lxd init --storage-backend=zfs --storage-create-loop=100 --storage-pool=lxd --auto <span class="line-numbers-rows" style="-webkit-print-color-adjust: exact; -webkit-user-select: none; border-right-color: rgb(153, 153, 153); border-right-style: solid; border-right-width: 1px; left: -3.8em; letter-spacing: -1px; pointer-events: none; position: absolute; top: 0px; width: 3em;"><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span></span></code></pre>
</div>
<h4 id="toc_8" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 16px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Step 3 configure the lxd bridge</h4>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
Modify the /etc/default/lxd-bridge file to set up the layer 2 network</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
I used:</div>
<table style="-webkit-print-color-adjust: exact; border-collapse: collapse; color: black; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding: 0px;"><thead style="-webkit-print-color-adjust: exact;">
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">MACHINE</th><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">STATICIP</th><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">CIDR MASK</th></tr>
</thead><tbody style="-webkit-print-color-adjust: exact;">
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">lxd1</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">172.16.0.1</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">16</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">lxd2</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">172.16.0.2</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">16</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">lxd3</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">172.16.0.3</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">16</td></tr>
</tbody></table>
<table style="-webkit-print-color-adjust: exact; border-collapse: collapse; color: black; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding: 0px;"><thead style="-webkit-print-color-adjust: exact;">
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">SETTING</th><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">VALUE</th></tr>
</thead><tbody style="-webkit-print-color-adjust: exact;">
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;"><code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">USE_LXD_BRIDGE</code></td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">true</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;"><code style="-webkit-print-color-adjust: exact; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">UPDATE_PROFILE</code></td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">true</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;"><code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_DOMAIN</code></td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">Your choice of domain</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;"><code style="-webkit-print-color-adjust: exact; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_IPV4_ADDR</code></td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">{STATICIP}</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;"><code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_IPV4_NETMASK</code></td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">calculated netmask from CIDR MASK e.g. 255.255.0.0</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;"><code style="-webkit-print-color-adjust: exact; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_IPV4_NETWORK</code></td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">{STATICIP}/{CIDR_MASK}</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;"><code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_IPV4_DHCP_RANGE</code></td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">The range for dnsmasq ip addresses e.g. "172.16.0.10,172.16.255.254", in this case we can have 9 hosts</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;"><code style="-webkit-print-color-adjust: exact; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_IPV4_DHCP_MAX</code></td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">The number of hosts in the range e.g. 65534 - 10 = 65524</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;"><code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_IPV4_NAT</code></td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">true</td></tr>
</tbody></table>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
The IPV6 setting are not used in this example.</div>
<h4 id="toc_9" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 16px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Step 4 we need to fix the DNSMASQ settings</h4>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
By default the lxd-bridge program creates a DNSMASQ service if the <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_IPV4_ADDR</code> or <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_IPV6_ADDR</code> (I think they should have used the DHCP entry myself), <strong style="-webkit-print-color-adjust: exact;">BUT</strong> we only want 1 running on the layer 2 network.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
There are a number of ways to fix this</div>
<ol style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding-left: 30px;">
<li style="-webkit-print-color-adjust: exact; margin: 0px;">Change the lxd-bridge program to check for the <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_IPV4_DHCP_RANGE</code> setting and only define that on 1 host</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">override the dnsmasq program in the /etc/default/lxd-bridge which is what I did</li>
</ol>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
So on 2 of the 3 hosts add this line to the /etc/default/lxd-bridge file alias dnsmasq="/bin/echo 'Not starting dnsmasq'"</div>
<h4 id="toc_10" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 16px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Step 4 Restart lxd-brigde</h4>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
On all hosts</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px;">
<pre class="line-numbers language-none" style="-webkit-print-color-adjust: exact; background: rgb(76, 63, 51); border-radius: 0.5em; border: 0.3em solid rgb(122, 102, 81); box-shadow: black 1px 1px 0.5em inset; color: white; counter-reset: linenumber 0; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; font-size: 13px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em 1em 1em 3.8em; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;"><code class=" language-none" style="-webkit-print-color-adjust: exact; background: none; border-radius: 3px; border: none; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; line-height: 1.5; margin: 0px; padding: 0px; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;">service lxd-brigde restart<span class="line-numbers-rows" style="-webkit-print-color-adjust: exact; -webkit-user-select: none; border-right-color: rgb(153, 153, 153); border-right-style: solid; border-right-width: 1px; left: -3.8em; letter-spacing: -1px; pointer-events: none; position: absolute; top: 0px; width: 3em;"><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span></span></code></pre>
</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
Now if you check the lxdbr0 device using ifconfig they should have the {STATICIP} assigned</div>
<h4 id="toc_11" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 16px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Step 5 Configure peervpn</h4>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
We're using peervpn to create the layer 2 network (you can use other methods, see the <a href="https://www.flockport.com/build-layer2-and-layer-3-overlay-networks-with-peervpn/" style="-webkit-print-color-adjust: exact; color: #4183c4;">Flockport article</a>.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
Each host must have an IP address for the VPN and the peers for the other hosts. Peers are of the format <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;"><ip> <port></code>, in my case we use the port_forwarded address,</div>
<table style="-webkit-print-color-adjust: exact; border-collapse: collapse; color: black; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding: 0px;"><thead style="-webkit-print-color-adjust: exact;">
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">MACHINE</th><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">IP</th><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">PEER1</th><th style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">PEER2</th></tr>
</thead><tbody style="-webkit-print-color-adjust: exact;">
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">lxd1</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">10.99.0.1</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">192.168.99.1 7002</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">192.168.99.1 7003</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">lxd2</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">10.99.0.2</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">192.168.99.1 7001</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">192.168.99.1 7003</td></tr>
<tr style="-webkit-print-color-adjust: exact; background-color: white; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; margin: 0px; padding: 0px;"><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">lxd3</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">10.99.0.3</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">192.168.99.1 7001</td><td style="-webkit-print-color-adjust: exact; border: 1px solid rgb(204, 204, 204); margin: 0px; padding: 6px 13px;">192.168.99.1 7002</td></tr>
</tbody></table>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
On each machine as root</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px;">
<pre class="line-numbers language-none" style="-webkit-print-color-adjust: exact; background: rgb(76, 63, 51); border-radius: 0.5em; border: 0.3em solid rgb(122, 102, 81); box-shadow: black 1px 1px 0.5em inset; color: white; counter-reset: linenumber 0; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; font-size: 13px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em 1em 1em 3.8em; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;"><code class=" language-none" style="-webkit-print-color-adjust: exact; background: none; border-radius: 3px; border: none; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; line-height: 1.5; margin: 0px; padding: 0px; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;">cd /etc
mkdir peervpn
cd peervpn
IP=<IP>
PEER1=<FORWARDED PORT FOR ANOTHER NODE>
PEER2=<FORWARDED PORT FOR ANOTHER NODE>
cat > peervpn.conf.l2 <<!
networkname PEERVPN
psk password
enabletunneling yes
interface peervpn0
ifconfig4 $IP/24
port 7000
initpeers $PEER1 $PEER2
upcmd brctl addif lxdbr0 peervpn0
!<span class="line-numbers-rows" style="-webkit-print-color-adjust: exact; -webkit-user-select: none; border-right-color: rgb(153, 153, 153); border-right-style: solid; border-right-width: 1px; left: -3.8em; letter-spacing: -1px; pointer-events: none; position: absolute; top: 0px; width: 3em;"><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span></span></code></pre>
</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
<strong style="-webkit-print-color-adjust: exact;">Notice The upcmd adds the tap device to the lxd bridge device</strong></div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
You can now test the peervpn by running</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px;">
<pre class="line-numbers language-none" style="-webkit-print-color-adjust: exact; background: rgb(76, 63, 51); border-radius: 0.5em; border: 0.3em solid rgb(122, 102, 81); box-shadow: black 1px 1px 0.5em inset; color: white; counter-reset: linenumber 0; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; font-size: 13px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em 1em 1em 3.8em; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;"><code class=" language-none" style="-webkit-print-color-adjust: exact; background: none; border-radius: 3px; border: none; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; line-height: 1.5; margin: 0px; padding: 0px; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;">/usr/local/sbin/peervpn /etc/peervpn/peervpn.conf.l2<span class="line-numbers-rows" style="-webkit-print-color-adjust: exact; -webkit-user-select: none; border-right-color: rgb(153, 153, 153); border-right-style: solid; border-right-width: 1px; left: -3.8em; letter-spacing: -1px; pointer-events: none; position: absolute; top: 0px; width: 3em;"><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span></span></code></pre>
</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
You should see the number of peers increasing after a few seconds</div>
<h4 id="toc_12" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 16px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Step 6 Make peervpn a service</h4>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
To make peervpn a service as root run:</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px;">
<pre class="line-numbers language-none" style="-webkit-print-color-adjust: exact; background: rgb(76, 63, 51); border-radius: 0.5em; border: 0.3em solid rgb(122, 102, 81); box-shadow: black 1px 1px 0.5em inset; color: white; counter-reset: linenumber 0; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; font-size: 13px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em 1em 1em 3.8em; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;"><code class=" language-none" style="-webkit-print-color-adjust: exact; background: none; border-radius: 3px; border: none; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; line-height: 1.5; margin: 0px; padding: 0px; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;">cd /lib/systemd/system
cat > peervpn.service <<!
[Unit]
Description=Start the VPN
Wants=lxd.service
After=lxd.service
[Service]
Type=simple
ExecStart=/usr/local/sbin/peervpn /etc/peervpn/peervpn.conf.l2
[Install]
WantedBy=multi-user.target
!
service peervpn start
systemctl enable peervpn<span class="line-numbers-rows" style="-webkit-print-color-adjust: exact; -webkit-user-select: none; border-right-color: rgb(153, 153, 153); border-right-style: solid; border-right-width: 1px; left: -3.8em; letter-spacing: -1px; pointer-events: none; position: absolute; top: 0px; width: 3em;"><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span></span></code></pre>
</div>
<h4 id="toc_13" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 16px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Step 6 Check stuff</h4>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
You should be able to ping the 172.16.0.[1,2,3] addresses from all machines if you can't then</div>
<ul style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding-left: 30px;">
<li style="-webkit-print-color-adjust: exact; margin: 0px;">use <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">brctl show</code> and make sure the peervpn0 is in the interface</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;"><code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">sudo service peervpn status</code> and make sure that there are peers</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;"><code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">ifconfig</code> and make sure that the peervpn0 and lxdbr0 devices have ip addresses</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;"><code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">ip route show</code> and make sure that there is a route for the peervpn network associated with the vpn ip</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;"><code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">ip route show</code> and make sure that there is a route for the lxd network associated with the lxd ip</li>
</ul>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
Example routes</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px;">
<pre class="line-numbers language-none" style="-webkit-print-color-adjust: exact; background: rgb(76, 63, 51); border-radius: 0.5em; border: 0.3em solid rgb(122, 102, 81); box-shadow: black 1px 1px 0.5em inset; color: white; counter-reset: linenumber 0; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; font-size: 13px; line-height: 1.5; margin-bottom: 0.5em; margin-top: 0.5em; overflow: auto; padding: 1em 1em 1em 3.8em; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;"><code class=" language-none" style="-webkit-print-color-adjust: exact; background: none; border-radius: 3px; border: none; font-family: Consolas, Monaco, 'Andale Mono', 'Ubuntu Mono', monospace; line-height: 1.5; margin: 0px; padding: 0px; position: relative; tab-size: 4; text-shadow: black 0px -0.1em 0.2em; word-break: normal; word-spacing: normal; word-wrap: normal;">ip route show
default via 10.0.2.2 dev eth0
10.0.2.0/24 dev eth0 proto kernel scope link src 10.0.2.15
10.99.0.0/24 dev peervpn0 proto kernel scope link src 10.99.0.1
172.16.0.0/16 dev lxdbr0 proto kernel scope link src 172.16.0.1<span class="line-numbers-rows" style="-webkit-print-color-adjust: exact; -webkit-user-select: none; border-right-color: rgb(153, 153, 153); border-right-style: solid; border-right-width: 1px; left: -3.8em; letter-spacing: -1px; pointer-events: none; position: absolute; top: 0px; width: 3em;"><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span><span style="-webkit-print-color-adjust: exact; counter-increment: linenumber 1; display: block; pointer-events: none;"></span></span></code></pre>
</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
Sometimes it will fail because the services were not started in the correct order, try rebooting the host if this was the case.</div>
<h4 id="toc_14" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 16px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Lets do some lxd tests</h4>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
Get and image from the image store on each host e.g.<code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">lxc image copy ubuntu:xenial local: --copy-aliases --auto-update</code></div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
<strong style="-webkit-print-color-adjust: exact;">You can set the <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">core.https_address</code> and <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">core.trust_password</code> and add remotes for the other hosts and copy the image between hosts if your internet is too slow</strong></div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
Got to each host and create a container make sure each container has a unique name.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
e.g <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">lxc launch xenial c1</code></div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
using <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">lxc list</code> the host should get an ip in the lxd network range <strong style="-webkit-print-color-adjust: exact;">(this may take a couple of seconds)</strong>.</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
You should be able to ping between containers using the ip address or the dns address (This is the e.g. c1.lxd if your <code style="-webkit-print-color-adjust: exact; background-color: #f8f8f8; border-radius: 3px; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; white-space: nowrap;">LXD_DOMAIN</code> was set to lxd).</div>
<div style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin-bottom: 15px; margin-top: 15px;">
This <a href="https://gist.github.com/jayjanssen/5697813" style="-webkit-print-color-adjust: exact; color: #4183c4;">gist</a> has an example of how to test multicast, which should work between containers.</div>
<h2 id="toc_15" style="-webkit-font-smoothing: antialiased; -webkit-print-color-adjust: exact; border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; cursor: text; font-family: Helvetica, arial, sans-serif; font-size: 24px; margin: 20px 0px 10px; padding: 0px; position: relative;">
Issues and improvements</h2>
<ul style="-webkit-print-color-adjust: exact; font-family: Helvetica, arial, sans-serif; font-size: 14px; line-height: 22.4px; margin: 15px 0px; padding-left: 30px;">
<li style="-webkit-print-color-adjust: exact; margin: 0px;">There is a single point of failure in dnsmasq</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">It is possible to create a container on 2 hosts with the same name, they get a unique IP address but only the last one is resolvable through DNS</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">A GUI for administering the cluster would be nice.</li>
<li style="-webkit-print-color-adjust: exact; margin: 0px;">Peervpn is not one of the standard ubuntu packages and forces encryption and overhead. Alternatives are mentioned in the Flockport article but I haven't experimented with them</li>
</ul>
Peter Lenderyouhttp://www.blogger.com/profile/03145232715670048365noreply@blogger.com3