Hetzner, Docker, and KVM: A New Beginning
This post has been in the planning phase for a long while now as I have lost interest in blogging over the last few months. As you may have noticed this blog has been moved about a few times and has changed the back-end blogging system just as much.
corrigan.xyz is now running on a hosted server somewhere in Germany by Hetzner Online. I got it for €52.44 a month from the server auction system. The full specifications of the box are as below, and it is running Ubuntu 18.04
Intel Xeon E3-1271V3
2x HDD SATA 4,0 TB Enterprise
4x RAM 8192 MB DDR3 ECC
NIC 1 Gbit – Intel I210
RAID Controller – LSI MegaRAID SAS 9260-4i
I acquired this server to replace a few other services I was paying for separately. First of was my Scaleway nodes, two to be exact, one in France another in Amsterdam. I have kept one of these around as a backup server, the above server rsyncs data to it just in case. The exact specifications of VPSs have changed over time, but the one remaining VPS is spec’d as below.
2 X86 64bit Cores
2GB memory
200Mbit/s Unmetered
These used to run my sites on one and my personal Mastodon instance on the other. My new server runs a lot more than just this blog and my mastodon instance. It also now runs my seedbox which is used to seed Linux Distros I use. This service used to hosted elsewhere, but I have moved that over to this new server. I also used IRCCloud and have replaced that too with ZNC and thelounge.
Another reason I purchased a server with a butt-load of storage was to offload my media hosted locally on my Gaming PC, this media mostly Movies, TV Shows and Anime with an old collection of Music. This media was digested either on this machine or over SMB to another device in my flat. Rather useless if I’m on a train to Glasgow with 4G or back home in Ireland with no remote access back to the flat. I wanted to self-host something like Plex or Emby, and I ended up with Plex as friends had recommended it and I was already in some other Plex libraries.
There was an opportunity here to move from traditionally installed software, one VPS was nginx/blogging system, and the other VPS was Mastodon, to more of a “DevOps” environment (a.k.a Docker). There was a learning curve to this migration and growth of my self-hosted life. Docker was entirely new for me and the setup and configuration I have used, in hindsight, probably wasn’t the best or even correct but it works okay for now. The reason for Docker was to allow me to reproduce my current self-hosted setup if I ever needed to move server, either due to upgrade or failure. I would need to copy over one folder containing all data and run a couple of commands and should be back online in no time. To get where I am now, where I can add new applications to my server with ease, took a lot of testing and failure.
I have no idea if it is correct how I configured my docker setup but it works for me. The only change I would make is to have one or several docker-compose files and not the collect of plain docker commands. Below is a list of docker images I am using currently.
Now the server I have is pretty over spec’d for just running a couple of Docker containers. I have also set up a few KVM machines that I can start up as needed. I initially tried to host a full Linux Desktop in docker that I could RDP or VNC on to if needed. This solution did not work at all, so I pulled up my sleeves and learnt how KVM/libvirt works. I now have 4 VMs that can all be spun up and accessed remotely (right now through an SSH tunnel but I hope to replace this with OpenVPN and Wireguard.) The VMs are as followed, Arch Linux, Windows 10, Windows Server 2016 and MacOS High Sierra. This setup allows me to always have a constant workspace in each OS and allows for testing and training remotely.
I have to say that a lot of the data on this server is entirely replaceable. For example, my media library can be replaced over time if it gets destroyed, same for the KVMs. I do a backup on the directory which includes configuration files for all dockerised applications as well as configs for my VMs. Each folder is tar.gz’d nightly excluding larger and replaceable data and then copied to another directory where it is rsync’d to my surviving Scaleway VPS. I only keep the last 5 days of backups. I could replicate the backups to another storage location like an S3 bucket or similar, but it’s not that important really, just nice to have.
In the penultimate paragraph of this post I will justify my actions. The main reason was to try to save some money overall or at least get better value for what I am paying. The removal of my local media hosting I assume has saved me some cash as electricity here in Scotland is not that expensive, but my gaming machine is a hog to sat the least so not having to keep that on more than it has to is a plus in my book. I now only have one VPS with Scaleway which cut my cost there in half, but this only amounts to about €3 a month so not that much. The most significant saving was the self-hosting of a seedbox; I originally paid about $22 for this a month, so that is saved too. As for the benefits I gain from this server. I now have a media server that allows me not to worry about the acquiring of media and give me a simple way to ingest it anywhere. I am now able to host applications that I could have never used or needed to pay for. I am learning more and more about docker, that will help me in my job in the future. I always like the idea of a Windows-based workspace in the cloud so I could have a constant experience no matter what machine I was on. This kind of service is quite pricey, but with my server and KVM, I now have this functionality.
I am happy with the move and the new abilities I have gained. The only problem I have now is that the 7.2 TBs of usable storage I got is getting filling up quickly but an upgrade shouldn’t be too hard, purchase a new server and rync with some docker magic should get me back to production
— Michael