Installing Virtual Machines over a Serial Line

Linux Installers and Serial Connections

From time to time I still find the need to install virtual machines over a serial connection. I configure a serial console on ALL of my virtual machines as an out-of-band management option for when I screw up their network config, so it seems natural to want to install them using a serial connection as well. One advantage this brings is that I don’t forget to leave them configured with an unnecessary and potentially problematic VNC server configured after install.

Some aspects of the install process are always going to be distribution specific, but I’ve found there is usually a way to get it going. The notes below are meant as a reminder to myself of what has worked for me in the past.

CentOS/RedHat/Fedora

Fedora (and so CentOS and RHEL) seem to have good support for non-GUI installs in general. They provide a text based installer which can handle basic installations, and they have kickstart (and VNC) to support more complicated installs. I’ll look at the text based install first, but then I’ll show an example using kickstart as that is how Fedora recommend doing non-GUI based installs (it is also much more flexible).

Text Based virt-install

Here an example for installing Fedora29

virt-install --name fedora29 --ram 2048 --disk \
path=/var/lib/libvirt/images/fedora29.qcow2,size=30,format=qcow2 \
 --os-type linux --os-variant fedora26 \
 --network network=default \
 --graphics none --console pty,target_type=serial \
 --location 'http://mirror.uoregon.edu/fedora/linux/releases/29/Server/x86_64/os/' \
 --extra-args 'console=tty0 console=ttyS0'

On some of my systems (qemu-kvm on debian) I see a long delay after the “Starting dracut initqueue” message on the console. From the other messages, it looks like it has to do with configuring the network interfaces. Anyway, the installer eventually continues and will prompt you to configure the details…

Starting installer, one moment...
anaconda 29.24.7-1.fc29 for Fedora 29 started.
 * installation log files are stored in /tmp during the installation
 * shell is available on TTY2
 * if the graphical installation interface fails to start, try again with the
   inst.text bootoption to start text installation
 * when reporting a bug add logs from /tmp as separate text/plain attachments
================================================================================
================================================================================

1) Start VNC
2) Use text mode

Please make a selection from the above ['c' to continue, 'q' to quit, 'r' to
refresh]:

From this point on the install should be familiar.

Kickstart Based virt-install

Kickstart based installs are very similar to the text based method above, but the command prompt is replaced by a kickstart file which documents exactly how you want the machine to be configured. The easiest way to come up with a kickstart file is to take one from an existing install. When the anaconda installer runs it will normally leave behind an anaconda-ks.cfg file documenting the decisions you made during the install in root’s homedir. Take one of these files and modify it to suit your needs. There are also some examples available on github, and the fedora kickstart documentation is excellent, as is the RHEL install manual. Here is a basic example.

auth --enableshadow --passalgo=sha512
text
url --url http://mirror.it.ubc.ca/centos/7/os/x86_64
ignoredisk --only-use=vda
network --device=eth0 --bootproto=dhcp --onboot=on --noipv6 --activate
rootpw --iscrypted A-CRYPTED-PASSWORD-GOES-HERE
timezone America/Vancouver
bootloader --append=" crashkernel=auto console=tty0 console=ttyS0"--location=mbr --boot-drive=vda
clearpart --all --initlabel --drives=vda
part / --fstype="xfs" --grow
%packages
@^minimal
@core
%end
reboot

The Kickstart file is usually hosted on a webserver somewhere, but if that isn’t an easy option, you can convince virt-install take this role. The trick is to use the --initrd-inject option to let the installer know where it can find the file. Create a file at /root/c7.ks with the contents above, then

virt-install --name centos7 --ram 2048 \
 --disk path=/var/lib/libvirt/images/centos7.qcow2,size=30,format=qcow2 \
 --os-type linux --os-variant rhel7 \
 --initrd-inject=/root/c7.ks \
 --network network=default \
 --graphics none --console pty,target_type=serial \
 --location 'http://mirror.it.ubc.ca/centos/7/os/x86_64/' \
 --extra-args 'console=tty0 console=ttyS0 ks=file:/c7.ks'

CentOS 8 / Alma 8 / RockyLinux 8

I revisited this process to get ready to migrate some of our infrastructure to whatever will replace CentOS 8. I ran into a few more problems (mostly dracut timeouts or missing kernel modules). I haven’t tracked down all of the individual problems but the command below should at least do a kickstart based install. Make sure you use the kickstart kernel for the --location in virt-install and in the url parameter of the kickstart.

$ virt-install --name alma8 --ram 2048 \
  --disk path=/var/lib/libvirt/images/alma8.qcow2,size=30,format=qcow2,bus=virtio \
  --os-type linux --os-variant rhel8-unknown \
  --network network=default \
  --graphics=none --console pty,target_type=serial \
  --location 'https://yum.tamu.edu/alma/8.3/BaseOS/x86_64/kickstart/' \
  --initrd-inject=/home/iana/alma8.ks \
  --extra-args="console=ttyS0 inst.ks=file:/alma8.ks inst.text"

And here is the corresponding kickstart. At the relevant points, replace the crypted password entries with the output of openssl passwd -6

>>>>>>> 3a5a54813fcbabda0fdce1d1e7bb5970559e40c0
text
url --url=https://repo.almalinux.org/almalinux/8/BaseOS/x86_64/kickstart/
repo --name="BaseOS" --mirrorlist=https://mirrors.almalinux.org/mirrorlist/8/baseos
repo --name="AppStream" --mirrorlist=https://mirrors.almalinux.org/mirrorlist/8/appstream
keyboard --vckeymap=us --xlayouts=''
lang en_US.UTF-8
ignoredisk --only-use=vda
bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=vda
autopart --type=lvm
clearpart --all --initlabel --drives=vda
# Network information
network  --bootproto=dhcp --device=ens2 --ipv6=auto --activate
network  --hostname=localhost.localdomain
# Root password
rootpw --iscrypted ### GENERATE A PW WITH `openssl passwd -6` ###
# Run the Setup Agent on first boot
firstboot --enable
# Do not configure the X Window System
skipx
# System services
services --enabled="chronyd"
# System timezone
timezone America/New_York --isUtc
user --groups=wheel --name=ptty2u --password=### GENERATE A PW WITH `openssl passwd -6 ### --iscrypted --gecos="ptty2u"

%packages
@^minimal-environment
kexec-tools

%end

%addon com_redhat_kdump --enable --reserve-mb='auto'

%end

%anaconda
pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty
pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok
pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty
%end
reboot

OpenBSD

OpenBSD can be installed in using a serial console, but the process is slightly more complicated. (N.B. These are old notes and the process may be simpler now, but this method still seems to work). The key is to change boot.conf to expect the console to be on a serial line.

First, grab a copy of the install media we wish to modify, loop mount it and make our edits.

  $ mkdir -p /mnt/OpenBSD69-in /mnt/OpenBSD69-out
  $ wget http://openbsd.cs.toronto.edu/pub/OpenBSD/6.9/amd64/install69.iso
  $ mount -o ro,loop ./install69.iso /mnt/OpenBSD69-in
  $ rsync -avp /mnt/OpenBSD69-in/ /mnt/OpenBSD69-out
  
  $ vi /mnt/OpenBSD69-out/etc/boot.conf
 +  stty com0 115200
 +  set tty com0
    set image /6.9/amd64/bsd.rd

Now generate a new ISO,

  $ cd /mnt/OpenBSD69-out
  $ genisoimage -v -r -L -l -d -D -N \
    -sysid OpenBSD -V OpenBSD -volset OpenBSD \
    -p "Ian" -P "PIMS" \
    -b 6.9/amd64/cdbr -no-emul-boot \
    -c 6.9/amd64/boot.catalog \
    -o ../unofficial-OpenBSD69.iso .

Here is an explanation of some of the options

Flag Argument Purpose
-v Be verbose
-r Generate SUSP and RR extensions and set ownership/mode to useful values
-L Allow leading dots in filenames
-l Allow full 31-character filenames - breaks ISO9660
-d Do not append a period to files that do not have one
-D Do not use deep directory relocation
-N Omit version numbers from ISO9660 filenames
-sysid OpenBSD Specifies the system ID
-V OpenBSD Specifies the volume ID
-volset OpenBSD Specifies the volume set ID
-p Ian Preparer ID
-P PIMS Publisher
-b 6.9/amd64/cdbr Path and filename for El Torito bootable CD for x86 PCs
-no-emul-boot load and execute the boot image without performing any disk emulation
-c 6.9/amd64/boot.catalog The path and filename of the boot catalog
-o ../unofficial-openbsd69.iso The output ISO image
. The final argument is the path to the root of the filesystem for the new image

Finally, install with virt-install using the new ISO as the location

virt-install -n OpenBSD69-1 -r 1024 \
  --disk path=/var/lib/libvirt/images/OpenBSD69-1.qcow2,format=qcow2,size=20,bus=virtio \
  --network network=default -c /mnt/unofficial-OpenBSD69.iso \
  -v --graphics none

You should be dropped to the standard OpenBSD installer.