Displaying CSV files in a readable way on the terminal

 Until this week I did not know about the column command.

$ head -5 zillow.csv
"Index", "Living Space (sq ft)", "Beds", "Baths", "Zip", "Year", "List Price ($)"
 1, 2222, 3, 3.5, 32312, 1981, 250000
 2, 1628, 3, 2,   32308, 2009, 185000
 3, 3824, 5, 4,   32312, 1954, 399000
 4, 1137, 3, 2,   32309, 1993, 150000

Turned out this file is much more readable with a good pipe (and a large screen)

$ head -5 zillow.csv | column --table --separator ,
"Index"   "Living Space (sq ft)"   "Beds"   "Baths"   "Zip"     "Year"   "List Price ($)"
 1        2222                     3        3.5       32312     1981     250000
 2        1628                     3        2           32308   2009     185000
 3        3824                     5        4           32312   1954     399000
 4        1137                     3        2           32309   1993     150000

column is part of util-linux and is thus available in all distributions.
Example file taken from this example list.

Playing Tetris over serial console

Today I played Tetris over a serial console connection, on a Vax 4000 running OpenBSD. I haven't felt that 1337 since a long time.
I am going to get rid of that Vax system though. If that's your stuff, contact me privately.

asciinema in its greatness:

How to move a single VM between cloud providers

I am running since a decade a small Debian VM, that I use for basic web and mail hosting. Since most of the VM setup is done manually and not following the Infrastructure As Code pattern, it is faster to simply copy the filesystem when switching providers instead of reconfiguring everything.
The steps involved are:

1. create a backup of the filesystem using tar of rsync, excluding dynamic content
rsync  --archive \
    --one-file-system --numeric-ids \
    --rsh "ssh -i private_key root@server:/ /local_dir
or
tar -cvpzf backup.tar.gz \
--numeric-owner \
--exclude=/backup.tar.gz \
--one-file-system /

Notice here the --one-file-system switch which avoids back'ing up the content of mount points like /proc, /dev.
If you have extra partitions with a mounted filesystem, like /boot or home you need do add a separate backup for those.

2. create a new VM on the new cloud provider, verify you have a working console access, and power it off.
3. boot on the new cloud provider a rescue image
4. partition the disk image on the new provider.
5. mount the new root partition, and untar your backup on it. You could for instance push the local backup via rsync, or download the tar archive using https.
6. update network configuration and /etc/fstab
7. chroot into the target system, and reinstall grub

This works surprisingly well, and you if made your backup locally, you can test the whole procedure by building a test VM with your backup. Just replace the deboostrap step with a command like tar -xvpzf /path/to/backup.tar.gz -C /mount_point --numeric-owner

Using this procedure, I moved from Hetzner (link in French language) to Digital Ocean, from Digital Ocean to Vultr, and now back at Hetzner.

Quick NetBSD serial console install on libvirt

I wanted to set up a small VM with NetBSD to test a couple of virt-install option. It turns out it you can get to the installer prompt quite fast.

get the NetBSD installer for serial console:

wget https://cdn.netbsd.org/pub/NetBSD/NetBSD-9.1/i386/installation/cdrom/boot-com.iso 

start the install  

$ virt-install \
--connect qemu:///session \
--name netbsd \
--ram 64 \
--vcpus 2 \
--disk path=$HOME/netbsd.qcow2,size=4,bus=scsi,format=qcow2 \
--controller type=scsi,model=virtio-scsi \
--cdrom=boot-com.iso \
--virt-type kvm \
--os-variant netbsd8.0 \
--graphics none \
--arch i686 \
--console pty,target_type=serial 

This will start a VM in usermode networking, so no need to be root, but the VM won’t be reachable from the outside world, except if you add qemu usermode port forwarding.

Using Debian and RHEL troubleshootings containers on Kubernetes & OpenShift

You can connect to a running pod with oc/kubectl rsh pod_name, or start a copy of a running pod with oc debug pod_name, but as best practises recommend unprivileged, slim container images, where do you get sosreport, kdump, dig and nmap for troubleshooting ? 

Fortunately you can start either a transient Debian troubleshooting container with:

oc run troubleshooting-pod --stdin --tty --rm --image=docker.io/library/debian:buster

or a Red Hat Entreprise Linux:

oc run troubleshooting-pod --stdin --tty --rm --image=registry.access.redhat.com/rhel7/rhel-tools

Learning openshift: a good moment to revisit awk too

I can’t believe I spent all these years using only grep.

Most of us know how to use awk to print the nth column of a file:

$ awk '{print $1}' /etc/hosts

will print all IP addresses from /etc/hosts

But you can also do filtering before printing the chosen column:

$ awk '$5 >= 2 {print $2}' /path/to/file

will print the second column of all lines, where the 5th column is greater than 2. That would have been hard with grep.

Now I can use that to find out all deployments on my openshift cluster, where the number of current replicas is greater than 2.

$ oc get deployments --all-namespaces | awk '$5 >= 2 {print $2}'
NAME
oauth-openshift
console
downloads
router-default
etcd-quorum-guard
prometheus-adapter
thanos-querier
packageserver

I know that openshift/kubernetes both have a powerful query selector syntax, but for the moment awk will do.

Test a webcam from the command line on Linux with VLC

Since this info was too well hidden on the internet, here is the information:

cvlc v4l2:///dev/video0

and there you go.
If you have multiple cameras connected, you can try /dev/video0 up to /dev/video5