a blog by Marius Gedminas

Switching to GitHub Actions

I am grateful to Travis CI for providing many years of free CI service to all of my FOSS projects. However the free lunch is over and I don’t want to constantly ask for free build credits by email (the first 10,000 ran out in 10 days).

I’ve chosen to migrate to GitHub Actions. There are already helpful resources about this:

In fact part of the difficulty with the migration is that there’s too much documentation available! And all the examples do things slightly differently.

So here’s a bunch of tips and discoveries from converting over 20 repositories:

  • The workflow is named build because the name of the workflow is used in the status badge, and I want my status badges to say

    build: passing

    like they used to.

    name: build
    
  • It’s placed in a file named .github/workflows/build.yml (which doesn’t really matter, but I had to pick some name, so I decided to be consistent).

  • I still ambivalent to all the examples restricting builds to one branch. I think I like that I’m not getting double builds (branch vs PR) that I used to get with Travis/Appveyor, especially since every matrix row (“Python 3.8” etc.) shows up as a separate check in the UI.

  • I’m testing multiple Python versions, and I’m putting the Python version in the pip cache key, because without it I still saw too many “building wheel …” messages during pip install.

        - name: Pip cache
          uses: actions/cache@v2
          with:
            path: ~/.cache/pip
            key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('tox.ini', 'setup.py') }}
            restore-keys: |
              ${{ runner.os }}-pip-${{ matrix.python-version }}-
              ${{ runner.os }}-pip-
    
  • I’m still using Coveralls, but not through the official action (which doesn’t support Python, what). Note that the coveralls package on PyPI gained GitHub Actions support only in version 2.1.0, which no longer supports Python 2, so I’m skipping Coveralls uploads for my 2.7 and pypy2 builds.

        - name: Report to coveralls
          run: coveralls
          if: "matrix.python-version != '2.7' && matrix.python-version != 'pypy2'"
          env:
            GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
            COVERALLS_PARALLEL: true
    
  • Note that in the GitHub Actions expression language '2.7' is a string, and 2.7 is a float, so you must quote Python versions in the strategy matrix or the above if condition will not skip 2.7!

      strategy:
        matrix:
          python-version:
            - "2.7"
            - "3.6"
            - "3.7"
            - "3.8"
            - "3.9"
            - "pypy2"
            - "pypy3"
    

    This is a good idea anyway, because Python 3.10 is coming, and in YAML 3.10 is just a spelling of the floating point number 3.1 with an unnecessary zero.

  • I want linters (flake8, mypy, isort, check-manifest, check-python-versions) to have their own matrix rows, and I don’t want to vary Python versions for these. The easiest solution turned out to be defining two jobs with their own matrices, and to duplicate most of the build steps.

    jobs:
      build:
        name: ${{ matrix.python-version }}
        strategy:
          matrix:
            python-version:
              - "3.6"
              ...
        steps:
          ...
    
          - name: Run tests
            run: tox -e py
      lint:
        name: ${{ matrix.toxenv }}
        strategy:
          matrix:
            toxenv:
              - flake8
              ...
        steps:
          ...
    
          - name: Run ${{ matrix.toxenv }}
            run: tox -e ${{ matrix.toxenv }}
    

    And yes, I’m putting ${{ matrix.toxenv }} into the pip cache key for these:

        - name: Pip cache
          uses: actions/cache@v2
          with:
            path: ~/.cache/pip
            key: ${{ runner.os }}-pip-${{ matrix.toxenv }}-${{ hashFiles('tox.ini') }}
            restore-keys: |
              ${{ runner.os }}-pip-${{ matrix.toxenv }}-
              ${{ runner.os }}-pip-
    
  • tox-gh-actions doesn’t work the way tox-travis does and runs nothing at all if you don’t add custom configuration to a tox.ini.

  • The workflow files are very verbose, compared to those of Travis CI!

  • The checks UI is flaky, misleading, and not up to GitHub’s usual standards.

If you’re curious, you can look at my workflow file for findimports (very simple test suite, no linters) or workflow file for project-summary (tox, two matrices), or workflow file for check-manifest (a real 2D matrix of python-version × version control system).

Booting ISO images, 2020 edition

Now that Ubuntu 20.04 LTS is out, it was time to update my bootable USB drive with a bunch of ISO images. Except I got tired of editing grub.cfg by hand. So I wrote a script.

Now all I have to do is plug in my USB drive, download a new ISO into it, run make in the /boot/grub subdirectory, and presto! It works.

screenshot of the GRUB menu

What the script does:

  • finds all ISO files in ../../ubuntu/*.iso
  • extracts /boot/grub/grub.cfg from inside the ISO image (yes, I wrote an ISO 9660 filesystem parser in Python)
  • extracts the kernel command line
  • adds the iso-scan/filename=… argument to make it boot from an ISO image
  • generates a new grub.cfg with a menu listing all Ubuntu versions and all image variants (desktop vs server)

This works for all Ubuntu ISO images based on casper, which is a tool Ubuntu uses to produce their LiveCD images.

Notably it doesn’t work for older Ubuntu server images based on debian-installer.

I don’t know about other distributions and Ubuntu variants. Ubuntu variants ought to work, but my script might generate wrong titles for them.

New laptop: ThinkPad X390

The time has come to replace my trusty ThinkPad X220 with something more modern (that has enough horsepower to run Subnautica). I picked a ThinkPad X390 for the usual reasons (it’s a ThinkPad, it’s small and light and portable, the keyboard is reasonably sized, it has a trackpoint).

I installed Ubuntu 19.04 from a USB stick. This time I did not wipe Windows (because Subnautica), which gave me an unpleasant surprise: the Ubuntu installer does not allow you to enable full disk encryption if you’re using dual-boot.

Everything works great on Linux, except for the fingerprint scanner (a Synaptics device with ID 06cb:00bd in lsusb). Good news: Huang Vincent from Synaptics is working on a driver, so stay tuned!

The 13” 1920x1080 screen at ~160 dpi is a bit uncomfortable to use with my poor eyesight, so first I tried to use GNOME Tweaks to scale fonts to 120%. This worked okay-ish (a shame that Firefox ignores this and I had to tweak it separately) until I plugged in an external monitor (~80 dpi) where the large fonts were cartoonishly too large. Next, I enabled GNOME’s experimental fractional scaling support (I use a Ubuntu on Wayland session instead of the default one) and (after a reboot) set the zoom level on the internal screen to 125% (after resetting font scaling back to 100%, of course). Wayland apps look nice and crisp, X11 apps (Firefox) look fuzzy, but shrug at least I can read the text without squinting.

Surprise: I don’t hate the new keyboard! It took a while to get used to (and I caught myself using vim’s 0/$/^F/^B instead of Home/End/PageDown/PageUp way more than I usually do). I’m not sure why the backlit keyboard has two different light levels instead of just one. The mute LEDs on the F1 and F4 keys are too bright, especially at night, so I turn down the volume to the lowest level instead of muting. (The power LED is also a bit too bright – why did Lenovo make it white instead of the traditional nice green?)

I like the nice little physical camera switch that moves a little builtin plastic cover over the lens so I can be sure nobody’s spying on me :)

Things were less smooth on Windows 10: first it didn’t want to reboot (I waited like 20 minutes at the blue “Do not turn off your computer” screen before YOLOing the Power button for 10 seconds). Next, Bluetooth didn’t work (scanning for visible devices to add failed after a fraction of a second) until I updated the driver (Device Manager -> Bluetooth something something -> right-click -> Update driver) and/or installed Windows updates and rebooted. Other than that Windows 10 was a pleasant surprise – not something I could love, but at least something I don’t hate.

Subnautica runs … at 20 fps, using the lowest graphics settings. That’s integrated Intel graphics for you. (This was a deliberate choice: I want Linux compatibility more than I want to run games, so I avoid NVidia.)

Overall verdict: a nice little machine, would buy again.

Update: In August of 2020 GNOME Software offered a firmware update for the Synaptics Prometheus fingerprint scanner. The fingerprint scanner now works!

Switching to HTTPS

It’s 2017, so it’s time to make this blog HTTPS-only, with permanent redirects and HSTS headers and everything.

Apologies to any planets that might get flooded when the RSS feed changes all permalinks to https.

(P.S. Hugo is a pain, as I expected. Content-Security-Policy headers are also made of pain, so I’m skipping one for this blog right now.)

Ansible is great

I haven’t used configuration management tools in a very long time because I didn’t know which tool to pick (Chef? Puppet? Ansible? Salt? cfengine?). Now I use Ansible and am very happy with it.

Here’s how you start – with an empty git repository:

$ mkdir my-infra
$ cd my-infra
$ git init .

Let’s start by making sure a couple of Ubuntu servers will have unattended upgrades enabled. First we tell Ansible what the servers are

$ vim hosts
$ cat hosts
debesis
ranka