This is an experiment to see how often this test would fail if we run it
on every commit. This depends on #2145 to fix a flaky part of the test.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
Okay, so, at a high level testing NixOS is a lot different than
other distros due to NixOS' determinism. Normally NixOS wants packages to
be defined in either an overlay, a custom packageOverrides or even
yolo-inline as a part of the system configuration. This is going to have
us take a different approach compared to other distributions. The overall
plan here is as following:
1. make the binaries as normal
2. template in their paths as raw strings to the nixos system module
3. run `nixos-generators -f qcow -o $CACHE_DIR/tailscale/nixos/version -c generated-config.nix`
4. pass that to the steps that make the virtual machine
It doesn't really make sense for us to use a premade virtual machine image
for this as that will make it harder to deterministically create the image.
Nix commands generate a lot of output, so their output is hidden behind the
`-verbose-nix-output` flag.
This unfortunately makes this test suite have a hard dependency on
Nix/NixOS, however the test suite has only ever been run on NixOS (and I
am not sure if it runs on other distros at all), so this probably isn't too
big of an issue.
Signed-off-by: Christine Dodrill <xe@tailscale.com>
This runner is in my homelab while we muse about a better, more
permanent home for these tests.
Updates #1988
Signed-off-by: Christine Dodrill <xe@tailscale.com>
