tailscale/tstest
Josh Bleecher Snyder 4bb2c6980d util/testingutil: new package with MinAllocsPerRun
testing.AllocsPerRun measures the total allocations performed
by the entire program while repeatedly executing a function f.
If some unrelated part of the rest of the program happens to
allocate a lot during that period, you end up with a test failure.

Ideally, the rest of the program would be silent while
testing.AllocsPerRun executes.

Realistically, that is often unachievable.

AllocsPerRun attempts to mitigate this by setting GOMAXPROCS to 1,
but that doesn't prevent other code from running;
it only makes it less likely.

You can also mitigate this by passing a large iteration count to
AllocsPerRun, but that is unreliable and needlessly expensive.

Unlike most of package testing, AllocsPerRun doesn't use any
toolchain magic, so we can just write a replacement.

One wild idea is to change how we count mallocs.
Instead of using runtime.MemStats, turn on memory profiling with a
memprofilerate of 1. Discard all samples from the profile whose stack
does not contain testing.AllocsPerRun. Count the remaining samples to
determine the number of mallocs.

That's fun, but overkill.

Instead, this change adds a simple API that attempts to get f to
run at least once with a target number of allocations.
This is useful when you know that f should allocate consistently.
We can then assume that any iterations with too many allocations
are probably due to one-time costs or background noise.

This suits most uses of AllocsPerRun.

Ratcheting tests tend to be significantly less flaky,
because they are biased towards success.
They can also be faster, because they can exit early,
once success has been reached.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-10-28 12:48:37 -07:00
..
integration control/controlclient,tailcfg: propagate registration errors to the frontend 2021-10-27 06:57:26 -07:00
natlab fix: typo spelling grammar 2021-08-24 07:55:04 -07:00
staticcheck tstest/staticcheck: import the main package to fix "go mod tidy" 2021-02-20 09:53:47 -08:00
allocs.go util/testingutil: new package with MinAllocsPerRun 2021-10-28 12:48:37 -07:00
clock.go tstest/natlab: add a stateful firewall. 2020-07-11 05:17:38 +00:00
log_test.go tstest: rename LogListener to LogLineTracker 2020-09-04 08:09:56 -07:00
log.go tstest: make MemLogger.String acquire its mutex 2021-09-09 11:38:06 -07:00
resource.go net/dns{,/resolver}: refactor DNS forwarder, send out of right link on macOS/iOS 2021-06-23 16:04:10 -07:00
tstest.go tstest: add WaitFor helper 2021-04-29 14:43:46 -07:00