mirror of
https://github.com/tailscale/tailscale.git
synced 2025-12-23 00:56:20 +00:00
221ca20a00
The filch implementation is fairly broken: * When Filch.cur exceeds MaxFileSize, it calls moveContents to copy the entirety of cur into alt (while holding the write lock). By nature, this is the movement of a lot of data in a hot path, meaning that all log calls will be globally blocked! It also means that log uploads will be blocked during the move. * The implementation of moveContents is buggy in that it copies data from cur into the start of alt, but fails to truncate alt to the number of bytes copied. Consequently, there are unrelated lines near the end, leading to out-of-order lines when being read back. * Data filched via stderr do not directly respect MaxFileSize, which is only checked every 100 Filch.Write calls. This means that it is possible that the file grows far beyond the specified max file size before moveContents is called. * If both log files have data when New is called, it also copies the entirety of cur into alt. This can block the startup of a process copying lots of data before the process can do any useful work. * TryReadLine is implemented using bufio.Scanner. Unfortunately, it will choke on any lines longer than bufio.MaxScanTokenSize, rather than gracefully skip over them. The re-implementation avoids a lot of these problems by fundamentally eliminating the need for moveContent. We enforce MaxFileSize by simply rotating the log files whenever the current file exceeds MaxFileSize/2. This is a constant-time operation regardless of file size. To more gracefully handle lines longer than bufio.MaxScanTokenSize, we skip over these lines (without growing the read buffer) and report an error. This allows subsequent lines to be read. In order to improve debugging, we add alot of metrics. Note that the the mechanism of dup2 with stderr is inherently racy with a the two file approach. The order of operations during a rotation is carefully chosen to reduce the race window to be as short as possible. Thus, this is slightly less racy than before. Updates tailscale/corp#21363 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
35 lines
517 B
Go
35 lines
517 B
Go
// Copyright (c) Tailscale Inc & AUTHORS
|
|
// SPDX-License-Identifier: BSD-3-Clause
|
|
|
|
//go:build ts_omit_logtail
|
|
|
|
package filch
|
|
|
|
import "os"
|
|
|
|
type Options struct {
|
|
ReplaceStderr bool
|
|
MaxLineSize int
|
|
MaxFileSize int
|
|
}
|
|
|
|
type Filch struct {
|
|
OrigStderr *os.File
|
|
}
|
|
|
|
func (*Filch) TryReadLine() ([]byte, error) {
|
|
return nil, nil
|
|
}
|
|
|
|
func (*Filch) Write(b []byte) (int, error) {
|
|
return len(b), nil
|
|
}
|
|
|
|
func (f *Filch) Close() error {
|
|
return nil
|
|
}
|
|
|
|
func New(string, Options) (*Filch, error) {
|
|
return new(Filch), nil
|
|
}
|