mirror of
https://github.com/tailscale/tailscale.git
synced 2025-08-23 11:27:29 +00:00
logtail/filch: preallocate a scanner buffer
Scanning log lines is a frequent source of allocations. Pre-allocate a re-usable buffer. This still doesn't help when there are giant log lines. Those will still be problematic from an iOS memory perspective. For more on that, see https://github.com/tailscale/corp/issues/2423. (For those who cannot follow that link, it is a discussion of particular problematic types of log lines for particular categories of customers. The "categories of customers" part is the reason that it is a private issue.) There is also a latent bug here. If we ever encounter a log line longer than bufio.MaxScanTokenSize, then bufio.Scan will return an error, and we'll truncate the file and discard the rest of the log. That's not good, but bufio.MaxScanTokenSize is really big, so it probably doesn't matter much in practice now. Unfortunately, it does prevent us from easily capping the potential memory usage here, on pain of losing log entries. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
This commit is contained in:

committed by
Josh Bleecher Snyder

parent
8ab44b339e
commit
93284209bc
@@ -12,6 +12,7 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
"unicode"
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
type filchTest struct {
|
||||
@@ -169,3 +170,10 @@ func TestFilchStderr(t *testing.T) {
|
||||
t.Errorf("unexpected write to fake stderr: %s", b)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSizeOf(t *testing.T) {
|
||||
s := unsafe.Sizeof(Filch{})
|
||||
if s > 4096 {
|
||||
t.Fatalf("Filch{} has size %d on %v, decrease size of buf field", s, runtime.GOARCH)
|
||||
}
|
||||
}
|
||||
|
Reference in New Issue
Block a user