TheArchive/tailscale
1
0
Fork 0
mirror of https://github.com/tailscale/tailscale.git synced 2024-12-02 14:35:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
f78e8f6ca6
tailscale/util/deephash/deephash_test.go

1141 lines
35 KiB
Go
Raw Normal View History

all: update copyright and license headers This updates all source files to use a new standard header for copyright and license declaration. Notably, copyright no longer includes a date, and we now use the standard SPDX-License-Identifier header. This commit was done almost entirely mechanically with perl, and then some minimal manual fixes. Updates #6865 Signed-off-by: Will Norris <will@tailscale.com>
2023-01-27 21:37:20 +00:00
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
internal/deepprint: add missing copyright headers
2020-06-30 02:36:45 +00:00
internal/deephash: rename from deepprint Yes, it printed, but that was an implementation detail for hashing. And coming optimization will make it print even less. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 19:09:25 +00:00
package deephash
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
import (
util/deephash: include type as part of hash for interfaces (#2476) A Go interface may hold any number of different concrete types. Just because two underlying values hash to the same thing does not mean the two values are identical if they have different concrete types. As such, include the type in the hash.
2021-07-21 17:26:04 +00:00
"archive/tar"
util/deephash: fix TestArrayAllocs Unfortunately this test fails on certain architectures. The problem comes down to inconsistencies in the Go escape analysis where specific variables are marked as escaping on certain architectures. The variables escaping to the heap are unfortunately in crypto/sha256, which makes it impossible to fixthis locally in deephash. For now, fix the test by compensating for the allocations that occur from calling sha256.digest.Sum. See golang/go#48055 Fixes #2727 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-30 17:47:21 +00:00
"crypto/sha256"
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
"encoding/binary"
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
"fmt"
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
"hash"
util/deephash: hash uint{8,16,32,64} explicitly (#2502) Instead of hashing the humanly formatted forms of a number, hash the native machine bits of the integers themselves. There is a small performance gain for this: name old time/op new time/op delta Hash-8 75.7µs ± 1% 76.0µs ± 2% ~ (p=0.315 n=10+9) HashMapAcyclic-8 63.1µs ± 3% 61.3µs ± 1% -2.77% (p=0.000 n=10+10) TailcfgNode-8 10.3µs ± 1% 10.2µs ± 1% -1.48% (p=0.000 n=10+10) HashArray-8 1.07µs ± 1% 1.05µs ± 1% -1.79% (p=0.000 n=10+10) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-03 04:44:13 +00:00
"math"
util/deephash: rely on direct memory hashing for primitive kinds (#5457) Rather than separate functions to hash each kind, just rely on the fact that these are direct memory hashable, thus simplifying the code. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 03:50:56 +00:00
"math/bits"
util/deephash: fix map hashing when key & element have the same type Regression from 09afb8e35b56c1150ae9872d781c7ae3c151b848, in which the same reflect.Value scratch value was being used as the map iterator copy destination. Also: make nil and empty maps hash differently, add test. Fixes #4871 Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com> Change-Id: I67f42524bc81f694c1b7259d6682200125ea4a66 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-16 20:21:32 +00:00
"math/rand"
all: use various net/netip parse funcs directly Mechanical change with perl+goimports. Changed {Must,}Parse{IP,IPPrefix,IPPort} to their netip variants, then goimports -d . Finally, removed the net/netaddr wrappers, to prevent future use. Updates #5162 Change-Id: I59c0e38b5fbca5a935d701645789cddf3d7863ad Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 03:55:44 +00:00
"net/netip"
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
"reflect"
util/deephash: fix TestArrayAllocs Unfortunately this test fails on certain architectures. The problem comes down to inconsistencies in the Go escape analysis where specific variables are marked as escaping on certain architectures. The variables escaping to the heap are unfortunately in crypto/sha256, which makes it impossible to fixthis locally in deephash. For now, fix the test by compensating for the allocations that occur from calling sha256.digest.Sum. See golang/go#48055 Fixes #2727 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-30 17:47:21 +00:00
"runtime"
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
"testing"
util/deephash: fix map hashing when key & element have the same type Regression from 09afb8e35b56c1150ae9872d781c7ae3c151b848, in which the same reflect.Value scratch value was being used as the map iterator copy destination. Also: make nil and empty maps hash differently, add test. Fixes #4871 Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com> Change-Id: I67f42524bc81f694c1b7259d6682200125ea4a66 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-16 20:21:32 +00:00
"testing/quick"
util/deephash: don't track cycles on non-recursive types name old time/op new time/op delta Hash-8 67.3µs ±20% 76.5µs ±16% ~ (p=0.143 n=10+10) HashMapAcyclic-8 63.0µs ± 2% 56.3µs ± 1% -10.65% (p=0.000 n=10+8) TailcfgNode-8 9.18µs ± 2% 6.52µs ± 3% -28.96% (p=0.000 n=9+10) HashArray-8 732ns ± 3% 709ns ± 1% -3.21% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I28642050d837dff66b2db54b2b0e6d272a930be8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
"time"
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
qt "github.com/frankban/quicktest"
all: replace tailcfg.DiscoKey with key.DiscoPublic. Signed-off-by: David Anderson <danderson@tailscale.com>
2021-11-02 21:41:56 +00:00
"go4.org/mem"
util/deephash: specialize for netip.Addr and drop AppendTo support (#5402) There are 5 types that we care about that implement AppendTo: key.DiscoPublic key.NodePublic netip.Prefix netipx.IPRange netip.Addr The key types are thin wrappers around [32]byte and are memory hashable. The netip.Prefix and netipx.IPRange types are thin wrappers over netip.Addr and are hashable by default if netip.Addr is hashable. The netip.Addr type is the only one with a complex structure where the default behavior of deephash does not hash it correctly due to the presence of the intern.Value type. Drop support for AppendTo and instead add specialized hashing for netip.Addr that would be semantically equivalent to == on the netip.Addr values. The AppendTo support was already broken prior to this change. It was fully removed (intentionally or not) in #4870. It was partially restored in #4858 for the fast path, but still broken in the slow path. Just drop support for it altogether. This does mean we lack any ability for types to self-hash themselves. In the future we can add support for types that implement: interface { DeepHash() Sum } Test and fuzz cases were added for the relevant types that used to rely on the AppendTo method. FuzzAddr has been executed on 1 billion samples without issues. Signed-off-by: Joe Tsai joetsai@digital-static.net
2022-08-19 05:54:56 +00:00
"go4.org/netipx"
internal/deepprint: improve benchmark This more closely matches our real usage of deepprint. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-05-11 16:45:12 +00:00
"tailscale.com/tailcfg"
types/dnstype: introduce new package for Resolver So the type can be used in net/dns without introducing a tailcfg dependency. For #2596 Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
2021-08-05 21:05:24 +00:00
"tailscale.com/types/dnstype"
util/deephash: don't reflect.Copy if element type is a defined uint8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-07 18:58:02 +00:00
"tailscale.com/types/ipproto"
wgengine/wgcfg: convert to use new node key type. Updates #3206 Signed-off-by: David Anderson <danderson@tailscale.com>
2021-10-28 00:42:33 +00:00
"tailscale.com/types/key"
types/ptr: move all the ptrTo funcs to one new package's ptr.To Change-Id: Ia0b820ffe7aa72897515f19bd415204b6fe743c7 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-12-01 01:33:29 +00:00
"tailscale.com/types/ptr"
util/deephash: fix unexported time.Time hashing Updates tailscale/corp#6311 Change-Id: I33cd7e4040966261c2f2eb3d32f29936aeb7f632 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-27 16:10:22 +00:00
"tailscale.com/util/deephash/testtype"
internal/deepprint: improve benchmark This more closely matches our real usage of deepprint. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-05-11 16:45:12 +00:00
"tailscale.com/util/dnsname"
util/deephash: implement SelfHasher to allow types to hash themselves Updates: corp#16409 Signed-off-by: Tom DNetto <tom@tailscale.com>
2024-01-31 23:33:59 +00:00
"tailscale.com/util/hashx"
util/deephash: skip alloc test under race detector Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-07 18:40:28 +00:00
"tailscale.com/version"
util/deephash: don't reflect.Copy if element type is a defined uint8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-07 18:58:02 +00:00
"tailscale.com/wgengine/filter"
many: gofmt. Signed-off-by: David Anderson <danderson@tailscale.com>
2021-03-26 00:41:51 +00:00
"tailscale.com/wgengine/router"
all: move wgcfg from wireguard-go This is mostly code movement from the wireguard-go repo. Most of the new wgcfg package corresponds to the wireguard-go wgcfg package. wgengine/wgcfg/device{_test}.go was device/config{_test}.go. There were substantive but simple changes to device_test.go to remove internal package device references. The API of device.Config (now wgcfg.DeviceConfig) grew an error return; we previously logged the error and threw it away. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-01-29 20:16:36 +00:00
"tailscale.com/wgengine/wgcfg"
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
)
util/deephash: disambiguate hashing of AppendTo (#2483) Prepend size to AppendTo output. Fixes #2443 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-07-21 18:29:08 +00:00
type appendBytes []byte
func (p appendBytes) AppendTo(b []byte) []byte {
return append(b, p...)
}
util/deephash: tighten up SelfHasher API (#11012) Providing a hash.Block512 is an implementation detail of how deephash works today, but providing an opaque type with mostly equivalent API (i.e., HashUint8, HashBytes, etc. methods) is still sensible. Thus, define a public Hasher type that exposes exactly the API that an implementation of SelfHasher would want to call. This gives us freedom to change the hashing algorithm of deephash at some point in the future. Also, this type is likely going to be called by types that are going to memoize their own hash results, we additionally add a HashSum method to simplify this use case. Add documentation to SelfHasher on how a type might implement it. Updates: corp#16409 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2024-02-02 01:07:41 +00:00
type selfHasherValueRecv struct {
util/deephash: implement SelfHasher to allow types to hash themselves Updates: corp#16409 Signed-off-by: Tom DNetto <tom@tailscale.com>
2024-01-31 23:33:59 +00:00
emit uint64
}
util/deephash: tighten up SelfHasher API (#11012) Providing a hash.Block512 is an implementation detail of how deephash works today, but providing an opaque type with mostly equivalent API (i.e., HashUint8, HashBytes, etc. methods) is still sensible. Thus, define a public Hasher type that exposes exactly the API that an implementation of SelfHasher would want to call. This gives us freedom to change the hashing algorithm of deephash at some point in the future. Also, this type is likely going to be called by types that are going to memoize their own hash results, we additionally add a HashSum method to simplify this use case. Add documentation to SelfHasher on how a type might implement it. Updates: corp#16409 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2024-02-02 01:07:41 +00:00
func (s selfHasherValueRecv) Hash(h *hashx.Block512) {
h.HashUint64(s.emit)
}
type selfHasherPointerRecv struct {
emit uint64
}
func (s *selfHasherPointerRecv) Hash(h *hashx.Block512) {
util/deephash: implement SelfHasher to allow types to hash themselves Updates: corp#16409 Signed-off-by: Tom DNetto <tom@tailscale.com>
2024-01-31 23:33:59 +00:00
h.HashUint64(s.emit)
}
util/deephash: include type as part of hash for interfaces (#2476) A Go interface may hold any number of different concrete types. Just because two underlying values hash to the same thing does not mean the two values are identical if they have different concrete types. As such, include the type in the hash.
2021-07-21 17:26:04 +00:00
func TestHash(t *testing.T) {
all: use any instead of interface{} My favorite part of generics. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2022-03-16 23:27:57 +00:00
type tuple [2]any
type iface struct{ X any }
util/deephash: hash uint{8,16,32,64} explicitly (#2502) Instead of hashing the humanly formatted forms of a number, hash the native machine bits of the integers themselves. There is a small performance gain for this: name old time/op new time/op delta Hash-8 75.7µs ± 1% 76.0µs ± 2% ~ (p=0.315 n=10+9) HashMapAcyclic-8 63.1µs ± 3% 61.3µs ± 1% -2.77% (p=0.000 n=10+10) TailcfgNode-8 10.3µs ± 1% 10.2µs ± 1% -1.48% (p=0.000 n=10+10) HashArray-8 1.07µs ± 1% 1.05µs ± 1% -1.79% (p=0.000 n=10+10) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-03 04:44:13 +00:00
type scalars struct {
I8 int8
I16 int16
I32 int32
I64 int64
I int
U8 uint8
U16 uint16
U32 uint32
U64 uint64
U uint
UP uintptr
F32 float32
F64 float64
C64 complex64
C128 complex128
}
util/deephash: include type as part of hash for interfaces (#2476) A Go interface may hold any number of different concrete types. Just because two underlying values hash to the same thing does not mean the two values are identical if they have different concrete types. As such, include the type in the hash.
2021-07-21 17:26:04 +00:00
type MyBool bool
type MyHeader tar.Header
util/deephash: fix collisions between different types Updates #4883 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-22 02:50:48 +00:00
var zeroFloat64 float64
util/deephash: include type as part of hash for interfaces (#2476) A Go interface may hold any number of different concrete types. Just because two underlying values hash to the same thing does not mean the two values are identical if they have different concrete types. As such, include the type in the hash.
2021-07-21 17:26:04 +00:00
tests := []struct {
in tuple
wantEq bool
}{
util/deephash: hash uint{8,16,32,64} explicitly (#2502) Instead of hashing the humanly formatted forms of a number, hash the native machine bits of the integers themselves. There is a small performance gain for this: name old time/op new time/op delta Hash-8 75.7µs ± 1% 76.0µs ± 2% ~ (p=0.315 n=10+9) HashMapAcyclic-8 63.1µs ± 3% 61.3µs ± 1% -2.77% (p=0.000 n=10+10) TailcfgNode-8 10.3µs ± 1% 10.2µs ± 1% -1.48% (p=0.000 n=10+10) HashArray-8 1.07µs ± 1% 1.05µs ± 1% -1.79% (p=0.000 n=10+10) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-03 04:44:13 +00:00
{in: tuple{false, true}, wantEq: false},
{in: tuple{true, true}, wantEq: true},
{in: tuple{false, false}, wantEq: true},
{
in: tuple{
scalars{-8, -16, -32, -64, -1234, 8, 16, 32, 64, 1234, 5678, 32.32, 64.64, 32 + 32i, 64 + 64i},
scalars{-8, -16, -32, -64, -1234, 8, 16, 32, 64, 1234, 5678, 32.32, 64.64, 32 + 32i, 64 + 64i},
},
wantEq: true,
},
{in: tuple{scalars{I8: math.MinInt8}, scalars{I8: math.MinInt8 / 2}}, wantEq: false},
{in: tuple{scalars{I16: math.MinInt16}, scalars{I16: math.MinInt16 / 2}}, wantEq: false},
{in: tuple{scalars{I32: math.MinInt32}, scalars{I32: math.MinInt32 / 2}}, wantEq: false},
{in: tuple{scalars{I64: math.MinInt64}, scalars{I64: math.MinInt64 / 2}}, wantEq: false},
{in: tuple{scalars{I: -1234}, scalars{I: -1234 / 2}}, wantEq: false},
{in: tuple{scalars{U8: math.MaxUint8}, scalars{U8: math.MaxUint8 / 2}}, wantEq: false},
{in: tuple{scalars{U16: math.MaxUint16}, scalars{U16: math.MaxUint16 / 2}}, wantEq: false},
{in: tuple{scalars{U32: math.MaxUint32}, scalars{U32: math.MaxUint32 / 2}}, wantEq: false},
{in: tuple{scalars{U64: math.MaxUint64}, scalars{U64: math.MaxUint64 / 2}}, wantEq: false},
{in: tuple{scalars{U: 1234}, scalars{U: 1234 / 2}}, wantEq: false},
{in: tuple{scalars{UP: 5678}, scalars{UP: 5678 / 2}}, wantEq: false},
{in: tuple{scalars{F32: 32.32}, scalars{F32: math.Nextafter32(32.32, 0)}}, wantEq: false},
{in: tuple{scalars{F64: 64.64}, scalars{F64: math.Nextafter(64.64, 0)}}, wantEq: false},
{in: tuple{scalars{F32: float32(math.NaN())}, scalars{F32: float32(math.NaN())}}, wantEq: true},
{in: tuple{scalars{F64: float64(math.NaN())}, scalars{F64: float64(math.NaN())}}, wantEq: true},
{in: tuple{scalars{C64: 32 + 32i}, scalars{C64: complex(math.Nextafter32(32, 0), 32)}}, wantEq: false},
{in: tuple{scalars{C128: 64 + 64i}, scalars{C128: complex(math.Nextafter(64, 0), 64)}}, wantEq: false},
util/deephash: handle slice edge-cases (#5471) It is unclear whether the lack of checking nil-ness of slices was an oversight or a deliberate feature. Lacking a comment, the assumption is that this was an oversight. Also, expand the logic to perform cycle detection for recursive slices. We do this on a per-element basis since a slice is semantically equivalent to a list of pointers. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-30 07:33:18 +00:00
{in: tuple{[]int(nil), []int(nil)}, wantEq: true},
{in: tuple{[]int{}, []int(nil)}, wantEq: false},
{in: tuple{[]int{}, []int{}}, wantEq: true},
{in: tuple{[]string(nil), []string(nil)}, wantEq: true},
{in: tuple{[]string{}, []string(nil)}, wantEq: false},
{in: tuple{[]string{}, []string{}}, wantEq: true},
util/deephash: disambiguate hashing of AppendTo (#2483) Prepend size to AppendTo output. Fixes #2443 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-07-21 18:29:08 +00:00
{in: tuple{[]appendBytes{{}, {0, 0, 0, 0, 0, 0, 0, 1}}, []appendBytes{{}, {0, 0, 0, 0, 0, 0, 0, 1}}}, wantEq: true},
{in: tuple{[]appendBytes{{}, {0, 0, 0, 0, 0, 0, 0, 1}}, []appendBytes{{0, 0, 0, 0, 0, 0, 0, 1}, {}}}, wantEq: false},
util/deephash: include type as part of hash for interfaces (#2476) A Go interface may hold any number of different concrete types. Just because two underlying values hash to the same thing does not mean the two values are identical if they have different concrete types. As such, include the type in the hash.
2021-07-21 17:26:04 +00:00
{in: tuple{iface{MyBool(true)}, iface{MyBool(true)}}, wantEq: true},
{in: tuple{iface{true}, iface{MyBool(true)}}, wantEq: false},
{in: tuple{iface{MyHeader{}}, iface{MyHeader{}}}, wantEq: true},
{in: tuple{iface{MyHeader{}}, iface{tar.Header{}}}, wantEq: false},
{in: tuple{iface{&MyHeader{}}, iface{&MyHeader{}}}, wantEq: true},
{in: tuple{iface{&MyHeader{}}, iface{&tar.Header{}}}, wantEq: false},
{in: tuple{iface{[]map[string]MyBool{}}, iface{[]map[string]MyBool{}}}, wantEq: true},
{in: tuple{iface{[]map[string]bool{}}, iface{[]map[string]MyBool{}}}, wantEq: false},
util/deephash: fix collisions between different types Updates #4883 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-22 02:50:48 +00:00
{in: tuple{zeroFloat64, -zeroFloat64}, wantEq: false}, // Issue 4883 (false alarm)
{in: tuple{[]any(nil), 0.0}, wantEq: false}, // Issue 4883
{in: tuple{[]any(nil), uint8(0)}, wantEq: false}, // Issue 4883
{in: tuple{nil, nil}, wantEq: true}, // Issue 4883
util/deephash: improve cycle detection (#2470) The previous algorithm used a map of all visited pointers. The strength of this approach is that it quickly prunes any nodes that we have ever visited before. The detriment of the approach is that pruning is heavily dependent on the order that pointers were visited. This is especially relevant for hashing a map where map entries are visited in a non-deterministic manner, which would cause the map hash to be non-deterministic (which defeats the point of a hash). This new algorithm uses a stack of all visited pointers, similar to how github.com/google/go-cmp performs cycle detection. When we visit a pointer, we push it onto the stack, and when we leave a pointer, we pop it from the stack. Before visiting a pointer, we first check whether the pointer exists anywhere in the stack. If yes, then we prune the node. The detriment of this approach is that we may hash a node more often than before since we do not prune as aggressively. The set of visited pointers up until any node is only the path of nodes up to that node and not any other pointers that may have been visited elsewhere. This provides us deterministic hashing regardless of visit order. We can now delete hashMapFallback and associated complexity, which only exists because the previous approach was non-deterministic in the presence of cycles. This fixes a failure of the old algorithm where obviously different values are treated as equal because the pruning was too aggresive. See https://github.com/tailscale/tailscale/issues/2443#issuecomment-883653534 The new algorithm is slightly slower since it prunes less aggresively: name old time/op new time/op delta Hash-8 66.1µs ± 1% 68.8µs ± 1% +4.09% (p=0.000 n=19+19) HashMapAcyclic-8 63.0µs ± 1% 62.5µs ± 1% -0.76% (p=0.000 n=18+19) TailcfgNode-8 9.79µs ± 2% 9.88µs ± 1% +0.95% (p=0.000 n=19+17) HashArray-8 643ns ± 1% 653ns ± 1% +1.64% (p=0.000 n=19+19) However, a slower but more correct algorithm seems more favorable than a faster but incorrect algorithm. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-07-22 22:22:48 +00:00
{
in: func() tuple {
i1 := 1
i2 := 2
v1 := [3]*int{&i1, &i2, &i1}
v2 := [3]*int{&i1, &i2, &i2}
return tuple{v1, v2}
}(),
wantEq: false,
},
util/deephash: specialize for netip.Addr and drop AppendTo support (#5402) There are 5 types that we care about that implement AppendTo: key.DiscoPublic key.NodePublic netip.Prefix netipx.IPRange netip.Addr The key types are thin wrappers around [32]byte and are memory hashable. The netip.Prefix and netipx.IPRange types are thin wrappers over netip.Addr and are hashable by default if netip.Addr is hashable. The netip.Addr type is the only one with a complex structure where the default behavior of deephash does not hash it correctly due to the presence of the intern.Value type. Drop support for AppendTo and instead add specialized hashing for netip.Addr that would be semantically equivalent to == on the netip.Addr values. The AppendTo support was already broken prior to this change. It was fully removed (intentionally or not) in #4870. It was partially restored in #4858 for the fast path, but still broken in the slow path. Just drop support for it altogether. This does mean we lack any ability for types to self-hash themselves. In the future we can add support for types that implement: interface { DeepHash() Sum } Test and fuzz cases were added for the relevant types that used to rely on the AppendTo method. FuzzAddr has been executed on 1 billion samples without issues. Signed-off-by: Joe Tsai joetsai@digital-static.net
2022-08-19 05:54:56 +00:00
{in: tuple{netip.Addr{}, netip.Addr{}}, wantEq: true},
{in: tuple{netip.Addr{}, netip.AddrFrom4([4]byte{})}, wantEq: false},
{in: tuple{netip.AddrFrom4([4]byte{}), netip.AddrFrom4([4]byte{})}, wantEq: true},
{in: tuple{netip.AddrFrom4([4]byte{192, 168, 0, 1}), netip.AddrFrom4([4]byte{192, 168, 0, 1})}, wantEq: true},
{in: tuple{netip.AddrFrom4([4]byte{192, 168, 0, 1}), netip.AddrFrom4([4]byte{192, 168, 0, 2})}, wantEq: false},
{in: tuple{netip.AddrFrom4([4]byte{}), netip.AddrFrom16([16]byte{})}, wantEq: false},
{in: tuple{netip.AddrFrom16([16]byte{}), netip.AddrFrom16([16]byte{})}, wantEq: true},
{in: tuple{netip.AddrPort{}, netip.AddrPort{}}, wantEq: true},
{in: tuple{netip.AddrPort{}, netip.AddrPortFrom(netip.AddrFrom4([4]byte{}), 0)}, wantEq: false},
{in: tuple{netip.AddrPortFrom(netip.AddrFrom4([4]byte{}), 0), netip.AddrPortFrom(netip.AddrFrom4([4]byte{}), 0)}, wantEq: true},
{in: tuple{netip.AddrPortFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 1234), netip.AddrPortFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 1234)}, wantEq: true},
{in: tuple{netip.AddrPortFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 1234), netip.AddrPortFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 1235)}, wantEq: false},
{in: tuple{netip.AddrPortFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 1234), netip.AddrPortFrom(netip.AddrFrom4([4]byte{192, 168, 0, 2}), 1234)}, wantEq: false},
{in: tuple{netip.Prefix{}, netip.Prefix{}}, wantEq: true},
util/deephash: cleanup TODO in TestHash (#11080) Updates #cleanup Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2024-02-09 01:33:25 +00:00
{in: tuple{netip.Prefix{}, netip.PrefixFrom(netip.Addr{}, 1)}, wantEq: true},
util/deephash: specialize for netip.Addr and drop AppendTo support (#5402) There are 5 types that we care about that implement AppendTo: key.DiscoPublic key.NodePublic netip.Prefix netipx.IPRange netip.Addr The key types are thin wrappers around [32]byte and are memory hashable. The netip.Prefix and netipx.IPRange types are thin wrappers over netip.Addr and are hashable by default if netip.Addr is hashable. The netip.Addr type is the only one with a complex structure where the default behavior of deephash does not hash it correctly due to the presence of the intern.Value type. Drop support for AppendTo and instead add specialized hashing for netip.Addr that would be semantically equivalent to == on the netip.Addr values. The AppendTo support was already broken prior to this change. It was fully removed (intentionally or not) in #4870. It was partially restored in #4858 for the fast path, but still broken in the slow path. Just drop support for it altogether. This does mean we lack any ability for types to self-hash themselves. In the future we can add support for types that implement: interface { DeepHash() Sum } Test and fuzz cases were added for the relevant types that used to rely on the AppendTo method. FuzzAddr has been executed on 1 billion samples without issues. Signed-off-by: Joe Tsai joetsai@digital-static.net
2022-08-19 05:54:56 +00:00
{in: tuple{netip.Prefix{}, netip.PrefixFrom(netip.AddrFrom4([4]byte{}), 0)}, wantEq: false},
{in: tuple{netip.PrefixFrom(netip.AddrFrom4([4]byte{}), 1), netip.PrefixFrom(netip.AddrFrom4([4]byte{}), 1)}, wantEq: true},
{in: tuple{netip.PrefixFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 1), netip.PrefixFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 1)}, wantEq: true},
{in: tuple{netip.PrefixFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 1), netip.PrefixFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 0)}, wantEq: false},
{in: tuple{netip.PrefixFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), 1), netip.PrefixFrom(netip.AddrFrom4([4]byte{192, 168, 0, 2}), 1)}, wantEq: false},
{in: tuple{netipx.IPRange{}, netipx.IPRange{}}, wantEq: true},
{in: tuple{netipx.IPRange{}, netipx.IPRangeFrom(netip.AddrFrom4([4]byte{}), netip.AddrFrom16([16]byte{}))}, wantEq: false},
{in: tuple{netipx.IPRangeFrom(netip.AddrFrom4([4]byte{}), netip.AddrFrom16([16]byte{})), netipx.IPRangeFrom(netip.AddrFrom4([4]byte{}), netip.AddrFrom16([16]byte{}))}, wantEq: true},
{in: tuple{netipx.IPRangeFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), netip.AddrFrom4([4]byte{192, 168, 0, 100})), netipx.IPRangeFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), netip.AddrFrom4([4]byte{192, 168, 0, 100}))}, wantEq: true},
{in: tuple{netipx.IPRangeFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), netip.AddrFrom4([4]byte{192, 168, 0, 100})), netipx.IPRangeFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), netip.AddrFrom4([4]byte{192, 168, 0, 101}))}, wantEq: false},
{in: tuple{netipx.IPRangeFrom(netip.AddrFrom4([4]byte{192, 168, 0, 1}), netip.AddrFrom4([4]byte{192, 168, 0, 100})), netipx.IPRangeFrom(netip.AddrFrom4([4]byte{192, 168, 0, 2}), netip.AddrFrom4([4]byte{192, 168, 0, 100}))}, wantEq: false},
{in: tuple{key.DiscoPublic{}, key.DiscoPublic{}}, wantEq: true},
{in: tuple{key.DiscoPublic{}, key.DiscoPublicFromRaw32(mem.B(func() []byte {
b := make([]byte, 32)
b[0] = 1
return b
}()))}, wantEq: false},
{in: tuple{key.NodePublic{}, key.NodePublic{}}, wantEq: true},
{in: tuple{key.NodePublic{}, key.NodePublicFromRaw32(mem.B(func() []byte {
b := make([]byte, 32)
b[0] = 1
return b
}()))}, wantEq: false},
util/deephash: tighten up SelfHasher API (#11012) Providing a hash.Block512 is an implementation detail of how deephash works today, but providing an opaque type with mostly equivalent API (i.e., HashUint8, HashBytes, etc. methods) is still sensible. Thus, define a public Hasher type that exposes exactly the API that an implementation of SelfHasher would want to call. This gives us freedom to change the hashing algorithm of deephash at some point in the future. Also, this type is likely going to be called by types that are going to memoize their own hash results, we additionally add a HashSum method to simplify this use case. Add documentation to SelfHasher on how a type might implement it. Updates: corp#16409 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2024-02-02 01:07:41 +00:00
{in: tuple{&selfHasherPointerRecv{}, &selfHasherPointerRecv{}}, wantEq: true},
{in: tuple{(*selfHasherPointerRecv)(nil), (*selfHasherPointerRecv)(nil)}, wantEq: true},
{in: tuple{(*selfHasherPointerRecv)(nil), &selfHasherPointerRecv{}}, wantEq: false},
{in: tuple{&selfHasherPointerRecv{emit: 1}, &selfHasherPointerRecv{emit: 2}}, wantEq: false},
{in: tuple{selfHasherValueRecv{emit: 1}, selfHasherValueRecv{emit: 2}}, wantEq: false},
{in: tuple{selfHasherValueRecv{emit: 2}, selfHasherValueRecv{emit: 2}}, wantEq: true},
util/deephash: include type as part of hash for interfaces (#2476) A Go interface may hold any number of different concrete types. Just because two underlying values hash to the same thing does not mean the two values are identical if they have different concrete types. As such, include the type in the hash.
2021-07-21 17:26:04 +00:00
}
for _, tt := range tests {
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
gotEq := Hash(&tt.in[0]) == Hash(&tt.in[1])
util/deephash: include type as part of hash for interfaces (#2476) A Go interface may hold any number of different concrete types. Just because two underlying values hash to the same thing does not mean the two values are identical if they have different concrete types. As such, include the type in the hash.
2021-07-21 17:26:04 +00:00
if gotEq != tt.wantEq {
util/deephash: fix collisions between different types Updates #4883 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-22 02:50:48 +00:00
t.Errorf("(Hash(%T %v) == Hash(%T %v)) = %v, want %v", tt.in[0], tt.in[0], tt.in[1], tt.in[1], gotEq, tt.wantEq)
util/deephash: include type as part of hash for interfaces (#2476) A Go interface may hold any number of different concrete types. Just because two underlying values hash to the same thing does not mean the two values are identical if they have different concrete types. As such, include the type in the hash.
2021-07-21 17:26:04 +00:00
}
}
}
internal/deephash: simplify API Reduce to just a single external endpoint. Convert from a variadic number of interfaces to a slice there. name old time/op new time/op delta Hash-8 14.4µs ± 0% 14.0µs ± 1% -3.08% (p=0.000 n=9+9) name old alloc/op new alloc/op delta Hash-8 873B ± 0% 793B ± 0% -9.16% (p=0.000 n=9+6) name old allocs/op new allocs/op delta Hash-8 18.0 ± 0% 14.0 ± 0% -22.22% (p=0.000 n=10+10) Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-05-18 20:16:37 +00:00
func TestDeepHash(t *testing.T) {
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
// v contains the types of values we care about for our current callers.
// Mostly we're just testing that we don't panic on handled types.
v := getVal()
util/deephash: export a Hash func for use by the control plane name old time/op new time/op delta Hash-6 69.4µs ± 6% 68.4µs ± 4% ~ (p=0.286 n=9+9) HashMapAcyclic-6 115µs ± 5% 115µs ± 4% ~ (p=1.000 n=10+10) name old alloc/op new alloc/op delta Hash-6 2.29kB ± 0% 1.88kB ± 0% -18.13% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (all equal) name old allocs/op new allocs/op delta Hash-6 58.0 ± 0% 54.0 ± 0% -6.90% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-05 04:25:15 +00:00
hash1 := Hash(v)
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
t.Logf("hash: %v", hash1)
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for range 20 {
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
v := getVal()
hash2 := Hash(v)
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
if hash1 != hash2 {
t.Error("second hash didn't match")
}
}
}
util/deephash: fix map hashing to actually hash elements Fixes #4868 Change-Id: I574fd139cb7f7033dd93527344e6aa0e625477c7 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-16 18:24:04 +00:00
// Tests that we actually hash map elements. Whoops.
func TestIssue4868(t *testing.T) {
m1 := map[int]string{1: "foo"}
m2 := map[int]string{1: "bar"}
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
if Hash(&m1) == Hash(&m2) {
util/deephash: fix map hashing to actually hash elements Fixes #4868 Change-Id: I574fd139cb7f7033dd93527344e6aa0e625477c7 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-16 18:24:04 +00:00
t.Error("bogus")
}
}
util/deephash: fix map hashing when key & element have the same type Regression from 09afb8e35b56c1150ae9872d781c7ae3c151b848, in which the same reflect.Value scratch value was being used as the map iterator copy destination. Also: make nil and empty maps hash differently, add test. Fixes #4871 Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com> Change-Id: I67f42524bc81f694c1b7259d6682200125ea4a66 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-16 20:21:32 +00:00
func TestIssue4871(t *testing.T) {
m1 := map[string]string{"": "", "x": "foo"}
m2 := map[string]string{}
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
if h1, h2 := Hash(&m1), Hash(&m2); h1 == h2 {
util/deephash: fix map hashing when key & element have the same type Regression from 09afb8e35b56c1150ae9872d781c7ae3c151b848, in which the same reflect.Value scratch value was being used as the map iterator copy destination. Also: make nil and empty maps hash differently, add test. Fixes #4871 Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com> Change-Id: I67f42524bc81f694c1b7259d6682200125ea4a66 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-16 20:21:32 +00:00
t.Errorf("bogus: h1=%x, h2=%x", h1, h2)
}
}
func TestNilVsEmptymap(t *testing.T) {
m1 := map[string]string(nil)
m2 := map[string]string{}
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
if h1, h2 := Hash(&m1), Hash(&m2); h1 == h2 {
util/deephash: fix map hashing when key & element have the same type Regression from 09afb8e35b56c1150ae9872d781c7ae3c151b848, in which the same reflect.Value scratch value was being used as the map iterator copy destination. Also: make nil and empty maps hash differently, add test. Fixes #4871 Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com> Change-Id: I67f42524bc81f694c1b7259d6682200125ea4a66 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-16 20:21:32 +00:00
t.Errorf("bogus: h1=%x, h2=%x", h1, h2)
}
}
func TestMapFraming(t *testing.T) {
m1 := map[string]string{"foo": "", "fo": "o"}
m2 := map[string]string{}
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
if h1, h2 := Hash(&m1), Hash(&m2); h1 == h2 {
util/deephash: fix map hashing when key & element have the same type Regression from 09afb8e35b56c1150ae9872d781c7ae3c151b848, in which the same reflect.Value scratch value was being used as the map iterator copy destination. Also: make nil and empty maps hash differently, add test. Fixes #4871 Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com> Change-Id: I67f42524bc81f694c1b7259d6682200125ea4a66 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-16 20:21:32 +00:00
t.Errorf("bogus: h1=%x, h2=%x", h1, h2)
}
}
func TestQuick(t *testing.T) {
initSeed()
err := quick.Check(func(v, w map[string]string) bool {
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
return (Hash(&v) == Hash(&w)) == reflect.DeepEqual(v, w)
util/deephash: fix map hashing when key & element have the same type Regression from 09afb8e35b56c1150ae9872d781c7ae3c151b848, in which the same reflect.Value scratch value was being used as the map iterator copy destination. Also: make nil and empty maps hash differently, add test. Fixes #4871 Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com> Change-Id: I67f42524bc81f694c1b7259d6682200125ea4a66 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-16 20:21:32 +00:00
}, &quick.Config{MaxCount: 1000, Rand: rand.New(rand.NewSource(int64(seed)))})
if err != nil {
t.Fatalf("seed=%v, err=%v", seed, err)
}
}
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
type tailscaleTypes struct {
WGConfig *wgcfg.Config
RouterConfig *router.Config
MapFQDNAddrs map[dnsname.FQDN][]netip.Addr
MapFQDNAddrPorts map[dnsname.FQDN][]netip.AddrPort
MapDiscoPublics map[key.DiscoPublic]bool
MapResponse *tailcfg.MapResponse
FilterMatch filter.Match
}
func getVal() *tailscaleTypes {
return &tailscaleTypes{
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
&wgcfg.Config{
wgengine/wgcfg: remove Config.ListenPort We don't use the port that wireguard-go passes to us (via magicsock.connBind.Open). We ignore it entirely and use the port we selected. When we tell wireguard-go that we're changing the listen_port, it calls connBind.Close and then connBind.Open. And in the meantime, it stops calling the receive functions, which means that we stop receiving and processing UDP and DERP packets. And that is Very Bad. That was never a problem prior to b3ceca1dd7d7a1a6f9ddab136a4e12900e976333, because we passed the SkipBindUpdate flag to our wireguard-go fork, which told wireguard-go not to re-bind on listen_port changes. That commit eliminated the SkipBindUpdate flag. We could write a bunch of code to work around the gap. We could add background readers that process UDP and DERP packets when wireguard-go isn't. But it's simpler to never create the conditions in which wireguard-go rebinds. The other scenario in which wireguard-go re-binds is device.Down. Conveniently, we never call device.Down. We go from device.Up to device.Close, and the latter only when we're shutting down a magicsock.Conn completely. Rubber-ducked-by: Avery Pennarun <apenwarr@tailscale.com> Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-04-03 01:04:39 +00:00
Name: "foo",
all: migrate more code code to net/netip directly Instead of going through the tailscale.com/net/netaddr transitional wrappers. Updates #5162 Change-Id: I3dafd1c2effa1a6caa9b7151ecf6edd1a3fda3dd Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-08-02 20:38:11 +00:00
Addresses: []netip.Prefix{netip.PrefixFrom(netip.AddrFrom16([16]byte{3: 3}).Unmap(), 5)},
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
Peers: []wgcfg.Peer{
{
wgengine/wgcfg: convert to use new node key type. Updates #3206 Signed-off-by: David Anderson <danderson@tailscale.com>
2021-10-28 00:42:33 +00:00
PublicKey: key.NodePublic{},
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
},
},
},
&router.Config{
all: convert more code to use net/netip directly perl -i -npe 's,netaddr.IPPrefixFrom,netip.PrefixFrom,' $(git grep -l -F netaddr.) perl -i -npe 's,netaddr.IPPortFrom,netip.AddrPortFrom,' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPrefix,netip.Prefix,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPort,netip.AddrPort,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IP\b,netip.Addr,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPv6Raw\b,netip.AddrFrom16,g' $(git grep -l -F netaddr. ) goimports -w . Then delete some stuff from the net/netaddr shim package which is no longer neeed. Updates #5162 Change-Id: Ia7a86893fe21c7e3ee1ec823e8aba288d4566cd8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 04:14:09 +00:00
Routes: []netip.Prefix{
all: use various net/netip parse funcs directly Mechanical change with perl+goimports. Changed {Must,}Parse{IP,IPPrefix,IPPort} to their netip variants, then goimports -d . Finally, removed the net/netaddr wrappers, to prevent future use. Updates #5162 Change-Id: I59c0e38b5fbca5a935d701645789cddf3d7863ad Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 03:55:44 +00:00
netip.MustParsePrefix("1.2.3.0/24"),
netip.MustParsePrefix("1234::/64"),
wgengine/router: dns: unify on *BSD, multimode on Linux, Magic DNS (#536) Signed-off-by: Dmytro Shynkevych <dmytro@tailscale.com>
2020-07-14 13:12:00 +00:00
},
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
},
all: convert more code to use net/netip directly perl -i -npe 's,netaddr.IPPrefixFrom,netip.PrefixFrom,' $(git grep -l -F netaddr.) perl -i -npe 's,netaddr.IPPortFrom,netip.AddrPortFrom,' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPrefix,netip.Prefix,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPort,netip.AddrPort,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IP\b,netip.Addr,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPv6Raw\b,netip.AddrFrom16,g' $(git grep -l -F netaddr. ) goimports -w . Then delete some stuff from the net/netaddr shim package which is no longer neeed. Updates #5162 Change-Id: Ia7a86893fe21c7e3ee1ec823e8aba288d4566cd8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 04:14:09 +00:00
map[dnsname.FQDN][]netip.Addr{
all: use various net/netip parse funcs directly Mechanical change with perl+goimports. Changed {Must,}Parse{IP,IPPrefix,IPPort} to their netip variants, then goimports -d . Finally, removed the net/netaddr wrappers, to prevent future use. Updates #5162 Change-Id: I59c0e38b5fbca5a935d701645789cddf3d7863ad Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 03:55:44 +00:00
dnsname.FQDN("a."): {netip.MustParseAddr("1.2.3.4"), netip.MustParseAddr("4.3.2.1")},
dnsname.FQDN("b."): {netip.MustParseAddr("8.8.8.8"), netip.MustParseAddr("9.9.9.9")},
dnsname.FQDN("c."): {netip.MustParseAddr("6.6.6.6"), netip.MustParseAddr("7.7.7.7")},
dnsname.FQDN("d."): {netip.MustParseAddr("6.7.6.6"), netip.MustParseAddr("7.7.7.8")},
dnsname.FQDN("e."): {netip.MustParseAddr("6.8.6.6"), netip.MustParseAddr("7.7.7.9")},
dnsname.FQDN("f."): {netip.MustParseAddr("6.9.6.6"), netip.MustParseAddr("7.7.7.0")},
internal/deepprint: improve benchmark This more closely matches our real usage of deepprint. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-05-11 16:45:12 +00:00
},
all: convert more code to use net/netip directly perl -i -npe 's,netaddr.IPPrefixFrom,netip.PrefixFrom,' $(git grep -l -F netaddr.) perl -i -npe 's,netaddr.IPPortFrom,netip.AddrPortFrom,' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPrefix,netip.Prefix,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPort,netip.AddrPort,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IP\b,netip.Addr,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPv6Raw\b,netip.AddrFrom16,g' $(git grep -l -F netaddr. ) goimports -w . Then delete some stuff from the net/netaddr shim package which is no longer neeed. Updates #5162 Change-Id: Ia7a86893fe21c7e3ee1ec823e8aba288d4566cd8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 04:14:09 +00:00
map[dnsname.FQDN][]netip.AddrPort{
all: use various net/netip parse funcs directly Mechanical change with perl+goimports. Changed {Must,}Parse{IP,IPPrefix,IPPort} to their netip variants, then goimports -d . Finally, removed the net/netaddr wrappers, to prevent future use. Updates #5162 Change-Id: I59c0e38b5fbca5a935d701645789cddf3d7863ad Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 03:55:44 +00:00
dnsname.FQDN("a."): {netip.MustParseAddrPort("1.2.3.4:11"), netip.MustParseAddrPort("4.3.2.1:22")},
dnsname.FQDN("b."): {netip.MustParseAddrPort("8.8.8.8:11"), netip.MustParseAddrPort("9.9.9.9:22")},
dnsname.FQDN("c."): {netip.MustParseAddrPort("8.8.8.8:12"), netip.MustParseAddrPort("9.9.9.9:23")},
dnsname.FQDN("d."): {netip.MustParseAddrPort("8.8.8.8:13"), netip.MustParseAddrPort("9.9.9.9:24")},
dnsname.FQDN("e."): {netip.MustParseAddrPort("8.8.8.8:14"), netip.MustParseAddrPort("9.9.9.9:25")},
internal/deepprint: improve benchmark This more closely matches our real usage of deepprint. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-05-11 16:45:12 +00:00
},
all: replace tailcfg.DiscoKey with key.DiscoPublic. Signed-off-by: David Anderson <danderson@tailscale.com>
2021-11-02 21:41:56 +00:00
map[key.DiscoPublic]bool{
key.DiscoPublicFromRaw32(mem.B([]byte{1: 1, 31: 0})): true,
key.DiscoPublicFromRaw32(mem.B([]byte{1: 2, 31: 0})): false,
key.DiscoPublicFromRaw32(mem.B([]byte{1: 3, 31: 0})): true,
key.DiscoPublicFromRaw32(mem.B([]byte{1: 4, 31: 0})): false,
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
},
util/deephash: add a benchmark and some benchmark data No code changes. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 04:21:52 +00:00
&tailcfg.MapResponse{
DERPMap: &tailcfg.DERPMap{
Regions: map[int]*tailcfg.DERPRegion{
all: gofmt -w -s (simplify) tests And it updates the build tag style on a couple files. Change-Id: I84478d822c8de3f84b56fa1176c99d2ea5083237 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-12-15 16:42:25 +00:00
1: {
util/deephash: add a benchmark and some benchmark data No code changes. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 04:21:52 +00:00
RegionID: 1,
RegionCode: "foo",
Nodes: []*tailcfg.DERPNode{
{
Name: "n1",
RegionID: 1,
HostName: "foo.com",
},
{
Name: "n2",
RegionID: 1,
HostName: "bar.com",
},
},
},
},
},
DNSConfig: &tailcfg.DNSConfig{
tailcfg,all: change structs to []*dnstype.Resolver Signed-off-by: Maisem Ali <maisem@tailscale.com>
2022-05-03 21:41:58 +00:00
Resolvers: []*dnstype.Resolver{
util/deephash: add a benchmark and some benchmark data No code changes. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 04:21:52 +00:00
{Addr: "10.0.0.1"},
},
},
PacketFilter: []tailcfg.FilterRule{
{
SrcIPs: []string{"1.2.3.4"},
DstPorts: []tailcfg.NetPortRange{
{
IP: "1.2.3.4/32",
util/deephash: use keyed struct literal, fix vet Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 04:31:30 +00:00
Ports: tailcfg.PortRange{First: 1, Last: 2},
util/deephash: add a benchmark and some benchmark data No code changes. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 04:21:52 +00:00
},
},
},
},
Peers: []*tailcfg.Node{
{
ID: 1,
},
{
ID: 2,
},
},
UserProfiles: []tailcfg.UserProfile{
{ID: 1, LoginName: "foo@bar.com"},
{ID: 2, LoginName: "bar@foo.com"},
},
},
util/deephash: don't reflect.Copy if element type is a defined uint8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-07 18:58:02 +00:00
filter.Match{
IPProto: []ipproto.Proto{1, 2, 3},
},
wgengine, internal/deepprint: replace UAPI usage as hash func; add deepprint The new deepprint package just walks a Go data structure and writes to an io.Writer. It's not pretty like go-spew, etc. We then use it to replace the use of UAPI (which we have a TODO to remove) to generate signatures of data structures to detect whether anything changed (without retaining the old copy). This was necessary because the UAPI conversion ends up trying to do DNS lookups which an upcoming change depends on not happening.
2020-06-28 17:58:21 +00:00
}
}
internal/deepprint: add BenchmarkHash deepprint currently accounts for 15% of allocs in tailscaled. This is a useful benchmark to have. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-05-08 01:05:04 +00:00
util/deephash: add canMemHash func + typeInfo property Currently unused. (breaking up a bigger change) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-25 19:46:01 +00:00
type IntThenByte struct {
all: cleanup unused code, part 1 (#10661) Run `staticcheck` with `U1000` to find unused code. This cleans up about a half of it. I'll do the other half separately to keep PRs manageable. Updates #cleanup Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2023-12-20 22:50:30 +00:00
_ int
_ byte
util/deephash: add canMemHash func + typeInfo property Currently unused. (breaking up a bigger change) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-25 19:46:01 +00:00
}
all: cleanup unused code, part 1 (#10661) Run `staticcheck` with `U1000` to find unused code. This cleans up about a half of it. I'll do the other half separately to keep PRs manageable. Updates #cleanup Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2023-12-20 22:50:30 +00:00
type TwoInts struct{ _, _ int }
util/deephash: add canMemHash func + typeInfo property Currently unused. (breaking up a bigger change) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-25 19:46:01 +00:00
type IntIntByteInt struct {
i1, i2 int32
b byte // padding after
i3 int32
}
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
func u8(n uint8) string { return string([]byte{n}) }
func u32(n uint32) string { return string(binary.LittleEndian.AppendUint32(nil, n)) }
func u64(n uint64) string { return string(binary.LittleEndian.AppendUint64(nil, n)) }
util/deephash: rely on direct memory hashing for primitive kinds (#5457) Rather than separate functions to hash each kind, just rely on the fact that these are direct memory hashable, thus simplifying the code. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 03:50:56 +00:00
func ux(n uint) string {
if bits.UintSize == 32 {
return u32(uint32(n))
} else {
return u64(uint64(n))
}
}
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
func TestGetTypeHasher(t *testing.T) {
switch runtime.GOARCH {
case "amd64", "arm64", "arm", "386", "riscv64":
default:
// Test outputs below are specifically for little-endian machines.
// Just skip everything else for now. Feel free to add more above if
// you have the hardware to test and it's little-endian.
t.Skipf("skipping on %v", runtime.GOARCH)
}
type typedString string
var (
someInt = int('A')
someComplex128 = complex128(1 + 2i)
all: use various net/netip parse funcs directly Mechanical change with perl+goimports. Changed {Must,}Parse{IP,IPPrefix,IPPort} to their netip variants, then goimports -d . Finally, removed the net/netaddr wrappers, to prevent future use. Updates #5162 Change-Id: I59c0e38b5fbca5a935d701645789cddf3d7863ad Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 03:55:44 +00:00
someIP = netip.MustParseAddr("1.2.3.4")
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
)
tests := []struct {
name string
val any
out string
out32 string // overwrites out if 32-bit
}{
{
name: "int",
val: int(1),
util/deephash: rely on direct memory hashing for primitive kinds (#5457) Rather than separate functions to hash each kind, just rely on the fact that these are direct memory hashable, thus simplifying the code. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 03:50:56 +00:00
out: ux(1),
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
{
name: "int_negative",
val: int(-1),
util/deephash: rely on direct memory hashing for primitive kinds (#5457) Rather than separate functions to hash each kind, just rely on the fact that these are direct memory hashable, thus simplifying the code. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 03:50:56 +00:00
out: ux(math.MaxUint),
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
{
name: "int8",
val: int8(1),
out: "\x01",
},
{
name: "float64",
val: float64(1.0),
out: "\x00\x00\x00\x00\x00\x00\xf0?",
},
{
name: "float32",
val: float32(1.0),
out: "\x00\x00\x80?",
},
{
name: "string",
val: "foo",
out: "\x03\x00\x00\x00\x00\x00\x00\x00foo",
},
{
name: "typedString",
val: typedString("foo"),
out: "\x03\x00\x00\x00\x00\x00\x00\x00foo",
},
{
name: "string_slice",
val: []string{"foo", "bar"},
util/deephash: handle slice edge-cases (#5471) It is unclear whether the lack of checking nil-ness of slices was an oversight or a deliberate feature. Lacking a comment, the assumption is that this was an oversight. Also, expand the logic to perform cycle detection for recursive slices. We do this on a per-element basis since a slice is semantically equivalent to a list of pointers. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-30 07:33:18 +00:00
out: "\x01\x02\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00foo\x03\x00\x00\x00\x00\x00\x00\x00bar",
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
{
name: "int_slice",
val: []int{1, 0, -1},
util/deephash: handle slice edge-cases (#5471) It is unclear whether the lack of checking nil-ness of slices was an oversight or a deliberate feature. Lacking a comment, the assumption is that this was an oversight. Also, expand the logic to perform cycle detection for recursive slices. We do this on a per-element basis since a slice is semantically equivalent to a list of pointers. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-30 07:33:18 +00:00
out: "\x01\x03\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff",
out32: "\x01\x03\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff",
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
{
name: "struct",
val: struct {
a, b int
c uint16
}{1, -1, 2},
util/deephash: always keep values addressable (#5328) The logic of deephash is both simpler and easier to reason about if values are always addressable. In Go, the composite kinds are slices, arrays, maps, structs, interfaces, pointers, channels, and functions, where we define "composite" as a Go value that encapsulates some other Go value (e.g., a map is a collection of key-value entries). In the cases of pointers and slices, the sub-values are always addressable. In the cases of arrays and structs, the sub-values are always addressable if and only if the parent value is addressable. In the case of maps and interfaces, the sub-values are never addressable. To make them addressable, we need to copy them onto the heap. For the purposes of deephash, we do not care about channels and functions. For all non-composite kinds (e.g., strings and ints), they are only addressable if obtained from one of the composite kinds that produce addressable values (i.e., pointers, slices, addressable arrays, and addressable structs). A non-addressible, non-composite kind can be made addressable by allocating it on the heap, obtaining a pointer to it, and dereferencing it. Thus, if we can ensure that values are addressable at the entry points, and shallow copy sub-values whenever we encounter an interface or map, then we can ensure that all values are always addressable and assume such property throughout all the logic. Performance: name old time/op new time/op delta Hash-24 21.5µs ± 1% 19.7µs ± 1% -8.29% (p=0.000 n=9+9) HashPacketFilter-24 2.61µs ± 1% 2.62µs ± 0% +0.29% (p=0.037 n=10+9) HashMapAcyclic-24 30.8µs ± 1% 30.9µs ± 1% ~ (p=0.400 n=9+10) TailcfgNode-24 1.84µs ± 1% 1.84µs ± 2% ~ (p=0.928 n=10+10) HashArray-24 324ns ± 2% 332ns ± 2% +2.45% (p=0.000 n=10+10) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-10 05:00:02 +00:00
out: "\x01\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\x02\x00",
out32: "\x01\x00\x00\x00\xff\xff\xff\xff\x02\x00",
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
{
name: "nil_int_ptr",
val: (*int)(nil),
out: "\x00",
},
{
name: "int_ptr",
val: &someInt,
out: "\x01A\x00\x00\x00\x00\x00\x00\x00",
out32: "\x01A\x00\x00\x00",
},
{
name: "nil_uint32_ptr",
val: (*uint32)(nil),
out: "\x00",
},
{
name: "complex128_ptr",
val: &someComplex128,
out: "\x01\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x00\x00@",
},
{
name: "packet_filter",
val: filterRules,
all: require Go 1.21 Updates #8419 Change-Id: I809b6a4d59d92a2ab6ec587ccbb9053376bf02c2 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-08-09 03:44:31 +00:00
out: "\x01\x04\x00\x00\x00\x00\x00\x00\x00\x01\x03\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00*\v\x00\x00\x00\x00\x00\x00\x0010.1.3.4/32\v\x00\x00\x00\x00\x00\x00\x0010.0.0.0/24\x01\x03\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\n\x00\x00\x00\x00\x00\x00\x001.2.3.4/32\x01 \x00\x00\x00\x00\x00\x00\x00\x01\x00\x02\x00\x01\x04\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x01\x02\x03\x04!\x01\x01\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00foo\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\v\x00\x00\x00\x00\x00\x00\x00foooooooooo\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\f\x00\x00\x00\x00\x00\x00\x00baaaaaarrrrr\x00\x01\x00\x02\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\v\x00\x00\x00\x00\x00\x00\x00foooooooooo\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\f\x00\x00\x00\x00\x00\x00\x00baaaaaarrrrr\x00\x01\x00\x02\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\v\x00\x00\x00\x00\x00\x00\x00foooooooooo\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\f\x00\x00\x00\x00\x00\x00\x00baaaaaarrrrr\x00\x01\x00\x02\x00\x00\x00",
out32: "\x01\x04\x00\x00\x00\x00\x00\x00\x00\x01\x03\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00*\v\x00\x00\x00\x00\x00\x00\x0010.1.3.4/32\v\x00\x00\x00\x00\x00\x00\x0010.0.0.0/24\x01\x03\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\n\x00\x00\x00\x00\x00\x00\x001.2.3.4/32\x01 \x00\x00\x00\x01\x00\x02\x00\x01\x04\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00\x04\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x01\x02\x03\x04!\x01\x01\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00foo\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\v\x00\x00\x00\x00\x00\x00\x00foooooooooo\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\f\x00\x00\x00\x00\x00\x00\x00baaaaaarrrrr\x00\x01\x00\x02\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\v\x00\x00\x00\x00\x00\x00\x00foooooooooo\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\f\x00\x00\x00\x00\x00\x00\x00baaaaaarrrrr\x00\x01\x00\x02\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\v\x00\x00\x00\x00\x00\x00\x00foooooooooo\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\f\x00\x00\x00\x00\x00\x00\x00baaaaaarrrrr\x00\x01\x00\x02\x00\x00\x00",
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
{
all: convert more code to use net/netip directly perl -i -npe 's,netaddr.IPPrefixFrom,netip.PrefixFrom,' $(git grep -l -F netaddr.) perl -i -npe 's,netaddr.IPPortFrom,netip.AddrPortFrom,' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPrefix,netip.Prefix,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPort,netip.AddrPort,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IP\b,netip.Addr,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPv6Raw\b,netip.AddrFrom16,g' $(git grep -l -F netaddr. ) goimports -w . Then delete some stuff from the net/netaddr shim package which is no longer neeed. Updates #5162 Change-Id: Ia7a86893fe21c7e3ee1ec823e8aba288d4566cd8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 04:14:09 +00:00
name: "netip.Addr",
all: use various net/netip parse funcs directly Mechanical change with perl+goimports. Changed {Must,}Parse{IP,IPPrefix,IPPort} to their netip variants, then goimports -d . Finally, removed the net/netaddr wrappers, to prevent future use. Updates #5162 Change-Id: I59c0e38b5fbca5a935d701645789cddf3d7863ad Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 03:55:44 +00:00
val: netip.MustParseAddr("fe80::123%foo"),
util/deephash: specialize for netip.Addr and drop AppendTo support (#5402) There are 5 types that we care about that implement AppendTo: key.DiscoPublic key.NodePublic netip.Prefix netipx.IPRange netip.Addr The key types are thin wrappers around [32]byte and are memory hashable. The netip.Prefix and netipx.IPRange types are thin wrappers over netip.Addr and are hashable by default if netip.Addr is hashable. The netip.Addr type is the only one with a complex structure where the default behavior of deephash does not hash it correctly due to the presence of the intern.Value type. Drop support for AppendTo and instead add specialized hashing for netip.Addr that would be semantically equivalent to == on the netip.Addr values. The AppendTo support was already broken prior to this change. It was fully removed (intentionally or not) in #4870. It was partially restored in #4858 for the fast path, but still broken in the slow path. Just drop support for it altogether. This does mean we lack any ability for types to self-hash themselves. In the future we can add support for types that implement: interface { DeepHash() Sum } Test and fuzz cases were added for the relevant types that used to rely on the AppendTo method. FuzzAddr has been executed on 1 billion samples without issues. Signed-off-by: Joe Tsai joetsai@digital-static.net
2022-08-19 05:54:56 +00:00
out: u64(16+3) + u64(0x80fe) + u64(0x2301<<48) + "foo",
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
{
all: convert more code to use net/netip directly perl -i -npe 's,netaddr.IPPrefixFrom,netip.PrefixFrom,' $(git grep -l -F netaddr.) perl -i -npe 's,netaddr.IPPortFrom,netip.AddrPortFrom,' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPrefix,netip.Prefix,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPort,netip.AddrPort,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IP\b,netip.Addr,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPv6Raw\b,netip.AddrFrom16,g' $(git grep -l -F netaddr. ) goimports -w . Then delete some stuff from the net/netaddr shim package which is no longer neeed. Updates #5162 Change-Id: Ia7a86893fe21c7e3ee1ec823e8aba288d4566cd8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 04:14:09 +00:00
name: "ptr-netip.Addr",
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
val: &someIP,
util/deephash: specialize for netip.Addr and drop AppendTo support (#5402) There are 5 types that we care about that implement AppendTo: key.DiscoPublic key.NodePublic netip.Prefix netipx.IPRange netip.Addr The key types are thin wrappers around [32]byte and are memory hashable. The netip.Prefix and netipx.IPRange types are thin wrappers over netip.Addr and are hashable by default if netip.Addr is hashable. The netip.Addr type is the only one with a complex structure where the default behavior of deephash does not hash it correctly due to the presence of the intern.Value type. Drop support for AppendTo and instead add specialized hashing for netip.Addr that would be semantically equivalent to == on the netip.Addr values. The AppendTo support was already broken prior to this change. It was fully removed (intentionally or not) in #4870. It was partially restored in #4858 for the fast path, but still broken in the slow path. Just drop support for it altogether. This does mean we lack any ability for types to self-hash themselves. In the future we can add support for types that implement: interface { DeepHash() Sum } Test and fuzz cases were added for the relevant types that used to rely on the AppendTo method. FuzzAddr has been executed on 1 billion samples without issues. Signed-off-by: Joe Tsai joetsai@digital-static.net
2022-08-19 05:54:56 +00:00
out: u8(1) + u64(4) + u32(0x04030201),
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
{
all: convert more code to use net/netip directly perl -i -npe 's,netaddr.IPPrefixFrom,netip.PrefixFrom,' $(git grep -l -F netaddr.) perl -i -npe 's,netaddr.IPPortFrom,netip.AddrPortFrom,' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPrefix,netip.Prefix,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPort,netip.AddrPort,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IP\b,netip.Addr,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPv6Raw\b,netip.AddrFrom16,g' $(git grep -l -F netaddr. ) goimports -w . Then delete some stuff from the net/netaddr shim package which is no longer neeed. Updates #5162 Change-Id: Ia7a86893fe21c7e3ee1ec823e8aba288d4566cd8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 04:14:09 +00:00
name: "ptr-nil-netip.Addr",
val: (*netip.Addr)(nil),
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
out: "\x00",
},
{
name: "time",
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
val: time.Unix(1234, 5678).In(time.UTC),
out: u64(1234) + u32(5678) + u32(0),
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
util/deephash: fix unexported time.Time hashing Updates tailscale/corp#6311 Change-Id: I33cd7e4040966261c2f2eb3d32f29936aeb7f632 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-27 16:10:22 +00:00
{
name: "time_ptr", // addressable, as opposed to "time" test above
types/ptr: move all the ptrTo funcs to one new package's ptr.To Change-Id: Ia0b820ffe7aa72897515f19bd415204b6fe743c7 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-12-01 01:33:29 +00:00
val: ptr.To(time.Unix(1234, 5678).In(time.UTC)),
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
out: u8(1) + u64(1234) + u32(5678) + u32(0),
util/deephash: fix unexported time.Time hashing Updates tailscale/corp#6311 Change-Id: I33cd7e4040966261c2f2eb3d32f29936aeb7f632 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-27 16:10:22 +00:00
},
{
name: "time_ptr_via_unexported",
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
val: testtype.NewUnexportedAddressableTime(time.Unix(1234, 5678).In(time.UTC)),
out: u8(1) + u64(1234) + u32(5678) + u32(0),
util/deephash: fix unexported time.Time hashing Updates tailscale/corp#6311 Change-Id: I33cd7e4040966261c2f2eb3d32f29936aeb7f632 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-27 16:10:22 +00:00
},
{
name: "time_ptr_via_unexported_value",
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
val: *testtype.NewUnexportedAddressableTime(time.Unix(1234, 5678).In(time.UTC)),
out: u64(1234) + u32(5678) + u32(0),
util/deephash: fix unexported time.Time hashing Updates tailscale/corp#6311 Change-Id: I33cd7e4040966261c2f2eb3d32f29936aeb7f632 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-27 16:10:22 +00:00
},
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
{
name: "time_custom_zone",
val: time.Unix(1655311822, 0).In(time.FixedZone("FOO", -60*60)),
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
out: u64(1655311822) + u32(0) + u32(math.MaxUint32-60*60+1),
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
{
name: "time_nil",
val: (*time.Time)(nil),
out: "\x00",
},
{
name: "array_memhash",
val: [4]byte{1, 2, 3, 4},
out: "\x01\x02\x03\x04",
},
{
name: "array_ptr_memhash",
types/ptr: move all the ptrTo funcs to one new package's ptr.To Change-Id: Ia0b820ffe7aa72897515f19bd415204b6fe743c7 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-12-01 01:33:29 +00:00
val: ptr.To([4]byte{1, 2, 3, 4}),
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
out: "\x01\x01\x02\x03\x04",
},
{
name: "ptr_to_struct_partially_memhashable",
val: &struct {
A int16
B int16
C *int
}{5, 6, nil},
out: "\x01\x05\x00\x06\x00\x00",
},
{
name: "struct_partially_memhashable_but_cant_addr",
val: struct {
A int16
B int16
C *int
}{5, 6, nil},
out: "\x05\x00\x06\x00\x00",
},
{
name: "array_elements",
val: [4]byte{1, 2, 3, 4},
out: "\x01\x02\x03\x04",
},
{
name: "bool",
val: true,
out: "\x01",
},
{
name: "IntIntByteInt",
val: IntIntByteInt{1, 2, 3, 4},
out: "\x01\x00\x00\x00\x02\x00\x00\x00\x03\x04\x00\x00\x00",
},
{
all: fix spelling mistakes Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-09-25 18:29:55 +00:00
name: "IntIntByteInt-canaddr",
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
val: &IntIntByteInt{1, 2, 3, 4},
out: "\x01\x01\x00\x00\x00\x02\x00\x00\x00\x03\x04\x00\x00\x00",
},
{
name: "array-IntIntByteInt",
val: [2]IntIntByteInt{
{1, 2, 3, 4},
{5, 6, 7, 8},
},
out: "\x01\x00\x00\x00\x02\x00\x00\x00\x03\x04\x00\x00\x00\x05\x00\x00\x00\x06\x00\x00\x00\a\b\x00\x00\x00",
},
{
name: "array-IntIntByteInt-canaddr",
val: &[2]IntIntByteInt{
{1, 2, 3, 4},
{5, 6, 7, 8},
},
out: "\x01\x01\x00\x00\x00\x02\x00\x00\x00\x03\x04\x00\x00\x00\x05\x00\x00\x00\x06\x00\x00\x00\a\b\x00\x00\x00",
},
{
all: start groundwork for using capver for localapi & peerapi Updates #7015 Change-Id: I3d4c11b42a727a62eaac3262a879f29bb4ce82dd Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-01-19 20:40:58 +00:00
name: "tailcfg.Node",
val: &tailcfg.Node{},
util/deephash: relax an annoyingly needy test I'd added a test case of deephash against a tailcfg.Node to make sure it worked at all more than anything. We don't care what the exact bytes are in this test, just that it doesn't fail. So adjust for that. Then when we make changes to tailcfg.Node and types under it, we don't need to keep adjusting this test. Updates #cleanup Change-Id: Ibf4fa42820aeab8f5292fe65f9f92ffdb0b4407b Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-08-20 02:47:34 +00:00
out: "ANY", // magic value; just check it doesn't fail to hash
out32: "ANY",
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
rv := reflect.ValueOf(tt.val)
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
va := reflect.New(rv.Type()).Elem()
util/deephash: always keep values addressable (#5328) The logic of deephash is both simpler and easier to reason about if values are always addressable. In Go, the composite kinds are slices, arrays, maps, structs, interfaces, pointers, channels, and functions, where we define "composite" as a Go value that encapsulates some other Go value (e.g., a map is a collection of key-value entries). In the cases of pointers and slices, the sub-values are always addressable. In the cases of arrays and structs, the sub-values are always addressable if and only if the parent value is addressable. In the case of maps and interfaces, the sub-values are never addressable. To make them addressable, we need to copy them onto the heap. For the purposes of deephash, we do not care about channels and functions. For all non-composite kinds (e.g., strings and ints), they are only addressable if obtained from one of the composite kinds that produce addressable values (i.e., pointers, slices, addressable arrays, and addressable structs). A non-addressible, non-composite kind can be made addressable by allocating it on the heap, obtaining a pointer to it, and dereferencing it. Thus, if we can ensure that values are addressable at the entry points, and shallow copy sub-values whenever we encounter an interface or map, then we can ensure that all values are always addressable and assume such property throughout all the logic. Performance: name old time/op new time/op delta Hash-24 21.5µs ± 1% 19.7µs ± 1% -8.29% (p=0.000 n=9+9) HashPacketFilter-24 2.61µs ± 1% 2.62µs ± 0% +0.29% (p=0.037 n=10+9) HashMapAcyclic-24 30.8µs ± 1% 30.9µs ± 1% ~ (p=0.400 n=9+10) TailcfgNode-24 1.84µs ± 1% 1.84µs ± 2% ~ (p=0.928 n=10+10) HashArray-24 324ns ± 2% 332ns ± 2% +2.45% (p=0.000 n=10+10) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-10 05:00:02 +00:00
va.Set(rv)
util/deephash: remove getTypeInfo (#5469) Add a new lookupTypeHasher function that is just a cached front-end around the makeTypeHasher function. We do not need to worry about the recursive type cycle issue that made getTypeInfo more complicated since makeTypeHasher is not directly recursive. All calls to itself happen lazily through a sync.Once upon first use. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-28 00:39:51 +00:00
fn := lookupTypeHasher(va.Type())
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
hb := &hashBuffer{Hash: sha256.New()}
h := new(hasher)
util/sha256x: rename Hash as Block512 (#5351) Rename Hash as Block512 to indicate that this is a general-purpose hash.Hash for any algorithm that operates on 512-bit block sizes. While we rename the package as hashx in this commit, a subsequent commit will move the sha256x package to hashx. This is done separately to avoid confusing git. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-16 16:49:48 +00:00
h.Block512.Hash = hb
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
fn(h, pointerOf(va.Addr()))
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
const ptrSize = 32 << uintptr(^uintptr(0)>>63)
if tt.out32 != "" && ptrSize == 32 {
tt.out = tt.out32
}
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
h.sum()
util/deephash: relax an annoyingly needy test I'd added a test case of deephash against a tailcfg.Node to make sure it worked at all more than anything. We don't care what the exact bytes are in this test, just that it doesn't fail. So adjust for that. Then when we make changes to tailcfg.Node and types under it, we don't need to keep adjusting this test. Updates #cleanup Change-Id: Ibf4fa42820aeab8f5292fe65f9f92ffdb0b4407b Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-08-20 02:47:34 +00:00
if got := string(hb.B); got != tt.out && tt.out != "ANY" {
util/deephash: generate type-specific hasher funcs name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-15 05:49:11 +00:00
t.Fatalf("got %q; want %q", got, tt.out)
}
})
}
}
util/deephash: handle slice edge-cases (#5471) It is unclear whether the lack of checking nil-ness of slices was an oversight or a deliberate feature. Lacking a comment, the assumption is that this was an oversight. Also, expand the logic to perform cycle detection for recursive slices. We do this on a per-element basis since a slice is semantically equivalent to a list of pointers. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-30 07:33:18 +00:00
func TestSliceCycle(t *testing.T) {
type S []S
c := qt.New(t)
all: fix spelling mistakes Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-09-25 18:29:55 +00:00
a := make(S, 1) // cyclic graph of 1 node
util/deephash: handle slice edge-cases (#5471) It is unclear whether the lack of checking nil-ness of slices was an oversight or a deliberate feature. Lacking a comment, the assumption is that this was an oversight. Also, expand the logic to perform cycle detection for recursive slices. We do this on a per-element basis since a slice is semantically equivalent to a list of pointers. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-30 07:33:18 +00:00
a[0] = a
all: fix spelling mistakes Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-09-25 18:29:55 +00:00
b := make(S, 1) // cyclic graph of 1 node
util/deephash: handle slice edge-cases (#5471) It is unclear whether the lack of checking nil-ness of slices was an oversight or a deliberate feature. Lacking a comment, the assumption is that this was an oversight. Also, expand the logic to perform cycle detection for recursive slices. We do this on a per-element basis since a slice is semantically equivalent to a list of pointers. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-30 07:33:18 +00:00
b[0] = b
ha := Hash(&a)
hb := Hash(&b)
c.Assert(ha, qt.Equals, hb)
c1 := make(S, 1) // cyclic graph of 2 nodes
c2 := make(S, 1) // cyclic graph of 2 nodes
c1[0] = c2
c2[0] = c1
hc1 := Hash(&c1)
hc2 := Hash(&c2)
c.Assert(hc1, qt.Equals, hc2)
c.Assert(ha, qt.Not(qt.Equals), hc1)
c.Assert(hb, qt.Not(qt.Equals), hc2)
c3 := make(S, 1) // graph of 1 node pointing to cyclic graph of 2 nodes
c3[0] = c1
hc3 := Hash(&c3)
c.Assert(hc1, qt.Not(qt.Equals), hc3)
c4 := make(S, 2) // cyclic graph of 3 nodes
c5 := make(S, 2) // cyclic graph of 3 nodes
c4[0] = nil
c4[1] = c4
c5[0] = c5
c5[1] = nil
hc4 := Hash(&c4)
hc5 := Hash(&c5)
c.Assert(hc4, qt.Not(qt.Equals), hc5) // cycle occurs through different indexes
}
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
func TestMapCycle(t *testing.T) {
type M map[string]M
c := qt.New(t)
all: fix spelling mistakes Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-09-25 18:29:55 +00:00
a := make(M) // cyclic graph of 1 node
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
a["self"] = a
all: fix spelling mistakes Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-09-25 18:29:55 +00:00
b := make(M) // cyclic graph of 1 node
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
b["self"] = b
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
ha := Hash(&a)
hb := Hash(&b)
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
c.Assert(ha, qt.Equals, hb)
c1 := make(M) // cyclic graph of 2 nodes
c2 := make(M) // cyclic graph of 2 nodes
c1["peer"] = c2
c2["peer"] = c1
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
hc1 := Hash(&c1)
hc2 := Hash(&c2)
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
c.Assert(hc1, qt.Equals, hc2)
c.Assert(ha, qt.Not(qt.Equals), hc1)
c.Assert(hb, qt.Not(qt.Equals), hc2)
c3 := make(M) // graph of 1 node pointing to cyclic graph of 2 nodes
c3["child"] = c1
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
hc3 := Hash(&c3)
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
c.Assert(hc1, qt.Not(qt.Equals), hc3)
util/deephash: handle slice edge-cases (#5471) It is unclear whether the lack of checking nil-ness of slices was an oversight or a deliberate feature. Lacking a comment, the assumption is that this was an oversight. Also, expand the logic to perform cycle detection for recursive slices. We do this on a per-element basis since a slice is semantically equivalent to a list of pointers. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-30 07:33:18 +00:00
c4 := make(M) // cyclic graph of 3 nodes
c5 := make(M) // cyclic graph of 3 nodes
c4["0"] = nil
c4["1"] = c4
c5["0"] = c5
c5["1"] = nil
hc4 := Hash(&c4)
hc5 := Hash(&c5)
c.Assert(hc4, qt.Not(qt.Equals), hc5) // cycle occurs through different keys
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
}
func TestPointerCycle(t *testing.T) {
type P *P
c := qt.New(t)
a := new(P) // cyclic graph of 1 node
*a = a
b := new(P) // cyclic graph of 1 node
*b = b
ha := Hash(&a)
hb := Hash(&b)
c.Assert(ha, qt.Equals, hb)
c1 := new(P) // cyclic graph of 2 nodes
c2 := new(P) // cyclic graph of 2 nodes
*c1 = c2
*c2 = c1
hc1 := Hash(&c1)
hc2 := Hash(&c2)
c.Assert(hc1, qt.Equals, hc2)
c.Assert(ha, qt.Not(qt.Equals), hc1)
c.Assert(hb, qt.Not(qt.Equals), hc2)
c3 := new(P) // graph of 1 node pointing to cyclic graph of 2 nodes
*c3 = c1
hc3 := Hash(&c3)
c.Assert(hc1, qt.Not(qt.Equals), hc3)
}
func TestInterfaceCycle(t *testing.T) {
type I struct{ v any }
c := qt.New(t)
a := new(I) // cyclic graph of 1 node
a.v = a
b := new(I) // cyclic graph of 1 node
b.v = b
ha := Hash(&a)
hb := Hash(&b)
c.Assert(ha, qt.Equals, hb)
c1 := new(I) // cyclic graph of 2 nodes
c2 := new(I) // cyclic graph of 2 nodes
c1.v = c2
c2.v = c1
hc1 := Hash(&c1)
hc2 := Hash(&c2)
c.Assert(hc1, qt.Equals, hc2)
c.Assert(ha, qt.Not(qt.Equals), hc1)
c.Assert(hb, qt.Not(qt.Equals), hc2)
c3 := new(I) // graph of 1 node pointing to cyclic graph of 2 nodes
c3.v = c1
hc3 := Hash(&c3)
c.Assert(hc1, qt.Not(qt.Equals), hc3)
}
util/deephash: always keep values addressable (#5328) The logic of deephash is both simpler and easier to reason about if values are always addressable. In Go, the composite kinds are slices, arrays, maps, structs, interfaces, pointers, channels, and functions, where we define "composite" as a Go value that encapsulates some other Go value (e.g., a map is a collection of key-value entries). In the cases of pointers and slices, the sub-values are always addressable. In the cases of arrays and structs, the sub-values are always addressable if and only if the parent value is addressable. In the case of maps and interfaces, the sub-values are never addressable. To make them addressable, we need to copy them onto the heap. For the purposes of deephash, we do not care about channels and functions. For all non-composite kinds (e.g., strings and ints), they are only addressable if obtained from one of the composite kinds that produce addressable values (i.e., pointers, slices, addressable arrays, and addressable structs). A non-addressible, non-composite kind can be made addressable by allocating it on the heap, obtaining a pointer to it, and dereferencing it. Thus, if we can ensure that values are addressable at the entry points, and shallow copy sub-values whenever we encounter an interface or map, then we can ensure that all values are always addressable and assume such property throughout all the logic. Performance: name old time/op new time/op delta Hash-24 21.5µs ± 1% 19.7µs ± 1% -8.29% (p=0.000 n=9+9) HashPacketFilter-24 2.61µs ± 1% 2.62µs ± 0% +0.29% (p=0.037 n=10+9) HashMapAcyclic-24 30.8µs ± 1% 30.9µs ± 1% ~ (p=0.400 n=9+10) TailcfgNode-24 1.84µs ± 1% 1.84µs ± 2% ~ (p=0.928 n=10+10) HashArray-24 324ns ± 2% 332ns ± 2% +2.45% (p=0.000 n=10+10) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-10 05:00:02 +00:00
var sink Sum
util/deephash: export a Hash func for use by the control plane name old time/op new time/op delta Hash-6 69.4µs ± 6% 68.4µs ± 4% ~ (p=0.286 n=9+9) HashMapAcyclic-6 115µs ± 5% 115µs ± 4% ~ (p=1.000 n=10+10) name old alloc/op new alloc/op delta Hash-6 2.29kB ± 0% 1.88kB ± 0% -18.13% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (all equal) name old allocs/op new allocs/op delta Hash-6 58.0 ± 0% 54.0 ± 0% -6.90% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-05 04:25:15 +00:00
internal/deepprint: add BenchmarkHash deepprint currently accounts for 15% of allocs in tailscaled. This is a useful benchmark to have. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-05-08 01:05:04 +00:00
func BenchmarkHash(b *testing.B) {
b.ReportAllocs()
v := getVal()
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for range b.N {
util/deephash: export a Hash func for use by the control plane name old time/op new time/op delta Hash-6 69.4µs ± 6% 68.4µs ± 4% ~ (p=0.286 n=9+9) HashMapAcyclic-6 115µs ± 5% 115µs ± 4% ~ (p=1.000 n=10+10) name old alloc/op new alloc/op delta Hash-6 2.29kB ± 0% 1.88kB ± 0% -18.13% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (all equal) name old allocs/op new allocs/op delta Hash-6 58.0 ± 0% 54.0 ± 0% -6.90% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-05 04:25:15 +00:00
sink = Hash(v)
internal/deepprint: add BenchmarkHash deepprint currently accounts for 15% of allocs in tailscaled. This is a useful benchmark to have. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-05-08 01:05:04 +00:00
}
}
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
util/deephash: add packet filter benchmark (breaking up parts of another change) This adds a PacketFilter hashing benchmark with an input that both contains every possible field, but also is somewhat representative in the shape of what real packet filters contain. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-25 19:26:54 +00:00
// filterRules is a packet filter that has both everything populated (in its
// first element) and also a few entries that are the typical shape for regular
// packet filters as sent to clients.
var filterRules = []tailcfg.FilterRule{
{
SrcIPs: []string{"*", "10.1.3.4/32", "10.0.0.0/24"},
SrcBits: []int{1, 2, 3},
DstPorts: []tailcfg.NetPortRange{{
IP: "1.2.3.4/32",
types/ptr: move all the ptrTo funcs to one new package's ptr.To Change-Id: Ia0b820ffe7aa72897515f19bd415204b6fe743c7 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-12-01 01:33:29 +00:00
Bits: ptr.To(32),
util/deephash: add packet filter benchmark (breaking up parts of another change) This adds a PacketFilter hashing benchmark with an input that both contains every possible field, but also is somewhat representative in the shape of what real packet filters contain. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-25 19:26:54 +00:00
Ports: tailcfg.PortRange{First: 1, Last: 2},
}},
IPProto: []int{1, 2, 3, 4},
CapGrant: []tailcfg.CapGrant{{
all: convert more code to use net/netip directly perl -i -npe 's,netaddr.IPPrefixFrom,netip.PrefixFrom,' $(git grep -l -F netaddr.) perl -i -npe 's,netaddr.IPPortFrom,netip.AddrPortFrom,' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPrefix,netip.Prefix,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPort,netip.AddrPort,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IP\b,netip.Addr,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPv6Raw\b,netip.AddrFrom16,g' $(git grep -l -F netaddr. ) goimports -w . Then delete some stuff from the net/netaddr shim package which is no longer neeed. Updates #5162 Change-Id: Ia7a86893fe21c7e3ee1ec823e8aba288d4566cd8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-26 04:14:09 +00:00
Dsts: []netip.Prefix{netip.MustParsePrefix("1.2.3.4/32")},
tailcfg,ipn/ipnlocal,wgengine: add values to PeerCapabilities Define PeerCapabilty and PeerCapMap as the new way of sending down inter-peer capability information. Previously, this was unstructured and you could only send down strings which got too limiting for certain usecases. Instead add the ability to send down raw JSON messages that are opaque to Tailscale but provide the applications to define them however they wish. Also update accessors to use the new values. Updates #4217 Signed-off-by: Maisem Ali <maisem@tailscale.com>
2023-07-25 04:07:00 +00:00
Caps: []tailcfg.PeerCapability{"foo"},
util/deephash: add packet filter benchmark (breaking up parts of another change) This adds a PacketFilter hashing benchmark with an input that both contains every possible field, but also is somewhat representative in the shape of what real packet filters contain. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-25 19:26:54 +00:00
}},
},
{
SrcIPs: []string{"foooooooooo"},
DstPorts: []tailcfg.NetPortRange{{
IP: "baaaaaarrrrr",
Ports: tailcfg.PortRange{First: 1, Last: 2},
}},
},
{
SrcIPs: []string{"foooooooooo"},
DstPorts: []tailcfg.NetPortRange{{
IP: "baaaaaarrrrr",
Ports: tailcfg.PortRange{First: 1, Last: 2},
}},
},
{
SrcIPs: []string{"foooooooooo"},
DstPorts: []tailcfg.NetPortRange{{
IP: "baaaaaarrrrr",
Ports: tailcfg.PortRange{First: 1, Last: 2},
}},
},
}
func BenchmarkHashPacketFilter(b *testing.B) {
b.ReportAllocs()
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for range b.N {
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
sink = Hash(&filterRules)
util/deephash: add packet filter benchmark (breaking up parts of another change) This adds a PacketFilter hashing benchmark with an input that both contains every possible field, but also is somewhat representative in the shape of what real packet filters contain. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-06-25 19:26:54 +00:00
}
}
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
func TestHashMapAcyclic(t *testing.T) {
m := map[int]string{}
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for i := range 100 {
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
m[i] = fmt.Sprint(i)
}
got := map[string]bool{}
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
hb := &hashBuffer{Hash: sha256.New()}
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
all: use reflect.TypeFor now available in Go 1.22 (#11078) Updates #cleanup Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2024-02-09 01:34:22 +00:00
hash := lookupTypeHasher(reflect.TypeFor[map[int]string]())
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for range 20 {
util/deephash: move map logic to separate function (#5464) This helps pprof better identify which Go kinds take the most time since the kind is always in the function name. There is a minor adjustment where we hash the length of the map to be more on the cautious side. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 22:49:26 +00:00
va := reflect.ValueOf(&m).Elem()
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
hb.Reset()
h := new(hasher)
util/sha256x: rename Hash as Block512 (#5351) Rename Hash as Block512 to indicate that this is a general-purpose hash.Hash for any algorithm that operates on 512-bit block sizes. While we rename the package as hashx in this commit, a subsequent commit will move the sha256x package to hashx. This is done separately to avoid confusing git. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-16 16:49:48 +00:00
h.Block512.Hash = hb
util/deephash: move map logic to separate function (#5464) This helps pprof better identify which Go kinds take the most time since the kind is always in the function name. There is a minor adjustment where we hash the length of the map to be more on the cautious side. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 22:49:26 +00:00
hash(h, pointerOf(va.Addr()))
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
h.sum()
if got[string(hb.B)] {
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
continue
}
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
got[string(hb.B)] = true
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
}
if len(got) != 1 {
t.Errorf("got %d results; want 1", len(got))
}
}
util/deephash: optimize hashing of byte arrays, reduce allocs in Hash name old time/op new time/op delta Hash-6 173µs ± 4% 101µs ± 3% -41.69% (p=0.000 n=10+9) HashMapAcyclic-6 101µs ± 5% 105µs ± 3% +3.52% (p=0.001 n=9+10) TailcfgNode-6 29.4µs ± 2% 16.4µs ± 3% -44.25% (p=0.000 n=8+10) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 1.13kB ± 0% -68.70% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.137 n=10+8) TailcfgNode-6 528B ± 0% 0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 40.0 ± 0% -52.38% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) Updates tailscale/corp#2130 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 05:13:33 +00:00
func TestPrintArray(t *testing.T) {
type T struct {
X [32]byte
}
util/deephash: improve cycle detection (#2470) The previous algorithm used a map of all visited pointers. The strength of this approach is that it quickly prunes any nodes that we have ever visited before. The detriment of the approach is that pruning is heavily dependent on the order that pointers were visited. This is especially relevant for hashing a map where map entries are visited in a non-deterministic manner, which would cause the map hash to be non-deterministic (which defeats the point of a hash). This new algorithm uses a stack of all visited pointers, similar to how github.com/google/go-cmp performs cycle detection. When we visit a pointer, we push it onto the stack, and when we leave a pointer, we pop it from the stack. Before visiting a pointer, we first check whether the pointer exists anywhere in the stack. If yes, then we prune the node. The detriment of this approach is that we may hash a node more often than before since we do not prune as aggressively. The set of visited pointers up until any node is only the path of nodes up to that node and not any other pointers that may have been visited elsewhere. This provides us deterministic hashing regardless of visit order. We can now delete hashMapFallback and associated complexity, which only exists because the previous approach was non-deterministic in the presence of cycles. This fixes a failure of the old algorithm where obviously different values are treated as equal because the pruning was too aggresive. See https://github.com/tailscale/tailscale/issues/2443#issuecomment-883653534 The new algorithm is slightly slower since it prunes less aggresively: name old time/op new time/op delta Hash-8 66.1µs ± 1% 68.8µs ± 1% +4.09% (p=0.000 n=19+19) HashMapAcyclic-8 63.0µs ± 1% 62.5µs ± 1% -0.76% (p=0.000 n=18+19) TailcfgNode-8 9.79µs ± 2% 9.88µs ± 1% +0.95% (p=0.000 n=19+17) HashArray-8 643ns ± 1% 653ns ± 1% +1.64% (p=0.000 n=19+19) However, a slower but more correct algorithm seems more favorable than a faster but incorrect algorithm. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-07-22 22:22:48 +00:00
x := T{X: [32]byte{1: 1, 31: 31}}
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
hb := &hashBuffer{Hash: sha256.New()}
h := new(hasher)
util/sha256x: rename Hash as Block512 (#5351) Rename Hash as Block512 to indicate that this is a general-purpose hash.Hash for any algorithm that operates on 512-bit block sizes. While we rename the package as hashx in this commit, a subsequent commit will move the sha256x package to hashx. This is done separately to avoid confusing git. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-16 16:49:48 +00:00
h.Block512.Hash = hb
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
va := reflect.ValueOf(&x).Elem()
util/deephash: remove getTypeInfo (#5469) Add a new lookupTypeHasher function that is just a cached front-end around the makeTypeHasher function. We do not need to worry about the recursive type cycle issue that made getTypeInfo more complicated since makeTypeHasher is not directly recursive. All calls to itself happen lazily through a sync.Once upon first use. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-28 00:39:51 +00:00
hash := lookupTypeHasher(va.Type())
hash(h, pointerOf(va.Addr()))
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
h.sum()
util/deephash: remove unnecessary formatting for structs and slices (#2571) The index for every struct field or slice element and the number of fields for the struct is unncessary. The hashing of Go values is unambiguous because every type (except maps) encodes in a parsable manner. So long as we know the type information, we could theoretically decode every value (except for maps). At a high level: * numbers are encoded as fixed-width records according to precision. * strings (and AppendTo output) are encoded with a fixed-width length, followed by the contents of the buffer. * slices are prefixed by a fixed-width length, followed by the encoding of each value. So long as we know the type of each element, we could theoretically decode each element. * arrays are encoded just like slices, but elide the length since it is determined from the Go type. * maps are encoded first with a byte indicating whether it is a cycle. If a cycle, it is followed by a fixed-width index for the pointer, otherwise followed by the SHA-256 hash of its contents. The encoding of maps is not decodeable, but a SHA-256 hash is sufficient to avoid ambiguities. * interfaces are encoded first with a byte indicating whether it is nil. If not nil, it is followed by a fixed-width index for the type, and then the encoding for the underlying value. Having the type be encoded first ensures that the value could theoretically be decoded next. * pointers are encoded first with a byte indicating whether it is 1) nil, 2) a cycle, or 3) newly seen. If a cycle, it is followed by a fixed-width index for the pointer. If newly seen, it is followed by the encoding for the pointed-at value. Removing unnecessary details speeds up hashing: name old time/op new time/op delta Hash-8 76.0µs ± 1% 55.8µs ± 2% -26.62% (p=0.000 n=10+10) HashMapAcyclic-8 61.9µs ± 0% 62.0µs ± 0% ~ (p=0.666 n=9+9) TailcfgNode-8 10.2µs ± 1% 7.5µs ± 1% -26.90% (p=0.000 n=10+9) HashArray-8 1.07µs ± 1% 0.70µs ± 1% -34.67% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-04 03:35:57 +00:00
const want = "\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x1f"
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
if got := hb.B; string(got) != want {
util/deephash: optimize hashing of byte arrays, reduce allocs in Hash name old time/op new time/op delta Hash-6 173µs ± 4% 101µs ± 3% -41.69% (p=0.000 n=10+9) HashMapAcyclic-6 101µs ± 5% 105µs ± 3% +3.52% (p=0.001 n=9+10) TailcfgNode-6 29.4µs ± 2% 16.4µs ± 3% -44.25% (p=0.000 n=8+10) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 1.13kB ± 0% -68.70% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.137 n=10+8) TailcfgNode-6 528B ± 0% 0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 40.0 ± 0% -52.38% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) Updates tailscale/corp#2130 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 05:13:33 +00:00
t.Errorf("wrong:\n got: %q\nwant: %q\n", got, want)
}
}
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
func BenchmarkHashMapAcyclic(b *testing.B) {
b.ReportAllocs()
m := map[int]string{}
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for i := range 100 {
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
m[i] = fmt.Sprint(i)
}
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
hb := &hashBuffer{Hash: sha256.New()}
util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 19:30:35 +00:00
va := reflect.ValueOf(&m).Elem()
util/deephash: remove getTypeInfo (#5469) Add a new lookupTypeHasher function that is just a cached front-end around the makeTypeHasher function. We do not need to worry about the recursive type cycle issue that made getTypeInfo more complicated since makeTypeHasher is not directly recursive. All calls to itself happen lazily through a sync.Once upon first use. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-28 00:39:51 +00:00
hash := lookupTypeHasher(va.Type())
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
h := new(hasher)
util/sha256x: rename Hash as Block512 (#5351) Rename Hash as Block512 to indicate that this is a general-purpose hash.Hash for any algorithm that operates on 512-bit block sizes. While we rename the package as hashx in this commit, a subsequent commit will move the sha256x package to hashx. This is done separately to avoid confusing git. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-16 16:49:48 +00:00
h.Block512.Hash = hb
util/deephash: move funcs to methods Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-07 05:37:32 +00:00
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for range b.N {
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
h.Reset()
util/deephash: move map logic to separate function (#5464) This helps pprof better identify which Go kinds take the most time since the kind is always in the function name. There is a minor adjustment where we hash the length of the map to be more on the cautious side. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 22:49:26 +00:00
hash(h, pointerOf(va.Addr()))
internal/deephash: hash maps without sorting in the acyclic common case Hash and xor each entry instead, then write final xor'ed result. name old time/op new time/op delta Hash-4 33.6µs ± 4% 34.6µs ± 3% +3.03% (p=0.013 n=10+9) name old alloc/op new alloc/op delta Hash-4 1.86kB ± 0% 1.77kB ± 0% -5.10% (p=0.000 n=10+9) name old allocs/op new allocs/op delta Hash-4 51.0 ± 0% 49.0 ± 0% -3.92% (p=0.000 n=10+10) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-05-11 20:17:12 +00:00
}
}
internal/deephash: 8 bits of output is not enough Running hex.Encode(b, b) is a bad idea. The first byte of input will overwrite the first two bytes of output. Subsequent bytes have no impact on the output. Not related to today's IPv6 bug, but...wh::ps. This caused us to spuriously ignore some wireguard config updates. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-07-02 20:45:23 +00:00
util/deephash: add a benchmark and some benchmark data No code changes. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 04:21:52 +00:00
func BenchmarkTailcfgNode(b *testing.B) {
b.ReportAllocs()
node := new(tailcfg.Node)
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for range b.N {
util/deephash: add a benchmark and some benchmark data No code changes. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 04:21:52 +00:00
sink = Hash(node)
}
}
internal/deephash: 8 bits of output is not enough Running hex.Encode(b, b) is a bad idea. The first byte of input will overwrite the first two bytes of output. Subsequent bytes have no impact on the output. Not related to today's IPv6 bug, but...wh::ps. This caused us to spuriously ignore some wireguard config updates. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-07-02 20:45:23 +00:00
func TestExhaustive(t *testing.T) {
util/deephash: make hash type opaque The fact that Hash returns a [sha256.Size]byte leaks details about the underlying hash implementation. This could very well be any other hashing algorithm with a possible different block size. Abstract this implementation detail away by declaring an opaque type that is comparable. While we are changing the signature of UpdateHash, rename it to just Update to reduce stutter (e.g., deephash.Update). Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-07-20 05:49:51 +00:00
seen := make(map[Sum]bool)
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for i := range 100000 {
util/deephash: require pointer in API (#5467) The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-27 23:08:31 +00:00
s := Hash(&i)
internal/deephash: 8 bits of output is not enough Running hex.Encode(b, b) is a bad idea. The first byte of input will overwrite the first two bytes of output. Subsequent bytes have no impact on the output. Not related to today's IPv6 bug, but...wh::ps. This caused us to spuriously ignore some wireguard config updates. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2021-07-02 20:45:23 +00:00
if seen[s] {
t.Fatalf("hash collision %v", i)
}
seen[s] = true
}
}
util/deephash: export a Hash func for use by the control plane name old time/op new time/op delta Hash-6 69.4µs ± 6% 68.4µs ± 4% ~ (p=0.286 n=9+9) HashMapAcyclic-6 115µs ± 5% 115µs ± 4% ~ (p=1.000 n=10+10) name old alloc/op new alloc/op delta Hash-6 2.29kB ± 0% 1.88kB ± 0% -18.13% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (all equal) name old allocs/op new allocs/op delta Hash-6 58.0 ± 0% 54.0 ± 0% -6.90% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-05 04:25:15 +00:00
util/deephash: prevent infinite loop on map cycle Fixes #2340 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-07 04:41:18 +00:00
// verify this doesn't loop forever, as it used to (Issue 2340)
func TestMapCyclicFallback(t *testing.T) {
type T struct {
all: use any instead of interface{} My favorite part of generics. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2022-03-16 23:27:57 +00:00
M map[string]any
util/deephash: prevent infinite loop on map cycle Fixes #2340 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-07 04:41:18 +00:00
}
v := &T{
all: use any instead of interface{} My favorite part of generics. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2022-03-16 23:27:57 +00:00
M: map[string]any{},
util/deephash: prevent infinite loop on map cycle Fixes #2340 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-07 04:41:18 +00:00
}
v.M["m"] = v.M
Hash(v)
}
util/deephash: optimize hashing of byte arrays, reduce allocs in Hash name old time/op new time/op delta Hash-6 173µs ± 4% 101µs ± 3% -41.69% (p=0.000 n=10+9) HashMapAcyclic-6 101µs ± 5% 105µs ± 3% +3.52% (p=0.001 n=9+10) TailcfgNode-6 29.4µs ± 2% 16.4µs ± 3% -44.25% (p=0.000 n=8+10) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 1.13kB ± 0% -68.70% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.137 n=10+8) TailcfgNode-6 528B ± 0% 0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 40.0 ± 0% -52.38% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) Updates tailscale/corp#2130 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 05:13:33 +00:00
func TestArrayAllocs(t *testing.T) {
util/deephash: skip alloc test under race detector Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-07 18:40:28 +00:00
if version.IsRace() {
t.Skip("skipping test under race detector")
}
util/deephash: fix TestArrayAllocs Unfortunately this test fails on certain architectures. The problem comes down to inconsistencies in the Go escape analysis where specific variables are marked as escaping on certain architectures. The variables escaping to the heap are unfortunately in crypto/sha256, which makes it impossible to fixthis locally in deephash. For now, fix the test by compensating for the allocations that occur from calling sha256.digest.Sum. See golang/go#48055 Fixes #2727 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-30 17:47:21 +00:00
// In theory, there should be no allocations. However, escape analysis on
// certain architectures fails to detect that certain cases do not escape.
all: fix spelling mistakes Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-09-25 18:29:55 +00:00
// This discrepancy currently affects sha256.digest.Sum.
util/deephash: fix TestArrayAllocs Unfortunately this test fails on certain architectures. The problem comes down to inconsistencies in the Go escape analysis where specific variables are marked as escaping on certain architectures. The variables escaping to the heap are unfortunately in crypto/sha256, which makes it impossible to fixthis locally in deephash. For now, fix the test by compensating for the allocations that occur from calling sha256.digest.Sum. See golang/go#48055 Fixes #2727 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-30 17:47:21 +00:00
// Measure the number of allocations in sha256 to ensure that Hash does
// not allocate on top of its usage of sha256.
// See https://golang.org/issue/48055.
var b []byte
h := sha256.New()
want := int(testing.AllocsPerRun(1000, func() {
b = h.Sum(b[:0])
}))
switch runtime.GOARCH {
case "amd64", "arm64":
want = 0 // ensure no allocations on popular architectures
}
util/deephash: optimize hashing of byte arrays, reduce allocs in Hash name old time/op new time/op delta Hash-6 173µs ± 4% 101µs ± 3% -41.69% (p=0.000 n=10+9) HashMapAcyclic-6 101µs ± 5% 105µs ± 3% +3.52% (p=0.001 n=9+10) TailcfgNode-6 29.4µs ± 2% 16.4µs ± 3% -44.25% (p=0.000 n=8+10) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 1.13kB ± 0% -68.70% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.137 n=10+8) TailcfgNode-6 528B ± 0% 0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 40.0 ± 0% -52.38% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) Updates tailscale/corp#2130 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 05:13:33 +00:00
type T struct {
X [32]byte
}
x := &T{X: [32]byte{1: 1, 2: 2, 3: 3, 4: 4}}
util/deephash: fix TestArrayAllocs Unfortunately this test fails on certain architectures. The problem comes down to inconsistencies in the Go escape analysis where specific variables are marked as escaping on certain architectures. The variables escaping to the heap are unfortunately in crypto/sha256, which makes it impossible to fixthis locally in deephash. For now, fix the test by compensating for the allocations that occur from calling sha256.digest.Sum. See golang/go#48055 Fixes #2727 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-30 17:47:21 +00:00
got := int(testing.AllocsPerRun(1000, func() {
util/deephash: optimize hashing of byte arrays, reduce allocs in Hash name old time/op new time/op delta Hash-6 173µs ± 4% 101µs ± 3% -41.69% (p=0.000 n=10+9) HashMapAcyclic-6 101µs ± 5% 105µs ± 3% +3.52% (p=0.001 n=9+10) TailcfgNode-6 29.4µs ± 2% 16.4µs ± 3% -44.25% (p=0.000 n=8+10) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 1.13kB ± 0% -68.70% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.137 n=10+8) TailcfgNode-6 528B ± 0% 0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 40.0 ± 0% -52.38% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) Updates tailscale/corp#2130 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 05:13:33 +00:00
sink = Hash(x)
}))
util/deephash: fix TestArrayAllocs Unfortunately this test fails on certain architectures. The problem comes down to inconsistencies in the Go escape analysis where specific variables are marked as escaping on certain architectures. The variables escaping to the heap are unfortunately in crypto/sha256, which makes it impossible to fixthis locally in deephash. For now, fix the test by compensating for the allocations that occur from calling sha256.digest.Sum. See golang/go#48055 Fixes #2727 Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2021-08-30 17:47:21 +00:00
if got > want {
t.Errorf("allocs = %v; want %v", got, want)
util/deephash: optimize hashing of byte arrays, reduce allocs in Hash name old time/op new time/op delta Hash-6 173µs ± 4% 101µs ± 3% -41.69% (p=0.000 n=10+9) HashMapAcyclic-6 101µs ± 5% 105µs ± 3% +3.52% (p=0.001 n=9+10) TailcfgNode-6 29.4µs ± 2% 16.4µs ± 3% -44.25% (p=0.000 n=8+10) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 1.13kB ± 0% -68.70% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.137 n=10+8) TailcfgNode-6 528B ± 0% 0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 40.0 ± 0% -52.38% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) Updates tailscale/corp#2130 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 05:13:33 +00:00
}
}
util/deephash: fix unexported time.Time hashing Updates tailscale/corp#6311 Change-Id: I33cd7e4040966261c2f2eb3d32f29936aeb7f632 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-27 16:10:22 +00:00
// Test for http://go/corp/6311 issue.
func TestHashThroughView(t *testing.T) {
type sshPolicyOut struct {
Rules []tailcfg.SSHRuleView
}
type mapResponseOut struct {
SSHPolicy *sshPolicyOut
}
// Just test we don't panic:
_ = Hash(&mapResponseOut{
SSHPolicy: &sshPolicyOut{
Rules: []tailcfg.SSHRuleView{
(&tailcfg.SSHRule{
types/ptr: move all the ptrTo funcs to one new package's ptr.To Change-Id: Ia0b820ffe7aa72897515f19bd415204b6fe743c7 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-12-01 01:33:29 +00:00
RuleExpires: ptr.To(time.Unix(123, 0)),
util/deephash: fix unexported time.Time hashing Updates tailscale/corp#6311 Change-Id: I33cd7e4040966261c2f2eb3d32f29936aeb7f632 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2022-07-27 16:10:22 +00:00
}).View(),
},
},
})
}
util/deephash: optimize hashing of byte arrays, reduce allocs in Hash name old time/op new time/op delta Hash-6 173µs ± 4% 101µs ± 3% -41.69% (p=0.000 n=10+9) HashMapAcyclic-6 101µs ± 5% 105µs ± 3% +3.52% (p=0.001 n=9+10) TailcfgNode-6 29.4µs ± 2% 16.4µs ± 3% -44.25% (p=0.000 n=8+10) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 1.13kB ± 0% -68.70% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.137 n=10+8) TailcfgNode-6 528B ± 0% 0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 40.0 ± 0% -52.38% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) Updates tailscale/corp#2130 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 05:13:33 +00:00
func BenchmarkHashArray(b *testing.B) {
b.ReportAllocs()
type T struct {
X [32]byte
}
x := &T{X: [32]byte{1: 1, 2: 2, 3: 3, 4: 4}}
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for range b.N {
util/deephash: optimize hashing of byte arrays, reduce allocs in Hash name old time/op new time/op delta Hash-6 173µs ± 4% 101µs ± 3% -41.69% (p=0.000 n=10+9) HashMapAcyclic-6 101µs ± 5% 105µs ± 3% +3.52% (p=0.001 n=9+10) TailcfgNode-6 29.4µs ± 2% 16.4µs ± 3% -44.25% (p=0.000 n=8+10) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 1.13kB ± 0% -68.70% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.137 n=10+8) TailcfgNode-6 528B ± 0% 0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 40.0 ± 0% -52.38% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) Updates tailscale/corp#2130 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2021-07-06 05:13:33 +00:00
sink = Hash(x)
}
}
util/deephash: use sha256x (#5339) Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 00:44:09 +00:00
// hashBuffer is a hash.Hash that buffers all written data.
type hashBuffer struct {
hash.Hash
B []byte
}
func (h *hashBuffer) Write(b []byte) (int, error) {
n, err := h.Hash.Write(b)
h.B = append(h.B, b[:n]...)
return n, err
}
func (h *hashBuffer) Reset() {
h.Hash.Reset()
h.B = h.B[:0]
}
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
func FuzzTime(f *testing.F) {
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(0), false, "", 0)
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(0), true, "", 0)
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(0), true, "hello", 0)
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(0), true, "", 1234)
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(0), true, "hello", 1234)
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(1), false, "", 0)
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(1), true, "", 0)
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(1), true, "hello", 0)
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(1), true, "", 1234)
f.Add(int64(0), int64(0), false, "", 0, int64(0), int64(1), true, "hello", 1234)
f.Add(int64(math.MaxInt64), int64(math.MaxInt64), false, "", 0, int64(math.MaxInt64), int64(math.MaxInt64), false, "", 0)
f.Add(int64(math.MaxInt64), int64(math.MaxInt64), false, "", 0, int64(math.MaxInt64), int64(math.MaxInt64), true, "", 0)
f.Add(int64(math.MaxInt64), int64(math.MaxInt64), false, "", 0, int64(math.MaxInt64), int64(math.MaxInt64), true, "hello", 0)
f.Add(int64(math.MaxInt64), int64(math.MaxInt64), false, "", 0, int64(math.MaxInt64), int64(math.MaxInt64), true, "", 1234)
f.Add(int64(math.MaxInt64), int64(math.MaxInt64), false, "", 0, int64(math.MaxInt64), int64(math.MaxInt64), true, "hello", 1234)
f.Add(int64(math.MinInt64), int64(math.MinInt64), false, "", 0, int64(math.MinInt64), int64(math.MinInt64), false, "", 0)
f.Add(int64(math.MinInt64), int64(math.MinInt64), false, "", 0, int64(math.MinInt64), int64(math.MinInt64), true, "", 0)
f.Add(int64(math.MinInt64), int64(math.MinInt64), false, "", 0, int64(math.MinInt64), int64(math.MinInt64), true, "hello", 0)
f.Add(int64(math.MinInt64), int64(math.MinInt64), false, "", 0, int64(math.MinInt64), int64(math.MinInt64), true, "", 1234)
f.Add(int64(math.MinInt64), int64(math.MinInt64), false, "", 0, int64(math.MinInt64), int64(math.MinInt64), true, "hello", 1234)
f.Fuzz(func(t *testing.T,
s1, ns1 int64, loc1 bool, name1 string, off1 int,
s2, ns2 int64, loc2 bool, name2 string, off2 int,
) {
t1 := time.Unix(s1, ns1)
if loc1 {
all: update to Go 1.20, use strings.CutPrefix/Suffix instead of our fork Updates #7123 Updates #5309 Change-Id: I90bcd87a2fb85a91834a0dd4be6e03db08438672 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-02-01 21:43:06 +00:00
_ = t1.In(time.FixedZone(name1, off1))
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
}
t2 := time.Unix(s2, ns2)
if loc2 {
all: update to Go 1.20, use strings.CutPrefix/Suffix instead of our fork Updates #7123 Updates #5309 Change-Id: I90bcd87a2fb85a91834a0dd4be6e03db08438672 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-02-01 21:43:06 +00:00
_ = t2.In(time.FixedZone(name2, off2))
util/deephash: use binary encoding of time.Time (#5352) Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2022-08-12 21:42:51 +00:00
}
got := Hash(&t1) == Hash(&t2)
want := t1.Format(time.RFC3339Nano) == t2.Format(time.RFC3339Nano)
if got != want {
t.Errorf("time.Time(%s) == time.Time(%s) mismatches hash equivalent", t1.Format(time.RFC3339Nano), t2.Format(time.RFC3339Nano))
}
})
}
util/deephash: specialize for netip.Addr and drop AppendTo support (#5402) There are 5 types that we care about that implement AppendTo: key.DiscoPublic key.NodePublic netip.Prefix netipx.IPRange netip.Addr The key types are thin wrappers around [32]byte and are memory hashable. The netip.Prefix and netipx.IPRange types are thin wrappers over netip.Addr and are hashable by default if netip.Addr is hashable. The netip.Addr type is the only one with a complex structure where the default behavior of deephash does not hash it correctly due to the presence of the intern.Value type. Drop support for AppendTo and instead add specialized hashing for netip.Addr that would be semantically equivalent to == on the netip.Addr values. The AppendTo support was already broken prior to this change. It was fully removed (intentionally or not) in #4870. It was partially restored in #4858 for the fast path, but still broken in the slow path. Just drop support for it altogether. This does mean we lack any ability for types to self-hash themselves. In the future we can add support for types that implement: interface { DeepHash() Sum } Test and fuzz cases were added for the relevant types that used to rely on the AppendTo method. FuzzAddr has been executed on 1 billion samples without issues. Signed-off-by: Joe Tsai joetsai@digital-static.net
2022-08-19 05:54:56 +00:00
func FuzzAddr(f *testing.F) {
f.Fuzz(func(t *testing.T,
u1a, u1b uint64, zone1 string,
u2a, u2b uint64, zone2 string,
) {
var b1, b2 [16]byte
binary.LittleEndian.PutUint64(b1[:8], u1a)
binary.LittleEndian.PutUint64(b1[8:], u1b)
binary.LittleEndian.PutUint64(b2[:8], u2a)
binary.LittleEndian.PutUint64(b2[8:], u2b)
var ips [4]netip.Addr
ips[0] = netip.AddrFrom4(*(*[4]byte)(b1[:]))
ips[1] = netip.AddrFrom4(*(*[4]byte)(b2[:]))
ips[2] = netip.AddrFrom16(b1)
if zone1 != "" {
ips[2] = ips[2].WithZone(zone1)
}
ips[3] = netip.AddrFrom16(b2)
if zone2 != "" {
ips[3] = ips[2].WithZone(zone2)
}
for _, ip1 := range ips[:] {
for _, ip2 := range ips[:] {
got := Hash(&ip1) == Hash(&ip2)
want := ip1 == ip2
if got != want {
t.Errorf("netip.Addr(%s) == netip.Addr(%s) mismatches hash equivalent", ip1.String(), ip2.String())
}
}
}
})
}
util/deephash: add AppendSum method (#5768) This method can be used to obtain the hex-formatted deephash.Sum instance without allocations. Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
2022-09-27 22:22:31 +00:00
func TestAppendTo(t *testing.T) {
v := getVal()
h := Hash(v)
sum := h.AppendTo(nil)
if s := h.String(); s != string(sum) {
t.Errorf("hash sum mismatch; h.String()=%q h.AppendTo()=%q", s, string(sum))
}
}
util/deephash: add IncludeFields, ExcludeFields HasherForType Options Updates tailscale/corp#6198 Change-Id: Iafc18c5b947522cf07a42a56f35c0319cc7b1c94 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2023-08-27 17:09:06 +00:00
func TestFilterFields(t *testing.T) {
type T struct {
A int
B int
C int
}
hashers := map[string]func(*T) Sum{
"all": HasherForType[T](),
"ac": HasherForType[T](IncludeFields[T]("A", "C")),
"b": HasherForType[T](ExcludeFields[T]("A", "C")),
}
tests := []struct {
hasher string
a, b T
wantEq bool
}{
{"all", T{1, 2, 3}, T{1, 2, 3}, true},
{"all", T{1, 2, 3}, T{0, 2, 3}, false},
{"all", T{1, 2, 3}, T{1, 0, 3}, false},
{"all", T{1, 2, 3}, T{1, 2, 0}, false},
{"ac", T{0, 0, 0}, T{0, 0, 0}, true},
{"ac", T{1, 0, 1}, T{1, 1, 1}, true},
{"ac", T{1, 1, 1}, T{1, 1, 0}, false},
{"b", T{0, 0, 0}, T{0, 0, 0}, true},
{"b", T{1, 0, 1}, T{1, 1, 1}, false},
{"b", T{1, 1, 1}, T{0, 1, 0}, true},
}
for _, tt := range tests {
f, ok := hashers[tt.hasher]
if !ok {
t.Fatalf("bad test: unknown hasher %q", tt.hasher)
}
sum1 := f(&tt.a)
sum2 := f(&tt.b)
got := sum1 == sum2
if got != tt.wantEq {
t.Errorf("hasher %q, for %+v and %v, got equal = %v; want %v", tt.hasher, tt.a, tt.b, got, tt.wantEq)
}
}
}
util/deephash: add AppendSum method (#5768) This method can be used to obtain the hex-formatted deephash.Sum instance without allocations. Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
2022-09-27 22:22:31 +00:00
func BenchmarkAppendTo(b *testing.B) {
b.ReportAllocs()
v := getVal()
h := Hash(v)
hashBuf := make([]byte, 0, 100)
b.ResetTimer()
all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2024-04-16 20:15:13 +00:00
for range b.N {
util/deephash: add AppendSum method (#5768) This method can be used to obtain the hex-formatted deephash.Sum instance without allocations. Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
2022-09-27 22:22:31 +00:00
hashBuf = h.AppendTo(hashBuf[:0])
}
}
Reference in New Issue Copy Permalink