72 Commits

Author SHA1 Message Date
Kristoffer Dalby
9515040161
make reauth test compat with tailscale head (#2167)
* make reauth test compat with tailscale head

tailscale/tailscale@1eaad7d broke our reauth test as it makes the client
retry with https/443 if it reconnects within 2 minutes.

This commit fixes this by running the test as a two part,
- with https, to confirm instant reconnect works
- with http, and a 3 min wait, to check that it work without.

The change is not a general consern as headscale in prod is ran
with https.

Updates #2164

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* sort test for stable order

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-10-03 12:01:48 +02:00
Kristoffer Dalby
218138afee
Redo OIDC configuration (#2020)
expand user, add claims to user

This commit expands the user table with additional fields that
can be retrieved from OIDC providers (and other places) and
uses this data in various tailscale response objects if it is
available.

This is the beginning of implementing
https://docs.google.com/document/d/1X85PMxIaVWDF6T_UPji3OeeUqVBcGj_uHRM5CI-AwlY/edit
trying to make OIDC more coherant and maintainable in addition
to giving the user a better experience and integration with a
provider.

remove usernames in magic dns, normalisation of emails

this commit removes the option to have usernames as part of MagicDNS
domains and headscale will now align with Tailscale, where there is a
root domain, and the machine name.

In addition, the various normalisation functions for dns names has been
made lighter not caring about username and special character that wont
occur.

Email are no longer normalised as part of the policy processing.

untagle oidc and regcache, use typed cache

This commits stops reusing the registration cache for oidc
purposes and switches the cache to be types and not use any
allowing the removal of a bunch of casting.

try to make reauth/register branches clearer in oidc

Currently there was a function that did a bunch of stuff,
finding the machine key, trying to find the node, reauthing
the node, returning some status, and it was called validate
which was very confusing.

This commit tries to split this into what to do if the node
exists, if it needs to register etc.

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-10-02 14:50:17 +02:00
Kristoffer Dalby
bc9e83b52e
use gorm serialiser instead of custom hooks (#2156)
* add sqlite to debug/test image

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* test using gorm serialiser instead of custom hooks

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-10-02 11:41:58 +02:00
Kristoffer Dalby
3964dec1c6
use tsaddr library and cleanups (#2150)
* resuse tsaddr code instead of handrolled

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* ensure we dont give out internal tailscale IPs

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* use prefix instead of string for routes

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* remove old custom compare func

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* trim unused util code

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-10-02 09:06:09 +02:00
Kristoffer Dalby
5eda9c8d2d
denormalise PreAuthKey tags (#2155)
this commit denormalises the Tags related to a Pre auth key
back onto the preauthkey table and struct as a string list.

There was not really any real normalisation here as we just added
a bunch of duplicate tags with new IDs and preauthkeyIDs, lots of
GORM cermony but no actual advantage.

This work is the start to fixup tags which currently are not working
as they should.

Updates #1369

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-09-29 13:00:27 +02:00
Kristoffer Dalby
064c46f2a5
move logic for validating node names (#2127)
* move logic for validating node names

this commits moves the generation of "given names" of nodes
into the registration function, and adds validation of renames
to RenameNode using the same logic.

Fixes #2121

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* fix double arg

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-09-11 18:27:49 +02:00
Kristoffer Dalby
f368ed01ed
2068 AutoApprovers tests (#2105)
* replace old suite approved routes test with table driven

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* add test to reproduce issue

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* add integration test for 2068

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-09-05 16:46:20 +02:00
Mike Poindexter
34361c6f82
Fix FKs on sqlite migrations (#2083) 2024-08-30 08:08:54 +02:00
Thomas Gautvedt
f4427dd29e
fix: correct a small spelling mistake (#2081) 2024-08-28 07:50:09 +00:00
Kristoffer Dalby
cf6a606d74
fix route table migration wiping routes 0.22 -> 0.23 (#2076) 2024-08-27 18:54:28 +02:00
nadongjun
fdc034e8ae
Integrate GORM Logger with Zerolog and Add Configuration Options for Logging and Performance (#2040)
* Integrate GORM logger with zerolog and add custom GORM configuration options

* Add GormConfig struct to group GORM-related settings

* Update debug mode instruction in config-example.yaml

Co-authored-by: Kristoffer Dalby <kristoffer@dalby.cc>

---------

Co-authored-by: Kristoffer Dalby <kristoffer@dalby.cc>
2024-08-19 11:47:52 +02:00
nadongjun
4ad3f3c484
Fix data race issues in EphemeralGarbageCollector tests (#2023)
* Fix data race issues in EphemeralGarbageCollector tests

* Add defer for mutex unlock in TestEphemeralGarbageCollectorOrder

* Fix mutex unlock order in closure by updating defer placement
2024-07-22 15:11:57 +00:00
greizgh
8571513e3c
reformat code (#2019)
* reformat code

This is mostly an automated change with `make lint`.
I had to manually please golangci-lint in routes_test because of a short
variable name.

* fix start -> strategy which was wrongly corrected by linter
2024-07-22 08:56:00 +02:00
Kristoffer Dalby
7e62031444
replace ephemeral deletion logic (#2008)
* replace ephemeral deletion logic

this commit replaces the way we remove ephemeral nodes,
currently they are deleted in a loop and we look at last seen
time. This time is now only set when a node disconnects and
there was a bug (#2006) where nodes that had never disconnected
was deleted since they did not have a last seen.

The new logic will start an expiry timer when the node disconnects
and delete the node from the database when the timer is up.

If the node reconnects within the expiry, the timer is cancelled.

Fixes #2006

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* use uint64 as authekyid and ptr helper in tests

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* add test db helper

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* add list ephemeral node func

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* schedule ephemeral nodes for removal on startup

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* fix gorm query for postgres

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* add godoc

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-07-18 10:01:59 +02:00
Pallab Pain
58bd38a609
feat: implements apis for managing headscale policy (#1792) 2024-07-18 07:38:25 +02:00
Kristoffer Dalby
4a34cfc4a6
Make write-ahead-log default and configurable for sqlite (#1985)
* correctly enable WAL log for sqlite

this commit makes headscale correctly enable write-ahead-log for
sqlite and adds an option to turn it on and off.

WAL is enabled by default and should make sqlite perform a lot better,
even further eliminating the need to use postgres.

It also adds a couple of other useful defaults.

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* update changelog

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-06-23 22:06:59 +02:00
Kristoffer Dalby
8f8f469c0a
Remove allocations of lists before use (#1989)
* policy: remove allocs before appends in acls

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* notifier: make batcher tests stable/non-flaky

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* {db,derp,mapper}: dont allocate until append

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-06-23 22:06:50 +02:00
Kristoffer Dalby
5a4e52b727
remove last_successful_update error check (#1959)
most of the time we dont even check this error and checking
the string for particular errors is very flake as different
databases (sqlite and psql) use different error messages, and
some users might have it in other languages.

Fixes #1956

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-05-27 11:53:37 +01:00
Stefan Majer
8185a70dc7
Fix typos (#1860)
* Fix typos

* trigger GitHub actions

* remove kdiff3 orig files

* fix unicode

* remove unnecessary function call

* remove unnecessary comment

* remove unnecessary comment

---------

Co-authored-by: ohdearaugustin <ohdearaugustin@users.noreply.github.com>
2024-05-19 23:49:27 +02:00
MichaelKo
7fd2485000
Restore foreign keys and add constraints (#1562)
* fix #1482, restore foregin keys, add constraints

* #1562, fix tests, fix formatting

* #1562: fix tests

* #1562: fix local run of test_integration
2024-05-15 20:40:14 -04:00
Kristoffer Dalby
ba614a5e6c
metrics, tuning in tests, db cleanups, fix concurrency issue (#1895) 2024-04-21 18:28:17 +02:00
Juan Font
c906aaf927 Allow to remove forced tags of a node
Set as empty StringList
2024-04-18 09:55:55 +02:00
Juan Font
c4c8cfe5ea Fix crash when a prefix family was empty 2024-04-17 15:28:06 +02:00
Kristoffer Dalby
40953727cf fix ip migration
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-04-17 12:36:12 +02:00
Kristoffer Dalby
2ce23df45a
Migrate IP fields in database to dedicated columns (#1869) 2024-04-17 07:03:06 +02:00
Kristoffer Dalby
60f0cf908c more log.Error -> fmt.Errorf cleanup 2024-04-15 12:31:53 +02:00
Kristoffer Dalby
1704977e76 improve testing of route failover logic
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-04-15 12:31:53 +02:00
Kristoffer Dalby
bf4fd078fc clean up use of log.Error where errors could be wrapped
Replace a lot of occurences of log.Error with fmt.Errorf,
bubbling the error up the chain instead.
2024-04-15 12:31:53 +02:00
Kristoffer Dalby
58c94d2bd3 Rework map session
This commit restructures the map session in to a struct
holding the state of what is needed during its lifetime.

For streaming sessions, the event loop is structured a
bit differently not hammering the clients with updates
but rather batching them over a short, configurable time
which should significantly improve cpu usage, and potentially
flakyness.

The use of Patch updates has been dialed back a little as
it does not look like its a 100% ready for prime time. Nodes
are now updated with full changes, except for a few things
like online status.

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-04-15 12:31:53 +02:00
Juan Font
a244eabd03 Ephemeral keys can now be reusable and non-reusable
Fixes the issue reported in #1712. In Tailscale SaaS, ephemeral keys can be single-user or reusable. Until now, our ephemerals were only reusable. This PR makes us adhere to the .com behaviour.
2024-03-03 12:37:37 +01:00
Kristoffer Dalby
c9966ba6c2
fix postgres migration (#1802) 2024-03-01 19:11:46 +01:00
Kristoffer Dalby
3f162c212c
drop unused last_successful_update field from node table (#1754) 2024-02-18 23:22:07 +01:00
Kristoffer Dalby
384ca03208
new IP allocator and add postgres to integration tests. (#1756) 2024-02-18 19:31:29 +01:00
Kristoffer Dalby
b60ee9db54
improve errors for missing directories (#1765)
* improve errors for missing directories

Fixes #1761
Updates #1760

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* update container docs

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* update changelog with /var changes

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-02-17 13:36:19 +01:00
Pallab Pain
9047c09871
feat: add pqsql configs for open and idle connections (#1583)
When Postgres is used as the backing database for headscale,
it does not set a limit on maximum open and idle connections
which leads to hundreds of open connections to the Postgres
server.

This commit introduces the configuration variables to set those
values and also sets default while opening a new postgres connection.
2024-02-09 17:34:28 +01:00
Kristoffer Dalby
94b30abf56
Restructure database config (#1700) 2024-02-09 07:27:00 +01:00
Kristoffer Dalby
83769ba715
Replace database locks with transactions (#1701)
This commits removes the locks used to guard data integrity for the
database and replaces them with Transactions, turns out that SQL had
a way to deal with this all along.

This reduces the complexity we had with multiple locks that might stack
or recurse (database, nofitifer, mapper). All notifications and state
updates are now triggered _after_ a database change.


Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-02-08 17:28:19 +01:00
derelm
4ea12f472a
Fix failover to disabled route #1706 (#1707)
* fix #1706 - failover should disregard disabled routes during failover

* fixe tests for failover; all current tests assume routes to be enabled

* add testcase for #1706 - failover to disabled route
2024-02-03 15:30:15 +01:00
dyz
a369d57a17
fix node expire error due to type in gorm model Update (#1692)
Fixes #1674

Signed-off-by: fortitude.zhang <fortitude.zhang@gmail.com>
2024-01-21 17:38:24 +01:00
Kristoffer Dalby
1e22f17f36
node selfupdate and fix subnet router when ACL is enabled (#1673)
Fixes #1604

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2024-01-18 17:30:25 +01:00
Kristoffer Dalby
65376e2842
ensure renabled auto-approve routes works (#1670) 2024-01-18 16:36:47 +01:00
Alexander Halbarth
7e8bf4bfe5
Add Customization Options to DERP Map entry of integrated DERP server (#1565)
Co-authored-by: Alexander Halbarth <alexander.halbarth@alite.at>
Co-authored-by: Bela Lemle <bela.lemle@alite.at>
Co-authored-by: Kristoffer Dalby <kristoffer@dalby.cc>
2024-01-16 16:04:03 +01:00
Kristoffer Dalby
3b103280ef
implement selfupdate and pass expiry (#1647) 2024-01-05 10:41:56 +01:00
Kristoffer Dalby
6049ec758c
add versioned migrations (#1644) 2023-12-10 15:46:14 +01:00
Kristoffer Dalby
f65f4eca35
ensure online status and route changes are propagated (#1564) 2023-12-09 18:09:24 +01:00
Kristoffer Dalby
a59aab2081
Remove support for non-noise clients (pre-1.32) (#1611) 2023-11-23 08:31:33 +01:00
Kristoffer Dalby
b918aa03fc
move to use tailscfg types over strings/custom types (#1612)
* rename database only fields

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* use correct endpoint type over string list

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* remove HostInfo wrapper

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* wrap errors in database hooks

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2023-11-21 18:20:06 +01:00
Kristoffer Dalby
ed4e19996b
Use tailscale key types instead of strings (#1609)
* upgrade tailscale

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* make Node object use actualy tailscale key types

This commit changes the Node struct to have both a field for strings
to store the keys in the database and a dedicated Key for each type
of key.

The keys are populated and stored with Gorm hooks to ensure the data
is stored in the db.

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* use key types throughout the code

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* make sure machinekey is concistently used

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* use machine key in auth url

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* fix web register

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* use key type in notifier

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

* fix relogin with webauth

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

---------

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2023-11-19 22:37:04 +01:00
Kristoffer Dalby
c0fd06e3f5
remove the use key stripping and store the proper keys (#1603) 2023-11-16 17:55:29 +01:00
Kristoffer Dalby
64c0a6523f
Set online status in lite requests (#1555) 2023-09-25 14:27:14 -07:00