From 85cf443ac6290022edd688e579d954516d0e3ec6 Mon Sep 17 00:00:00 2001 From: Adrien Raffin-Caboisse Date: Tue, 8 Feb 2022 16:59:35 +0100 Subject: [PATCH 01/37] docs(acls): Issues with ACL and proposition --- docs/acls.md | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 docs/acls.md diff --git a/docs/acls.md b/docs/acls.md new file mode 100644 index 00000000..d0708110 --- /dev/null +++ b/docs/acls.md @@ -0,0 +1,55 @@ + +# ACLs + +A key component of tailscale is the notion of Tailnet. This notion is hidden but the implications that it have on how to use tailscale are not. + +For tailscale an [tailnet](https://tailscale.com/kb/1136/tailnet/) is the following: + +> For personal users, you are a tailnet of many devices and one person. Each device gets a private Tailscale IP address in the CGNAT range and every device can talk directly to every other device, wherever they are on the internet. +> +> For businesses and organizations, a tailnet is many devices and many users. It can be based on your Microsoft Active Directory, your Google Workspace, a GitHub organization, Okta tenancy, or other identity provider namespace. All of the devices and users in your tailnet can be seen by the tailnet administrators in the Tailscale admin console. There you can apply tailnet-wide configuration, such as ACLs that affect visibility of devices inside your tailnet, DNS settings, and more. + +## Current implementation and issues + +Currently in headscale, the namespaces are used both as tailnet and users. The issue is that if we want to use the ACL's we can't use both at the same time. + +Tailnet's cannot communicate with each others. So we can't have an ACL that authorize tailnet (namespace) A to talk to tailnet (namespace) B. + +We also can't write ACLs based on the users (namespaces in headscale) since all devices belong to the same user. + +With the current implementation the only ACL that we can user is to associate each headscale IP to a host manually then write the ACLs according to this manual mapping. + +```json +{ + "hosts":{ + "host1": "100.64.0.1", + "server": "100.64.0.2" + }, + "acls": [ + {"action": "accept", "users":["host1"], "ports":["host2:80,443"]} + ] +} +``` + +While this works, it requires a lot of manual editing on the configuration and to keep track of all devices IP address. + +## Proposition for a next implementation + +In order to ease the use of ACL's we need to split the tailnet and users notion. + +A solution could be to consider a headscale server (in it's entirety) as a tailnet. + +For personal users the default behavior could either allow all communications between all namespaces (like tailscale) or dissallow all communications between namespaces (current behavior). + +For businesses and organisations, viewing a headscale instance a single tailnet would allow users (namespace) to talk to each other with the ACLs. As described in tailscale's documentation [[1]], a server should be tagged and personnal devices should be tied to a user. Translated in headscale's terms each user can have multiple devices and all those devices should be in the same namespace. The servers should be tagged and used as such. + +This implementation would render useless the sharing feature that is currently implemented since an ACL could do the same. + +What could be improved would be to peer different headscale installation and allow `sharing`. This would raises issues about compatible network IPs range. + +[1]: https://tailscale.com/kb/1068/acl-tags/ + + +## Get the better of both worlds + +If the current behavior has a lot of use cases we could maybe have a flag to trigger one behavior or the other. Or enabling the ACL's behavior if an ACL file is defined. From 04262123483f1294d52d22f0451c9d05a4b753f9 Mon Sep 17 00:00:00 2001 From: Adrien Raffin-Caboisse Date: Thu, 10 Feb 2022 10:42:26 +0100 Subject: [PATCH 02/37] docs(acls): add example use case --- docs/acls.md | 228 ++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 226 insertions(+), 2 deletions(-) diff --git a/docs/acls.md b/docs/acls.md index d0708110..1d1c56d6 100644 --- a/docs/acls.md +++ b/docs/acls.md @@ -49,7 +49,231 @@ What could be improved would be to peer different headscale installation and all [1]: https://tailscale.com/kb/1068/acl-tags/ +## Example -## Get the better of both worlds +Let's build an example use case for a small business (It may be the place where +ACL's are the most useful). -If the current behavior has a lot of use cases we could maybe have a flag to trigger one behavior or the other. Or enabling the ACL's behavior if an ACL file is defined. +We have a small company with a boss, an admin, two developper and an intern. + +The boss should have access to all servers but not to the users hosts. Admin +should also have access to all hosts except that their permissions should be +limited to maintaining the hosts (for example purposes). The developers can do +anything they want on dev hosts, but only watch on productions hosts. Intern +can only interact with the development servers. + +Each user have at least a device connected to the network and we have some +servers. + +- database.prod +- database.dev +- app-server1.prod +- app-server1.dev +- billing.internal + +### Current headscale implementation + +Let's create some namespaces + +```bash +headscale namespaces create prod +headscale namespaces create dev +headscale namespaces create internal +headscale namespaces create users + +headscale nodes register -n users boss-computer +headscale nodes register -n users admin1-computer +headscale nodes register -n users dev1-computer +headscale nodes register -n users dev1-phone +headscale nodes register -n users dev2-computer +headscale nodes register -n users intern1-computer + +headscale nodes register -n prod database +headscale nodes register -n prod app-server1 + +headscale nodes register -n dev database +headscale nodes register -n dev app-server1 + +headscale nodes register -n internal billing + +headscale nodes list +ID | Name | Namespace | IP address +1 | boss-computer | users | 100.64.0.1 +2 | admin1-computer | users | 100.64.0.2 +3 | dev1-computer | users | 100.64.0.3 +4 | dev1-phone | users | 100.64.0.4 +5 | dev2-computer | users | 100.64.0.5 +6 | intern1-computer | users | 100.64.0.6 +7 | database | prod | 100.64.0.7 +8 | app-server1 | prod | 100.64.0.8 +9 | database | dev | 100.64.0.9 +10 | app-server1 | dev | 100.64.0.10 +11 | internal | internal | 100.64.0.11 +``` + +In order to only allow the communications related to our description above we +need to add the following ACLs + +```json +{ + "hosts":{ + "boss-computer": "100.64.0.1", + "admin1-computer": "100.64.0.2", + "dev1-computer": "100.64.0.3", + "dev1-phone": "100.64.0.4", + "dev2-computer": "100.64.0.5", + "intern1-computer": "100.64.0.6", + "prod-app-server1": "100.64.0.8", + }, + "groups":{ + "group:dev": ["dev1-computer", "dev1-phone", "dev2-computer"], + "group:admin": ["admin1-computer"], + "group:boss": ["boss-computer"], + "group:intern": ["intern1-computer"], + }, + "acls":[ + // boss have access to all servers but no users hosts + {"action":"accept", "users":["group:boss"], "ports":["prod:*","dev:*","internal:*"]}, + + // admin have access to adminstration port (lets only consider port 22 here) + {"action":"accept", "users":["group:admin"], "ports":["prod:22","dev:22","internal:22"]}, + + // dev can do anything on dev servers and check access on prod servers + {"action":"accept", "users":["group:dev"], "ports":["dev:*","prod-app-server1:80,443"]}, + + // interns only have access to port 80 and 443 on dev servers (lame internship) + {"action":"accept", "users":["group:intern"], "ports":["dev:80,443"]}, + + // users can access their own devices + {"action":"accept", "users":["dev1-computer"], "ports":["dev1-phone:*"]}, + {"action":"accept", "users":["dev1-phone"], "ports":["dev1-computer:*"]}, + ] +} +``` + +Since communications between namespace isn't possible we also have to share the +devices between the namespaces. + +```bash + +// add boss host to prod, dev and internal network +headscale nodes share -i 1 -n prod +headscale nodes share -i 1 -n dev +headscale nodes share -i 1 -n internal + +// add admin computer to prod, dev and internal network +headscale nodes share -i 2 -n prod +headscale nodes share -i 2 -n dev +headscale nodes share -i 2 -n internal + +// add all dev to prod and dev network +headscale nodes share -i 3 -n dev +headscale nodes share -i 4 -n dev +headscale nodes share -i 3 -n prod +headscale nodes share -i 4 -n prod +headscale nodes share -i 5 -n dev +headscale nodes share -i 5 -n prod + +headscale nodes share -i 6 -n dev +``` + +This fake network have not been tested but it should work. Operating it could +be quite tedious if the company grows. Each time a new user join we have to add +it to a group, and share it to the correct namespaces. If the user want +multiple devices we have to allow communication to each of them one by one. If +business conduct a change in the organisations we may have to rewrite all acls +and reorganise all namespaces. + +If we add servers in production we should also update the ACLs to allow dev access to certain category of them (only app servers for example). + +### example based on the proposition in this document + +Let's create the namespaces + +```bash +headscale namespaces create boss +headscale namespaces create admin1 +headscale namespaces create dev1 +headscale namespaces create dev2 +headscale namespaces create intern1 +``` + +We don't need to create namespaces for the servers because the servers will be +tagged. When registering the servers we will need to add the flag +`--advertised-tags=tag:,tag:`, and the user (namespace) that is +registering the server should be allowed to do it. Since anyone can add tags to +a server they can register, the check of the tags is done on headscale server +and only valid tags are applied. A tag is valid if the namespace that is +registering it is allowed to do it. + +Here are the ACL's to implement the same permissions as above: + +```json +{ + // groups are simpler and only list the namespaces name + "groups": { + "group:boss": ["boss"], + "group:dev": ["dev1","dev2"], + "group:admin": ["admin1"], + "group:intern": ["intern1"], + }, + "tagOwners": { + // the administrators can add servers in production + "tag:prod-databases": ["group:admin"], + "tag:prod-app-servers": ["group:admin"], + + // the boss can tag any server as internal + "tag:internal": ["group:boss"], + + // dev can add servers for dev purposes as well as admins + "tag:dev-databases": ["group:admin","group:dev"], + "tag:dev-app-servers": ["group:admin", "group:dev"], + + // interns cannot add servers + }, + "acls": [ + // boss have access to all servers + {"action":"accept", + "users":["group:boss"], + "ports":[ + "tag:prod-databases:*", + "tag:prod-app-servers:*", + "tag:internal:*", + "tag:dev-databases:*", + "tag:dev-app-servers:*", + ] + }, + + // admin have only access to administrative ports of the servers + {"action":"accept", + "users":["group:admin"], + "ports":[ + "tag:prod-databases:22", + "tag:prod-app-servers:22", + "tag:internal:22", + "tag:dev-databases:22", + "tag:dev-app-servers:22", + ] + }, + + {"action":"accept", "users":["group:dev"], "ports":[ + "tag:dev-databases:*", + "tag:dev-app-servers:*", + "tag:prod-app-servers:80,443", + ] + }, + + // interns have access to dev-app-servers only in reading mode + {"action":"accept", "users":["group:intern"], "ports":["tag:dev-app-servers:80,443"]}, + + // we still have to allow internal namespaces communications since nothing guarantees that each user have their own namespaces. This could be talked over. + {"action":"accept", "users":["boss"], "ports":["boss:*"]}, + {"action":"accept", "users":["dev1"], "ports":["dev1:*"]}, + {"action":"accept", "users":["dev2"], "ports":["dev2:*"]}, + {"action":"accept", "users":["admin1"], "ports":["admin1:*"]}, + {"action":"accept", "users":["intern1"], "ports":["intern1:*"]}, + ] +} +``` + +With this implementation, the sharing step is not necessary. Maintenance cost of the ACL file is lower and less tedious (no need to map hostname and IP's into it). From 7bdd7748e4cec82206369d972240363bca644527 Mon Sep 17 00:00:00 2001 From: Adrien Raffin-Caboisse Date: Thu, 10 Feb 2022 12:03:03 +0100 Subject: [PATCH 03/37] fix(acl): add missing internal namespace communications --- docs/acls.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/acls.md b/docs/acls.md index 1d1c56d6..31b1c8a5 100644 --- a/docs/acls.md +++ b/docs/acls.md @@ -147,6 +147,11 @@ need to add the following ACLs // users can access their own devices {"action":"accept", "users":["dev1-computer"], "ports":["dev1-phone:*"]}, {"action":"accept", "users":["dev1-phone"], "ports":["dev1-computer:*"]}, + + // internal namespace communications should still be allowed within the namespace + {"action":"accept", "users":["dev"], "ports":["dev:*"]}, + {"action":"accept", "users":["prod"], "ports":["prod:*"]}, + {"action":"accept", "users":["internal"], "ports":["internal:*"]}, ] } ``` @@ -263,6 +268,10 @@ Here are the ACL's to implement the same permissions as above: ] }, + // servers should be able to talk to database. Database should not be able to initiate connections to server + {"action":"accept", "users":["tag:dev-app-servers"], "ports":["tag:dev-databases:5432"]}, + {"action":"accept", "users":["tag:prod-app-servers"], "ports":["tag:prod-databases:5432"]}, + // interns have access to dev-app-servers only in reading mode {"action":"accept", "users":["group:intern"], "ports":["tag:dev-app-servers:80,443"]}, From 86b329d8bfeb5af56d847b3085ac286cbf935676 Mon Sep 17 00:00:00 2001 From: Adrien Raffin-Caboisse Date: Tue, 15 Feb 2022 09:27:33 +0100 Subject: [PATCH 04/37] chore(docs): create proposals directory --- docs/{acls.md => proposals/001-acls.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/{acls.md => proposals/001-acls.md} (100%) diff --git a/docs/acls.md b/docs/proposals/001-acls.md similarity index 100% rename from docs/acls.md rename to docs/proposals/001-acls.md From e540679dbd6d4d9d11d9838ecf6e522d3d8a9a04 Mon Sep 17 00:00:00 2001 From: Adrien Raffin-Caboisse Date: Tue, 15 Feb 2022 09:52:05 +0100 Subject: [PATCH 05/37] docs(acl-proposals): integrate comments --- docs/proposals/001-acls.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/proposals/001-acls.md b/docs/proposals/001-acls.md index 31b1c8a5..92e000a7 100644 --- a/docs/proposals/001-acls.md +++ b/docs/proposals/001-acls.md @@ -43,9 +43,12 @@ For personal users the default behavior could either allow all communications be For businesses and organisations, viewing a headscale instance a single tailnet would allow users (namespace) to talk to each other with the ACLs. As described in tailscale's documentation [[1]], a server should be tagged and personnal devices should be tied to a user. Translated in headscale's terms each user can have multiple devices and all those devices should be in the same namespace. The servers should be tagged and used as such. -This implementation would render useless the sharing feature that is currently implemented since an ACL could do the same. +This implementation would render useless the sharing feature that is currently +implemented since an ACL could do the same. Simplifying to only one user +interface to do one thing is easier and less confusing for the users. -What could be improved would be to peer different headscale installation and allow `sharing`. This would raises issues about compatible network IPs range. +As a sidenote, users would like to write ACLs as YAML. We should offer users +the ability to rules in either format (HuJSON or YAML). [1]: https://tailscale.com/kb/1068/acl-tags/ From c364c2a38206855ad8bbf7264d687cf09c0c40cd Mon Sep 17 00:00:00 2001 From: Adrien Raffin-Caboisse Date: Tue, 15 Feb 2022 09:53:22 +0100 Subject: [PATCH 06/37] chore(acl-proposals): apply prettier --- docs/proposals/001-acls.md | 293 ++++++++++++++++++++++--------------- 1 file changed, 179 insertions(+), 114 deletions(-) diff --git a/docs/proposals/001-acls.md b/docs/proposals/001-acls.md index 92e000a7..23435a2a 100644 --- a/docs/proposals/001-acls.md +++ b/docs/proposals/001-acls.md @@ -1,47 +1,72 @@ - # ACLs -A key component of tailscale is the notion of Tailnet. This notion is hidden but the implications that it have on how to use tailscale are not. +A key component of tailscale is the notion of Tailnet. This notion is hidden +but the implications that it have on how to use tailscale are not. -For tailscale an [tailnet](https://tailscale.com/kb/1136/tailnet/) is the following: +For tailscale an [tailnet](https://tailscale.com/kb/1136/tailnet/) is the +following: -> For personal users, you are a tailnet of many devices and one person. Each device gets a private Tailscale IP address in the CGNAT range and every device can talk directly to every other device, wherever they are on the internet. +> For personal users, you are a tailnet of many devices and one person. Each +> device gets a private Tailscale IP address in the CGNAT range and every +> device can talk directly to every other device, wherever they are on the +> internet. > -> For businesses and organizations, a tailnet is many devices and many users. It can be based on your Microsoft Active Directory, your Google Workspace, a GitHub organization, Okta tenancy, or other identity provider namespace. All of the devices and users in your tailnet can be seen by the tailnet administrators in the Tailscale admin console. There you can apply tailnet-wide configuration, such as ACLs that affect visibility of devices inside your tailnet, DNS settings, and more. +> For businesses and organizations, a tailnet is many devices and many users. +> It can be based on your Microsoft Active Directory, your Google Workspace, a +> GitHub organization, Okta tenancy, or other identity provider namespace. All +> of the devices and users in your tailnet can be seen by the tailnet +> administrators in the Tailscale admin console. There you can apply +> tailnet-wide configuration, such as ACLs that affect visibility of devices +> inside your tailnet, DNS settings, and more. ## Current implementation and issues -Currently in headscale, the namespaces are used both as tailnet and users. The issue is that if we want to use the ACL's we can't use both at the same time. +Currently in headscale, the namespaces are used both as tailnet and users. The +issue is that if we want to use the ACL's we can't use both at the same time. -Tailnet's cannot communicate with each others. So we can't have an ACL that authorize tailnet (namespace) A to talk to tailnet (namespace) B. +Tailnet's cannot communicate with each others. So we can't have an ACL that +authorize tailnet (namespace) A to talk to tailnet (namespace) B. -We also can't write ACLs based on the users (namespaces in headscale) since all devices belong to the same user. +We also can't write ACLs based on the users (namespaces in headscale) since all +devices belong to the same user. -With the current implementation the only ACL that we can user is to associate each headscale IP to a host manually then write the ACLs according to this manual mapping. +With the current implementation the only ACL that we can user is to associate +each headscale IP to a host manually then write the ACLs according to this +manual mapping. ```json { - "hosts":{ - "host1": "100.64.0.1", - "server": "100.64.0.2" - }, - "acls": [ - {"action": "accept", "users":["host1"], "ports":["host2:80,443"]} - ] + "hosts": { + "host1": "100.64.0.1", + "server": "100.64.0.2" + }, + "acls": [ + { "action": "accept", "users": ["host1"], "ports": ["host2:80,443"] } + ] } ``` -While this works, it requires a lot of manual editing on the configuration and to keep track of all devices IP address. +While this works, it requires a lot of manual editing on the configuration and +to keep track of all devices IP address. ## Proposition for a next implementation -In order to ease the use of ACL's we need to split the tailnet and users notion. +In order to ease the use of ACL's we need to split the tailnet and users +notion. -A solution could be to consider a headscale server (in it's entirety) as a tailnet. +A solution could be to consider a headscale server (in it's entirety) as a +tailnet. -For personal users the default behavior could either allow all communications between all namespaces (like tailscale) or dissallow all communications between namespaces (current behavior). +For personal users the default behavior could either allow all communications +between all namespaces (like tailscale) or dissallow all communications between +namespaces (current behavior). -For businesses and organisations, viewing a headscale instance a single tailnet would allow users (namespace) to talk to each other with the ACLs. As described in tailscale's documentation [[1]], a server should be tagged and personnal devices should be tied to a user. Translated in headscale's terms each user can have multiple devices and all those devices should be in the same namespace. The servers should be tagged and used as such. +For businesses and organisations, viewing a headscale instance a single tailnet +would allow users (namespace) to talk to each other with the ACLs. As described +in tailscale's documentation [[1]], a server should be tagged and personnal +devices should be tied to a user. Translated in headscale's terms each user can +have multiple devices and all those devices should be in the same namespace. +The servers should be tagged and used as such. This implementation would render useless the sharing feature that is currently implemented since an ACL could do the same. Simplifying to only one user @@ -119,43 +144,63 @@ need to add the following ACLs ```json { - "hosts":{ - "boss-computer": "100.64.0.1", - "admin1-computer": "100.64.0.2", - "dev1-computer": "100.64.0.3", - "dev1-phone": "100.64.0.4", - "dev2-computer": "100.64.0.5", - "intern1-computer": "100.64.0.6", - "prod-app-server1": "100.64.0.8", + "hosts": { + "boss-computer": "100.64.0.1", + "admin1-computer": "100.64.0.2", + "dev1-computer": "100.64.0.3", + "dev1-phone": "100.64.0.4", + "dev2-computer": "100.64.0.5", + "intern1-computer": "100.64.0.6", + "prod-app-server1": "100.64.0.8" + }, + "groups": { + "group:dev": ["dev1-computer", "dev1-phone", "dev2-computer"], + "group:admin": ["admin1-computer"], + "group:boss": ["boss-computer"], + "group:intern": ["intern1-computer"] + }, + "acls": [ + // boss have access to all servers but no users hosts + { + "action": "accept", + "users": ["group:boss"], + "ports": ["prod:*", "dev:*", "internal:*"] }, - "groups":{ - "group:dev": ["dev1-computer", "dev1-phone", "dev2-computer"], - "group:admin": ["admin1-computer"], - "group:boss": ["boss-computer"], - "group:intern": ["intern1-computer"], + + // admin have access to adminstration port (lets only consider port 22 here) + { + "action": "accept", + "users": ["group:admin"], + "ports": ["prod:22", "dev:22", "internal:22"] }, - "acls":[ - // boss have access to all servers but no users hosts - {"action":"accept", "users":["group:boss"], "ports":["prod:*","dev:*","internal:*"]}, - // admin have access to adminstration port (lets only consider port 22 here) - {"action":"accept", "users":["group:admin"], "ports":["prod:22","dev:22","internal:22"]}, + // dev can do anything on dev servers and check access on prod servers + { + "action": "accept", + "users": ["group:dev"], + "ports": ["dev:*", "prod-app-server1:80,443"] + }, - // dev can do anything on dev servers and check access on prod servers - {"action":"accept", "users":["group:dev"], "ports":["dev:*","prod-app-server1:80,443"]}, + // interns only have access to port 80 and 443 on dev servers (lame internship) + { "action": "accept", "users": ["group:intern"], "ports": ["dev:80,443"] }, - // interns only have access to port 80 and 443 on dev servers (lame internship) - {"action":"accept", "users":["group:intern"], "ports":["dev:80,443"]}, + // users can access their own devices + { + "action": "accept", + "users": ["dev1-computer"], + "ports": ["dev1-phone:*"] + }, + { + "action": "accept", + "users": ["dev1-phone"], + "ports": ["dev1-computer:*"] + }, - // users can access their own devices - {"action":"accept", "users":["dev1-computer"], "ports":["dev1-phone:*"]}, - {"action":"accept", "users":["dev1-phone"], "ports":["dev1-computer:*"]}, - - // internal namespace communications should still be allowed within the namespace - {"action":"accept", "users":["dev"], "ports":["dev:*"]}, - {"action":"accept", "users":["prod"], "ports":["prod:*"]}, - {"action":"accept", "users":["internal"], "ports":["internal:*"]}, - ] + // internal namespace communications should still be allowed within the namespace + { "action": "accept", "users": ["dev"], "ports": ["dev:*"] }, + { "action": "accept", "users": ["prod"], "ports": ["prod:*"] }, + { "action": "accept", "users": ["internal"], "ports": ["internal:*"] } + ] } ``` @@ -192,7 +237,8 @@ multiple devices we have to allow communication to each of them one by one. If business conduct a change in the organisations we may have to rewrite all acls and reorganise all namespaces. -If we add servers in production we should also update the ACLs to allow dev access to certain category of them (only app servers for example). +If we add servers in production we should also update the ACLs to allow dev +access to certain category of them (only app servers for example). ### example based on the proposition in this document @@ -218,74 +264,93 @@ Here are the ACL's to implement the same permissions as above: ```json { - // groups are simpler and only list the namespaces name - "groups": { - "group:boss": ["boss"], - "group:dev": ["dev1","dev2"], - "group:admin": ["admin1"], - "group:intern": ["intern1"], + // groups are simpler and only list the namespaces name + "groups": { + "group:boss": ["boss"], + "group:dev": ["dev1", "dev2"], + "group:admin": ["admin1"], + "group:intern": ["intern1"] + }, + "tagOwners": { + // the administrators can add servers in production + "tag:prod-databases": ["group:admin"], + "tag:prod-app-servers": ["group:admin"], + + // the boss can tag any server as internal + "tag:internal": ["group:boss"], + + // dev can add servers for dev purposes as well as admins + "tag:dev-databases": ["group:admin", "group:dev"], + "tag:dev-app-servers": ["group:admin", "group:dev"] + + // interns cannot add servers + }, + "acls": [ + // boss have access to all servers + { + "action": "accept", + "users": ["group:boss"], + "ports": [ + "tag:prod-databases:*", + "tag:prod-app-servers:*", + "tag:internal:*", + "tag:dev-databases:*", + "tag:dev-app-servers:*" + ] }, - "tagOwners": { - // the administrators can add servers in production - "tag:prod-databases": ["group:admin"], - "tag:prod-app-servers": ["group:admin"], - // the boss can tag any server as internal - "tag:internal": ["group:boss"], - - // dev can add servers for dev purposes as well as admins - "tag:dev-databases": ["group:admin","group:dev"], - "tag:dev-app-servers": ["group:admin", "group:dev"], - - // interns cannot add servers + // admin have only access to administrative ports of the servers + { + "action": "accept", + "users": ["group:admin"], + "ports": [ + "tag:prod-databases:22", + "tag:prod-app-servers:22", + "tag:internal:22", + "tag:dev-databases:22", + "tag:dev-app-servers:22" + ] }, - "acls": [ - // boss have access to all servers - {"action":"accept", - "users":["group:boss"], - "ports":[ - "tag:prod-databases:*", - "tag:prod-app-servers:*", - "tag:internal:*", - "tag:dev-databases:*", - "tag:dev-app-servers:*", - ] - }, - // admin have only access to administrative ports of the servers - {"action":"accept", - "users":["group:admin"], - "ports":[ - "tag:prod-databases:22", - "tag:prod-app-servers:22", - "tag:internal:22", - "tag:dev-databases:22", - "tag:dev-app-servers:22", - ] - }, + { + "action": "accept", + "users": ["group:dev"], + "ports": [ + "tag:dev-databases:*", + "tag:dev-app-servers:*", + "tag:prod-app-servers:80,443" + ] + }, - {"action":"accept", "users":["group:dev"], "ports":[ - "tag:dev-databases:*", - "tag:dev-app-servers:*", - "tag:prod-app-servers:80,443", - ] - }, + // servers should be able to talk to database. Database should not be able to initiate connections to server + { + "action": "accept", + "users": ["tag:dev-app-servers"], + "ports": ["tag:dev-databases:5432"] + }, + { + "action": "accept", + "users": ["tag:prod-app-servers"], + "ports": ["tag:prod-databases:5432"] + }, - // servers should be able to talk to database. Database should not be able to initiate connections to server - {"action":"accept", "users":["tag:dev-app-servers"], "ports":["tag:dev-databases:5432"]}, - {"action":"accept", "users":["tag:prod-app-servers"], "ports":["tag:prod-databases:5432"]}, + // interns have access to dev-app-servers only in reading mode + { + "action": "accept", + "users": ["group:intern"], + "ports": ["tag:dev-app-servers:80,443"] + }, - // interns have access to dev-app-servers only in reading mode - {"action":"accept", "users":["group:intern"], "ports":["tag:dev-app-servers:80,443"]}, - - // we still have to allow internal namespaces communications since nothing guarantees that each user have their own namespaces. This could be talked over. - {"action":"accept", "users":["boss"], "ports":["boss:*"]}, - {"action":"accept", "users":["dev1"], "ports":["dev1:*"]}, - {"action":"accept", "users":["dev2"], "ports":["dev2:*"]}, - {"action":"accept", "users":["admin1"], "ports":["admin1:*"]}, - {"action":"accept", "users":["intern1"], "ports":["intern1:*"]}, - ] + // we still have to allow internal namespaces communications since nothing guarantees that each user have their own namespaces. This could be talked over. + { "action": "accept", "users": ["boss"], "ports": ["boss:*"] }, + { "action": "accept", "users": ["dev1"], "ports": ["dev1:*"] }, + { "action": "accept", "users": ["dev2"], "ports": ["dev2:*"] }, + { "action": "accept", "users": ["admin1"], "ports": ["admin1:*"] }, + { "action": "accept", "users": ["intern1"], "ports": ["intern1:*"] } + ] } ``` -With this implementation, the sharing step is not necessary. Maintenance cost of the ACL file is lower and less tedious (no need to map hostname and IP's into it). +With this implementation, the sharing step is not necessary. Maintenance cost +of the ACL file is lower and less tedious (no need to map hostname and IP's +into it). From 55d746d3f5323f661238db9ecb2da93ad95e7430 Mon Sep 17 00:00:00 2001 From: Adrien Raffin-Caboisse Date: Wed, 16 Feb 2022 09:16:25 +0100 Subject: [PATCH 07/37] docs(acls-proposal): wording comment A hidden thing was implied in this document is that each person should have his own namespace. Hidden information in spicification isn't good. Thank's @kradalby for pointing it out. --- docs/proposals/001-acls.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/proposals/001-acls.md b/docs/proposals/001-acls.md index 23435a2a..8a02e836 100644 --- a/docs/proposals/001-acls.md +++ b/docs/proposals/001-acls.md @@ -72,6 +72,12 @@ This implementation would render useless the sharing feature that is currently implemented since an ACL could do the same. Simplifying to only one user interface to do one thing is easier and less confusing for the users. +To better suit the ACLs in this proposition, it's advised to consider that each +namespaces belong to one person. This person can have multiple devices, they +will all be considered as the same user in the ACLs. OIDC feature wouldn't need +to map people to namespace, just create a namespace if the person isn't +registered yet. + As a sidenote, users would like to write ACLs as YAML. We should offer users the ability to rules in either format (HuJSON or YAML). From 897d480f4dd723ad0c197e9c9a782da8a4503c7f Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Fri, 4 Mar 2022 00:01:31 +0100 Subject: [PATCH 08/37] Add an embedded DERP server to Headscale This series of commit will be adding an embedded DERP server (and STUN) to Headscale, thus making it completely self-contained and not dependant in other infrastructure. --- app.go | 58 ++++++++++-- cmd/headscale/cli/utils.go | 6 ++ derp_embedded.go | 178 +++++++++++++++++++++++++++++++++++++ 3 files changed, 235 insertions(+), 7 deletions(-) create mode 100644 derp_embedded.go diff --git a/app.go b/app.go index 58185093..ee38ed93 100644 --- a/app.go +++ b/app.go @@ -119,6 +119,7 @@ type OIDCConfig struct { } type DERPConfig struct { + EmbeddedDERP bool URLs []url.URL Paths []string AutoUpdate bool @@ -141,7 +142,8 @@ type Headscale struct { dbDebug bool privateKey *key.MachinePrivate - DERPMap *tailcfg.DERPMap + DERPMap *tailcfg.DERPMap + EmbeddedDerpServer *EmbeddedDerpServer aclPolicy *ACLPolicy aclRules []tailcfg.FilterRule @@ -238,6 +240,38 @@ func NewHeadscale(cfg Config) (*Headscale, error) { } } + if cfg.DERP.EmbeddedDERP { + embeddedDerpServer, err := app.NewEmbeddedDerpServer() + if err != nil { + return nil, err + } + app.EmbeddedDerpServer = embeddedDerpServer + + // If we are using the embedded DERP, there is no reason to use Tailscale's DERP infrastructure + serverURL, err := url.Parse(app.cfg.ServerURL) + if err != nil { + return nil, err + } + app.DERPMap = &tailcfg.DERPMap{ + Regions: map[int]*tailcfg.DERPRegion{ + 1: { + RegionID: 1, + RegionCode: "headscale", + RegionName: "Headscale Embedded DERP", + Avoid: false, + Nodes: []*tailcfg.DERPNode{ + { + Name: "1a", + RegionID: 1, + HostName: serverURL.Host, + }, + }, + }, + }, + OmitDefaultRegions: false, + } + } + return &app, nil } @@ -454,6 +488,12 @@ func (h *Headscale) createRouter(grpcMux *runtime.ServeMux) *gin.Engine { router.GET("/swagger", SwaggerUI) router.GET("/swagger/v1/openapiv2.json", SwaggerAPIv1) + if h.cfg.DERP.EmbeddedDERP { + router.Any("/derp", h.EmbeddedDerpHandler) + router.Any("/derp/probe", h.EmbeddedDerpProbeHandler) + router.Any("/bootstrap-dns", h.EmbeddedDerpBootstrapDNSHandler) + } + api := router.Group("/api") api.Use(h.httpAuthenticationMiddleware) { @@ -469,13 +509,17 @@ func (h *Headscale) createRouter(grpcMux *runtime.ServeMux) *gin.Engine { func (h *Headscale) Serve() error { var err error - // Fetch an initial DERP Map before we start serving - h.DERPMap = GetDERPMap(h.cfg.DERP) + if h.cfg.DERP.EmbeddedDERP { + go h.ServeSTUN() + } else { + // Fetch an initial DERP Map before we start serving + h.DERPMap = GetDERPMap(h.cfg.DERP) - if h.cfg.DERP.AutoUpdate { - derpMapCancelChannel := make(chan struct{}) - defer func() { derpMapCancelChannel <- struct{}{} }() - go h.scheduledDERPMapUpdateWorker(derpMapCancelChannel) + if h.cfg.DERP.AutoUpdate { + derpMapCancelChannel := make(chan struct{}) + defer func() { derpMapCancelChannel <- struct{}{} }() + go h.scheduledDERPMapUpdateWorker(derpMapCancelChannel) + } } go h.expireEphemeralNodes(updateInterval) diff --git a/cmd/headscale/cli/utils.go b/cmd/headscale/cli/utils.go index 97c2440e..cff31f32 100644 --- a/cmd/headscale/cli/utils.go +++ b/cmd/headscale/cli/utils.go @@ -117,6 +117,12 @@ func LoadConfig(path string) error { } func GetDERPConfig() headscale.DERPConfig { + if viper.GetBool("derp.embedded_derp") { + return headscale.DERPConfig{ + EmbeddedDERP: true, + } + } + urlStrs := viper.GetStringSlice("derp.urls") urls := make([]url.URL, len(urlStrs)) diff --git a/derp_embedded.go b/derp_embedded.go new file mode 100644 index 00000000..0631fff8 --- /dev/null +++ b/derp_embedded.go @@ -0,0 +1,178 @@ +package headscale + +import ( + "context" + "encoding/json" + "fmt" + "net" + "net/http" + "strings" + "sync/atomic" + "time" + + "github.com/gin-gonic/gin" + "github.com/rs/zerolog/log" + "tailscale.com/derp" + "tailscale.com/net/stun" + "tailscale.com/types/key" +) + +// fastStartHeader is the header (with value "1") that signals to the HTTP +// server that the DERP HTTP client does not want the HTTP 101 response +// headers and it will begin writing & reading the DERP protocol immediately +// following its HTTP request. +const fastStartHeader = "Derp-Fast-Start" + +var ( + dnsCache atomic.Value // of []byte + bootstrapDNS = "derp.tailscale.com" +) + +type EmbeddedDerpServer struct { + tailscaleDerp *derp.Server +} + +func (h *Headscale) NewEmbeddedDerpServer() (*EmbeddedDerpServer, error) { + s := derp.NewServer(key.NodePrivate(*h.privateKey), log.Info().Msgf) + return &EmbeddedDerpServer{s}, nil + +} + +func (h *Headscale) EmbeddedDerpHandler(ctx *gin.Context) { + up := strings.ToLower(ctx.Request.Header.Get("Upgrade")) + if up != "websocket" && up != "derp" { + if up != "" { + log.Warn().Caller().Msgf("Weird websockets connection upgrade: %q", up) + } + ctx.String(http.StatusUpgradeRequired, "DERP requires connection upgrade") + return + } + + fastStart := ctx.Request.Header.Get(fastStartHeader) == "1" + + hijacker, ok := ctx.Writer.(http.Hijacker) + if !ok { + log.Error().Caller().Msg("DERP requires Hijacker interface from Gin") + ctx.String(http.StatusInternalServerError, "HTTP does not support general TCP support") + return + } + + netConn, conn, err := hijacker.Hijack() + if err != nil { + log.Error().Caller().Err(err).Msgf("Hijack failed") + ctx.String(http.StatusInternalServerError, "HTTP does not support general TCP support") + return + } + + if !fastStart { + pubKey := h.privateKey.Public() + fmt.Fprintf(conn, "HTTP/1.1 101 Switching Protocols\r\n"+ + "Upgrade: DERP\r\n"+ + "Connection: Upgrade\r\n"+ + "Derp-Version: %v\r\n"+ + "Derp-Public-Key: %s\r\n\r\n", + derp.ProtocolVersion, + pubKey.UntypedHexString()) + } + + h.EmbeddedDerpServer.tailscaleDerp.Accept(netConn, conn, netConn.RemoteAddr().String()) +} + +// EmbeddedDerpProbeHandler is the endpoint that js/wasm clients hit to measure +// DERP latency, since they can't do UDP STUN queries. +func (h *Headscale) EmbeddedDerpProbeHandler(ctx *gin.Context) { + switch ctx.Request.Method { + case "HEAD", "GET": + ctx.Writer.Header().Set("Access-Control-Allow-Origin", "*") + default: + ctx.String(http.StatusMethodNotAllowed, "bogus probe method") + } +} + +func (h *Headscale) EmbeddedDerpBootstrapDNSHandler(ctx *gin.Context) { + ctx.Header("Content-Type", "application/json") + j, _ := dnsCache.Load().([]byte) + // Bootstrap DNS requests occur cross-regions, + // and are randomized per request, + // so keeping a connection open is pointlessly expensive. + ctx.Header("Connection", "close") + ctx.Writer.Write(j) +} + +// ServeSTUN starts a STUN server on udp/3478 +func (h *Headscale) ServeSTUN() { + pc, err := net.ListenPacket("udp", "0.0.0.0:3478") + if err != nil { + log.Fatal().Msgf("failed to open STUN listener: %v", err) + } + log.Printf("running STUN server on %v", pc.LocalAddr()) + serverSTUNListener(context.Background(), pc.(*net.UDPConn)) +} + +func serverSTUNListener(ctx context.Context, pc *net.UDPConn) { + var buf [64 << 10]byte + var ( + n int + ua *net.UDPAddr + err error + ) + for { + n, ua, err = pc.ReadFromUDP(buf[:]) + if err != nil { + if ctx.Err() != nil { + return + } + log.Printf("STUN ReadFrom: %v", err) + time.Sleep(time.Second) + continue + } + pkt := buf[:n] + if !stun.Is(pkt) { + continue + } + txid, err := stun.ParseBindingRequest(pkt) + if err != nil { + continue + } + + res := stun.Response(txid, ua.IP, uint16(ua.Port)) + pc.WriteTo(res, ua) + } +} + +// Shamelessly taken from +// https://github.com/tailscale/tailscale/blob/main/cmd/derper/bootstrap_dns.go +func refreshBootstrapDNSLoop() { + if bootstrapDNS == "" { + return + } + for { + refreshBootstrapDNS() + time.Sleep(10 * time.Minute) + } +} + +func refreshBootstrapDNS() { + if bootstrapDNS == "" { + return + } + dnsEntries := make(map[string][]net.IP) + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + names := strings.Split(bootstrapDNS, ",") + var r net.Resolver + for _, name := range names { + addrs, err := r.LookupIP(ctx, "ip", name) + if err != nil { + log.Printf("bootstrap DNS lookup %q: %v", name, err) + continue + } + dnsEntries[name] = addrs + } + j, err := json.MarshalIndent(dnsEntries, "", "\t") + if err != nil { + // leave the old values in place + return + } + dnsCache.Store(j) +} From 9d43f589ae5ac9b902247b5d00ec28cc632b9168 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Fri, 4 Mar 2022 00:04:28 +0100 Subject: [PATCH 09/37] Added missing deps --- go.mod | 3 +++ go.sum | 7 +++++++ 2 files changed, 10 insertions(+) diff --git a/go.mod b/go.mod index a121826d..d6754f98 100644 --- a/go.mod +++ b/go.mod @@ -46,6 +46,7 @@ require ( github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect github.com/Microsoft/go-winio v0.5.2 // indirect github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 // indirect + github.com/akutz/memconn v0.1.0 // indirect github.com/atomicgo/cursor v0.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/cenkalti/backoff/v4 v4.1.2 // indirect @@ -100,6 +101,7 @@ require ( github.com/mattn/go-sqlite3 v1.14.11 // indirect github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect + github.com/mitchellh/go-ps v1.0.0 // indirect github.com/mitchellh/mapstructure v1.4.3 // indirect github.com/moby/term v0.0.0-20210619224110-3f7ff695adc6 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect @@ -134,6 +136,7 @@ require ( golang.org/x/sys v0.0.0-20220227234510-4e6760a101f9 // indirect golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 // indirect golang.org/x/text v0.3.7 // indirect + golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 // indirect google.golang.org/appengine v1.6.7 // indirect gopkg.in/ini.v1 v1.66.4 // indirect gopkg.in/square/go-jose.v2 v2.6.0 // indirect diff --git a/go.sum b/go.sum index 558d3d7c..c23db380 100644 --- a/go.sum +++ b/go.sum @@ -82,6 +82,8 @@ github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 h1:TngWCqHvy9oXAN6lEV github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5/go.mod h1:lmUJ/7eu/Q8D7ML55dXQrVaamCz2vxCfdQBasLZfHKk= github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= github.com/PuerkitoBio/goquery v1.5.1/go.mod h1:GsLWisAFVj4WgDibEWF4pvYnkVQBpKBKeU+7zCJoLcc= +github.com/akutz/memconn v0.1.0 h1:NawI0TORU4hcOMsMr11g7vwlCdkYeLKXBcxWu2W/P8A= +github.com/akutz/memconn v0.1.0/go.mod h1:Jo8rI7m0NieZyLI5e2CDlRdRqRRB4S7Xp77ukDjH+Fw= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= @@ -205,6 +207,7 @@ github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYF github.com/fatih/set v0.2.1 h1:nn2CaJyknWE/6txyUDGwysr3G5QC6xWB/PtVjPBbeaA= github.com/fatih/set v0.2.1/go.mod h1:+RKtMCH+favT2+3YecHGxcc0b4KyVWA1QWWJUs4E0CI= github.com/frankban/quicktest v1.11.3/go.mod h1:wRf/ReqHper53s+kmmSZizM8NamnL3IM0I9ntUbOk+k= +github.com/frankban/quicktest v1.14.0 h1:+cqqvzZV87b4adx/5ayVOaYZ2CrvM4ejQvUdBzPPUss= github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo= github.com/fsnotify/fsnotify v1.5.1 h1:mZcQUHVQUQWoPXXtuf9yuEXKudkV2sx1E06UadKWpgI= github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU= @@ -570,6 +573,8 @@ github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceT github.com/mitchellh/cli v1.1.0/go.mod h1:xcISNoH86gajksDmfB23e/pu+B+GeFRMYmoHXxx3xhI= github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0= +github.com/mitchellh/go-ps v1.0.0 h1:i6ampVEEF4wQFF+bkYfwYgY+F/uYJDktmvLPf7qIgjc= +github.com/mitchellh/go-ps v1.0.0/go.mod h1:J4lOc8z8yJs6vUwklHw2XEIiT4z4C40KtWVN3nvg8Pg= github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI= github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg= github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY= @@ -1083,6 +1088,8 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11 h1:GZokNIeuVkl3aZHJchRrr13WCsols02MLUcz1U9is6M= +golang.org/x/time v0.0.0-20211116232009-f0f3c7e86c11/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= From 607c1eb3163999fb310170a5ee67a83c634e8196 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Fri, 4 Mar 2022 11:31:41 +0100 Subject: [PATCH 10/37] Be consistent with uppercase DERP --- app.go | 12 ++++++------ derp_embedded.go | 18 +++++++++--------- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/app.go b/app.go index 3115b994..eb3b8ade 100644 --- a/app.go +++ b/app.go @@ -144,7 +144,7 @@ type Headscale struct { privateKey *key.MachinePrivate DERPMap *tailcfg.DERPMap - EmbeddedDerpServer *EmbeddedDerpServer + EmbeddedDERPServer *EmbeddedDERPServer aclPolicy *ACLPolicy aclRules []tailcfg.FilterRule @@ -242,11 +242,11 @@ func NewHeadscale(cfg Config) (*Headscale, error) { } if cfg.DERP.EmbeddedDERP { - embeddedDerpServer, err := app.NewEmbeddedDerpServer() + embeddedDERPServer, err := app.NewEmbeddedDERPServer() if err != nil { return nil, err } - app.EmbeddedDerpServer = embeddedDerpServer + app.EmbeddedDERPServer = embeddedDERPServer // If we are using the embedded DERP, there is no reason to use Tailscale's DERP infrastructure serverURL, err := url.Parse(app.cfg.ServerURL) @@ -496,9 +496,9 @@ func (h *Headscale) createRouter(grpcMux *runtime.ServeMux) *gin.Engine { router.GET("/swagger/v1/openapiv2.json", SwaggerAPIv1) if h.cfg.DERP.EmbeddedDERP { - router.Any("/derp", h.EmbeddedDerpHandler) - router.Any("/derp/probe", h.EmbeddedDerpProbeHandler) - router.Any("/bootstrap-dns", h.EmbeddedDerpBootstrapDNSHandler) + router.Any("/derp", h.EmbeddedDERPHandler) + router.Any("/derp/probe", h.EmbeddedDERPProbeHandler) + router.Any("/bootstrap-dns", h.EmbeddedDERPBootstrapDNSHandler) } api := router.Group("/api") diff --git a/derp_embedded.go b/derp_embedded.go index 0631fff8..1d4fb0b5 100644 --- a/derp_embedded.go +++ b/derp_embedded.go @@ -28,17 +28,17 @@ var ( bootstrapDNS = "derp.tailscale.com" ) -type EmbeddedDerpServer struct { - tailscaleDerp *derp.Server +type EmbeddedDERPServer struct { + tailscaleDERP *derp.Server } -func (h *Headscale) NewEmbeddedDerpServer() (*EmbeddedDerpServer, error) { +func (h *Headscale) NewEmbeddedDERPServer() (*EmbeddedDERPServer, error) { s := derp.NewServer(key.NodePrivate(*h.privateKey), log.Info().Msgf) - return &EmbeddedDerpServer{s}, nil + return &EmbeddedDERPServer{s}, nil } -func (h *Headscale) EmbeddedDerpHandler(ctx *gin.Context) { +func (h *Headscale) EmbeddedDERPHandler(ctx *gin.Context) { up := strings.ToLower(ctx.Request.Header.Get("Upgrade")) if up != "websocket" && up != "derp" { if up != "" { @@ -75,12 +75,12 @@ func (h *Headscale) EmbeddedDerpHandler(ctx *gin.Context) { pubKey.UntypedHexString()) } - h.EmbeddedDerpServer.tailscaleDerp.Accept(netConn, conn, netConn.RemoteAddr().String()) + h.EmbeddedDERPServer.tailscaleDERP.Accept(netConn, conn, netConn.RemoteAddr().String()) } -// EmbeddedDerpProbeHandler is the endpoint that js/wasm clients hit to measure +// EmbeddedDERPProbeHandler is the endpoint that js/wasm clients hit to measure // DERP latency, since they can't do UDP STUN queries. -func (h *Headscale) EmbeddedDerpProbeHandler(ctx *gin.Context) { +func (h *Headscale) EmbeddedDERPProbeHandler(ctx *gin.Context) { switch ctx.Request.Method { case "HEAD", "GET": ctx.Writer.Header().Set("Access-Control-Allow-Origin", "*") @@ -89,7 +89,7 @@ func (h *Headscale) EmbeddedDerpProbeHandler(ctx *gin.Context) { } } -func (h *Headscale) EmbeddedDerpBootstrapDNSHandler(ctx *gin.Context) { +func (h *Headscale) EmbeddedDERPBootstrapDNSHandler(ctx *gin.Context) { ctx.Header("Content-Type", "application/json") j, _ := dnsCache.Load().([]byte) // Bootstrap DNS requests occur cross-regions, From 22d2443281466006500f3a0ceb2e1f87b21d174d Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Fri, 4 Mar 2022 13:26:45 +0100 Subject: [PATCH 11/37] Move more stuff to common --- integration_common_test.go | 7 +++++++ integration_test.go | 7 ------- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/integration_common_test.go b/integration_common_test.go index de304d0d..de9fdd99 100644 --- a/integration_common_test.go +++ b/integration_common_test.go @@ -18,8 +18,15 @@ const DOCKER_EXECUTE_TIMEOUT = 10 * time.Second var ( IpPrefix4 = netaddr.MustParseIPPrefix("100.64.0.0/10") IpPrefix6 = netaddr.MustParseIPPrefix("fd7a:115c:a1e0::/48") + + tailscaleVersions = []string{"1.20.4", "1.18.2", "1.16.2", "1.14.3", "1.12.3"} ) +type TestNamespace struct { + count int + tailscales map[string]dockertest.Resource +} + type ExecuteCommandConfig struct { timeout time.Duration } diff --git a/integration_test.go b/integration_test.go index 03d6d2f2..523a9a99 100644 --- a/integration_test.go +++ b/integration_test.go @@ -29,13 +29,6 @@ import ( "tailscale.com/ipn/ipnstate" ) -var tailscaleVersions = []string{"1.20.4", "1.18.2", "1.16.2", "1.14.3", "1.12.3"} - -type TestNamespace struct { - count int - tailscales map[string]dockertest.Resource -} - type IntegrationTestSuite struct { suite.Suite stats *suite.SuiteInformation From 09d78c7a05b6a9dc90b1a365d0a98ee34f62b112 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Fri, 4 Mar 2022 13:54:59 +0100 Subject: [PATCH 12/37] Even more stuff moved to common --- integration_common_test.go | 33 +++++++++++++++++++++++++++++++++ integration_test.go | 32 -------------------------------- 2 files changed, 33 insertions(+), 32 deletions(-) diff --git a/integration_common_test.go b/integration_common_test.go index de9fdd99..a3417121 100644 --- a/integration_common_test.go +++ b/integration_common_test.go @@ -6,6 +6,7 @@ package headscale import ( "bytes" "fmt" + "strings" "time" "github.com/ory/dockertest/v3" @@ -126,3 +127,35 @@ func DockerAllowNetworkAdministration(config *docker.HostConfig) { Target: "/dev/net/tun", }) } + +func getIPs( + tailscales map[string]dockertest.Resource, +) (map[string][]netaddr.IP, error) { + ips := make(map[string][]netaddr.IP) + for hostname, tailscale := range tailscales { + command := []string{"tailscale", "ip"} + + result, err := ExecuteCommand( + &tailscale, + command, + []string{}, + ) + if err != nil { + return nil, err + } + + for _, address := range strings.Split(result, "\n") { + address = strings.TrimSuffix(address, "\n") + if len(address) < 1 { + continue + } + ip, err := netaddr.ParseIP(address) + if err != nil { + return nil, err + } + ips[hostname] = append(ips[hostname], ip) + } + } + + return ips, nil +} diff --git a/integration_test.go b/integration_test.go index 523a9a99..1649f322 100644 --- a/integration_test.go +++ b/integration_test.go @@ -680,38 +680,6 @@ func (s *IntegrationTestSuite) TestMagicDNS() { } } -func getIPs( - tailscales map[string]dockertest.Resource, -) (map[string][]netaddr.IP, error) { - ips := make(map[string][]netaddr.IP) - for hostname, tailscale := range tailscales { - command := []string{"tailscale", "ip"} - - result, err := ExecuteCommand( - &tailscale, - command, - []string{}, - ) - if err != nil { - return nil, err - } - - for _, address := range strings.Split(result, "\n") { - address = strings.TrimSuffix(address, "\n") - if len(address) < 1 { - continue - } - ip, err := netaddr.ParseIP(address) - if err != nil { - return nil, err - } - ips[hostname] = append(ips[hostname], ip) - } - } - - return ips, nil -} - func getAPIURLs( tailscales map[string]dockertest.Resource, ) (map[netaddr.IP]string, error) { From 758b1ba1cbcd13fcff10954a301b7773f5921f20 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sat, 5 Mar 2022 16:22:02 +0100 Subject: [PATCH 13/37] Renamed configuration items of the DERP server --- app.go | 50 ++++++++++++++++++++++++++------------ cmd/headscale/cli/utils.go | 9 +++---- config-example.yaml | 8 ++++++ 3 files changed, 47 insertions(+), 20 deletions(-) diff --git a/app.go b/app.go index eb3b8ade..b739a052 100644 --- a/app.go +++ b/app.go @@ -13,6 +13,7 @@ import ( "os" "os/signal" "sort" + "strconv" "strings" "sync" "syscall" @@ -120,7 +121,8 @@ type OIDCConfig struct { } type DERPConfig struct { - EmbeddedDERP bool + ServerEnabled bool + ServerInsecure bool URLs []url.URL Paths []string AutoUpdate bool @@ -143,8 +145,8 @@ type Headscale struct { dbDebug bool privateKey *key.MachinePrivate - DERPMap *tailcfg.DERPMap - EmbeddedDERPServer *EmbeddedDERPServer + DERPMap *tailcfg.DERPMap + DERPServer *DERPServer aclPolicy *ACLPolicy aclRules []tailcfg.FilterRule @@ -180,7 +182,6 @@ func LookupTLSClientAuthMode(mode string) (tls.ClientAuthType, bool) { } } -// NewHeadscale returns the Headscale app. func NewHeadscale(cfg Config) (*Headscale, error) { privKey, err := readOrCreatePrivateKey(cfg.PrivateKeyPath) if err != nil { @@ -241,30 +242,49 @@ func NewHeadscale(cfg Config) (*Headscale, error) { } } - if cfg.DERP.EmbeddedDERP { - embeddedDERPServer, err := app.NewEmbeddedDERPServer() + if cfg.DERP.ServerEnabled { + embeddedDERPServer, err := app.NewDERPServer() if err != nil { return nil, err } - app.EmbeddedDERPServer = embeddedDERPServer + app.DERPServer = embeddedDERPServer - // If we are using the embedded DERP, there is no reason to use Tailscale's DERP infrastructure serverURL, err := url.Parse(app.cfg.ServerURL) if err != nil { return nil, err } + var host string + var port int + host, portStr, err := net.SplitHostPort(serverURL.Host) + if err != nil { + if serverURL.Scheme == "https" { + host = serverURL.Host + port = 443 + } else { + host = serverURL.Host + port = 80 + } + } else { + port, err = strconv.Atoi(portStr) + if err != nil { + return nil, err + } + } + app.DERPMap = &tailcfg.DERPMap{ Regions: map[int]*tailcfg.DERPRegion{ - 1: { - RegionID: 1, + 999: { + RegionID: 999, RegionCode: "headscale", RegionName: "Headscale Embedded DERP", Avoid: false, Nodes: []*tailcfg.DERPNode{ { - Name: "1a", - RegionID: 1, - HostName: serverURL.Host, + Name: "999a", + RegionID: 999, + HostName: host, + DERPPort: port, + InsecureForTests: cfg.DERP.ServerInsecure, }, }, }, @@ -495,7 +515,7 @@ func (h *Headscale) createRouter(grpcMux *runtime.ServeMux) *gin.Engine { router.GET("/swagger", SwaggerUI) router.GET("/swagger/v1/openapiv2.json", SwaggerAPIv1) - if h.cfg.DERP.EmbeddedDERP { + if h.cfg.DERP.ServerEnabled { router.Any("/derp", h.EmbeddedDERPHandler) router.Any("/derp/probe", h.EmbeddedDERPProbeHandler) router.Any("/bootstrap-dns", h.EmbeddedDERPBootstrapDNSHandler) @@ -516,7 +536,7 @@ func (h *Headscale) createRouter(grpcMux *runtime.ServeMux) *gin.Engine { func (h *Headscale) Serve() error { var err error - if h.cfg.DERP.EmbeddedDERP { + if h.cfg.DERP.ServerEnabled { go h.ServeSTUN() } else { // Fetch an initial DERP Map before we start serving diff --git a/cmd/headscale/cli/utils.go b/cmd/headscale/cli/utils.go index 98ffe2ec..06b9ca98 100644 --- a/cmd/headscale/cli/utils.go +++ b/cmd/headscale/cli/utils.go @@ -117,11 +117,8 @@ func LoadConfig(path string) error { } func GetDERPConfig() headscale.DERPConfig { - if viper.GetBool("derp.embedded_derp") { - return headscale.DERPConfig{ - EmbeddedDERP: true, - } - } + enabled := viper.GetBool("derp.server.enabled") + insecure := viper.GetBool("derp.server.insecure") urlStrs := viper.GetStringSlice("derp.urls") @@ -144,6 +141,8 @@ func GetDERPConfig() headscale.DERPConfig { updateFrequency := viper.GetDuration("derp.update_frequency") return headscale.DERPConfig{ + ServerEnabled: enabled, + ServerInsecure: insecure, URLs: urls, Paths: paths, AutoUpdate: autoUpdate, diff --git a/config-example.yaml b/config-example.yaml index c28b6089..84b1c906 100644 --- a/config-example.yaml +++ b/config-example.yaml @@ -55,6 +55,14 @@ ip_prefixes: # headscale needs a list of DERP servers that can be presented # to the clients. derp: + server: + # If enabled, runs the embedded DERP server and merges it into the rest of the DERP config + enabled: false + + # Insecure mode is recommended only for tests. It indicates the tailscale clients + # to use insecure connections to this server. + insecure: false + # List of externally available DERP maps encoded in JSON urls: - https://controlplane.tailscale.com/derpmap/default From df37d1a639b81e7b9305bba3ef941d38a64604a1 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sat, 5 Mar 2022 19:19:21 +0100 Subject: [PATCH 14/37] Do not offer the option to be DERP insecure Websockets, in which DERP is based, requires a TLS certificate. At the same time, if we use a certificate it must be valid... otherwise Tailscale wont connect (does not have an Insecure option). So there is no option to expose insecure here --- app.go | 16 +++++++--------- cmd/headscale/cli/utils.go | 2 -- config-example.yaml | 5 +---- 3 files changed, 8 insertions(+), 15 deletions(-) diff --git a/app.go b/app.go index b739a052..34602d60 100644 --- a/app.go +++ b/app.go @@ -122,7 +122,6 @@ type OIDCConfig struct { type DERPConfig struct { ServerEnabled bool - ServerInsecure bool URLs []url.URL Paths []string AutoUpdate bool @@ -280,11 +279,10 @@ func NewHeadscale(cfg Config) (*Headscale, error) { Avoid: false, Nodes: []*tailcfg.DERPNode{ { - Name: "999a", - RegionID: 999, - HostName: host, - DERPPort: port, - InsecureForTests: cfg.DERP.ServerInsecure, + Name: "999a", + RegionID: 999, + HostName: host, + DERPPort: port, }, }, }, @@ -516,9 +514,9 @@ func (h *Headscale) createRouter(grpcMux *runtime.ServeMux) *gin.Engine { router.GET("/swagger/v1/openapiv2.json", SwaggerAPIv1) if h.cfg.DERP.ServerEnabled { - router.Any("/derp", h.EmbeddedDERPHandler) - router.Any("/derp/probe", h.EmbeddedDERPProbeHandler) - router.Any("/bootstrap-dns", h.EmbeddedDERPBootstrapDNSHandler) + router.Any("/derp", h.DERPHandler) + router.Any("/derp/probe", h.DERPProbeHandler) + router.Any("/bootstrap-dns", h.DERPBootstrapDNSHandler) } api := router.Group("/api") diff --git a/cmd/headscale/cli/utils.go b/cmd/headscale/cli/utils.go index 06b9ca98..7277723f 100644 --- a/cmd/headscale/cli/utils.go +++ b/cmd/headscale/cli/utils.go @@ -118,7 +118,6 @@ func LoadConfig(path string) error { func GetDERPConfig() headscale.DERPConfig { enabled := viper.GetBool("derp.server.enabled") - insecure := viper.GetBool("derp.server.insecure") urlStrs := viper.GetStringSlice("derp.urls") @@ -142,7 +141,6 @@ func GetDERPConfig() headscale.DERPConfig { return headscale.DERPConfig{ ServerEnabled: enabled, - ServerInsecure: insecure, URLs: urls, Paths: paths, AutoUpdate: autoUpdate, diff --git a/config-example.yaml b/config-example.yaml index 84b1c906..08cc6c12 100644 --- a/config-example.yaml +++ b/config-example.yaml @@ -57,12 +57,9 @@ ip_prefixes: derp: server: # If enabled, runs the embedded DERP server and merges it into the rest of the DERP config + # The Headscale server_url defined above MUST be using https, DERP requires TLS to be in place enabled: false - # Insecure mode is recommended only for tests. It indicates the tailscale clients - # to use insecure connections to this server. - insecure: false - # List of externally available DERP maps encoded in JSON urls: - https://controlplane.tailscale.com/derpmap/default From b7423796278aa4036bca3e64d587801d8a1fd0eb Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sat, 5 Mar 2022 19:30:30 +0100 Subject: [PATCH 15/37] Do not use the term embedded --- derp_embedded.go | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/derp_embedded.go b/derp_embedded.go index 1d4fb0b5..d8abbc81 100644 --- a/derp_embedded.go +++ b/derp_embedded.go @@ -28,17 +28,18 @@ var ( bootstrapDNS = "derp.tailscale.com" ) -type EmbeddedDERPServer struct { +type DERPServer struct { tailscaleDERP *derp.Server } -func (h *Headscale) NewEmbeddedDERPServer() (*EmbeddedDERPServer, error) { +func (h *Headscale) NewDERPServer() (*DERPServer, error) { s := derp.NewServer(key.NodePrivate(*h.privateKey), log.Info().Msgf) - return &EmbeddedDERPServer{s}, nil + return &DERPServer{s}, nil } -func (h *Headscale) EmbeddedDERPHandler(ctx *gin.Context) { +func (h *Headscale) DERPHandler(ctx *gin.Context) { + log.Trace().Caller().Msgf("/derp request from %v", ctx.ClientIP()) up := strings.ToLower(ctx.Request.Header.Get("Upgrade")) if up != "websocket" && up != "derp" { if up != "" { @@ -75,12 +76,12 @@ func (h *Headscale) EmbeddedDERPHandler(ctx *gin.Context) { pubKey.UntypedHexString()) } - h.EmbeddedDERPServer.tailscaleDERP.Accept(netConn, conn, netConn.RemoteAddr().String()) + h.DERPServer.tailscaleDERP.Accept(netConn, conn, netConn.RemoteAddr().String()) } -// EmbeddedDERPProbeHandler is the endpoint that js/wasm clients hit to measure +// DERPProbeHandler is the endpoint that js/wasm clients hit to measure // DERP latency, since they can't do UDP STUN queries. -func (h *Headscale) EmbeddedDERPProbeHandler(ctx *gin.Context) { +func (h *Headscale) DERPProbeHandler(ctx *gin.Context) { switch ctx.Request.Method { case "HEAD", "GET": ctx.Writer.Header().Set("Access-Control-Allow-Origin", "*") @@ -89,7 +90,7 @@ func (h *Headscale) EmbeddedDERPProbeHandler(ctx *gin.Context) { } } -func (h *Headscale) EmbeddedDERPBootstrapDNSHandler(ctx *gin.Context) { +func (h *Headscale) DERPBootstrapDNSHandler(ctx *gin.Context) { ctx.Header("Content-Type", "application/json") j, _ := dnsCache.Load().([]byte) // Bootstrap DNS requests occur cross-regions, @@ -105,7 +106,7 @@ func (h *Headscale) ServeSTUN() { if err != nil { log.Fatal().Msgf("failed to open STUN listener: %v", err) } - log.Printf("running STUN server on %v", pc.LocalAddr()) + log.Trace().Msgf("STUN server started at %s", pc.LocalAddr()) serverSTUNListener(context.Background(), pc.(*net.UDPConn)) } @@ -122,10 +123,11 @@ func serverSTUNListener(ctx context.Context, pc *net.UDPConn) { if ctx.Err() != nil { return } - log.Printf("STUN ReadFrom: %v", err) + log.Error().Caller().Err(err).Msgf("STUN ReadFrom") time.Sleep(time.Second) continue } + log.Trace().Caller().Msgf("STUN request from %v", ua) pkt := buf[:n] if !stun.Is(pkt) { continue @@ -164,7 +166,7 @@ func refreshBootstrapDNS() { for _, name := range names { addrs, err := r.LookupIP(ctx, "ip", name) if err != nil { - log.Printf("bootstrap DNS lookup %q: %v", name, err) + log.Trace().Caller().Err(err).Msgf("bootstrap DNS lookup %q: %v", name) continue } dnsEntries[name] = addrs From 88378c22fb41383273f43c9d84aba1e44271b1f6 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sat, 5 Mar 2022 19:31:50 +0100 Subject: [PATCH 16/37] Rename the file to derp_server.go for coherence --- derp_embedded.go => derp_server.go | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename derp_embedded.go => derp_server.go (100%) diff --git a/derp_embedded.go b/derp_server.go similarity index 100% rename from derp_embedded.go rename to derp_server.go From e9eb90fa7691f1f45b0ad60180dd87ae59374ae7 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sat, 5 Mar 2022 19:34:06 +0100 Subject: [PATCH 17/37] Added integration tests for the embedded DERP server --- .dockerignore | 1 + Dockerfile.tailscale | 7 +- Makefile | 3 + integration_embedded_derp_test.go | 384 ++++++++++++++++++ .../etc_embedded_derp/tls/server.crt | 18 + 5 files changed, 412 insertions(+), 1 deletion(-) create mode 100644 integration_embedded_derp_test.go create mode 100644 integration_test/etc_embedded_derp/tls/server.crt diff --git a/.dockerignore b/.dockerignore index 057a20e7..e3acf996 100644 --- a/.dockerignore +++ b/.dockerignore @@ -3,6 +3,7 @@ // development integration_test.go integration_test/ +!integration_test/etc_embedded_derp/tls/server.crt Dockerfile* docker-compose* diff --git a/Dockerfile.tailscale b/Dockerfile.tailscale index f96c6b9c..fded8375 100644 --- a/Dockerfile.tailscale +++ b/Dockerfile.tailscale @@ -7,5 +7,10 @@ RUN apt-get update \ && curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/focal.gpg | apt-key add - \ && curl -fsSL https://pkgs.tailscale.com/stable/ubuntu/focal.list | tee /etc/apt/sources.list.d/tailscale.list \ && apt-get update \ - && apt-get install -y tailscale=${TAILSCALE_VERSION} dnsutils \ + && apt-get install -y ca-certificates tailscale=${TAILSCALE_VERSION} dnsutils \ && rm -rf /var/lib/apt/lists/* + +ADD integration_test/etc_embedded_derp/tls/server.crt /usr/local/share/ca-certificates/ +RUN chmod 644 /usr/local/share/ca-certificates/server.crt + +RUN update-ca-certificates \ No newline at end of file diff --git a/Makefile b/Makefile index 266dadb8..73630d3f 100644 --- a/Makefile +++ b/Makefile @@ -23,6 +23,9 @@ test_integration: test_integration_cli: go test -tags integration -v integration_cli_test.go integration_common_test.go +test_integration_derp: + go test -tags integration -v integration_embedded_derp_test.go integration_common_test.go + coverprofile_func: go tool cover -func=coverage.out diff --git a/integration_embedded_derp_test.go b/integration_embedded_derp_test.go new file mode 100644 index 00000000..d95c460b --- /dev/null +++ b/integration_embedded_derp_test.go @@ -0,0 +1,384 @@ +//go:build integration + +package headscale + +import ( + "bytes" + "context" + "crypto/tls" + "encoding/json" + "fmt" + "io/ioutil" + "log" + "net/http" + "os" + "path" + "strings" + "sync" + "testing" + "time" + + v1 "github.com/juanfont/headscale/gen/go/headscale/v1" + "github.com/ory/dockertest/v3" + "github.com/ory/dockertest/v3/docker" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/suite" +) + +const ( + headscaleHostname = "headscale-derp" + namespaceName = "derpnamespace" + totalContainers = 3 +) + +type IntegrationDERPTestSuite struct { + suite.Suite + stats *suite.SuiteInformation + + pool dockertest.Pool + networks map[int]dockertest.Network // so we keep the containers isolated + headscale dockertest.Resource + + tailscales map[string]dockertest.Resource + joinWaitGroup sync.WaitGroup +} + +func TestDERPIntegrationTestSuite(t *testing.T) { + s := new(IntegrationDERPTestSuite) + + s.tailscales = make(map[string]dockertest.Resource) + s.networks = make(map[int]dockertest.Network) + + suite.Run(t, s) + + // HandleStats, which allows us to check if we passed and save logs + // is called after TearDown, so we cannot tear down containers before + // we have potentially saved the logs. + for _, tailscale := range s.tailscales { + if err := s.pool.Purge(&tailscale); err != nil { + log.Printf("Could not purge resource: %s\n", err) + } + } + + if !s.stats.Passed() { + err := s.saveLog(&s.headscale, "test_output") + if err != nil { + log.Printf("Could not save log: %s\n", err) + } + } + if err := s.pool.Purge(&s.headscale); err != nil { + log.Printf("Could not purge resource: %s\n", err) + } + + for _, network := range s.networks { + if err := network.Close(); err != nil { + log.Printf("Could not close network: %s\n", err) + } + } +} + +func (s *IntegrationDERPTestSuite) SetupSuite() { + if ppool, err := dockertest.NewPool(""); err == nil { + s.pool = *ppool + } else { + log.Fatalf("Could not connect to docker: %s", err) + } + + for i := 0; i < totalContainers; i++ { + if pnetwork, err := s.pool.CreateNetwork(fmt.Sprintf("headscale-derp-%d", i)); err == nil { + s.networks[i] = *pnetwork + } else { + log.Fatalf("Could not create network: %s", err) + } + } + + headscaleBuildOptions := &dockertest.BuildOptions{ + Dockerfile: "Dockerfile", + ContextDir: ".", + } + + currentPath, err := os.Getwd() + if err != nil { + log.Fatalf("Could not determine current path: %s", err) + } + + headscaleOptions := &dockertest.RunOptions{ + Name: headscaleHostname, + Mounts: []string{ + fmt.Sprintf("%s/integration_test/etc_embedded_derp:/etc/headscale", currentPath), + }, + Cmd: []string{"headscale", "serve"}, + ExposedPorts: []string{"8443/tcp", "3478/udp"}, + PortBindings: map[docker.Port][]docker.PortBinding{ + "8443/tcp": {{HostPort: "8443"}}, + "3478/udp": {{HostPort: "3478"}}, + }, + } + + log.Println("Creating headscale container") + if pheadscale, err := s.pool.BuildAndRunWithBuildOptions(headscaleBuildOptions, headscaleOptions, DockerRestartPolicy); err == nil { + s.headscale = *pheadscale + } else { + log.Fatalf("Could not start resource: %s", err) + } + log.Println("Created headscale container to test DERP") + + log.Println("Creating tailscale containers") + + for i := 0; i < totalContainers; i++ { + version := tailscaleVersions[i%len(tailscaleVersions)] + hostname, container := s.tailscaleContainer( + fmt.Sprint(i), + version, + s.networks[i], + ) + s.tailscales[hostname] = *container + } + + log.Println("Waiting for headscale to be ready") + hostEndpoint := fmt.Sprintf("localhost:%s", s.headscale.GetPort("8443/tcp")) + + if err := s.pool.Retry(func() error { + url := fmt.Sprintf("https://%s/health", hostEndpoint) + insecureTransport := http.DefaultTransport.(*http.Transport).Clone() + insecureTransport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true} + client := &http.Client{Transport: insecureTransport} + resp, err := client.Get(url) + if err != nil { + return err + } + + if resp.StatusCode != http.StatusOK { + return fmt.Errorf("status code not OK") + } + + return nil + }); err != nil { + // TODO(kradalby): If we cannot access headscale, or any other fatal error during + // test setup, we need to abort and tear down. However, testify does not seem to + // support that at the moment: + // https://github.com/stretchr/testify/issues/849 + return // fmt.Errorf("Could not connect to headscale: %s", err) + } + log.Println("headscale container is ready") + + log.Printf("Creating headscale namespace: %s\n", namespaceName) + result, err := ExecuteCommand( + &s.headscale, + []string{"headscale", "namespaces", "create", namespaceName}, + []string{}, + ) + log.Println("headscale create namespace result: ", result) + assert.Nil(s.T(), err) + + log.Printf("Creating pre auth key for %s\n", namespaceName) + preAuthResult, err := ExecuteCommand( + &s.headscale, + []string{ + "headscale", + "--namespace", + namespaceName, + "preauthkeys", + "create", + "--reusable", + "--expiration", + "24h", + "--output", + "json", + }, + []string{"LOG_LEVEL=error"}, + ) + assert.Nil(s.T(), err) + + var preAuthKey v1.PreAuthKey + err = json.Unmarshal([]byte(preAuthResult), &preAuthKey) + assert.Nil(s.T(), err) + assert.True(s.T(), preAuthKey.Reusable) + + headscaleEndpoint := fmt.Sprintf("https://headscale:%s", s.headscale.GetPort("8443/tcp")) + + log.Printf( + "Joining tailscale containers to headscale at %s\n", + headscaleEndpoint, + ) + for hostname, tailscale := range s.tailscales { + s.joinWaitGroup.Add(1) + go s.Join(headscaleEndpoint, preAuthKey.Key, hostname, tailscale) + } + + s.joinWaitGroup.Wait() + + // The nodes need a bit of time to get their updated maps from headscale + // TODO: See if we can have a more deterministic wait here. + time.Sleep(60 * time.Second) +} + +func (s *IntegrationDERPTestSuite) Join( + endpoint, key, hostname string, + tailscale dockertest.Resource, +) { + defer s.joinWaitGroup.Done() + + command := []string{ + "tailscale", + "up", + "-login-server", + endpoint, + "--authkey", + key, + "--hostname", + hostname, + } + + log.Println("Join command:", command) + log.Printf("Running join command for %s\n", hostname) + _, err := ExecuteCommand( + &tailscale, + command, + []string{}, + ) + assert.Nil(s.T(), err) + log.Printf("%s joined\n", hostname) +} + +func (s *IntegrationDERPTestSuite) tailscaleContainer(identifier, version string, network dockertest.Network, +) (string, *dockertest.Resource) { + tailscaleBuildOptions := &dockertest.BuildOptions{ + Dockerfile: "Dockerfile.tailscale", + ContextDir: ".", + BuildArgs: []docker.BuildArg{ + { + Name: "TAILSCALE_VERSION", + Value: version, + }, + }, + } + hostname := fmt.Sprintf( + "tailscale-%s-%s", + strings.Replace(version, ".", "-", -1), + identifier, + ) + tailscaleOptions := &dockertest.RunOptions{ + Name: hostname, + Networks: []*dockertest.Network{&network}, + Cmd: []string{ + "tailscaled", "--tun=tsdev", + }, + + // expose the host IP address, so we can access it from inside the container + ExtraHosts: []string{"host.docker.internal:host-gateway", "headscale:host-gateway"}, + } + + pts, err := s.pool.BuildAndRunWithBuildOptions( + tailscaleBuildOptions, + tailscaleOptions, + DockerRestartPolicy, + DockerAllowLocalIPv6, + DockerAllowNetworkAdministration, + ) + if err != nil { + log.Fatalf("Could not start resource: %s", err) + } + log.Printf("Created %s container\n", hostname) + + return hostname, pts +} + +func (s *IntegrationDERPTestSuite) TearDownSuite() { +} + +func (s *IntegrationDERPTestSuite) HandleStats( + suiteName string, + stats *suite.SuiteInformation, +) { + s.stats = stats +} + +func (s *IntegrationDERPTestSuite) saveLog( + resource *dockertest.Resource, + basePath string, +) error { + err := os.MkdirAll(basePath, os.ModePerm) + if err != nil { + return err + } + + var stdout bytes.Buffer + var stderr bytes.Buffer + + err = s.pool.Client.Logs( + docker.LogsOptions{ + Context: context.TODO(), + Container: resource.Container.ID, + OutputStream: &stdout, + ErrorStream: &stderr, + Tail: "all", + RawTerminal: false, + Stdout: true, + Stderr: true, + Follow: false, + Timestamps: false, + }, + ) + if err != nil { + return err + } + + log.Printf("Saving logs for %s to %s\n", resource.Container.Name, basePath) + + err = ioutil.WriteFile( + path.Join(basePath, resource.Container.Name+".stdout.log"), + []byte(stdout.String()), + 0o644, + ) + if err != nil { + return err + } + + err = ioutil.WriteFile( + path.Join(basePath, resource.Container.Name+".stderr.log"), + []byte(stdout.String()), + 0o644, + ) + if err != nil { + return err + } + + return nil +} + +func (s *IntegrationDERPTestSuite) TestPingAllPeersByHostname() { + ips, err := getIPs(s.tailscales) + assert.Nil(s.T(), err) + for hostname, tailscale := range s.tailscales { + for peername := range ips { + if peername == hostname { + continue + } + s.T().Run(fmt.Sprintf("%s-%s", hostname, peername), func(t *testing.T) { + command := []string{ + "tailscale", "ping", + "--timeout=10s", + "--c=5", + "--until-direct=false", + peername, + } + + log.Printf( + "Pinging using hostname from %s to %s\n", + hostname, + peername, + ) + log.Println(command) + result, err := ExecuteCommand( + &tailscale, + command, + []string{}, + ) + assert.Nil(t, err) + log.Printf("Result for %s: %s\n", hostname, result) + assert.Contains(t, result, "via DERP") + }) + } + } +} diff --git a/integration_test/etc_embedded_derp/tls/server.crt b/integration_test/etc_embedded_derp/tls/server.crt new file mode 100644 index 00000000..48953883 --- /dev/null +++ b/integration_test/etc_embedded_derp/tls/server.crt @@ -0,0 +1,18 @@ +-----BEGIN CERTIFICATE----- +MIIC8jCCAdqgAwIBAgIULbu+UbSTMG/LtxooLLh7BgSEyqEwDQYJKoZIhvcNAQEL +BQAwFDESMBAGA1UEAwwJaGVhZHNjYWxlMCAXDTIyMDMwNTE2NDgwM1oYDzI1MjEx +MTA0MTY0ODAzWjAUMRIwEAYDVQQDDAloZWFkc2NhbGUwggEiMA0GCSqGSIb3DQEB +AQUAA4IBDwAwggEKAoIBAQDqcfpToLZUF0rlNwXkkt3lbyw4Cl4TJdx36o2PKaOK +U+tze/IjRsCWeMwrcR1o9TNZcxsD+c2J48D1WATuQJlMeg+2UJXGaTGRKkkbPMy3 +5m7AFf/Q16UEOgm2NYjZaQ8faRGIMYURG/6sXmNeETJvBixpBev9yKJuVXgqHNS4 +NpEkNwdOCuAZXrmw0HCbiusawJOay4tFvhH14rav8Uimonl8UTNVXufMzyUOuoaQ +TGflmzYX3hIoswRnTPlIWFoqObvx2Q8H+of3uQJXy0m8I6OrIoXLNxnqYMfFls79 +9SYgVc2jPsCbh5fwyRbx2Hof7sIZ1K/mNgxJRG1E3ZiLAgMBAAGjOjA4MBQGA1Ud +EQQNMAuCCWhlYWRzY2FsZTALBgNVHQ8EBAMCB4AwEwYDVR0lBAwwCgYIKwYBBQUH +AwEwDQYJKoZIhvcNAQELBQADggEBANGlVN7NCsJaKz0k0nhlRGK+tcxn2p1PXN/i +Iy+JX8ahixPC4ocRwOhrXgb390ZXLLwq08HrWYRB/Wi1VUzCp5d8dVxvrR43dJ+v +L2EOBiIKgcu2C3pWW1qRR46/EoXUU9kSH2VNBvIhNufi32kEOidoDzxtQf6qVCoF +guUt1JkAqrynv1UvR/2ZRM/WzM/oJ8qfECwrwDxyYhkqU5Z5jCWg0C6kPIBvNdzt +B0eheWS+ZxVwkePTR4e17kIafwknth3lo+orxVrq/xC+OVM1bGrt2ZyD64ZvEqQl +w6kgbzBdLScAQptWOFThwhnJsg0UbYKimZsnYmjVEuN59TJv92M= +-----END CERTIFICATE----- From 992efbd84adc8adbb3259476983876f87b552cc8 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sat, 5 Mar 2022 19:35:15 +0100 Subject: [PATCH 18/37] Added missing private TLS key --- .../etc_embedded_derp/tls/server.key | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 integration_test/etc_embedded_derp/tls/server.key diff --git a/integration_test/etc_embedded_derp/tls/server.key b/integration_test/etc_embedded_derp/tls/server.key new file mode 100644 index 00000000..8a2df34b --- /dev/null +++ b/integration_test/etc_embedded_derp/tls/server.key @@ -0,0 +1,28 @@ +-----BEGIN PRIVATE KEY----- +MIIEwAIBADANBgkqhkiG9w0BAQEFAASCBKowggSmAgEAAoIBAQDqcfpToLZUF0rl +NwXkkt3lbyw4Cl4TJdx36o2PKaOKU+tze/IjRsCWeMwrcR1o9TNZcxsD+c2J48D1 +WATuQJlMeg+2UJXGaTGRKkkbPMy35m7AFf/Q16UEOgm2NYjZaQ8faRGIMYURG/6s +XmNeETJvBixpBev9yKJuVXgqHNS4NpEkNwdOCuAZXrmw0HCbiusawJOay4tFvhH1 +4rav8Uimonl8UTNVXufMzyUOuoaQTGflmzYX3hIoswRnTPlIWFoqObvx2Q8H+of3 +uQJXy0m8I6OrIoXLNxnqYMfFls799SYgVc2jPsCbh5fwyRbx2Hof7sIZ1K/mNgxJ +RG1E3ZiLAgMBAAECggEBALu1Ni/u5Qy++YA8ZcN0s6UXNdhItLmv/q0kZuLQ+9et +CT8VZfFInLndTdsaXenDKLHdryunviFA8SV+q7P2lMbek+Xs735EiyMnMBFWxLIZ +FWNGOeQERGL19QCmLEOmEi2b+iWJQHlKaMWpbPXL3w11a+lKjIBNO4ALfoJ5QveZ +cGMKsJdm/mpqBvLeNeh2eAFk3Gp6sT1g80Ge8NkgyzFBNIqnut0eerM15kPTc6Qz +12JLaOXUuV3PrcB4PN4nOwrTDg88GDNOQtc1Pc9r4nOHyLfr8X7QEtj1wXSwmOuK +d6ynMnAmoxVA9wEnupLbil1bzohRzpsTpkmDruYaBEECgYEA/Z09I8D6mt2NVqIE +KyvLjBK39ijSV9r3/lvB2Ple2OOL5YQEd+yTrIFy+3zdUnDgD1zmNnXjmjvHZ9Lc +IFf2o06AF84QLNB5gLPdDQkGNFdDqUxljBrfAfE3oANmPS/B0SijMGOOOiDO2FtO +xl1nfRr78mswuRs9awoUWCdNRKUCgYEA7KaTYKIQW/FEjw9lshp74q5vbn6zoXF5 +7N8VkwI+bBVNvRbM9XZ8qhfgRdu9eXs5oL/N4mSYY54I8fA//pJ0Z2vpmureMm1V +mL5WBUmSD9DIbAchoK+sRiQhVmNMBQC6cHMABA7RfXvBeGvWrm9pKCS6ZLgLjkjp +PsmAcaXQcW8CgYEA2inAxljjOwUK6FNGsrxhxIT1qtNC3kCGxE+6WSNq67gSR8Vg +8qiX//T7LEslOB3RIGYRwxd2St7RkgZZRZllmOWWWuPwFhzf6E7RAL2akLvggGov +kG4tGEagSw2hjVDfsUT73ExHtMk0Jfmlsg33UC8+PDLpHtLH6qQpDAwC8+ECgYEA +o+AqOIWhvHmT11l7O915Ip1WzvZwYADbxLsrDnVEUsZh4epTHjvh0kvcY6PqTqCV +ZIrOANNWb811Nkz/k8NJVoD08PFp0xPBbZeIq/qpachTsfMyRzq/mobUiyUR9Hjv +ooUQYr78NOApNsG+lWbTNBhS9wI4BlzZIECbcJe5g4MCgYEAndRoy8S+S0Hx/S8a +O3hzXeDmivmgWqn8NVD4AKOovpkz4PaIVVQbAQkiNfAx8/DavPvjEKAbDezJ4ECV +j7IsOWtDVI7pd6eF9fTcECwisrda8aUoiOap8AQb48153Vx+g2N4Vy3uH0xJs4cz +TDALZPOBg8VlV+HEFDP43sp9Bf0= +-----END PRIVATE KEY----- From e78c002f5a0c4e941d6a7b822b3b926f5c6922b4 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sat, 5 Mar 2022 19:48:30 +0100 Subject: [PATCH 19/37] Fix minor issue --- derp_server.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/derp_server.go b/derp_server.go index d8abbc81..e9009860 100644 --- a/derp_server.go +++ b/derp_server.go @@ -166,7 +166,7 @@ func refreshBootstrapDNS() { for _, name := range names { addrs, err := r.LookupIP(ctx, "ip", name) if err != nil { - log.Trace().Caller().Err(err).Msgf("bootstrap DNS lookup %q: %v", name) + log.Trace().Caller().Err(err).Msgf("bootstrap DNS lookup %q", name) continue } dnsEntries[name] = addrs From 54c3e00a1ffb89d2b96d988f8fe21b3280a577f6 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sat, 5 Mar 2022 20:04:31 +0100 Subject: [PATCH 20/37] Merge local DERP server region with other configured DERP sources --- app.go | 60 ++++++++------------------------------------------ derp.go | 1 + derp_server.go | 49 ++++++++++++++++++++++++++++++++++++++++- 3 files changed, 58 insertions(+), 52 deletions(-) diff --git a/app.go b/app.go index 4568558d..8cb987a9 100644 --- a/app.go +++ b/app.go @@ -13,7 +13,6 @@ import ( "os" "os/signal" "sort" - "strconv" "strings" "sync" "syscall" @@ -247,48 +246,6 @@ func NewHeadscale(cfg Config) (*Headscale, error) { return nil, err } app.DERPServer = embeddedDERPServer - - serverURL, err := url.Parse(app.cfg.ServerURL) - if err != nil { - return nil, err - } - var host string - var port int - host, portStr, err := net.SplitHostPort(serverURL.Host) - if err != nil { - if serverURL.Scheme == "https" { - host = serverURL.Host - port = 443 - } else { - host = serverURL.Host - port = 80 - } - } else { - port, err = strconv.Atoi(portStr) - if err != nil { - return nil, err - } - } - - app.DERPMap = &tailcfg.DERPMap{ - Regions: map[int]*tailcfg.DERPRegion{ - 999: { - RegionID: 999, - RegionCode: "headscale", - RegionName: "Headscale Embedded DERP", - Avoid: false, - Nodes: []*tailcfg.DERPNode{ - { - Name: "999a", - RegionID: 999, - HostName: host, - DERPPort: port, - }, - }, - }, - }, - OmitDefaultRegions: false, - } } return &app, nil @@ -536,17 +493,18 @@ func (h *Headscale) createRouter(grpcMux *runtime.ServeMux) *gin.Engine { func (h *Headscale) Serve() error { var err error + // Fetch an initial DERP Map before we start serving + h.DERPMap = GetDERPMap(h.cfg.DERP) + if h.cfg.DERP.ServerEnabled { go h.ServeSTUN() - } else { - // Fetch an initial DERP Map before we start serving - h.DERPMap = GetDERPMap(h.cfg.DERP) + h.DERPMap.Regions[h.DERPServer.region.RegionID] = &h.DERPServer.region + } - if h.cfg.DERP.AutoUpdate { - derpMapCancelChannel := make(chan struct{}) - defer func() { derpMapCancelChannel <- struct{}{} }() - go h.scheduledDERPMapUpdateWorker(derpMapCancelChannel) - } + if h.cfg.DERP.AutoUpdate { + derpMapCancelChannel := make(chan struct{}) + defer func() { derpMapCancelChannel <- struct{}{} }() + go h.scheduledDERPMapUpdateWorker(derpMapCancelChannel) } go h.expireEphemeralNodes(updateInterval) diff --git a/derp.go b/derp.go index 63e448db..7a9b2367 100644 --- a/derp.go +++ b/derp.go @@ -148,6 +148,7 @@ func (h *Headscale) scheduledDERPMapUpdateWorker(cancelChan <-chan struct{}) { case <-ticker.C: log.Info().Msg("Fetching DERPMap updates") h.DERPMap = GetDERPMap(h.cfg.DERP) + h.DERPMap.Regions[h.DERPServer.region.RegionID] = &h.DERPServer.region namespaces, err := h.ListNamespaces() if err != nil { diff --git a/derp_server.go b/derp_server.go index e9009860..81cb1e29 100644 --- a/derp_server.go +++ b/derp_server.go @@ -6,6 +6,8 @@ import ( "fmt" "net" "net/http" + "net/url" + "strconv" "strings" "sync/atomic" "time" @@ -14,6 +16,7 @@ import ( "github.com/rs/zerolog/log" "tailscale.com/derp" "tailscale.com/net/stun" + "tailscale.com/tailcfg" "tailscale.com/types/key" ) @@ -30,12 +33,56 @@ var ( type DERPServer struct { tailscaleDERP *derp.Server + region tailcfg.DERPRegion } func (h *Headscale) NewDERPServer() (*DERPServer, error) { s := derp.NewServer(key.NodePrivate(*h.privateKey), log.Info().Msgf) - return &DERPServer{s}, nil + region, err := h.generateRegionLocalDERP() + if err != nil { + return nil, err + } + return &DERPServer{s, region}, nil +} +func (h *Headscale) generateRegionLocalDERP() (tailcfg.DERPRegion, error) { + serverURL, err := url.Parse(h.cfg.ServerURL) + if err != nil { + return tailcfg.DERPRegion{}, err + } + var host string + var port int + host, portStr, err := net.SplitHostPort(serverURL.Host) + if err != nil { + if serverURL.Scheme == "https" { + host = serverURL.Host + port = 443 + } else { + host = serverURL.Host + port = 80 + } + } else { + port, err = strconv.Atoi(portStr) + if err != nil { + return tailcfg.DERPRegion{}, err + } + } + + localDERPregion := tailcfg.DERPRegion{ + RegionID: 999, + RegionCode: "headscale", + RegionName: "Headscale Embedded DERP", + Avoid: false, + Nodes: []*tailcfg.DERPNode{ + { + Name: "999a", + RegionID: 999, + HostName: host, + DERPPort: port, + }, + }, + } + return localDERPregion, nil } func (h *Headscale) DERPHandler(ctx *gin.Context) { From 70910c459548304d00b5520525455910920dd353 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sun, 6 Mar 2022 01:23:35 +0100 Subject: [PATCH 21/37] Working /bootstrap-dns DERP helper --- derp_server.go | 73 +++++++++++++++----------------------------------- 1 file changed, 22 insertions(+), 51 deletions(-) diff --git a/derp_server.go b/derp_server.go index 81cb1e29..dcda63e9 100644 --- a/derp_server.go +++ b/derp_server.go @@ -2,14 +2,12 @@ package headscale import ( "context" - "encoding/json" "fmt" "net" "net/http" "net/url" "strconv" "strings" - "sync/atomic" "time" "github.com/gin-gonic/gin" @@ -26,11 +24,6 @@ import ( // following its HTTP request. const fastStartHeader = "Derp-Fast-Start" -var ( - dnsCache atomic.Value // of []byte - bootstrapDNS = "derp.tailscale.com" -) - type DERPServer struct { tailscaleDERP *derp.Server region tailcfg.DERPRegion @@ -137,14 +130,29 @@ func (h *Headscale) DERPProbeHandler(ctx *gin.Context) { } } +// DERPBootstrapDNSHandler implements the /bootsrap-dns endpoint +// Described in https://github.com/tailscale/tailscale/issues/1405, +// this endpoint provides a way to help a client when it fails to start up +// because its DNS are broken. +// The initial implementation is here https://github.com/tailscale/tailscale/pull/1406 +// They have a cache, but not clear if that is really necessary at Headscale, uh, scale. func (h *Headscale) DERPBootstrapDNSHandler(ctx *gin.Context) { - ctx.Header("Content-Type", "application/json") - j, _ := dnsCache.Load().([]byte) - // Bootstrap DNS requests occur cross-regions, - // and are randomized per request, - // so keeping a connection open is pointlessly expensive. - ctx.Header("Connection", "close") - ctx.Writer.Write(j) + dnsEntries := make(map[string][]net.IP) + + resolvCtx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + var r net.Resolver + for _, region := range h.DERPMap.Regions { + for _, node := range region.Nodes { // we don't care if we override some nodes + addrs, err := r.LookupIP(resolvCtx, "ip", node.HostName) + if err != nil { + log.Trace().Caller().Err(err).Msgf("bootstrap DNS lookup failed %q", node.HostName) + continue + } + dnsEntries[node.HostName] = addrs + } + } + ctx.JSON(http.StatusOK, dnsEntries) } // ServeSTUN starts a STUN server on udp/3478 @@ -188,40 +196,3 @@ func serverSTUNListener(ctx context.Context, pc *net.UDPConn) { pc.WriteTo(res, ua) } } - -// Shamelessly taken from -// https://github.com/tailscale/tailscale/blob/main/cmd/derper/bootstrap_dns.go -func refreshBootstrapDNSLoop() { - if bootstrapDNS == "" { - return - } - for { - refreshBootstrapDNS() - time.Sleep(10 * time.Minute) - } -} - -func refreshBootstrapDNS() { - if bootstrapDNS == "" { - return - } - dnsEntries := make(map[string][]net.IP) - ctx, cancel := context.WithTimeout(context.Background(), time.Minute) - defer cancel() - names := strings.Split(bootstrapDNS, ",") - var r net.Resolver - for _, name := range names { - addrs, err := r.LookupIP(ctx, "ip", name) - if err != nil { - log.Trace().Caller().Err(err).Msgf("bootstrap DNS lookup %q", name) - continue - } - dnsEntries[name] = addrs - } - j, err := json.MarshalIndent(dnsEntries, "", "\t") - if err != nil { - // leave the old values in place - return - } - dnsCache.Store(j) -} From dc909ba6d7ce48e0ed3d6f6415f301ef2653a903 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sun, 6 Mar 2022 16:54:19 +0100 Subject: [PATCH 22/37] Improved logging on startup --- cmd/headscale/cli/server.go | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/cmd/headscale/cli/server.go b/cmd/headscale/cli/server.go index 6d9ad194..b741f295 100644 --- a/cmd/headscale/cli/server.go +++ b/cmd/headscale/cli/server.go @@ -1,7 +1,7 @@ package cli import ( - "log" + "github.com/rs/zerolog/log" "github.com/spf13/cobra" ) @@ -19,12 +19,12 @@ var serveCmd = &cobra.Command{ Run: func(cmd *cobra.Command, args []string) { h, err := getHeadscaleApp() if err != nil { - log.Fatalf("Error initializing: %s", err) + log.Fatal().Caller().Err(err).Msg("Error initializing") } err = h.Serve() if err != nil { - log.Fatalf("Error initializing: %s", err) + log.Fatal().Caller().Err(err).Msg("Error starting server") } }, } From eb500155e84847774d79b8454c0327e37c23c0ff Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sun, 6 Mar 2022 17:00:56 +0100 Subject: [PATCH 23/37] Make STUN server configurable --- app.go | 6 +++++- cmd/headscale/cli/utils.go | 4 ++++ config-example.yaml | 6 ++++++ derp_server.go | 22 ++++++++++++++++++---- 4 files changed, 33 insertions(+), 5 deletions(-) diff --git a/app.go b/app.go index 8cb987a9..82e87cf0 100644 --- a/app.go +++ b/app.go @@ -121,6 +121,8 @@ type OIDCConfig struct { type DERPConfig struct { ServerEnabled bool + STUNEnabled bool + STUNAddr string URLs []url.URL Paths []string AutoUpdate bool @@ -497,8 +499,10 @@ func (h *Headscale) Serve() error { h.DERPMap = GetDERPMap(h.cfg.DERP) if h.cfg.DERP.ServerEnabled { - go h.ServeSTUN() h.DERPMap.Regions[h.DERPServer.region.RegionID] = &h.DERPServer.region + if h.cfg.DERP.STUNEnabled { + go h.ServeSTUN() + } } if h.cfg.DERP.AutoUpdate { diff --git a/cmd/headscale/cli/utils.go b/cmd/headscale/cli/utils.go index 7277723f..e6dce3a1 100644 --- a/cmd/headscale/cli/utils.go +++ b/cmd/headscale/cli/utils.go @@ -118,6 +118,8 @@ func LoadConfig(path string) error { func GetDERPConfig() headscale.DERPConfig { enabled := viper.GetBool("derp.server.enabled") + stunEnabled := viper.GetBool("derp.server.stun.enabled") + stunAddr := viper.GetString("derp.server.stun.listen_addr") urlStrs := viper.GetStringSlice("derp.urls") @@ -141,6 +143,8 @@ func GetDERPConfig() headscale.DERPConfig { return headscale.DERPConfig{ ServerEnabled: enabled, + STUNEnabled: stunEnabled, + STUNAddr: stunAddr, URLs: urls, Paths: paths, AutoUpdate: autoUpdate, diff --git a/config-example.yaml b/config-example.yaml index 6f4060ac..57b43fd4 100644 --- a/config-example.yaml +++ b/config-example.yaml @@ -60,6 +60,12 @@ derp: # The Headscale server_url defined above MUST be using https, DERP requires TLS to be in place enabled: false + # If enabled, also listens in the configured address for STUN connections to help on NAT traversal + # For more details on how this works, check this great article: https://tailscale.com/blog/how-tailscale-works/ + stun: + enabled: false + listen_addr: "0.0.0.0:3478" + # List of externally available DERP maps encoded in JSON urls: - https://controlplane.tailscale.com/derpmap/default diff --git a/derp_server.go b/derp_server.go index dcda63e9..aeb4877a 100644 --- a/derp_server.go +++ b/derp_server.go @@ -75,6 +75,19 @@ func (h *Headscale) generateRegionLocalDERP() (tailcfg.DERPRegion, error) { }, }, } + + if h.cfg.DERP.STUNEnabled { + _, portStr, err := net.SplitHostPort(h.cfg.DERP.STUNAddr) + if err != nil { + return tailcfg.DERPRegion{}, err + } + port, err := strconv.Atoi(portStr) + if err != nil { + return tailcfg.DERPRegion{}, err + } + localDERPregion.Nodes[0].STUNPort = port + } + return localDERPregion, nil } @@ -136,6 +149,7 @@ func (h *Headscale) DERPProbeHandler(ctx *gin.Context) { // because its DNS are broken. // The initial implementation is here https://github.com/tailscale/tailscale/pull/1406 // They have a cache, but not clear if that is really necessary at Headscale, uh, scale. +// An example implementation is found here https://derp.tailscale.com/bootstrap-dns func (h *Headscale) DERPBootstrapDNSHandler(ctx *gin.Context) { dnsEntries := make(map[string][]net.IP) @@ -155,14 +169,14 @@ func (h *Headscale) DERPBootstrapDNSHandler(ctx *gin.Context) { ctx.JSON(http.StatusOK, dnsEntries) } -// ServeSTUN starts a STUN server on udp/3478 +// ServeSTUN starts a STUN server on the configured addr func (h *Headscale) ServeSTUN() { - pc, err := net.ListenPacket("udp", "0.0.0.0:3478") + packetConn, err := net.ListenPacket("udp", h.cfg.DERP.STUNAddr) if err != nil { log.Fatal().Msgf("failed to open STUN listener: %v", err) } - log.Trace().Msgf("STUN server started at %s", pc.LocalAddr()) - serverSTUNListener(context.Background(), pc.(*net.UDPConn)) + log.Info().Msgf("STUN server started at %s", packetConn.LocalAddr()) + serverSTUNListener(context.Background(), packetConn.(*net.UDPConn)) } func serverSTUNListener(ctx context.Context, pc *net.UDPConn) { From eb06054a7b1e9d429b69d1c205ea81058468f60d Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sun, 6 Mar 2022 17:25:21 +0100 Subject: [PATCH 24/37] Make DERP Region configurable --- app.go | 17 ++++++++++------- cmd/headscale/cli/utils.go | 22 ++++++++++++++-------- config-example.yaml | 9 +++++++++ derp_server.go | 14 ++++++++------ 4 files changed, 41 insertions(+), 21 deletions(-) diff --git a/app.go b/app.go index 82e87cf0..f1426bbb 100644 --- a/app.go +++ b/app.go @@ -120,13 +120,16 @@ type OIDCConfig struct { } type DERPConfig struct { - ServerEnabled bool - STUNEnabled bool - STUNAddr string - URLs []url.URL - Paths []string - AutoUpdate bool - UpdateFrequency time.Duration + ServerEnabled bool + ServerRegionID int + ServerRegionCode string + ServerRegionName string + STUNEnabled bool + STUNAddr string + URLs []url.URL + Paths []string + AutoUpdate bool + UpdateFrequency time.Duration } type CLIConfig struct { diff --git a/cmd/headscale/cli/utils.go b/cmd/headscale/cli/utils.go index e6dce3a1..dc7a4e9f 100644 --- a/cmd/headscale/cli/utils.go +++ b/cmd/headscale/cli/utils.go @@ -117,7 +117,10 @@ func LoadConfig(path string) error { } func GetDERPConfig() headscale.DERPConfig { - enabled := viper.GetBool("derp.server.enabled") + serverEnabled := viper.GetBool("derp.server.enabled") + serverRegionID := viper.GetInt("derp.server.region_id") + serverRegionCode := viper.GetString("derp.server.region_code") + serverRegionName := viper.GetString("derp.server.region_name") stunEnabled := viper.GetBool("derp.server.stun.enabled") stunAddr := viper.GetString("derp.server.stun.listen_addr") @@ -142,13 +145,16 @@ func GetDERPConfig() headscale.DERPConfig { updateFrequency := viper.GetDuration("derp.update_frequency") return headscale.DERPConfig{ - ServerEnabled: enabled, - STUNEnabled: stunEnabled, - STUNAddr: stunAddr, - URLs: urls, - Paths: paths, - AutoUpdate: autoUpdate, - UpdateFrequency: updateFrequency, + ServerEnabled: serverEnabled, + ServerRegionID: serverRegionID, + ServerRegionCode: serverRegionCode, + ServerRegionName: serverRegionName, + STUNEnabled: stunEnabled, + STUNAddr: stunAddr, + URLs: urls, + Paths: paths, + AutoUpdate: autoUpdate, + UpdateFrequency: updateFrequency, } } diff --git a/config-example.yaml b/config-example.yaml index 57b43fd4..1ab92dca 100644 --- a/config-example.yaml +++ b/config-example.yaml @@ -60,6 +60,15 @@ derp: # The Headscale server_url defined above MUST be using https, DERP requires TLS to be in place enabled: false + # Region ID to use for the embedded DERP server. + # The local DERP prevails if the region ID collides with other region ID coming from + # the regular DERP config. + region_id: 999 + + # Region code and name are displayed in the Tailscale UI to identify a DERP region + region_code: "headscale" + region_name: "Headscale Embedded DERP" + # If enabled, also listens in the configured address for STUN connections to help on NAT traversal # For more details on how this works, check this great article: https://tailscale.com/blog/how-tailscale-works/ stun: diff --git a/derp_server.go b/derp_server.go index aeb4877a..8995ca8b 100644 --- a/derp_server.go +++ b/derp_server.go @@ -62,14 +62,14 @@ func (h *Headscale) generateRegionLocalDERP() (tailcfg.DERPRegion, error) { } localDERPregion := tailcfg.DERPRegion{ - RegionID: 999, - RegionCode: "headscale", - RegionName: "Headscale Embedded DERP", + RegionID: h.cfg.DERP.ServerRegionID, + RegionCode: h.cfg.DERP.ServerRegionCode, + RegionName: h.cfg.DERP.ServerRegionName, Avoid: false, Nodes: []*tailcfg.DERPNode{ { - Name: "999a", - RegionID: 999, + Name: fmt.Sprintf("%d", h.cfg.DERP.ServerRegionID), + RegionID: h.cfg.DERP.ServerRegionID, HostName: host, DERPPort: port, }, @@ -108,6 +108,7 @@ func (h *Headscale) DERPHandler(ctx *gin.Context) { if !ok { log.Error().Caller().Msg("DERP requires Hijacker interface from Gin") ctx.String(http.StatusInternalServerError, "HTTP does not support general TCP support") + return } @@ -115,6 +116,7 @@ func (h *Headscale) DERPHandler(ctx *gin.Context) { if err != nil { log.Error().Caller().Err(err).Msgf("Hijack failed") ctx.String(http.StatusInternalServerError, "HTTP does not support general TCP support") + return } @@ -169,7 +171,7 @@ func (h *Headscale) DERPBootstrapDNSHandler(ctx *gin.Context) { ctx.JSON(http.StatusOK, dnsEntries) } -// ServeSTUN starts a STUN server on the configured addr +// ServeSTUN starts a STUN server on the configured addr. func (h *Headscale) ServeSTUN() { packetConn, err := net.ListenPacket("udp", h.cfg.DERP.STUNAddr) if err != nil { From de2ea83b3b87c0d043f8dec45d788159722198f2 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sun, 6 Mar 2022 17:35:54 +0100 Subject: [PATCH 25/37] Linting here and there --- acls.go | 11 +++++------ cmd/headscale/cli/server.go | 1 - derp_server.go | 39 ++++++++++++++++++++++++------------- 3 files changed, 31 insertions(+), 20 deletions(-) diff --git a/acls.go b/acls.go index 24aadf5b..95484806 100644 --- a/acls.go +++ b/acls.go @@ -17,12 +17,11 @@ import ( ) const ( - errEmptyPolicy = Error("empty policy") - errInvalidAction = Error("invalid action") - errInvalidUserSection = Error("invalid user section") - errInvalidGroup = Error("invalid group") - errInvalidTag = Error("invalid tag") - errInvalidPortFormat = Error("invalid port format") + errEmptyPolicy = Error("empty policy") + errInvalidAction = Error("invalid action") + errInvalidGroup = Error("invalid group") + errInvalidTag = Error("invalid tag") + errInvalidPortFormat = Error("invalid port format") ) const ( diff --git a/cmd/headscale/cli/server.go b/cmd/headscale/cli/server.go index b741f295..c19580b9 100644 --- a/cmd/headscale/cli/server.go +++ b/cmd/headscale/cli/server.go @@ -2,7 +2,6 @@ package cli import ( "github.com/rs/zerolog/log" - "github.com/spf13/cobra" ) diff --git a/derp_server.go b/derp_server.go index 8995ca8b..9e1b7e54 100644 --- a/derp_server.go +++ b/derp_server.go @@ -30,12 +30,13 @@ type DERPServer struct { } func (h *Headscale) NewDERPServer() (*DERPServer, error) { - s := derp.NewServer(key.NodePrivate(*h.privateKey), log.Info().Msgf) + server := derp.NewServer(key.NodePrivate(*h.privateKey), log.Info().Msgf) region, err := h.generateRegionLocalDERP() if err != nil { return nil, err } - return &DERPServer{s, region}, nil + + return &DERPServer{server, region}, nil } func (h *Headscale) generateRegionLocalDERP() (tailcfg.DERPRegion, error) { @@ -99,6 +100,7 @@ func (h *Headscale) DERPHandler(ctx *gin.Context) { log.Warn().Caller().Msgf("Weird websockets connection upgrade: %q", up) } ctx.String(http.StatusUpgradeRequired, "DERP requires connection upgrade") + return } @@ -122,13 +124,14 @@ func (h *Headscale) DERPHandler(ctx *gin.Context) { if !fastStart { pubKey := h.privateKey.Public() + pubKeyStr := pubKey.UntypedHexString() // nolint fmt.Fprintf(conn, "HTTP/1.1 101 Switching Protocols\r\n"+ "Upgrade: DERP\r\n"+ "Connection: Upgrade\r\n"+ "Derp-Version: %v\r\n"+ "Derp-Public-Key: %s\r\n\r\n", derp.ProtocolVersion, - pubKey.UntypedHexString()) + pubKeyStr) } h.DERPServer.tailscaleDERP.Accept(netConn, conn, netConn.RemoteAddr().String()) @@ -163,6 +166,7 @@ func (h *Headscale) DERPBootstrapDNSHandler(ctx *gin.Context) { addrs, err := r.LookupIP(resolvCtx, "ip", node.HostName) if err != nil { log.Trace().Caller().Err(err).Msgf("bootstrap DNS lookup failed %q", node.HostName) + continue } dnsEntries[node.HostName] = addrs @@ -178,28 +182,34 @@ func (h *Headscale) ServeSTUN() { log.Fatal().Msgf("failed to open STUN listener: %v", err) } log.Info().Msgf("STUN server started at %s", packetConn.LocalAddr()) - serverSTUNListener(context.Background(), packetConn.(*net.UDPConn)) + + udpConn, ok := packetConn.(*net.UDPConn) + if !ok { + log.Fatal().Msg("STUN listener is not a UDP listener") + } + serverSTUNListener(context.Background(), udpConn) } -func serverSTUNListener(ctx context.Context, pc *net.UDPConn) { +func serverSTUNListener(ctx context.Context, packetConn *net.UDPConn) { var buf [64 << 10]byte var ( - n int - ua *net.UDPAddr - err error + bytesRead int + udpAddr *net.UDPAddr + err error ) for { - n, ua, err = pc.ReadFromUDP(buf[:]) + bytesRead, udpAddr, err = packetConn.ReadFromUDP(buf[:]) if err != nil { if ctx.Err() != nil { return } log.Error().Caller().Err(err).Msgf("STUN ReadFrom") time.Sleep(time.Second) + continue } - log.Trace().Caller().Msgf("STUN request from %v", ua) - pkt := buf[:n] + log.Trace().Caller().Msgf("STUN request from %v", udpAddr) + pkt := buf[:bytesRead] if !stun.Is(pkt) { continue } @@ -208,7 +218,10 @@ func serverSTUNListener(ctx context.Context, pc *net.UDPConn) { continue } - res := stun.Response(txid, ua.IP, uint16(ua.Port)) - pc.WriteTo(res, ua) + res := stun.Response(txid, udpAddr.IP, uint16(udpAddr.Port)) + _, err = packetConn.WriteTo(res, udpAddr) + if err != nil { + continue + } } } From e1fcf0da262244ccde3f6c9117acf50bd326c532 Mon Sep 17 00:00:00 2001 From: Juan Font Date: Sun, 6 Mar 2022 20:40:55 +0100 Subject: [PATCH 26/37] Added more version Co-authored-by: Kristoffer Dalby --- integration_common_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/integration_common_test.go b/integration_common_test.go index a3417121..70285fc4 100644 --- a/integration_common_test.go +++ b/integration_common_test.go @@ -20,7 +20,7 @@ var ( IpPrefix4 = netaddr.MustParseIPPrefix("100.64.0.0/10") IpPrefix6 = netaddr.MustParseIPPrefix("fd7a:115c:a1e0::/48") - tailscaleVersions = []string{"1.20.4", "1.18.2", "1.16.2", "1.14.3", "1.12.3"} + tailscaleVersions = []string{"1.22.0", "1.20.4", "1.18.2", "1.16.2", "1.14.3", "1.12.3"} ) type TestNamespace struct { From b47de07eea7bf0ec637f24bb5ad0c1aad9ab2961 Mon Sep 17 00:00:00 2001 From: Juan Font Date: Sun, 6 Mar 2022 20:42:27 +0100 Subject: [PATCH 27/37] Update Dockerfile.tailscale Co-authored-by: Kristoffer Dalby --- Dockerfile.tailscale | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Dockerfile.tailscale b/Dockerfile.tailscale index fded8375..32a8ce7b 100644 --- a/Dockerfile.tailscale +++ b/Dockerfile.tailscale @@ -13,4 +13,4 @@ RUN apt-get update \ ADD integration_test/etc_embedded_derp/tls/server.crt /usr/local/share/ca-certificates/ RUN chmod 644 /usr/local/share/ca-certificates/server.crt -RUN update-ca-certificates \ No newline at end of file +RUN update-ca-certificates From 580db9b58fe786fc23d1506ab3907dc8d4f5235d Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sun, 6 Mar 2022 23:19:21 +0100 Subject: [PATCH 28/37] Mention that STUN is UDP --- config-example.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/config-example.yaml b/config-example.yaml index 1ab92dca..2075e69a 100644 --- a/config-example.yaml +++ b/config-example.yaml @@ -69,7 +69,7 @@ derp: region_code: "headscale" region_name: "Headscale Embedded DERP" - # If enabled, also listens in the configured address for STUN connections to help on NAT traversal + # If enabled, also listens in UDP at the configured address for STUN connections to help on NAT traversal # For more details on how this works, check this great article: https://tailscale.com/blog/how-tailscale-works/ stun: enabled: false From a27b386123a815f9841b7109e01349e55f91c723 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sun, 6 Mar 2022 23:45:01 +0100 Subject: [PATCH 29/37] Clarified expiration dates --- integration_test/etc_embedded_derp/tls/server.crt | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/integration_test/etc_embedded_derp/tls/server.crt b/integration_test/etc_embedded_derp/tls/server.crt index 48953883..95556495 100644 --- a/integration_test/etc_embedded_derp/tls/server.crt +++ b/integration_test/etc_embedded_derp/tls/server.crt @@ -1,3 +1,4 @@ + -----BEGIN CERTIFICATE----- MIIC8jCCAdqgAwIBAgIULbu+UbSTMG/LtxooLLh7BgSEyqEwDQYJKoZIhvcNAQEL BQAwFDESMBAGA1UEAwwJaGVhZHNjYWxlMCAXDTIyMDMwNTE2NDgwM1oYDzI1MjEx @@ -16,3 +17,6 @@ guUt1JkAqrynv1UvR/2ZRM/WzM/oJ8qfECwrwDxyYhkqU5Z5jCWg0C6kPIBvNdzt B0eheWS+ZxVwkePTR4e17kIafwknth3lo+orxVrq/xC+OVM1bGrt2ZyD64ZvEqQl w6kgbzBdLScAQptWOFThwhnJsg0UbYKimZsnYmjVEuN59TJv92M= -----END CERTIFICATE----- + +(Expires on Nov 4 16:48:03 2521 GMT) + From b3fa66dbd2aebf4faf048bab563e1a5dad952ad7 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sun, 6 Mar 2022 23:46:16 +0100 Subject: [PATCH 30/37] Check for DERP in test --- integration_embedded_derp_test.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/integration_embedded_derp_test.go b/integration_embedded_derp_test.go index d95c460b..e68da013 100644 --- a/integration_embedded_derp_test.go +++ b/integration_embedded_derp_test.go @@ -377,7 +377,7 @@ func (s *IntegrationDERPTestSuite) TestPingAllPeersByHostname() { ) assert.Nil(t, err) log.Printf("Result for %s: %s\n", hostname, result) - assert.Contains(t, result, "via DERP") + assert.Contains(t, result, "via DERP(headscale)") }) } } From 05df8e947aa61a87d9e49086a0b2d44fb47d3ccd Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Sun, 6 Mar 2022 23:47:14 +0100 Subject: [PATCH 31/37] Added missing file --- .../etc_embedded_derp/config.yaml | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100644 integration_test/etc_embedded_derp/config.yaml diff --git a/integration_test/etc_embedded_derp/config.yaml b/integration_test/etc_embedded_derp/config.yaml new file mode 100644 index 00000000..00f63325 --- /dev/null +++ b/integration_test/etc_embedded_derp/config.yaml @@ -0,0 +1,30 @@ +log_level: trace +acl_policy_path: "" +db_type: sqlite3 +ephemeral_node_inactivity_timeout: 30m +ip_prefixes: + - fd7a:115c:a1e0::/48 + - 100.64.0.0/10 +dns_config: + base_domain: headscale.net + magic_dns: true + domains: [] + nameservers: + - 1.1.1.1 +db_path: /tmp/integration_test_db.sqlite3 +private_key_path: private.key +listen_addr: 0.0.0.0:8443 +server_url: https://headscale:8443 +tls_cert_path: "/etc/headscale/tls/server.crt" +tls_key_path: "/etc/headscale/tls/server.key" +tls_client_auth_mode: disabled + +derp: + server: + enabled: true + region_id: 999 + region_code: "headscale" + region_name: "Headscale Embedded DERP" + stun: + enabled: true + listen_addr: "0.0.0.0:3478" \ No newline at end of file From 03452a8dca9a4b6d956cd319b8d07c216bcb64db Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Mon, 7 Mar 2022 00:29:40 +0100 Subject: [PATCH 32/37] Prettied --- integration_test/etc_embedded_derp/config.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/integration_test/etc_embedded_derp/config.yaml b/integration_test/etc_embedded_derp/config.yaml index 00f63325..6e5291fa 100644 --- a/integration_test/etc_embedded_derp/config.yaml +++ b/integration_test/etc_embedded_derp/config.yaml @@ -18,7 +18,6 @@ server_url: https://headscale:8443 tls_cert_path: "/etc/headscale/tls/server.crt" tls_key_path: "/etc/headscale/tls/server.key" tls_client_auth_mode: disabled - derp: server: enabled: true From c06689dec15cf2642b30c38e7f661aafe8e02889 Mon Sep 17 00:00:00 2001 From: e-zk Date: Tue, 8 Mar 2022 18:34:46 +1000 Subject: [PATCH 33/37] fix: make register html/template consistent with other html - makes the html/template for /register follow the same formatting as /apple and /windows - adds a element - minor change for consistency's sake --- api.go | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/api.go b/api.go index b5e885a1..1023e6fc 100644 --- a/api.go +++ b/api.go @@ -45,22 +45,21 @@ type registerWebAPITemplateConfig struct { } var registerWebAPITemplate = template.Must( - template.New("registerweb").Parse(`<html> + template.New("registerweb").Parse(` +<html> + <head> + <title>Registration - Headscale + -

headscale

-

- Run the command below in the headscale server to add this machine to your network: -

- -

- - headscale -n NAMESPACE nodes register --key {{.Key}} - -

- +

headscale

+

Machine registration

+

+ Run the command below in the headscale server to add this machine to your network: +

+ 
headscale -n NAMESPACE nodes register --key {{.Key}}
- `), -) + +`)) // RegisterWebAPI shows a simple message in the browser to point to the CLI // Listens in /register. From cc0c88a63ab193dc075bf0cac4962025bfccab04 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Tue, 8 Mar 2022 12:11:51 +0100 Subject: [PATCH 34/37] Added small integration test for stun --- derp_server.go | 6 ++++++ go.mod | 1 + go.sum | 2 ++ integration_embedded_derp_test.go | 12 ++++++++++++ 4 files changed, 21 insertions(+) diff --git a/derp_server.go b/derp_server.go index 9e1b7e54..11e3eb14 100644 --- a/derp_server.go +++ b/derp_server.go @@ -211,16 +211,22 @@ func serverSTUNListener(ctx context.Context, packetConn *net.UDPConn) { log.Trace().Caller().Msgf("STUN request from %v", udpAddr) pkt := buf[:bytesRead] if !stun.Is(pkt) { + log.Trace().Caller().Msgf("UDP packet is not STUN") + continue } txid, err := stun.ParseBindingRequest(pkt) if err != nil { + log.Trace().Caller().Err(err).Msgf("STUN parse error") + continue } res := stun.Response(txid, udpAddr.IP, uint16(udpAddr.Port)) _, err = packetConn.WriteTo(res, udpAddr) if err != nil { + log.Trace().Caller().Err(err).Msgf("Issue writing to UDP") + continue } } diff --git a/go.mod b/go.mod index d6754f98..1ec291c3 100644 --- a/go.mod +++ b/go.mod @@ -49,6 +49,7 @@ require ( github.com/akutz/memconn v0.1.0 // indirect github.com/atomicgo/cursor v0.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect + github.com/ccding/go-stun/stun v0.0.0-20200514191101-4dc67bcdb029 // indirect github.com/cenkalti/backoff/v4 v4.1.2 // indirect github.com/cespare/xxhash/v2 v2.1.2 // indirect github.com/containerd/continuity v0.2.2 // indirect diff --git a/go.sum b/go.sum index c23db380..6d254a25 100644 --- a/go.sum +++ b/go.sum @@ -108,6 +108,8 @@ github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kB github.com/bits-and-blooms/bitset v1.2.0/go.mod h1:gIdJ4wp64HaoK2YrL1Q5/N7Y16edYb8uY+O0FJTyyDA= github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJmJgSg28kpZDP6UIiPt0e0Oz0kqKNGyRaWEPv84= github.com/bufbuild/buf v0.37.0/go.mod h1:lQ1m2HkIaGOFba6w/aC3KYBHhKEOESP3gaAEpS3dAFM= +github.com/ccding/go-stun/stun v0.0.0-20200514191101-4dc67bcdb029 h1:POmUHfxXdeyM8Aomg4tKDcwATCFuW+cYLkj6pwsw9pc= +github.com/ccding/go-stun/stun v0.0.0-20200514191101-4dc67bcdb029/go.mod h1:Rpr5n9cGHYdM3S3IK8ROSUUUYjQOu+MSUCZDcJbYWi8= github.com/cenkalti/backoff/v4 v4.1.2 h1:6Yo7N8UP2K6LWZnW94DLVSSrbobcWdVzAYOisuDPIFo= github.com/cenkalti/backoff/v4 v4.1.2/go.mod h1:scbssz8iZGpm3xbr14ovlUdkxfGXNInqkPWOWmG2CLw= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= diff --git a/integration_embedded_derp_test.go b/integration_embedded_derp_test.go index e68da013..a1737173 100644 --- a/integration_embedded_derp_test.go +++ b/integration_embedded_derp_test.go @@ -23,6 +23,8 @@ import ( "github.com/ory/dockertest/v3/docker" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/suite" + + "github.com/ccding/go-stun/stun" ) const ( @@ -382,3 +384,13 @@ func (s *IntegrationDERPTestSuite) TestPingAllPeersByHostname() { } } } + +func (s *IntegrationDERPTestSuite) TestDERPSTUN() { + headscaleSTUNAddr := fmt.Sprintf("localhost:%s", s.headscale.GetPort("3478/udp")) + client := stun.NewClient() + client.SetVerbose(true) + client.SetVVerbose(true) + client.SetServerAddr(headscaleSTUNAddr) + _, _, err := client.Discover() + assert.Nil(s.T(), err) +} From 05c5e2280b5e6fbe801195c3d1f1575d9f63b434 Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Tue, 8 Mar 2022 12:15:05 +0100 Subject: [PATCH 35/37] Updated CHANGELOG and README --- CHANGELOG.md | 1 + README.md | 1 + 2 files changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8535b45f..6ce0c35b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,6 +19,7 @@ - Users can now use emails in ACL's groups [#372](https://github.com/juanfont/headscale/issues/372) - Add shorthand aliases for commands and subcommands [#376](https://github.com/juanfont/headscale/pull/376) - Add `/windows` endpoint for Windows configuration instructions + registry file download [#392](https://github.com/juanfont/headscale/pull/392) +- Added embedded DERP server into Headscale [#388](https://github.com/juanfont/headscale/pull/388) ### Changes diff --git a/README.md b/README.md index 1b97b1aa..ec66b086 100644 --- a/README.md +++ b/README.md @@ -63,6 +63,7 @@ one of the maintainers. - Dual stack (IPv4 and IPv6) - Routing advertising (including exit nodes) - Ephemeral nodes +- Embedded [DERP server](https://tailscale.com/blog/how-tailscale-works/#encrypted-tcp-relays-derp) ## Client OS support From b803240dc1045e16da2bc79337c1bd33f8941bbb Mon Sep 17 00:00:00 2001 From: Juan Font Alonso Date: Tue, 8 Mar 2022 12:21:08 +0100 Subject: [PATCH 36/37] Added new line for prettier --- integration_test/etc_embedded_derp/config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/integration_test/etc_embedded_derp/config.yaml b/integration_test/etc_embedded_derp/config.yaml index 6e5291fa..1531d347 100644 --- a/integration_test/etc_embedded_derp/config.yaml +++ b/integration_test/etc_embedded_derp/config.yaml @@ -26,4 +26,4 @@ derp: region_name: "Headscale Embedded DERP" stun: enabled: true - listen_addr: "0.0.0.0:3478" \ No newline at end of file + listen_addr: "0.0.0.0:3478" From c47fb1ae5445581fa1babe0eeeddfbf41b22b7a1 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 8 Mar 2022 16:50:11 +0000 Subject: [PATCH 37/37] docs(README): update contributors --- README.md | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index ec66b086..d74b19f4 100644 --- a/README.md +++ b/README.md @@ -183,6 +183,15 @@ make build ohdearaugustin + + + e-zk/ +
+ e-zk + + + + Justin @@ -190,8 +199,6 @@ make build Justin Angel - - Alessandro @@ -199,13 +206,6 @@ make build Alessandro (Ale) Segala - - - e-zk/ -
- e-zk - - unreality/ @@ -382,6 +382,13 @@ make build rcursaru + + + WhiteSource +
+ WhiteSource Renovate + + Ryan @@ -403,6 +410,8 @@ make build Tanner + + Teteros/ @@ -410,8 +419,6 @@ make build Teteros - - The @@ -447,6 +454,8 @@ make build ZiYuan + + derelm/ @@ -454,8 +463,6 @@ make build derelm - - ignoramous/