Commit Graph

1676 Commits

Author SHA1 Message Date
Livio Spring
0e99c8356a
fix: correctly check user auth methods and enable button (#8342)
# Which Problems Are Solved

#8291 added backwards compatibilty for users who were created through
the user V2 API and want to sign in to the login UI.
There were however to issues, where users might be prompted to set a
password even if they already had one set or they would not be able to
submit the email verification code.

# How the Problems Are Solved

- Replaced `SearchUserAuthMethods `with `ListUserAuthMethodTypes` to
check for set up auth methods.
- Fixed page / javascript to disable submit button.

# Additional Changes

- Changed `ListActiveUserAuthMethodTypes ` to `ListUserAuthMethodTypes`
and a `activeOnly` boolean parameter

# Additional Context

- relates to #8291
- noticed internally on QA
2024-07-22 14:46:27 +02:00
Zach H
6a97be7bde
fix: always use pgxpool config if MaxOpenConns isn't set (#8328)
# Which Problems Are Solved

- `pgxpool -> pgx` dependency throws "MaxSize must be >= 1" on init if
`postgres.MaxOpenConns` isn't set in the ZItadel config

# How the Problems Are Solved

Only override the `MaxConns` with the Zitadel configured `MaxOpenConns`
if greater than 0 (default value). The default `MaxConns` [is derived by
`pgxpool`](ea9610f672/pgxpool/pool.go (L309-L324))
itself in a sensible way, but somewhat undocumented: checks for explicit
config in connection url or config and falls back on max(num_cpus, 4).

# Additional Changes

Applied same check in cockroach config

# Additional Context

This is likely a regression from the changes in
https://github.com/zitadel/zitadel/pull/8325

(cherry picked from commit e009ed9fe4)
2024-07-22 13:29:09 +02:00
Silvan
93be062d30
refactor(database): exchange connection pool (#8325)
# Which Problems Are Solved

The connection pool of go uses a high amount of database connections.

# How the Problems Are Solved

The standard lib connection pool was replaced by `pgxpool.Pool`

# Additional Changes

The `db.BeginTx`-spans are removed because they cause to much noise in
the traces.

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/7639

(cherry picked from commit 99c645cc60)
2024-07-22 13:28:57 +02:00
Livio Spring
d85af6ad9f
fix: correctly differ between grpc and grpc-web (#8292)
# Which Problems Are Solved

While #8285 also checked for `+proto` and `+json` grpc content types, it
accidentally matched all grpc-web requests to grpc.

# How the Problems Are Solved

- fixed the regex by checking for an exact match (added start `^` and
end `$` anchors)

# Additional Changes

None

# Additional Context

- relates to #8285

(cherry picked from commit d7c0ec282a)
2024-07-22 13:27:07 +02:00
Silvan
7cb16ef982
fix(api): grpc content type matcher all grpc types (#8285)
# Which Problems Are Solved

ZITADEL returned a 404 Unimplemented error if the client sent
'application/grpc+proto' or 'application/grpc+json' which are both valid
content types.

# How the Problems Are Solved

changed the header matcher to regexp

# Additional Context

Problem occured in
https://github.com/zitadel/typescript/tree/grpc-transport

(cherry picked from commit aa273ad000)
2024-07-22 13:27:06 +02:00
Zach H
e009ed9fe4
fix: always use pgxpool config if MaxOpenConns isn't set (#8328)
# Which Problems Are Solved

- `pgxpool -> pgx` dependency throws "MaxSize must be >= 1" on init if
`postgres.MaxOpenConns` isn't set in the ZItadel config

# How the Problems Are Solved

Only override the `MaxConns` with the Zitadel configured `MaxOpenConns`
if greater than 0 (default value). The default `MaxConns` [is derived by
`pgxpool`](ea9610f672/pgxpool/pool.go (L309-L324))
itself in a sensible way, but somewhat undocumented: checks for explicit
config in connection url or config and falls back on max(num_cpus, 4).

# Additional Changes

Applied same check in cockroach config 

# Additional Context

This is likely a regression from the changes in
https://github.com/zitadel/zitadel/pull/8325
2024-07-22 05:29:20 +00:00
Silvan
99c645cc60
refactor(database): exchange connection pool (#8325)
# Which Problems Are Solved

The connection pool of go uses a high amount of database connections.

# How the Problems Are Solved

The standard lib connection pool was replaced by `pgxpool.Pool`

# Additional Changes

The `db.BeginTx`-spans are removed because they cause to much noise in
the traces.

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/7639
2024-07-17 15:16:02 +00:00
Livio Spring
07b2bac463
fix: allow login with user created through v2 api without password (#8291)
# Which Problems Are Solved

User created through the User V2 API without any authentication method
and possibly unverified email address was not able to login through the
current hosted login UI.

An unverified email address would result in a mail verification and not
an initialization mail like it would with the management API. Also the
login UI would then require the user to enter the init code, which the
user never received.

# How the Problems Are Solved

- When verifying the email through the login UI, it will check for
existing auth methods (password, IdP, passkeys). In case there are none,
the user will be prompted to set a password.
- When a user was created through the V2 API with a verified email and
no auth method, the user will be prompted to set a password in the login
UI.
- Since setting a password requires a corresponding code, the code will
be generated and sent when login in.

# Additional Changes

- Changed `RequestSetPassword` to get the codeGenerator from the
eventstore instead of getting it from query.

# Additional Context

- closes https://github.com/zitadel/zitadel/issues/6600
- closes https://github.com/zitadel/zitadel/issues/8235

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-07-17 06:43:07 +02:00
Livio Spring
e126ccc9aa
feat(IDP): use single callback endpoint (#8295)
# Which Problems Are Solved

Both the login UI and the IdP intent flow have their own IdP callback
endpoints.

This makes configuration hard to impossible (e.g. Github only allows one
endpoint) for customers.

# How the Problems Are Solved

- The login UI prefixes the `state` parameter when creating an auth /
SAML request.
- All requests now use the `/idp/callback` or the corresponding
variation (e.g. SAML)
- On callback, the state, resp. its prefix is checked. In case of the
login UI prefix, the request will be forwarded to the existing login UI
handler without the prefix state.
Existing setups will therefore not be affected and also requests started
before this release can be handled without any impact.
- Console only lists the "new" endpoint(s). Any
`/login/externalidp/callback` is removed.

# Additional Changes

- Cleaned up some images  from the IdP documentation.
- fix the error handling in `handleExternalNotFoundOptionCheck`

# Additional Context

- closes #8236
2024-07-16 13:58:22 +00:00
Stefan Benz
19561a092c
fix: add OrgIDQuery to the queries for organizations (#8312)
# Which Problems Are Solved

ListOrgs has no option to select for organizations specific to Ids.

# How the Problems Are Solved

Add OrgIDQuery to ListOrgs.

# Additional Changes

Clean up double mapping for the OrgQueries.

# Additional Context

- noted internally while checking performance issues (in Console)
2024-07-16 13:12:19 +00:00
Norman-Lee
1e3b350042
feat(cmd): Added machine ID mode to zitadel start up logs (#8251)
# Which Problems Are Solved

Logs the type of sonyflake strategy used for generating unique machine
IDs

# How the Problems Are Solved

- Created function to log machine id strategy on the start up logs

# Additional Changes
- Added public function for retrieving current strategy set by
configuration

# Additional Context
- Closes #7750
2024-07-16 09:53:57 +00:00
borisroman
7d0c7e5b54
feat: allow machine user id to be set during creation (#8265)
# Which Problems Are Solved
Solves the problem described in #8264.

# How the Problems Are Solved

Added a UserID field which can be set during Machine User creation.

# Additional Changes

Added addition unit and integration tests to cover the cases where a
UserID field is present.

# Additional Context

- Closes #8264

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-07-16 09:27:37 +02:00
Zhang Zhe
8afdcd99c6
fix: Correct misspelling in zh.yaml file (#8299)
# Which Problems Are Solved

- Corrected a typo in the file
`internal/api/ui/login/static/i18n/zh.yaml` where "Migrosoft" was
changed to "Microsoft".

# How the Problems Are Solved

- Updated the misspelled word "Migrosoft" to "Microsoft" for consistency
and accuracy.

# Additional Changes

- None

# Additional Context

- None
2024-07-15 10:13:55 +02:00
Livio Spring
d7c0ec282a
fix: correctly differ between grpc and grpc-web (#8292)
# Which Problems Are Solved

While #8285 also checked for `+proto` and `+json` grpc content types, it
accidentally matched all grpc-web requests to grpc.

# How the Problems Are Solved

- fixed the regex by checking for an exact match (added start `^` and
end `$` anchors)

# Additional Changes

None

# Additional Context

- relates to #8285
2024-07-12 11:54:02 +02:00
Silvan
aa273ad000
fix(api): grpc content type matcher all grpc types (#8285)
# Which Problems Are Solved

ZITADEL returned a 404 Unimplemented error if the client sent
'application/grpc+proto' or 'application/grpc+json' which are both valid
content types.

# How the Problems Are Solved

changed the header matcher to regexp

# Additional Context

Problem occured in
https://github.com/zitadel/typescript/tree/grpc-transport
2024-07-11 13:22:01 +00:00
Stefan Benz
fb2d4545b9
fix: add details to ListUsers for user results (#8255)
# Which Problems Are Solved

In User v2 API, the ListUsers endpoint doesn't provide the information
to which organization the user belongs to.

# How the Problems Are Solved

Add the details to the user results from the ListUsers endpoint, so that
the OrgID is also included as ResourceOwner.

# Additional Changes
 
None

# Additional Context

Closes #8172
2024-07-10 17:49:35 +02:00
Stefan Benz
19a8ab02ad
fix: add missing totp remove endpoint in user v2 API (#8256)
# Which Problems Are Solved

TOTP remove endpoint available in management API, not in user v2 API.

# How the Problems Are Solved

Add endpoint RemoveTOTP to user v2 API.

# Additional Changes

None

# Additional Context

close #6605

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-07-10 12:31:28 +00:00
adlerhurst
7a552bd495 Merge branch 'main' into next-rc 2024-07-05 10:17:43 +02:00
Silvan
5ca8ad2075
fix(eventstore): early return if no events in field handler (#8254)
# Which Problems Are Solved

Fixes a panic which can occur if there are no events to reduce in the fields handler

# How the Problems Are Solved

Check if there are any events to reduce

# Additional Context

- Panic was added in https://github.com/zitadel/zitadel/pull/8191
2024-07-05 10:16:12 +02:00
Tim Möhlmann
7967e6f98b
perf(import): optimize search for domains claimed by other organizations (#8200)
# Which Problems Are Solved

Improve the performance of human imports by optimizing the query that
finds domains claimed by other organizations.

# How the Problems Are Solved

Use the fields search table introduced in
https://github.com/zitadel/zitadel/pull/8191 by storing each
organization domain as Object ID and the verified status as field value.

# Additional Changes

- Feature flag for this optimization

# Additional Context

- Performance improvements for import are evaluated and acted upon
internally at the moment

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2024-07-05 09:36:00 +02:00
Tim Möhlmann
ecfb9d0d6d
perf(command): user grant pre-condition check using the search table (#8230)
# Which Problems Are Solved

Imporve the performance of user grant addition, especially for import.

# How the Problems Are Solved

Use the search table to query for the project grant state. 
This could easily be done by making the search used in
`checkProjectGrantPreCondition` reusable.

# Additional Changes

Chanded event declerations to `const` in the
`internal/repository/project` package.

# Additional Context

- Performance improvements for import are evaluated and acted upon
internally at the moment

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-07-04 16:18:43 +00:00
Livio Spring
d705cb11b7
fix: error handling to prevent panics (#8248)
# Which Problems Are Solved

We found multiple cases where either the error was not properly handled,
which led to panics.

# How the Problems Are Solved

Handle the errors.

# Additional Changes

None.

# Additional Context

- noticed internally
2024-07-04 14:11:06 +00:00
Elio Bischof
7573e0ea8f
feat: add http_server_return_code_counter metric to gateway (#8173)
# Which Problems Are Solved

The metric `http_server_return_code_counter` doesn't record calls to the
gRPC gateway.

# How the Problems Are Solved

The DefaultMetricsHandler that is used for the gPRC gateway doesn't
record `http_server_return_code_counter`.
Instead of the DefaultMetricsHandler, a custom metrics handler which
includes `http_server_return_code_counter` is created for the gRPC
gateway

# Additional Changes

The DefaultMetricsHandler function is removed, as it is no longer used.

# Additional Context

Reported by a customer

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2024-07-04 09:37:23 +00:00
Brian Tajuddin
32b707cf46
feat(6222): remove @ and project from OIDC client ID (#8178)
# Which Problems Are Solved

The client ID for OIDC applications has an `@` in it, which is not
allowed in some 3rd-party systems (such as AWS).

# How the Problems Are Solved

Per @fforootd and @hifabienne in #6222, remove the project suffix and
the `@` from the client ID and just use the generated ID.

# Additional Changes

N/A

# Additional Context

- Closes #6222

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-07-04 08:31:40 +00:00
Livio Spring
9e72c67f72
Merge branch 'refs/heads/main' into next-rc 2024-07-04 10:26:55 +02:00
RedstonePfalz
02c98f570b
fix: Fixed spelling and grammar misstakes (#8240)
# Which Problems Are Solved

I fixed spelling, grammar and translation misstakes in the german
translation file.

I noticed something else. Some strings use the formal form ("sie") of
address and some strings use the informal (du) form of address.

# Additional Context

- Discussion #8211
2024-07-04 08:02:04 +00:00
Silvan
1d84635836
feat(eventstore): add search table (#8191)
# Which Problems Are Solved

To improve performance a new table and method is implemented on
eventstore. The goal of this table is to index searchable fields on
command side to use it on command and query side.

The table allows to store one primitive value (numeric, text) per row.

The eventstore framework is extended by the `Search`-method which allows
to search for objects.
The `Command`-interface is extended by the `SearchOperations()`-method
which does manipulate the the `search`-table.

# How the Problems Are Solved

This PR adds the capability of improving performance for command and
query side by using the `Search`-method of the eventstore instead of
using one of the `Filter`-methods.

# Open Tasks

- [x] Add feature flag
- [x] Unit tests
- [ ] ~~Benchmarks if needed~~
- [x] Ensure no behavior change
- [x] Add setup step to fill table with current data
- [x] Add projection which ensures data added between setup and start of
the new version are also added to the table

# Additional Changes

The `Search`-method is currently used by `ProjectGrant`-command side.

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/8094
2024-07-03 15:00:56 +00:00
Livio Spring
08a75635d2
fix: correctly set user agent / fingerprint id on user sessions (#8231)
# Which Problems Are Solved

When we switched to V2 tokens (#7822), the user agent was incorrectly
set for sessions created though the login UI.
Additionally, when calling the ListMyUserSessions from the AuthService,
any session without the fingerprint ID (e.g. created through the session
API) would be listed.

# How the Problems Are Solved

- Use the intended ID of the user agent (fingerprint)
- Ignore empty user agent IDs when listing the user sessions

# Additional Changes

None.

# Additional Context

- relates #7822 
- closes #8213
2024-07-03 09:43:34 +02:00
Miguel Cabrerizo
c8e168ac69
fix: remove non-existent colums in getMembershipFromQuery queries (#7869)
In issue #7841 @mahmoodfathy commented an issue when the API call for
Listing My ZITADEL Manager Roles is called with any kind of query
(orgQuery, projectQuery, projectGrantQuery...). A column XXXXXX does not
exist (SQLSTATE 42703) error is thrown.

The issue was focused in getMembershipFromQuery where filtering queries
functions are called: prepareOrgMember, prepareIAMMember,
prepareProjectMember and prepareProjectGrantMember

Those functions allow queries for columns that are not members of the
table to be queried so I've added a conditional clause to avoid using
the queries that cannot be called.

For example, for prepareOrgMember, member.id, member.project_id and
member.grant_id columns are not added to the filter queries

```
for _, q := range query.Queries {
		if q.Col().table.name == membershipAlias.name &&
			!slices.Contains([]string{membershipIAMID.name, membershipProjectID.name, membershipGrantID.name}, q.Col().name) {
			builder = q.toQuery(builder)
		}
	}
	return builder.MustSql()
```

Here I show one screenshot where the error "column XXXXXX does not exist
(SQLSTATE 42703)" is no longer thrown using an orgQuery.


![image](https://github.com/zitadel/zitadel/assets/30386061/77621e69-71df-42de-b3c5-fa9b4dbf1b89)

Should close #7841 

### Definition of Ready

- [X] I am happy with the code
- [X] Short description of the feature/issue is added in the pr
description
- [X] PR is linked to the corresponding user story
- [X] Acceptance criteria are met
- [ ] All open todos and follow ups are defined in a new ticket and
justified
- [ ] Deviations from the acceptance criteria and design are agreed with
the PO and documented.
- [X] No debug or dead code
- [X] My code has no repetitions
- [X] Critical parts are tested automatically
- [ ] Where possible E2E tests are implemented
- [ ] Documentation/examples are up-to-date
- [ ] All non-functional requirements are met
- [X] Functionality of the acceptance criteria is checked manually on
the dev system.

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-07-02 12:08:28 +00:00
Miguel Cabrerizo
cd6b452fc9
fix: add missing SAML type for GetActiveIdentityProviders (#8186)
# Which Problems Are Solved

- When the endpoint http://{CUSTOM-DOMAIN}/v2beta/settings/login/idps is
called the type for an activated SAML provider is not sent.
- The IDENTITY_PROVIDER_TYPE_SAML is missing

# How the Problems Are Solved

- Adds the missing IDENTITY_PROVIDER_TYPE_SAML to the
IdentityProviderType proto definition
- Adds the missing case for idpTypeToPb 
- Adds the missing test case for idpTypeToPb

Here's a screenshot showing the endpoint response:


![image](https://github.com/zitadel/zitadel/assets/30386061/6e3e9c41-543c-472e-96ab-3d40736a2699)

# Additional Context

- Closes #7885

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-06-28 15:00:56 +00:00
Stefan Benz
406fa44b63
Merge remote-tracking branch 'origin/main' into next-rc 2024-06-28 15:50:18 +02:00
Stefan Benz
d92717a1c6
fix: encode ldap values to make valid UTF8 (#8210)
# Which Problems Are Solved

UUIDs stored in LDAP are Octet Strings and have to be parsed, so that
they can be stored as IDs as they are not valid UTF8.

# How the Problems Are Solved

Try to parse the RawValue from LDAP as UUID, otherwise try to base64
decode and then parse as UUID, else use the data as string as before.

# Additional Changes

None

# Additional Context

Closes #7601
2024-06-28 13:46:54 +00:00
Miguel Cabrerizo
14aeb42cc2
fix: RFC1342 encode sender name that goes in from header (#8193)
# Which Problems Are Solved

- Some smtp server/client combination may have problems with non-ASCII
sender names for example using an umlaut

# How the Problems Are Solved

- The same RFC1342 mechanism that was added in
#https://github.com/zitadel/zitadel/pull/6637 and later improved by
@eliobischof with BEncoding has been added to the sender name that goes
in the From header

# Additional Context

- Closes #7976
2024-06-28 08:33:42 +00:00
Livio Spring
12ac53df1a
Merge branch 'main' into next-rc 2024-06-25 13:10:35 -07:00
Livio Spring
1b0e773ceb
fix(ldap): add more logs (#8197)
# Which Problems Are Solved

In case the user bind (user password check for LDAP IdP) fails, there's
no information about what went wrong.
This makes it hard to even impossible to find the cause.

# How the Problems Are Solved

Added logging of the error.

# Additional Changes

Additionally added a log in case no single user (none / multiple) are
found.

# Additional Context

- reported internally
2024-06-25 19:04:10 +00:00
Tim Möhlmann
dc170dc46e
feat(crypto): support md5 plain for imported password hashes (#8189)
# Which Problems Are Solved

Allow verification of imported passwords hashed with plain md5, without
salt. These are password digests typically created by one of:

- `printf "password" | md5sum` on most linux systems.
- PHP's `md5("password")`
- Python3's `hashlib.md5(b"password").hexdigest()`

# How the Problems Are Solved

- Upgrade passwap to
[v0.6.0](https://github.com/zitadel/passwap/releases/tag/v0.6.0)
- Add md5plain as a new verfier option in `defaults.yaml`

# Additional Changes

- Updated documentation to explain difference between `md5` (crypt) and
`md5plain` verifiers.

# Additional Context

- Requested by customer for import case
2024-06-25 11:10:49 +03:00
Livio Spring
9a9753a911
Merge branch 'main' into next
# Conflicts:
#	go.mod
#	internal/query/user_auth_method_test.go
2024-06-21 13:30:08 +02:00
Miguel Cabrerizo
3635320ce8
feat: send test mail (#7968)
# Which Problems Are Solved

- Zitadel doesn't have a way to test SMTP settings either before
creating a new provider or once the SMTP provider has been created.
- Zitadel SMTP messages can be more informative for usual errors

# How the Problems Are Solved

- A new step is added to the new/update SMTP provider wizard that allows
us to test a configuration. The result is shown in a text area.
- From the table of SMTP providers you can test your settings too.
- The email address to send the email is by default the email address
for the logged in user as suggested.
- Some of the SMTP error messages have been changed to give more
information about the possible situation. For example: could not contact
with the SMTP server, check the port, firewall issues... instead of
could not dial

Here's a video showing this new option in action:


https://github.com/zitadel/zitadel/assets/30386061/50128ba1-c9fa-4481-8eec-e79a3ca69bda

# Additional Changes

Replace this example text with a concise list of additional changes that
this PR introduces, that are not directly solving the initial problem
but are related.
For example:
- The docs explicitly describe that the property XY is mandatory
- Adds missing translations for validations.

# Additional Context

- Closes #4504
2024-06-20 19:51:42 +00:00
Livio Spring
00b5e55565
fix: set certificate validity for SAML IdPs (#8170)
# Which Problems Are Solved

Certificates created for a SAML IdP (used for metadata and request
singing) did not have any validity set. While it's not required for
SAML, when trying to import the certificate into a (keychain) tool it
might fail.

# How the Problems Are Solved

The validity is set based on the `CertificateLifetime` set in the
runtime config.

## After the fix:
If an IdP was created with a certificate without validity, an admin can
regenerate the certificate:
- for instance wide IdPs:
https://zitadel.com/docs/apis/resources/admin/admin-service-regenerate-saml-provider-certificate#regenerate-saml-identity-provider-certificate
- for organization specific IdPs:
https://zitadel.com/docs/apis/resources/mgmt/management-service-regenerate-saml-provider-certificate#regenerate-saml-identity-provider-certificate

Due to the new certificate, the metadata will change and will need to be
updated at the external IdP.

# Additional Changes

Additionally the `CertificateSize` instead of the `Size` (used for keys)
is used for generating the certificate, resp. the underlying key pair.

# Additional Context

- noted by a customer
- needs backports

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-06-20 14:09:20 +00:00
Tim Möhlmann
669ac6bda2
perf(import): do not check for existing grant ID (#8164)
# Which Problems Are Solved

Improve the performance of the `admin/v1/import` API endpoint.
Specifaclly the import of large amount of project grants.

# How the Problems Are Solved

`AddProjectGrantWithID` and `AddProjectGrantMember` methods of
`Commands` used to get the current state of the Writemodel to check if
the current GrantID or the combination of GrantID & UserID wasn't
already used. However, the Added events already have protection against
duplication by the `UniqueConstaint` methods.

The queries become very slow when there is a great amount of project
grants. Because all the events are pushed to the aggregate ID of the
project, we had to obtain all related project events, including events
of grantIDs we do not care about. This O(n) duration for bached import
jobs adding many organization granted to a single project.

This change removes the unnecesary state query to improve performance.

# Additional Changes

- Add integration tests for import

# Additional Context

- reported internally
2024-06-20 13:31:58 +00:00
Tim Möhlmann
f9742a58f4
fix(import): add tracing spans to all import related functions (#8160)
# Which Problems Are Solved

This fix adds tracing spans to all V1 API import related functions. This
is to troubleshoot import related performance issues reported to us.

# How the Problems Are Solved

Add a tracing span to `api/grpc/admin/import.go` and all related
functions that are called in the `command` package.

# Additional Changes

- none

# Additional Context

- Reported by internal communication
2024-06-19 12:56:33 +02:00
Livio Spring
fb8cd18f93
feat: password age policy (#8132)
# Which Problems Are Solved

Some organizations / customers have the requirement, that there users
regularly need to change their password.
ZITADEL already had the possibility to manage a `password age policy` (
thought the API) with the maximum amount of days a password should be
valid, resp. days after with the user should be warned of the upcoming
expiration.
The policy could not be managed though the Console UI and was not
checked in the Login UI.

# How the Problems Are Solved

- The policy can be managed in the Console UI's settings sections on an
instance and organization level.
- During an authentication in the Login UI, if a policy is set with an
expiry (>0) and the user's last password change exceeds the amount of
days set, the user will be prompted to change their password.
- The prompt message of the Login UI can be customized in the Custom
Login Texts though the Console and API on the instance and each
organization.
- The information when the user last changed their password is returned
in the Auth, Management and User V2 API.
- The policy can be retrieved in the settings service as `password
expiry settings`.

# Additional Changes

None.

# Additional Context

- closes #8081

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-06-18 11:27:44 +00:00
Stefan Benz
04fc59f538
fix: empty custom text changes push no events (#8054)
# Which Problems Are Solved

If there is no custom text given, the call ends in an internal error as
no events have to be pushed.

# How the Problems Are Solved

If no events have to be pushed, no trying to push an empty list of
events.

# Additional Changes

No additional changes.

# Additional Context

Closes #6954
2024-06-17 12:47:38 +00:00
Tim Möhlmann
1aa8c49e41
fix(oidc): store requested response_mode (#8145)
# Which Problems Are Solved

Zitadel never stored or returned the requested `response_mode` in oidc
Auth Requests. This caused the oidc library to fallback to the default
based on the response_type.

# How the Problems Are Solved

- Store the `response_mode` in the Auth request repo
- Store the `response_mode` in the Auth request v2 events
- Return the `resonse_mode` from the Auth Request v1 and v2
`ResponseMode()` methods. (Was hard-coded to an empty string)

# Additional Changes

- Populate the `response_modes_supported` to the oidc Discovery
Configuration. When it was empty, the standard specifies the default of
`query` and `fragment`. However, our oidc library also supports
`form_post` and by this fix, zitadel now also supports this.

# Additional Context

- Closes #6586
- Reported
https://discord.com/channels/927474939156643850/1151508313717084220

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-06-17 09:50:12 +00:00
Livio Spring
85d7536d44
fix(oauth): ensure client error is prioritized over token error (#8133)
# Which Problems Are Solved

Introduced with #6909, the authentication check (API client) and the
token verification on the introspection endpoint where parallelized to
improve performance. Only the first error would be considered and
returned (and the second completely ignored).
This could lead to situations where both the client authentication and
token verification failed and the response would result in a 200 OK with
`active: false`.

# How the Problems Are Solved

- The client authentication check error will always be prioritized.
- An error in the token check will no longer terminate the client
authentication check.

# Additional Changes

None.

# Additional Context

- reported in Discord:
https://discord.com/channels/927474939156643850/1242770807105781760
2024-06-17 09:09:00 +00:00
Jesper We
18222008b6
feat: Contribute i18n swedish language (#8105)
# Which Problems Are Solved

- Swedish speakers cannot use their beautiful native language ;-)

# How the Problems Are Solved

- Contributes Swedish language for Login, Console, common texts and
Emails


# Additional Changes

- none

# Additional Context

- The PR currently provides all translation files according to
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#contribute-internationalization.

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-06-15 13:45:19 +00:00
Silvan
8b4531c28d
fix(v2): correct eventstore query (#8124)
# Which Problems Are Solved

Postgres versions < 16 require an alias for subqueries. The query
executed in the new eventstore didn't add this alias.

# How the Problems Are Solved

Added the alias to the subquery
2024-06-15 16:12:21 +03:00
Tim Möhlmann
120ed0af73
feat(oidc): organization roles scope (#8120)
# Which Problems Are Solved

An admin / application might want to be able to reduce the amount of
roles returned in the token, for example if a user is granted to many
organizations or for specific cases where the application want to narrow
down the access for that token to a specific organization or multiple.
This can now be achieved by providing a scope with the id of the
organization, resp. multiple scopes for every organization, which should
be included.

```
urn:zitadel:iam:org:roles🆔{orgID}
```

**Note:** the new scope does not work when Introspection / Userinfo are
set to legacy mode.

# How the Problems Are Solved

The user info query now has two variants:

1. Variant that returns all organization authorization grants if the new
scope wasn't provided for backward compatibility.
2. Variant that filters the organizations based on the IDs passed in one
or more of the above scopes and returns only those authorization grants.

The query is defined as a `text/template` and both variants are rendered
once in package `init()`.

# Additional Changes

- In the integration tests `assertProjectRoleClaims` now also checks the
org IDs in the roles.

# Additional Context

- Closes #7996
2024-06-14 10:00:43 +02:00
Tim Möhlmann
869ebceac6
fix(oidc): upgrade zitadel/oidc to allow scope without openid (#8109)
# Which Problems Are Solved

Drop the requirement of the `openid` scope in all auth and token request
types.

# How the Problems Are Solved

This change upgrades the oidc package to include
https://github.com/zitadel/oidc/pull/613.

# Additional Changes

- Fix a typo in a go doc string
- upgrade otel modules to match the version from oidc.

# Additional Context

- https://github.com/zitadel/zitadel/pull/7822 started doing scope
validation for machine user authentication on the token endpoint.
- https://github.com/zitadel/zitadel/discussions/8068 reports breakage
of machine authentication without `openid` scope.
- Merge after https://github.com/zitadel/oidc/pull/613 is released.

(cherry picked from commit 81cc7c62cb)
2024-06-13 13:55:05 +02:00
Yxnt
2450736a34
fix(login): ldap login page js file not found (#8101)
# Which Problems Are Solved
* fix ldap login page js file not found

![image](https://github.com/zitadel/zitadel/assets/10323352/47640ae9-3aa2-4a62-aa95-e23750cb8eb7)

Signed-off-by: Yxnt <jyxnt1@gmail.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 1374f17c8d)
2024-06-13 13:55:05 +02:00
Livio Spring
a3fcf6431a
fix(oidc): remove MFA requirement on ZITADEL API based on user auth methods (#8069)
# Which Problems Are Solved

Request to the ZITADEL API currently require multi factor authentication
if the user has set up any second factor.
However, the login UI will only prompt the user to check factors that
are allowed by the login policy.
This can lead to situations, where the user has set up a factor (e.g.
some OTP) which was not allowed by the policy, therefore will not have
to verify the factor, the ZITADEL API however will require the check
since the user has set it up.

# How the Problems Are Solved

The requirement for multi factor authentication based on the user's
authentication methods is removed when accessing the ZITADEL APIs.
Those requests will only require MFA in case the login policy does so
because of `requireMFA` or `requireMFAForLocalUsers`.

# Additional Changes

None.

# Additional Context

- a customer reached out to support
- discussed internally
- relates #7822
- backport to 2.53.x

(cherry picked from commit fb2b1610f9)
2024-06-13 13:55:05 +02:00
Livio Spring
e927eeaaed
fix(oidc): only return state in access token response for implicit flow (#8108)
# Which Problems Are Solved

Introduced with #7822 the access token response incorrectly returned the
`state` parameter.

# How the Problems Are Solved

The `state` will only be returned for access token responses in an
implicit_flow.

# Additional Changes

None.

# Additional Context

- relates to #7822
- relates to
https://github.com/zitadel/oidc/issues/446#issuecomment-2144999644
- backport to 2.53.x

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
(cherry picked from commit fb95f4a8a2)
2024-06-13 13:55:04 +02:00
Joakim Lodén
2cbcdf4ba9
fix: remove duplicate otel span processors (#8104)
(cherry picked from commit 1d13d41139)
2024-06-13 13:55:04 +02:00
Livio Spring
9361a7f0dd
perf(authZ): improve oidc session check (#8091)
# Which Problems Are Solved

Access token checks make sure that there have not been any termination
events (user locked, deactivated, signed out, ...) in the meantime. This
events were filtered based on the creation date of the last session
event, which might cause latency issues in the database.

# How the Problems Are Solved

- Changed the query to use `position` instead of `created_at`.
- removed `AwaitOpenTransactions`

# Additional Changes

Added the `position` field to the `ReadModel`.

# Additional Context

- relates to #8088
- part of #7639
- backport to 2.53.x

(cherry picked from commit 931a6c7cce)
2024-06-13 13:55:04 +02:00
Tim Möhlmann
81cc7c62cb
fix(oidc): upgrade zitadel/oidc to allow scope without openid (#8109)
# Which Problems Are Solved

Drop the requirement of the `openid` scope in all auth and token request
types.

# How the Problems Are Solved

This change upgrades the oidc package to include
https://github.com/zitadel/oidc/pull/613.

# Additional Changes

- Fix a typo in a go doc string
- upgrade otel modules to match the version from oidc.

# Additional Context

- https://github.com/zitadel/zitadel/pull/7822 started doing scope
validation for machine user authentication on the token endpoint.
- https://github.com/zitadel/zitadel/discussions/8068 reports breakage
of machine authentication without `openid` scope.
- Merge after https://github.com/zitadel/oidc/pull/613 is released.
2024-06-13 09:32:27 +00:00
Yxnt
1374f17c8d
fix(login): ldap login page js file not found (#8101)
# Which Problems Are Solved
* fix ldap login page js file not found

![image](https://github.com/zitadel/zitadel/assets/10323352/47640ae9-3aa2-4a62-aa95-e23750cb8eb7)

Signed-off-by: Yxnt <jyxnt1@gmail.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-06-13 07:05:59 +00:00
Livio Spring
fb2b1610f9
fix(oidc): remove MFA requirement on ZITADEL API based on user auth methods (#8069)
# Which Problems Are Solved

Request to the ZITADEL API currently require multi factor authentication
if the user has set up any second factor.
However, the login UI will only prompt the user to check factors that
are allowed by the login policy.
This can lead to situations, where the user has set up a factor (e.g.
some OTP) which was not allowed by the policy, therefore will not have
to verify the factor, the ZITADEL API however will require the check
since the user has set it up.

# How the Problems Are Solved

The requirement for multi factor authentication based on the user's
authentication methods is removed when accessing the ZITADEL APIs.
Those requests will only require MFA in case the login policy does so
because of `requireMFA` or `requireMFAForLocalUsers`.

# Additional Changes

None.

# Additional Context

- a customer reached out to support
- discussed internally
- relates #7822 
- backport to 2.53.x
2024-06-12 12:24:17 +00:00
Livio Spring
fb95f4a8a2
fix(oidc): only return state in access token response for implicit flow (#8108)
# Which Problems Are Solved

Introduced with #7822 the access token response incorrectly returned the
`state` parameter.

# How the Problems Are Solved

The `state` will only be returned for access token responses in an
implicit_flow.

# Additional Changes

None.

# Additional Context

- relates to #7822 
- relates to
https://github.com/zitadel/oidc/issues/446#issuecomment-2144999644
- backport to 2.53.x

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-06-12 11:53:56 +00:00
Joakim Lodén
1d13d41139
fix: remove duplicate otel span processors (#8104) 2024-06-12 10:18:48 +00:00
Livio Spring
931a6c7cce
perf(authZ): improve oidc session check (#8091)
# Which Problems Are Solved

Access token checks make sure that there have not been any termination
events (user locked, deactivated, signed out, ...) in the meantime. This
events were filtered based on the creation date of the last session
event, which might cause latency issues in the database.

# How the Problems Are Solved

- Changed the query to use `position` instead of `created_at`.
- removed `AwaitOpenTransactions`

# Additional Changes

Added the `position` field to the `ReadModel`.

# Additional Context

- relates to #8088
- part of #7639
- backport to 2.53.x
2024-06-12 09:11:36 +00:00
Livio Spring
fb5b4cff58
fix(oauth2): correctly return an error on client_credentials and jwt_profile (#8092)
# Which Problems Are Solved

When an error occurred during the oidc session creation from
client_credentials or jwt_profile, the error was ignored.

# How the Problems Are Solved

Return the error.

# Additional Changes

None.

# Additional Context

- relates to #7822
- noticed internally
- backport to 2.53.x

(cherry picked from commit 448f8f2c11)
2024-06-12 08:43:48 +02:00
Livio Spring
663484e1fb
fix: consider oidc session events for authN milestones (#8089)
# Which Problems Are Solved

After migrating the access token events in #7822, milestones based on
authentication, resp. theses events would not be reached.

# How the Problems Are Solved

Additionally use the `oidc_session.Added` event to check for
`milestone.AuthenticationSucceededOnInstance` and
`milestone.AuthenticationSucceededOnApplication`.

# Additional Changes

None.

# Additional Context

- relates to #7822
- noticed internally

(cherry picked from commit b6c10c4c83)
2024-06-12 08:43:48 +02:00
Livio Spring
4552fe7d99
fix: potential panics in login and return proper http 405 (#8065)
# Which Problems Are Solved

We identified some parts in the code, which could panic with a nil
pointer when accessed without auth request.
Additionally, if a GRPC method was called with an unmapped HTTP method,
e.g. POST instead of GET a 501 instead of a 405 was returned.

# How the Problems Are Solved

- Additional checks for existing authRequest
- custom http status code mapper for gateway

# Additional Changes

None.

# Additional Context

- noted internally in OPS

(cherry picked from commit 26c7d95c88)
2024-06-12 08:43:47 +02:00
Livio Spring
448f8f2c11
fix(oauth2): correctly return an error on client_credentials and jwt_profile (#8092)
# Which Problems Are Solved

When an error occurred during the oidc session creation from
client_credentials or jwt_profile, the error was ignored.

# How the Problems Are Solved

Return the error.

# Additional Changes

None.

# Additional Context

- relates to #7822 
- noticed internally
- backport to 2.53.x
2024-06-12 06:42:50 +00:00
Livio Spring
b6c10c4c83
fix: consider oidc session events for authN milestones (#8089)
# Which Problems Are Solved

After migrating the access token events in #7822, milestones based on
authentication, resp. theses events would not be reached.

# How the Problems Are Solved

Additionally use the `oidc_session.Added` event to check for
`milestone.AuthenticationSucceededOnInstance` and
`milestone.AuthenticationSucceededOnApplication`.

# Additional Changes

None.

# Additional Context

- relates to #7822 
- noticed internally
2024-06-12 06:49:14 +02:00
Elio Bischof
b055d1d9e6
feat(init): configurable existing postgres db (#8045)
# Which Problems Are Solved

The init job fails if no database called *postgres* or *defaultdb* for
cockroach respectively exists.

# How the Problems Are Solved

The value is now configurable, for example by env variable
*ZITADEL_DATABASE_POSTGRES_ADMIN_EXISTINGDATABASE*

# Additional Context

- Closes #5810
2024-06-10 10:49:30 +00:00
Livio Spring
26c7d95c88
fix: potential panics in login and return proper http 405 (#8065)
# Which Problems Are Solved

We identified some parts in the code, which could panic with a nil
pointer when accessed without auth request.
Additionally, if a GRPC method was called with an unmapped HTTP method,
e.g. POST instead of GET a 501 instead of a 405 was returned.

# How the Problems Are Solved

- Additional checks for existing authRequest
- custom http status code mapper for gateway

# Additional Changes

None.

# Additional Context

- noted internally in OPS
2024-06-07 07:30:04 +00:00
Livio Spring
33235a5cbe fix(oidc): make sure id_token does not contain any info from access token actions (#8053)
# Which Problems Are Solved

During tests of 2.53.3 we noticed that in cases where the
`idTokenRoleAssertion` was disabled, claims set in the
preAccessTokenTrigger where also set in the id_token.

# How the Problems Are Solved

The userinfo of the id_token now uses a correct copy of their own.

# Additional Changes

None.

# Additional Context

- relates to #7822 
- relates to #8046
2024-05-31 15:08:00 +02:00
Livio Spring
4fa9de4314
fix(oidc): make sure id_token does not contain any info from access token actions (#8053)
# Which Problems Are Solved

During tests of 2.53.3 we noticed that in cases where the
`idTokenRoleAssertion` was disabled, claims set in the
preAccessTokenTrigger where also set in the id_token.

# How the Problems Are Solved

The userinfo of the id_token now uses a correct copy of their own.

# Additional Changes

None.

# Additional Context

- relates to #7822 
- relates to #8046
2024-05-31 13:06:59 +00:00
Livio Spring
50e0e7d564
Merge branch 'main' into next 2024-05-31 12:12:02 +02:00
Livio Spring
f065b42a97
fix(oidc): respect role assertion and idTokenInfo flags and trigger preAccessToken trigger (#8046)
# Which Problems Are Solved

After deployment of 2.53.x, customers noted that the roles claims where
always present in the tokens even if the corresponding option on the
client (accessTokenRoleAssertion, idTokenRoleAsseriton) was disabled.
Only the project flag (assertRolesOnAuthentication) would be considered.

Further it was noted, that the action on the preAccessTokenCreation
trigger would not be executed.

Additionally, while testing those issues we found out, that the user
information (name, givenname, family name, ...) where always present in
the id_token even if the option (idTokenUserInfo) was not enabled.

# How the Problems Are Solved

- The `getUserinfoOnce` which was used for access and id_tokens is
refactored to `getUserInfo` and no longer only queries the info once
from the database, but still provides a mechanism to be reused for
access and id_token where the corresponding `roleAssertion` and action
`triggerType` can be passed.
- `userInfo` on the other hand now directly makes sure the information
is only queried once from the database. Role claims are asserted every
time and action triggers are executed on every call.
- `userInfo` now also checks if the profile information need to be
returned.

# Additional Changes

None.

# Additional Context

- relates to #7822 
- reported by customers
2024-05-31 10:10:18 +00:00
Livio Spring
bc885632fb
fix(logging): reduce log level of errors in introspection and login UI (#8047)
# Which Problems Are Solved

Introspection errors such as invalid audience and errors in the login UI
such as invalid user agents where all logged as severity error.

# How the Problems Are Solved

Log level for both general loggers is changed to `info`.

# Additional Changes

None

# Additional Context

- internal discussion
2024-05-31 08:11:32 +00:00
Livio Spring
aabefb9382
feat(session api): respect lockout policy (#8027)
# Which Problems Are Solved

The session API was designed to be flexible enough for multiple use
cases / login scenarios, where the login could respect the login policy
or not. The session API itself does not have a corresponding policy and
would not check for a required MFA or alike. It therefore also did not
yet respect the lockout policy and would leave it to the login UI to
handle that.
Since the lockout policy is related to the user and not the login
itself, we decided to handle the lockout also on calls of the session
API.

# How the Problems Are Solved

If a lockout policy is set for either password or (T)OTP checks, the
corresponding check on the session API be run against the lockout check.
This means that any failed check, regardless if occurred in the session
API or the current hosted login will be counted against the maximum
allowed checks of that authentication mechanism. TOTP, OTP SMS and OTP
Email are each treated as a separate mechanism.

For implementation:
- The existing lockout check functions were refactored to be usable for
session API calls.
- `SessionCommand` type now returns not only an error, but also
`[]eventstore.Command`
  - these will be executed in case of an error

# Additional Changes

None.

# Additional Context

Closes #7967

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-05-30 22:08:48 +00:00
adlerhurst
e3523ea0f2 Merge branch 'main' into next 2024-05-30 11:42:47 +02:00
Silvan
2243306ef6
feat(cmd): mirror (#7004)
# Which Problems Are Solved

Adds the possibility to mirror an existing database to a new one. 

For that a new command was added `zitadel mirror`. Including it's
subcommands for a more fine grained mirror of the data.

Sub commands:

* `zitadel mirror eventstore`: copies only events and their unique
constraints
* `zitadel mirror system`: mirrors the data of the `system`-schema
*  `zitadel mirror projections`: runs all projections
*  `zitadel mirror auth`: copies auth requests
* `zitadel mirror verify`: counts the amount of rows in the source and
destination database and prints the diff.

The command requires one of the following flags:
* `--system`: copies all instances of the system
* `--instance <instance-id>`, `--instance <comma separated list of
instance ids>`: copies only the defined instances

The command is save to execute multiple times by adding the
`--replace`-flag. This replaces currently existing data except of the
`events`-table

# Additional Changes

A `--for-mirror`-flag was added to `zitadel setup` to prepare the new
database. The flag skips the creation of the first instances and initial
run of projections.

It is now possible to skip the creation of the first instance during
setup by setting `FirstInstance.Skip` to true in the steps
configuration.

# Additional info

It is currently not possible to merge multiple databases. See
https://github.com/zitadel/zitadel/issues/7964 for more details.

It is currently not possible to use files. See
https://github.com/zitadel/zitadel/issues/7966 for more information.

closes https://github.com/zitadel/zitadel/issues/7586
closes https://github.com/zitadel/zitadel/issues/7486

### Definition of Ready

- [x] I am happy with the code
- [x] Short description of the feature/issue is added in the pr
description
- [x] PR is linked to the corresponding user story
- [x] Acceptance criteria are met
- [x] All open todos and follow ups are defined in a new ticket and
justified
- [x] Deviations from the acceptance criteria and design are agreed with
the PO and documented.
- [x] No debug or dead code
- [x] My code has no repetitions
- [x] Critical parts are tested automatically
- [ ] Where possible E2E tests are implemented
- [x] Documentation/examples are up-to-date
- [x] All non-functional requirements are met
- [x] Functionality of the acceptance criteria is checked manually on
the dev system.

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-05-30 09:35:30 +00:00
Livio Spring
4f114e4bf3
fix: session idp intent check (#8040)
# Which Problems Are Solved

If an IdP intent succeeded with the user was not linked yet, the IdP
link was then added, the following IdP check on the session API would
then fail with `Intent meant for another user (COMMAND-O8xk3w)`.
This issue was introduced with when allowing IdP intents from other
organizations (https://github.com/zitadel/zitadel/pull/7871)

# How the Problems Are Solved

The IdP link is now correctly checked in the session API (using the
user's organization instead of the one from the intent).

# Additional Changes

- Improved the corresponding integration test to cover the exact
bahvior.
- Tests, which had to be updated with newer cases where additionally
changed to use expectEventstore instead of deprecated eventstoreExpect
and the two eventstore mocks of the session_tests.go where combined.

# Additional Context

- Relates to #7871
- This issue was reported by a customer.
- will be back ported to 2.52.x

(cherry picked from commit d254828d47)
2024-05-30 09:25:56 +02:00
Livio Spring
d254828d47
fix: session idp intent check (#8040)
# Which Problems Are Solved

If an IdP intent succeeded with the user was not linked yet, the IdP
link was then added, the following IdP check on the session API would
then fail with `Intent meant for another user (COMMAND-O8xk3w)`.
This issue was introduced with when allowing IdP intents from other
organizations (https://github.com/zitadel/zitadel/pull/7871)

# How the Problems Are Solved

The IdP link is now correctly checked in the session API (using the
user's organization instead of the one from the intent).

# Additional Changes

- Improved the corresponding integration test to cover the exact
bahvior.
- Tests, which had to be updated with newer cases where additionally
changed to use expectEventstore instead of deprecated eventstoreExpect
and the two eventstore mocks of the session_tests.go where combined.

# Additional Context

- Relates to #7871 
- This issue was reported by a customer.
- will be back ported to 2.52.x
2024-05-30 07:06:32 +00:00
Livio Spring
0b82fc1ed0
fix(login): correctly set preferred login name in the login ui (#8038)
# Which Problems Are Solved

A customer noted that after upgrade to 2.53.0, users were no longer able
to reset their passwords through the login UI.
This was due to a accidental change in
https://github.com/zitadel/zitadel/pull/7969

# How the Problems Are Solved

The `preferred_login_name` is now correctly read from the database.

# Additional Changes

None.

# Additional Context

relates to #7969

(cherry picked from commit eca8ffda70)
2024-05-29 17:47:37 +02:00
Livio Spring
eca8ffda70
fix(login): correctly set preferred login name in the login ui (#8038)
# Which Problems Are Solved

A customer noted that after upgrade to 2.53.0, users were no longer able
to reset their passwords through the login UI.
This was due to a accidental change in
https://github.com/zitadel/zitadel/pull/7969

# How the Problems Are Solved

The `preferred_login_name` is now correctly read from the database.

# Additional Changes

None.

# Additional Context

relates to #7969
2024-05-29 15:45:46 +00:00
Livio Spring
8ce3af2f9d
Merge branch 'main' into next 2024-05-28 14:33:54 +02:00
Livio Spring
ec222a13d7
fix(oidc): IDP and passwordless user auth methods (#7998)
# Which Problems Are Solved

As already mentioned and (partially) fixed in #7992 we discovered,
issues with v2 tokens that where obtained through an IDP, with
passwordless authentication or with password authentication (wihtout any
2FA set up) using the v1 login for zitadel API calls
- (Previous) authentication through an IdP is now correctly treated as
auth method in case of a reauth even when the user is not redirected to
the IdP
- There were some cases where passwordless authentication was
successfully checked but not correctly set as auth method, which denied
access to ZITADEL API
- Users with password and passwordless, but no 2FA set up which
authenticate just wich password can access the ZITADEL API again

Additionally while testing we found out that because of #7969 the login
UI could completely break / block with the following error:
`sql: Scan error on column index 3, name "state": converting NULL to
int32 is unsupported (Internal)`
# How the Problems Are Solved

- IdP checks are treated the same way as other factors and it's ensured
that a succeeded check within the configured timeframe will always
provide the idp auth method
- `MFATypesAllowed` checks for possible passwordless authentication
- As with the v1 login, the token check now only requires MFA if the
policy is set or the user has 2FA set up
- UserAuthMethodsRequirements now always uses the correctly policy to
check for MFA enforcement
- `State` column is handled as nullable and additional events set the
state to active (as before #7969)

# Additional Changes

- Console now also checks for 403 (mfa required) errors (e.g. after
setting up the first 2FA in console) and redirects the user to the login
UI (with the current id_token as id_token_hint)
- Possible duplicates in auth methods / AMRs are removed now as well.

# Additional Context

- Bugs were introduced in #7822 and # and 7969 and only part of a
pre-release.
- partially already fixed with #7992
- Reported internally.
2024-05-28 08:59:49 +00:00
Livio Spring
1ac7a37e62 fix(login): ensure auth request (#8004)
# Which Problems Are Solved

Potential nil pointers leading to a panic in the login UI.

# How the Problems Are Solved

As of now the login UI did not always check if the authRequest was
actually retrieved form the database, which is ok for some endpoints
which can also be called outside of an auth request.
There are now methods added to ensure the request is loaded.

# Additional Changes

None

# Additional Context

Closes https://github.com/zitadel/DevOps/issues/55
2024-05-28 08:51:15 +02:00
Silvan
43404d960c fix(eventstore): order by sequence if filter by aggregate id (#8019)
# Which Problems Are Solved

Queriying events by an aggregate id can produce high loads on the
database if the aggregate id contains many events (count > 1000000).

# How the Problems Are Solved

Instead of using the postion and in_tx_order columns we use the sequence
column which guarantees correct ordering in a single aggregate and uses
more optimised indexes.

# Additional Context

Closes https://github.com/zitadel/DevOps/issues/50

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-05-28 08:50:42 +02:00
Silvan
4dc86c2415
fix(eventstore): order by sequence if filter by aggregate id (#8019)
# Which Problems Are Solved

Queriying events by an aggregate id can produce high loads on the
database if the aggregate id contains many events (count > 1000000).

# How the Problems Are Solved

Instead of using the postion and in_tx_order columns we use the sequence
column which guarantees correct ordering in a single aggregate and uses
more optimised indexes.

# Additional Context

Closes https://github.com/zitadel/DevOps/issues/50

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-05-28 06:49:30 +00:00
Silvan
cff0f73e24
fix(init): add setting to enable durable locks on crdb (#7982)
feat(init): add setting to enable durable locks on crdb
2024-05-27 09:03:34 +00:00
Livio Spring
d058a2bc8a
fix(login): ensure auth request (#8004)
# Which Problems Are Solved

Potential nil pointers leading to a panic in the login UI.

# How the Problems Are Solved

As of now the login UI did not always check if the authRequest was
actually retrieved form the database, which is ok for some endpoints
which can also be called outside of an auth request.
There are now methods added to ensure the request is loaded.

# Additional Changes

None

# Additional Context

Closes https://github.com/zitadel/DevOps/issues/55
2024-05-24 14:58:45 +00:00
Silvan
5cfccb1442
fix(actions): correct statements to query targets (#8006)
# Which Problems Are Solved

Curretly loading targets fails on cockraochdb because the order of
`with`-statements is wrong.

# How the Problems Are Solved

Changed the order of queries in the statements.
2024-05-24 14:13:36 +00:00
Silvan
0bfcf2c317
refactor(query): use new packages for org by id query (#7826)
If the feature is enabled the new packages are used to query org by id

Part of: https://github.com/zitadel/zitadel/issues/7639

### Definition of Ready

- [x] I am happy with the code
- [x] Short description of the feature/issue is added in the pr
description
- [x] PR is linked to the corresponding user story
- [ ] Acceptance criteria are met
- [ ] All open todos and follow ups are defined in a new ticket and
justified
- [ ] Deviations from the acceptance criteria and design are agreed with
the PO and documented.
- [x] No debug or dead code
- [x] My code has no repetitions
- [ ] Critical parts are tested automatically
- [ ] Where possible E2E tests are implemented
- [ ] Documentation/examples are up-to-date
- [ ] All non-functional requirements are met
- [x] Functionality of the acceptance criteria is checked manually on
the dev system.
2024-05-24 13:32:57 +02:00
Stefan Benz
e58869c090
fix: setup instance with human an machine user at creation (#7997)
# Which Problems Are Solved

Currently on instance setup there is only a possibility to either use a
human or a machine user and not both at creation.

# How the Problems Are Solved

The logic in the instance setup is refactored and changed so there is
not an exclusion.

# Additional Changes

Refactoring, so that unit testing is possible to add for the different
elements of an instance setup.

# Additional Context

Closes #6430
2024-05-23 12:28:46 +02:00
Sujoy B
cfa3d013a4
feat: show max password length in complexity components (#7810)
# Which Problems Are Solved

There is confusing ambiguity in the error messages for setting too long
passwords in different places.

# How the Problems Are Solved

A check for maximum password length is added so it's clear that
passwords can't exceed a maximum length of 70 or 72 bytes.
Password validation now provides a live updating check mark or cross
mark to indicate if the maximum length requirement is met.

# Additional Changes

Clarified requirement descriptions on the registration page with
complete sentences.

# Additional Context

Closes #6301

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-05-23 08:07:36 +00:00
Tim Möhlmann
f5e9d4f57f
fix(oidc): IDP and machine user auth methods (#7992)
# Which Problems Are Solved

After https://github.com/zitadel/zitadel/pull/7822 was merged we
discovered that
v2 tokens that where obtained through an IDP using the v1 login, can't
be used for
zitadel API calls.

- Because we used to store the AMR claim on the auth request, but
internally use the domain.UserAuthMethod type. AMR has no notion of an
IDP login, so that "factor" was lost
during conversion. Rendering those v2 tokens invalid on the zitadel API.
- A wrong check on machine user tokens falsly allowed some tokens to be
valid
- The client ID was set to tokens from client credentials and JWT
profile, which made client queries fail in the validation middleware.
The middleware expects client ID unset for machine users.

# How the Problems Are Solved

Store the domain.AuthMethods directly in  the auth requests and session,
instead of using AMR claims with lossy conversion.

- IDPs have seperate auth method, which is not an AMR claim
- Machine users are treated specialy, eg auth methods are not required.
- Do not set the client ID for client credentials and JWT profile

# Additional Changes

Cleaned up mostly unused `oidc.getInfoFromRequest()`.

# Additional Context

- Bugs were introduced in https://github.com/zitadel/zitadel/pull/7822
and not yet part of a release.
- Reported internally.
2024-05-23 05:35:10 +00:00
Livio Spring
e57a9b57c8
feat(saml): allow setting nameid-format and alternative mapping for transient format (#7979)
# Which Problems Are Solved

ZITADEL currently always uses
`urn:oasis:names:tc:SAML:2.0:nameid-format:persistent` in SAML requests,
relying on the IdP to respect that flag and always return a peristent
nameid in order to be able to map the external user with an existing
user (idp link) in ZITADEL.
In case the IdP however returns a
`urn:oasis:names:tc:SAML:2.0:nameid-format:transient` (transient)
nameid, the attribute will differ between each request and it will not
be possible to match existing users.

# How the Problems Are Solved

This PR adds the following two options on SAML IdP:
- **nameIDFormat**: allows to set the nameid-format used in the SAML
Request
- **transientMappingAttributeName**: allows to set an attribute name,
which will be used instead of the nameid itself in case the returned
nameid-format is transient

# Additional Changes

To reduce impact on current installations, the `idp_templates6_saml`
table is altered with the two added columns by a setup job. New
installations will automatically get the table with the two columns
directly.
All idp unit tests are updated to use `expectEventstore` instead of the
deprecated `eventstoreExpect`.

# Additional Context

Closes #7483
Closes #7743

---------

Co-authored-by: peintnermax <max@caos.ch>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-05-23 05:04:07 +00:00
Silvan
12be21a3ff
refactor(v2): init events (#7823)
creates events structures for initial projections and read models
2024-05-23 06:36:08 +02:00
Stefan Benz
f37113194d
fix: actions v2 circular check for includes (#7563)
Add a check for circular includes in action v2 executions, so that no
self-includes or infinite loops can happen.

Closes #7445 

### Definition of Ready

- [x] I am happy with the code
- [x] Short description of the feature/issue is added in the pr
description
- [x] PR is linked to the corresponding user story
- [x] Acceptance criteria are met
- [x] All open todos and follow ups are defined in a new ticket and
justified
- [x] Deviations from the acceptance criteria and design are agreed with
the PO and documented.
- [x] No debug or dead code
- [x] My code has no repetitions
- [x] Critical parts are tested automatically
- [x] Where possible E2E tests are implemented
- [x] Documentation/examples are up-to-date
- [x] All non-functional requirements are met
- [x] Functionality of the acceptance criteria is checked manually on
the dev system.

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-05-22 16:05:06 +00:00
Livio Spring
fb162a7d75
fix(login): improve auth handlers (#7969)
# Which Problems Are Solved

During the implementation of #7486 it was noticed, that projections in
the `auth` database schema could be blocked.
Investigations suggested, that this is due to the use of
[GORM](https://gorm.io/index.html) and it's inability to use an existing
(sql) transaction.
With the improved / simplified handling (see below) there should also be
a minimal improvement in performance, resp. reduced database update
statements.

# How the Problems Are Solved

The handlers in `auth` are exchanged to proper (sql) statements and gorm
usage is removed for any writing part.
To further improve / simplify the handling of the users, a new
`auth.users3` table is created, where only attributes are handled, which
are not yet available from the `projections.users`,
`projections.login_name` and `projections.user_auth_methods` do not
provide. This reduces the events handled in that specific handler by a
lot.

# Additional Changes

None

# Additional Context

relates to #7486
2024-05-22 15:26:02 +00:00
Miguel Cabrerizo
cca342187b
feat(console): add new step to activate SMTP provider (#7956)
# Which Problems Are Solved

- In #7929 it was detected that it would be better to show the
activate/deactivate action for a SMTP configuration in the wizard as for
some users it'd not be intuitive that the SMTP provider must be
activated so Zitadel can use it to send notifications.

# How the Problems Are Solved

- When a new SMTP provider is added or updated, the wizard has a new
step that allow us to activate or deactivate the provider configured in
the previous step. The following video shows the new wizard:


https://github.com/zitadel/zitadel/assets/30386061/178234d6-73dc-4719-af0b-1d6f19bf3f7d

# Additional Context

- Closes #7929
2024-05-22 11:23:35 +02:00
Tim Möhlmann
5b1160de1e
feat(session): allow update of session without token (#7963)
# Which Problems Are Solved

The session update requires the current session token as argument.
Since this adds extra complexity but no real additional security and
prevents case like magic links, we want to remove this requirement.

We still require the session token on other resouces / endpoints, e.g.
for finalizing the auth request or on idp intents.

# How the Problems Are Solved

- Removed the session token verifier in the Update Session GRPc call.
- Removed the session token from login UI examples session update calls

# Additional Changes

- none

# Additional Context

- Closes #7883
2024-05-22 05:56:11 +00:00
Livio Spring
07f91e4f16
fix(login): ignore select_account in case there aren't any session to choose (#7981)
# Which Problems Are Solved

When poviding `select_account` in an OIDC auth request, ZITADEL would
always show the account selection page even if there aren't any user
sessions to choose and the user would then need to click the `Other
User` button to be presented the login page.

# How the Problems Are Solved

This PR changes the behavior and ignores the `select_account` prompt in
case there aren't any existing user sessions and will directly present
the login page.

# Additional Changes

None

# Additional Context

Closes #7213
2024-05-21 12:53:31 +00:00
Aurel004
6858ed7b21
fix: Update fr translations (#7910)
update fr translations

Co-authored-by: Max Peintner <max@caos.ch>
2024-05-17 13:16:55 +02:00
Livio Spring
b924fd59a5
fix: allow sorting of MyProjectOrgs (by name) (#7970) 2024-05-17 09:52:15 +02:00