Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: OIDC support #660

Merged
merged 32 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
2f4af4d
fix: Correctly encode user given content, such as passwords (#627)
sbernauer Jun 21, 2024
a31fde1
add vscode debugging profile
labrenbe Jun 28, 2024
c612bf4
wip: add integration test for oidc
labrenbe Jun 28, 2024
4054c17
Merge remote-tracking branch 'origin/main' into feat/oidc-support
labrenbe Jul 8, 2024
8e4ca19
Merge remote-tracking branch 'origin/main' into feat/oidc-support
labrenbe Jul 23, 2024
7b89547
wip: debug oidc & jwts
labrenbe Jul 24, 2024
d96503c
wip
labrenbe Jul 30, 2024
6a005ae
wip: map over ContainerBuilders
NickLarsenNZ Jul 30, 2024
ec83a52
fix oidc test
labrenbe Jul 31, 2024
1ac981b
fix oidc test
labrenbe Aug 5, 2024
46cebd1
fix clippy, update CRD and remove debug files
labrenbe Aug 5, 2024
e6ecd09
Merge remote-tracking branch 'origin/main' into feat/oidc-support
labrenbe Aug 5, 2024
5a6ad47
run cargo fmt
labrenbe Aug 5, 2024
f6b5dcb
address clippy and yamllint feedback
labrenbe Aug 6, 2024
9baf17e
remove unneccessary return
labrenbe Aug 6, 2024
e007773
reenable all tests
labrenbe Aug 6, 2024
07e8d1a
add docs and fix oidc test
labrenbe Aug 6, 2024
d357343
remove reporting task from oidc test
labrenbe Aug 6, 2024
458cd29
add debug logging
labrenbe Aug 6, 2024
0247445
fix test logging
labrenbe Aug 6, 2024
3087d46
use nifi-latest in oidc test
labrenbe Aug 6, 2024
cd49351
add comment why nifi-latest is used
labrenbe Aug 6, 2024
3c98ad5
clean up code and add comment
labrenbe Aug 7, 2024
ce13a4a
Merge branch 'main' into feat/oidc-support
labrenbe Aug 14, 2024
9d11920
address feedback from review
labrenbe Aug 15, 2024
d7e9c82
improve oidc integration test
labrenbe Aug 15, 2024
dce0f7c
fix oidc test for nifi 2.0.0-M4
labrenbe Aug 16, 2024
36dde26
Merge remote-tracking branch 'origin/main' into feat/oidc-support
labrenbe Aug 16, 2024
0959c75
increase timeout on test job creation
labrenbe Aug 17, 2024
a4e5fab
fix docs on oidc
labrenbe Aug 26, 2024
a07ec4f
move config for debugger to operator-templating
labrenbe Aug 27, 2024
447ed26
add comment to test job assert
labrenbe Aug 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .vscode/launch.json
sbernauer marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"version": "0.2.0",
"configurations": [
{
"type": "lldb",
"request": "launch",
"name": "Debug executable 'stackable-nifi-operator'",
"cargo": {
"args": [
"build",
"--bin=stackable-nifi-operator",
"--package=stackable-nifi-operator"
],
"filter": {
"name": "stackable-nifi-operator",
"kind": "bin"
}
},
"args": ["run"],
"cwd": "${workspaceFolder}"
}
]
}
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git",
strum = { version = "0.26", features = ["derive"] }
tokio = { version = "1.39", features = ["full"] }
tracing = "0.1"
url = { version = "2.5.2" }
xml-rs = "0.8"

# [patch."https://github.com/stackabletech/operator-rs.git"]
Expand Down
21 changes: 20 additions & 1 deletion deploy/helm/nifi-operator/crds/crds.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,27 @@ spec:
items:
properties:
authenticationClass:
description: Name of the [AuthenticationClass](https://docs.stackable.tech/home/nightly/concepts/authentication) used to authenticate users. Supported providers are `static` and `ldap`. For `static` the "admin" user needs to be present in the referenced secret, and only this user will be added to NiFi, other users are ignored.
description: A name/key which references an authentication class. To get the concrete [`AuthenticationClass`], we must resolve it. This resolution can be achieved by using [`ClientAuthenticationDetails::resolve_class`].
sbernauer marked this conversation as resolved.
Show resolved Hide resolved
type: string
oidc:
description: |-
This field contains authentication provider specific configuration.

Use [`ClientAuthenticationDetails::oidc_or_error`] to get the value or report an error to the user.
nullable: true
properties:
clientCredentialsSecret:
description: A reference to the OIDC client credentials secret. The secret contains the client id and secret.
type: string
extraScopes:
default: []
description: An optional list of extra scopes which get merged with the scopes defined in the [`AuthenticationClass`].
items:
type: string
type: array
required:
- clientCredentialsSecret
type: object
required:
- authenticationClass
type: object
Expand Down
81 changes: 74 additions & 7 deletions docs/modules/nifi/pages/usage_guide/security.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ spec:
serverSecretClass: non-default-secret-class # <1>
----

<1> The name of the `SecretClass` that will be used for certificates for the NiFi UI.
<1> The name of the SecretClass that will be used for certificates for the NiFi UI.

== Authentication

Expand All @@ -49,8 +49,8 @@ spec:
name: nifi-admin-credentials # <2>
----

<1> The name of the `AuthenticationClass` that will be referenced in the NiFi cluster.
<2> The name of the `Secret` containing the admin credentials.
<1> The name of the AuthenticationClass that will be referenced in the NiFi cluster.
<2> The name of the Secret containing the admin credentials.

[source,yaml]
----
Expand All @@ -63,9 +63,9 @@ stringData:
bob: bob # <3>
----

<1> The name of the `Secret` containing the admin user credentials.
<2> The user and password combination of the admin user. The username *must* be "admin" and cannot be changed. The NiFi pods will not start if they cannot mount the "admin" entry from the secret. The password can be adapted.
<3> The secret maybe used by other products of the Stackable Data Platform that allow more than one user. The Stackable Operator for Apache NiFi will ignore all users except for "admin".
<1> The name of the Secret containing the admin user credentials.
<2> The user and password combination of the admin user. The username *must* be "admin" and cannot be changed. The NiFi pods will not start if they cannot mount the "admin" entry from the Secret. The password can be adapted.
<3> The Secret maybe used by other products of the Stackable Data Platform that allow more than one user. The Stackable Operator for Apache NiFi will ignore all users except for "admin".

[source,yaml]
----
Expand All @@ -75,7 +75,7 @@ spec:
- authenticationClass: simple-nifi-users # <1>
----

<1> The reference to an `AuthenticationClass`. NiFi only supports one authentication mechanism at a time.
<1> The reference to an AuthenticationClass. NiFi only supports one authentication mechanism at a time.

[#authentication-ldap]
=== LDAP
Expand All @@ -100,13 +100,72 @@ spec:

You can follow the xref:tutorials:authentication_with_openldap.adoc[] tutorial to learn how to set up an AuthenticationClass for an LDAP server, as well as consulting the {crd-docs}/authentication.stackable.tech/authenticationclass/v1alpha1/[AuthenticationClass reference {external-link-icon}^].

[#authentication-oidc]
=== OIDC

NiFi supports xref:concepts:authentication.adoc[authentication] of users against an OIDC provider.
This requires setting up an AuthenticationClass for the OIDC provider and specifying a Secret containing OIDC client and OIDC client Secret as part of the NiFi configuration.
The AuthenticationClass and the OIDC client credentials Secret are then referenced in the NifiCluster resource:

[source,yaml]
----
apiVersion: nifi.stackable.tech/v1alpha1
kind: NifiCluster
metadata:
name: test-nifi
spec:
clusterConfig:
authentication:
- authenticationClass: oidc # <1>
oidc:
clientCredentialsSecret: nifi-oidc-client # <2>
----

<1> The reference to an AuthenticationClass called `oidc`
<2> The reference to an existing Secret called `nifi-oidc-client`

[source,yaml]
----
apiVersion: authentication.stackable.tech/v1alpha1
kind: AuthenticationClass
metadata:
name: oidc
spec:
provider:
oidc:
hostname: keycloak.example.com
rootPath: /realms/test/ # <1>
principalClaim: preferred_username
scopes:
- openid
- email
- profile
port: 8080
tls: null
[...]
----

<1> A trailing slash in `rootPath` is necessary.

[source,yaml]
----
apiVersion: v1
kind: Secret
metadata:
name: nifi-oidc-client
stringData:
clientId: <client-id>
clientSecret: <client-secret>
----

[#authorization]
== Authorization

NiFi supports {nifi-docs-authorization}[multiple authorization methods], the available authorization methods depend on the chosen authentication method.

Authorization is not fully implemented by the Stackable Operator for Apache NiFi.

[#authorization-single-user]
=== Single user

With this authorization method, a single user has administrator capabilities.
Expand All @@ -118,6 +177,14 @@ The operator uses the {nifi-docs-fileusergroupprovider}[`FileUserGroupProvider`]
This user is then able to create and modify groups and policies in the web interface.
These changes local to the Pod running NiFi and are *not* persistent.

[#authorization-oidc]
=== OIDC

With this authorization method, all authenticated users have administrator capabilities.

An admin user with an auto-generated password is created that can access the NiFi API.
The password for this user is stored in a Kubernetes Secret called `<nifi-name>-oidc-admin-password`.

[#encrypting-sensitive-properties]
== Encrypting sensitive properties on disk

Expand Down
153 changes: 95 additions & 58 deletions rust/crd/src/authentication.rs
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
use serde::{Deserialize, Serialize};
use std::future::Future;

use snafu::{ResultExt, Snafu};
use stackable_operator::commons::authentication::AuthenticationClassProvider;
use stackable_operator::commons::authentication::{
ldap, oidc, static_, AuthenticationClassProvider, ClientAuthenticationDetails,
};
use stackable_operator::kube::ResourceExt;
use stackable_operator::{
client::Client,
commons::authentication::AuthenticationClass,
client::Client, commons::authentication::AuthenticationClass,
kube::runtime::reflector::ObjectRef,
schemars::{self, JsonSchema},
};

use crate::NifiCluster;

#[derive(Snafu, Debug)]
pub enum Error {
#[snafu(display("Failed to retrieve AuthenticationClass {authentication_class}"))]
AuthenticationClassRetrieval {
#[snafu(display("failed to retrieve AuthenticationClass"))]
AuthenticationClassRetrievalFailed {
source: stackable_operator::client::Error,
authentication_class: ObjectRef<AuthenticationClass>,
},

#[snafu(display("The nifi-operator does not support running Nifi without any authentication. Please provide a AuthenticationClass to use."))]
Expand All @@ -33,67 +35,102 @@ pub enum Error {
NoLdapTlsVerificationNotSupported {
authentication_class: ObjectRef<AuthenticationClass>,
},

#[snafu(display("invalid OIDC configuration"))]
OidcConfigurationInvalid {
sbernauer marked this conversation as resolved.
Show resolved Hide resolved
source: stackable_operator::commons::authentication::Error,
},
}

type Result<T, E = Error> = std::result::Result<T, E>;

#[derive(Clone, Debug, Deserialize, JsonSchema, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct NifiAuthenticationClassRef {
/// Name of the [AuthenticationClass](DOCS_BASE_URL_PLACEHOLDER/concepts/authentication) used to authenticate users.
/// Supported providers are `static` and `ldap`.
/// For `static` the "admin" user needs to be present in the referenced secret, and only this user will be added to NiFi, other users are ignored.
pub authentication_class: String,
#[derive(Clone)]
pub enum AuthenticationClassResolved {
Static {
provider: static_::AuthenticationProvider,
},
Ldap {
provider: ldap::AuthenticationProvider,
},
Oidc {
provider: oidc::AuthenticationProvider,
oidc: oidc::ClientAuthenticationOptions<()>,
nifi: NifiCluster,
},
}

/// Retrieve all provided `AuthenticationClass` references.
pub async fn resolve_authentication_classes(
client: &Client,
authentication_class_refs: &Vec<NifiAuthenticationClassRef>,
) -> Result<Vec<AuthenticationClass>> {
let mut resolved_auth_classes = vec![];

match authentication_class_refs.len() {
0 => NoAuthenticationNotSupportedSnafu.fail()?,
1 => {}
_ => MultipleAuthenticationClassesNotSupportedSnafu.fail()?,
impl AuthenticationClassResolved {
pub async fn from(
nifi: &NifiCluster,
client: &Client,
) -> Result<Vec<AuthenticationClassResolved>> {
let resolve_auth_class = |auth_details: ClientAuthenticationDetails| async move {
auth_details.resolve_class(client).await
};
AuthenticationClassResolved::resolve(nifi, resolve_auth_class).await
}

for auth_class in authentication_class_refs {
let resolved_auth_class =
AuthenticationClass::resolve(client, &auth_class.authentication_class)
/// Retrieve all provided `AuthenticationClass` references.
pub async fn resolve<R>(
nifi: &NifiCluster,
resolve_auth_class: impl Fn(ClientAuthenticationDetails) -> R,
) -> Result<Vec<AuthenticationClassResolved>>
where
R: Future<Output = Result<AuthenticationClass, stackable_operator::client::Error>>,
{
let mut resolved_auth_classes = vec![];
let auth_details = &nifi.spec.cluster_config.authentication;

match auth_details.len() {
0 => NoAuthenticationNotSupportedSnafu.fail()?,
1 => {}
_ => MultipleAuthenticationClassesNotSupportedSnafu.fail()?,
}

for entry in auth_details {
let auth_class = resolve_auth_class(entry.clone())
.await
.context(AuthenticationClassRetrievalSnafu {
authentication_class: ObjectRef::<AuthenticationClass>::new(
&auth_class.authentication_class,
),
})?;

let resolved_auth_class_name = resolved_auth_class.name_any();

match &resolved_auth_class.spec.provider {
AuthenticationClassProvider::Static(_) => {}
AuthenticationClassProvider::Ldap(ldap) => {
if ldap.tls.uses_tls() && !ldap.tls.uses_tls_verification() {
NoLdapTlsVerificationNotSupportedSnafu {
authentication_class: ObjectRef::<AuthenticationClass>::new(
&resolved_auth_class_name,
),
.context(AuthenticationClassRetrievalFailedSnafu)?;

let auth_class_name = auth_class.name_any();

match &auth_class.spec.provider {
AuthenticationClassProvider::Static(provider) => {
resolved_auth_classes.push(AuthenticationClassResolved::Static {
provider: provider.to_owned(),
})
}
AuthenticationClassProvider::Ldap(provider) => {
if provider.tls.uses_tls() && !provider.tls.uses_tls_verification() {
NoLdapTlsVerificationNotSupportedSnafu {
authentication_class: ObjectRef::<AuthenticationClass>::new(
&auth_class_name,
),
}
.fail()?
}
.fail()?
resolved_auth_classes.push(AuthenticationClassResolved::Ldap {
provider: provider.to_owned(),
})
}
}
_ => AuthenticationClassProviderNotSupportedSnafu {
authentication_class_provider: resolved_auth_class.spec.provider.to_string(),
authentication_class: ObjectRef::<AuthenticationClass>::new(
&resolved_auth_class_name,
),
}
.fail()?,
};
AuthenticationClassProvider::Oidc(provider) => {
resolved_auth_classes.push(Ok(AuthenticationClassResolved::Oidc {
provider: provider.to_owned(),
oidc: entry
.oidc_or_error(&auth_class_name)
.context(OidcConfigurationInvalidSnafu)?
.clone(),
nifi: nifi.clone(),
})?)
}
_ => AuthenticationClassProviderNotSupportedSnafu {
authentication_class_provider: auth_class.spec.provider.to_string(),
authentication_class: ObjectRef::<AuthenticationClass>::new(&auth_class_name),
}
.fail()?,
};
}

resolved_auth_classes.push(resolved_auth_class);
Ok(resolved_auth_classes)
}

Ok(resolved_auth_classes)
}
Loading
Loading