Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix broken links in documentation #6305

Merged
merged 13 commits into from
Aug 3, 2023
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ docs-serve-docker: ### Serve local docs from Docker
--volume="$$PWD/docs:/srv/jekyll:Z" \
--volume="$$PWD/docs/.jekyll-bundle-cache:/usr/local/bundle:Z" \
--interactive --tty \
jekyll/jekyll:3.8 \
jekyll/jekyll:4.2.2 \
jekyll serve --livereload

gen-docs: ## Generate CLI docs automatically
Expand Down
6 changes: 3 additions & 3 deletions docs/cloud/auditing.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ lakeFS Cloud
{: .label .label-green }

{: .note}
> Auditing is only available for [lakeFS Cloud]({{ site.baseurl }}/cloud/).
> Auditing is only available for [lakeFS Cloud]({% link cloud/index.md %})

The lakeFS audit log allows you to view all relevant user action information in a clear and organized table, including when the action was performed, by whom, and what it was they did.

![audit log]({{ site.baseurl }}/assets/img/audit-log.png)
![audit log]({% link assets/img/audit-log.png %})

This can be useful for several purposes, including:

Expand Down Expand Up @@ -50,4 +50,4 @@ The table fields can be filtered by the following operators
- Region - `=`,`!=`
- Status(Number) - `<`,`>`,`<=`,`>=`,`=`,`!=`
- Action - `=`,`!=`
- Resource - `=`,`!=`
- Resource - `=`,`!=`
2 changes: 1 addition & 1 deletion docs/cloud/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ has_toc: false
[lakeFS Cloud](https://lakefs.cloud) is a fully-managed lakeFS solution provided by Treeverse, implemented using our best practices, providing high availability, auto-scaling, support and enterprise-ready features.

## lakeFS Cloud Features
* [Role-Based Access Control]({{ site.baseurl }}/reference/rbac.md)
* [Role-Based Access Control]({% link reference/rbac.md %})
* [Auditing](./auditing.md)
* [Single-Sign-On](./sso.md) (including support for SAML, OIDC, AD FS, Okta, and Azure AD)
* [Managed Garbage Collection](./managed-gc.md)
Expand Down
5 changes: 2 additions & 3 deletions docs/cloud/managed-gc.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,7 @@ lakeFS Cloud
{: .label .label-green }

{: .note}
> Managed GC is only available for [lakeFS Cloud]({{ site.baseurl }}/cloud/).
If you are using the self-managed lakeFS, garbage collection is [available to run manually]({{ site.baseurl }}/howto/garbage-collection-index.md).
> Managed GC is only available for [lakeFS Cloud]({% link cloud/index.md %}). If you are using the self-managed lakeFS, garbage collection is [available to run manually]({% link howto/garbage-collection-index.md %}).

## Benefits of using managed GC
* The quick and safe way to delete your unnecessary objects
Expand All @@ -21,7 +20,7 @@ If you are using the self-managed lakeFS, garbage collection is [available to ru
* Support from the Treeverse team

## How it works
Similarly to the self-managed lakeFS, managed GC uses [garbage collection rules]({{ site.baseurl }}/howto/garbage-collection-index.md) to determine which objects to delete.
Similarly to the self-managed lakeFS, managed GC uses [garbage collection rules]({% link howto/garbage-collection-index.md %}) to determine which objects to delete.
However, it uses our super-fast and efficient engine to detect stale objects and branches (depends on your configuration) and prioritize them for deletion.

## Setting up
Expand Down
2 changes: 1 addition & 1 deletion docs/cloud/sso.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ lakeFS Cloud
{: .label .label-green }

{: .note}
> SSO is also available for [lakeFS Enterprise]({{ site.baseurl }}/enterprise/sso.html). Using the open-source version of lakeFS? Read more on [authentication]({{ site.baseurl }}/reference/authentication.html).
> SSO is also available for [lakeFS Enterprise]({% link enterprise/sso.md %}). Using the open-source version of lakeFS? Read more on [authentication]({% link reference/authentication.md %}).

lakeFS Cloud uses Auth0 for authentication and thus support the same identity providers as Auth0 including Active Directory/LDAP, ADFS, Azure Active Directory Native, Google Workspace, OpenID Connect, Okta, PingFederate, SAML, and Azure Active Directory.

Expand Down
4 changes: 2 additions & 2 deletions docs/enterprise/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ has_toc: false
# lakeFS Enterprise

lakeFS Enterprise is an enterprise-ready lakeFS solution that provides a support SLA and additional features to the open-source version of lakeFS. The additional features are:
* [RBAC]({{ site.baseurl }}/reference/rbac.html)
* [SSO]({{ site.baseurl }}/enterprise/sso.html)
* [RBAC]({% link reference/rbac.md%})
* [SSO]({% link enterprise/sso.md%})
* Support SLA

6 changes: 4 additions & 2 deletions docs/enterprise/sso.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ lakeFS Enterprise
{: .label .label-purple }

{: .note}
> SSO is also available on [lakeFS Cloud]({{ site.baseurl }}/cloud/sso.md). Using the open-source version of lakeFS? Read more on [authentication]({{ site.baseurl }}/reference/authentication.md).
> SSO is also available on [lakeFS Cloud]({% link cloud/sso.md %}). Using the open-source version of lakeFS? Read more about [authentication]({% link reference/authentication.md %}).

Authentication in lakeFS Enterprise is handled by a secondary service which runs side-by-side with lakeFS. With a nod to Hogwarts and their security system, we've named this service _Fluffy_. Details for configuring the supported identity providers with Fluffy are shown below. In addition, please review the necessary [Helm configuration](#helm) to configure Fluffy.

Expand Down Expand Up @@ -101,7 +101,7 @@ auth:

In order for Fluffy to work, the following values must be configured. Update (or override) the following attributes in the chart's `values.yaml` file.
1. Replace `lakefsConfig.friendly_name_claim_name` with the right claim name.
1. Replace `lakefsConfig.default_initial_groups` with desired claim name (See [pre-configured]({{ site.baseurl }}/reference/rbac.md#preconfigured-groups) groups for enterprise)
1. Replace `lakefsConfig.default_initial_groups` with desired claim name (See [pre-configured][rbac-preconfigured] groups for enterprise)
2. Replace `fluffyConfig.auth.logout_redirect_url` with your full OIDC logout URL (e.g `https://oidc-provider-url.com/logout/path`)
3. Replace `fluffyConfig.auth.oidc.url` with your OIDC provider URL (e.g `https://oidc-provider-url.com`)
4. Replace `fluffyConfig.auth.oidc.logout_endpoint_query_parameters` with parameters you'd like to pass to the OIDC provider for logout.
Expand Down Expand Up @@ -218,3 +218,5 @@ Notes:
* Change the `ingress.hosts[0]` from `lakefs.company.com` to a real host (usually same as lakeFS), also update additional references in the file (note: URL path after host if provided should stay unchanged).
* Update the `ingress` configuration with other optional fields if used
* Fluffy docker image: replace the `fluffy.image.privateRegistry.secretToken` with real token to dockerhub for the fluffy docker image.

[rbac-preconfigured]: {% link reference/rbac.md %}#preconfigured-groups
25 changes: 16 additions & 9 deletions docs/howto/deploy/aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ By default, lakeFS will create the required DynamoDB table if it does not alread
}
```

💡 You can also use lakeFS with PostgreSQL instead of DynamoDB! See the [configuration reference]({{ site.baseurl }}/reference/configuration.md) for more information.
💡 You can also use lakeFS with PostgreSQL instead of DynamoDB! See the [configuration reference]({% link reference/configuration.md %}) for more information.
{: .note }

## Run the lakeFS server
Expand Down Expand Up @@ -93,9 +93,9 @@ Connect to your EC2 instance using SSH:
blockstore:
type: s3
```
1. [Download the binary]({{ site.baseurl }}/index.md#downloads) to the EC2 instance.
1. [Download the binary][downloads] to run on the EC2 instance.
1. Run the `lakefs` binary on the EC2 instance:

```sh
lakefs --config config.yaml run
```
Expand Down Expand Up @@ -131,7 +131,7 @@ To install lakeFS with Helm:
```
1. Fill in the missing values and save the file as `conf-values.yaml`. For more configuration options, see our Helm chart [README](https://github.com/treeverse/charts/blob/master/charts/lakefs/README.md#custom-configuration){:target="_blank"}.

The `lakefsConfig` parameter is the lakeFS configuration documented [here](https://docs.lakefs.io/reference/configuration.html) but without sensitive information.
The `lakefsConfig` parameter is the lakeFS configuration documented [here]({% link reference/configuration.md%}) but without sensitive information.
Sensitive information like `databaseConnectionString` is given through separate parameters, and the chart will inject it into Kubernetes secrets.
{: .note }

Expand All @@ -156,8 +156,8 @@ To configure a load balancer to direct requests to the lakeFS servers you can us
By default, lakeFS operates on port 8000 and exposes a `/_health` endpoint that you can use for health checks.

💡 The NGINX Ingress Controller by default limits the client body size to 1 MiB.
Some clients use bigger chunks to upload objects - for example, multipart upload to lakeFS using the [S3 Gateway]({{ site.baseurl }}/understand/architecture.md#s3-gateway) or
a simple PUT request using the [OpenAPI Server]({{ site.baseurl }}/understand/architecture.md#openapi-server).
Some clients use bigger chunks to upload objects - for example, multipart upload to lakeFS using the [S3 Gateway][s3-gateway] or
a simple PUT request using the [OpenAPI Server][openapi].
Checkout Nginx [documentation](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-max-body-size) for increasing the limit, or an example of Nginx configuration with [MinIO](https://docs.min.io/docs/setup-nginx-proxy-with-minio.html).
{: .note }

Expand Down Expand Up @@ -214,15 +214,15 @@ Checkout Nginx [documentation](https://kubernetes.github.io/ingress-nginx/user-g

* Replace `[BUCKET_NAME]`, `[ACCOUNT_ID]` and `[IAM_ROLE]` with values relevant to your environment.
* `[BUCKET_NAME_AND_PREFIX]` can be the bucket name. If you want to minimize the bucket policy permissions, use the bucket name together with a prefix (e.g. `example-bucket/a/b/c`).
This way, lakeFS will be able to create repositories only under this specific path (see: [Storage Namespace]({{ site.baseurl }}/understand/model.md#repository)).
This way, lakeFS will be able to create repositories only under this specific path (see: [Storage Namespace][understand-repository]).
* lakeFS will try to assume the role `[IAM_ROLE]`.
</div>
<div markdown="1" id="bucket-policy-minimal">
This permission is useful if you are using the [lakeFS Hadoop FileSystem Spark integration]({{ site.baseurl }}/integrations/spark.md#use-the-lakefs-hadoop-filesystem).
This permission is useful if you are using the [lakeFS Hadoop FileSystem Spark integration][integration-hadoopfs].
Since this FileSystem performs many operations directly on the storage, lakeFS requires less permissive permissions, resulting in increased security.

lakeFS always requires permissions to access the `_lakefs` prefix under your storage namespace, in which metadata
is stored ([learn more]({{ site.baseurl }}/understand/how/versioning-internals.md#constructing-a-consistent-view-of-the-keyspace-ie-a-commit)).
is stored ([learn more][understand-commits]).
By setting this policy you'll be able to perform only metadata operations through lakeFS, meaning that you'll **not** be able
to use lakeFS to upload or download objects. Specifically you won't be able to:
* Upload objects using the lakeFS GUI
Expand Down Expand Up @@ -278,3 +278,10 @@ lakeFS can authenticate with your AWS account using an AWS user, using an access
```

{% include_relative includes/setup.md %}

[downloads]: {% link index.md %}#downloads
[openapi]: {% link understand/architecture.md %}#openapi-server
[s3-gateway]: {% link understand/architecture.md %}#s3-gateway
[understand-repository]: {% link understand/model.md %}#repository
[integration-hadoopfs]: {% link integrations/spark.md %}#lakefs-hadoop-filesystem
[understand-commits]: {% link understand/how/versioning-internals.md %}#constructing-a-consistent-view-of-the-keyspace-ie-a-commit
23 changes: 16 additions & 7 deletions docs/howto/deploy/azure.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ lakeFS stores metadata in a database for its versioning engine. This is done via
1. Create a new container in the database and select type
`partitionKey` as the Partition key (case sensitive).
1. Pass the endpoint, database name and container name to lakeFS as
described in the [configuration guide]({{ site.baseurl }}/reference/configuration.md#example--azure-blob-storage).
described in the [configuration guide][config-reference-azure-block].
You can either pass the CosmosDB's account read-write key to lakeFS, or
use a managed identity to authenticate to CosmosDB, as described
[earlier](#identity-based-credentials).
Expand Down Expand Up @@ -203,7 +203,7 @@ Connect to your VM instance using SSH:
type: azure
azure:
```
1. [Download the binary]({{ site.baseurl }}/index.md#downloads) to the VM.
1. [Download the binary][downloads] to run on the VM.
1. Run the `lakefs` binary:

```sh
Expand Down Expand Up @@ -232,7 +232,7 @@ docker run \
treeverse/lakefs:latest run
```

See the [reference]({{ site.baseurl }}/reference/configuration.md#using-environment-variables) for a complete list of environment variables.
See the [reference][config-envariables] for a complete list of environment variables.


</div>
Expand Down Expand Up @@ -282,14 +282,23 @@ To configure a load balancer to direct requests to the lakeFS servers you can us
By default, lakeFS operates on port 8000 and exposes a `/_health` endpoint that you can use for health checks.

💡 The NGINX Ingress Controller by default limits the client body size to 1 MiB.
Some clients use bigger chunks to upload objects - for example, multipart upload to lakeFS using the [S3-compatible Gateway]({{ site.baseurl }}/understand/architecture.md#s3-gateway) or
a simple PUT request using the [OpenAPI Server]({{ site.baseurl }}/understand/architecture.md#openapi-server).
Checkout Nginx [documentation](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-max-body-size) for increasing the limit, or an example of Nginx configuration with [MinIO](https://docs.min.io/docs/setup-nginx-proxy-with-minio.html).
Some clients use bigger chunks to upload objects - for example, multipart upload to lakeFS using the [S3-compatible Gateway][s3-gateway] or
a simple PUT request using the [OpenAPI Server][openapi].
Check out Nginx [documentation](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-max-body-size) for increasing the limit, or an example of Nginx configuration with [MinIO](https://docs.min.io/docs/setup-nginx-proxy-with-minio.html).
{: .note }

</div>
</div>



{% include_relative includes/setup.md %}
{% include_relative includes/setup.md %}

[config-envariables]: {% link reference/configuration.md %}#using-environment-variables %}
[config-reference-azure-block]: {% link reference/configuration.md %}#example-azure-blob-storage
[downloads]: {% link index.md %}#downloads
[openapi]: {% link understand/architecture.md %}#openapi-server
[s3-gateway]: {% link understand/architecture.md %}#s3-gateway
[understand-repository]: {% link understand/model.md %}#repository
[integration-hadoopfs]: {% link integrations/spark.md %}#lakefs-hadoop-filesystem
[understand-commits]: {% link understand/how/versioning-internals.md %}#constructing-a-consistent-view-of-the-keyspace-ie-a-commit
17 changes: 12 additions & 5 deletions docs/howto/deploy/gcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ If you already have a database, take note of the connection string and skip to t
# credentials_json: [YOUR SERVICE ACCOUNT JSON STRING]
```

1. [Download the binary]({{ site.baseurl }}/index.md#downloads) to the GCE instance.
1. [Download the binary][downloads] to run on the GCE instance.
1. Run the `lakefs` binary on the GCE machine:
```bash
lakefs --config config.yaml run
Expand All @@ -87,7 +87,7 @@ docker run \
treeverse/lakefs:latest run
```

See the [reference]({{ site.baseurl }}/reference/configuration.md#using-environment-variables) for a complete list of environment variables.
See the [reference][config-envariables] for a complete list of environment variables.

</div>
<div markdown="3" id="gke">
Expand Down Expand Up @@ -136,14 +136,21 @@ To configure a load balancer to direct requests to the lakeFS servers you can us
By default, lakeFS operates on port 8000 and exposes a `/_health` endpoint that you can use for health checks.

💡 The NGINX Ingress Controller by default limits the client body size to 1 MiB.
Some clients use bigger chunks to upload objects - for example, multipart upload to lakeFS using the [S3-compatible Gateway]({{ site.baseurl }}/understand/architecture.md#s3-gateway) or
a simple PUT request using the [OpenAPI Server]({{ site.baseurl }}/understand/architecture.md#openapi-server).
Some clients use bigger chunks to upload objects - for example, multipart upload to lakeFS using the [S3-compatible Gateway][s3-gateway] or
a simple PUT request using the [OpenAPI Server][openapi].
Checkout Nginx [documentation](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#custom-max-body-size) for increasing the limit, or an example of Nginx configuration with [MinIO](https://docs.min.io/docs/setup-nginx-proxy-with-minio.html).
{: .note }

</div>
</div>


{% include_relative includes/setup.md %}

{% include_relative includes/setup.md %}
[config-envariables]: {% link reference/configuration.md %}#using-environment-variables %}
[downloads]: {% link index.md %}#downloads
[openapi]: {% link understand/architecture.md %}#openapi-server
[s3-gateway]: {% link understand/architecture.md %}#s3-gateway
[understand-repository]: {% link understand/model.md %}#repository
[integration-hadoopfs]: {% link integrations/spark.md %}#lakefs-hadoop-filesystem
[understand-commits]: {% link understand/how/versioning-internals.md %}#constructing-a-consistent-view-of-the-keyspace-ie-a-commit
2 changes: 1 addition & 1 deletion docs/howto/deploy/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ For a hosted lakeFS service with guaranteed SLAs, try [lakeFS Cloud](https://lak

lakeFS releases include [binaries](https://github.com/treeverse/lakeFS/releases) for common operating systems, a [containerized option](https://hub.docker.com/r/treeverse/lakefs) or a [Helm chart](https://artifacthub.io/packages/helm/lakefs/lakefs).

Check out our guides for running lakeFS on [AWS]({{ site.baseurl }}/howto/deploy/aws.md), [GCP]({{ site.baseurl }}/howto/deploy/gcp.md) and [more]({{ site.baseurl }}/howto/deploy).
Check out our guides for running lakeFS on [AWS]({% link howto/deploy/aws.md %}), [GCP]({% link howto/deploy/gcp.md %}) and [more]({% link howto/deploy/index.md %}}).
Loading
Loading