Skip to content

Commit

Permalink
docs: update docs for instrumentation rule
Browse files Browse the repository at this point in the history
  • Loading branch information
blumamir committed Sep 8, 2024
1 parent aa1f9df commit 03d7790
Show file tree
Hide file tree
Showing 2 changed files with 85 additions and 44 deletions.
47 changes: 45 additions & 2 deletions docs/pipeline/rules/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,49 @@ title: "Instrumentation Rules Overview"
sidebarTitle: "Overview"
---

Instrumentation rules are a set of flexible configuration options that describe how Odigos should instrument your application.
They are applied at the data recording stage of the pipeline, and offer fine-grained control over the collection of telemetry data.
Instrumentation Rules control how telemetry is recorded from your application.

## Rule Types:

- [Payload Collection](/pipeline/rules/payloadcollection)

## Scope

A rule can be applied to a set of Odigos sources (k8s workloads) and instrumentation libraries.

### Sources

- To apply a rule to all sources in the cluster, omit the `workloads` field.
- To limit the rule to specific sources, provide an array of workload objects (name, namespace, kind).

### Instrumentation Libraries

This is an advanced feature that allows you to apply a rule to a specific instrumentation library.
It is recommended to set the same rules to all instrumentation libraries, and only use this feature when you need fine-grained control.

- To apply the rule to all instrumentation libraries, omit the `instrumentationLibraries` field.
- Alternatively, you can specify a list of instrumentation libraries to which the rule should be applied. Instrumentation library is identified by a combination of it's unique name, and language (and span kind for golang).

Any rule on a specific library will take precedence over a rule that does not specify a library.

## Configuration Options

- `ruleName` (optional): Allows you to attach a meaningful name to the rule for convenience and documentation. Odigos does not use or assume any meaning from this field.

- `notes` (optional): A free-form text field that allows you to attach notes regarding the rule for convenience. For example: why it was added. Odigos does not use or assume any meaning from this field.

- `disabled` (optional, default is `false`, e.g. enabled): A boolean field allowing to temporarily disable the rule, but keep it around for future use

- `workloads` (optional, default is omitted, e.g. all workloads): An array of workload objects (name, namespace, kind) to which the rule should be applied. If not specified, the rule will be applied to all workloads. empty array will make the rule ineffective.

- `instrumentationLibraries` (optional, default is omitted, e.g. all libraries): An array of instrumentation library id objects to which the rule should be applied. If not specified, the rule will be applied to all instrumentation libraries. empty array will make the rule ineffective.

## Merging Rules

You can specify multiple rules in your cluster:

- When multiple teams work on the same cluster, they can each define their own rules.
- To define some global default behavior and override it for specific cases.
- If you need to create a temporary rule for a givin task which will be removed later.

Odigos allows you to define multiple rules, and will merge them together to create a single rule per source.
82 changes: 40 additions & 42 deletions docs/pipeline/rules/payloadcollection.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,34 +3,36 @@ title: "Payload Collection Instrumentation Rule"
sidebarTitle: "Payload Collection"
---

The "Payload Collection" Rule can be used to instruct Odigos to collect and record payload data from operations into spans.
The "Payload Collection" Rule can be used to add span attributes containing payload data to traces.

Payload can supply valuable context to trace signal consumers, which can help in understanding the behavior of the application.
In many cases, the payload data can provide valuable context for understanding the behavior of your application.

## Considerations

Before enabling payload collection, please note the following:

- If your payload contains PII data (Personally Identifiable Information), this data can be exposed in the traces. Evaluate the risk of collecting this data, and consider using the [PII Masking Action](/pipeline/actions/attributes/piimasking) to remove sensitive data.
- Payload data can be large and may increase the size of your traces. This can impact the performance of your application and the cost of storing and analyzing traces.
- PII (Personally Identifiable Information) and other potentially sensitive data may be present in the payload. Evaluate the risk of collecting this data, and consider using the [PII Masking Action](/pipeline/actions/attributes/piimasking) to mask sensitive data.
- Payload data can be large and may increase the size of your traces. This can impact the performance of your application and the cost of processing, storing and analyzing traces.
- The support for payload collection varies between instrumentation libraries and languages. Not all libraries support payload collection, and the supported payload types and formats may differ. Consult the documentation of the instrumentation library you are using for more information.

## Basic Example

The following example demonstrates how to enable payload collection any supported payload, for all workloads in the cluster and all supported instrumentation Libraries.
The following example demonstrates how to enable payload collection for all supported workloads and instrumentation libraries in the cluster.

Create a file named `payloadcollection.yaml` with the following content:

```yaml
apiVersion: rules.odigos.io/v1alpha1
kind: PayloadCollection
apiVersion: odigos.io/v1alpha1
kind: InstrumentationRule
metadata:
name: collect-all-payloads
namespace: odigos-system
spec:
ruleName: "collect all supported payloads"
httpRequest: {}
httpResponse: {}
dbQuery: {}
payloadCollection:
httpRequest: {}
httpResponse: {}
dbQuery: {}
```
Apply the action to the cluster:
Expand All @@ -44,11 +46,11 @@ kubectl apply -f payloadcollection.yaml
The following example is a demonstration of all the options available in the "Payload Collection" Rule.
It is not meant to be used "as is", but rather as a reference to customize the rule to your needs.

Create a file named `fullpayloadcollection.yaml` with the following content:
Create a file named `full-payload-collection-example.yaml` with the following content:

```yaml
apiVersion: rules.odigos.io/v1alpha1
kind: PayloadCollection
apiVersion: odigos.io/v1alpha1
kind: InstrumentationRule
metadata:
name: full-payload-collection-example
namespace: odigos-system
Expand All @@ -70,36 +72,32 @@ spec:
- language: go
name: "database/sql"
spanKind: client
httpRequest:
mimeTypes:
- "application/json"
- "text/plain"
maxPayloadLength: 2048
dropPartialPayloads: true
httpResponse:
mimeTypes:
- "application/json"
- "text/plain"
maxPayloadLength: 2048
dropPartialPayloads: true
dbQuery:
maxPayloadLength: 1024
dropPartialPayloads: true
payloadCollection:
httpRequest:
mimeTypes:
- "application/json"
maxPayloadLength: 2048
dropPartialPayloads: true
httpResponse:
mimeTypes:
- "application/json"
- "text/plain"
maxPayloadLength: 8096
dropPartialPayloads: true
dbQuery:
maxPayloadLength: 1024
dropPartialPayloads: true
```
### Configuration Options
The full list of configuration options for the "Payload Collection" Rule are:
- `ruleName` (optional): Allows you to attach a meaningful name to the rule for convenience. Odigos does not use or assume any meaning from this field.

- `notes` (optional): A free-form text field that allows you to attach notes regarding the rule for convenience. For example: why it was added. Odigos does not use or assume any meaning from this field.
Apply the action to the cluster:
- `disabled` (optional, default `false` - enabled): A boolean field allowing to temporarily disable the rule, but keep it around for future use
```shell
kubectl apply -f full-payload-collection-example.yaml
```

- `workloads` (optional, default is nil - all workloads): An array of workload objects (name, namespace, kind) to which the rule should be applied. If not specified, the rule will be applied to all workloads. empty array will make the rule ineffective.
### Configuration Options

- `instrumentationLibraries` (optional, default is nil - all libraries): An array of instrumentation library id objects to which the rule should be applied. If not specified, the rule will be applied to all instrumentation libraries. empty array will make the rule ineffective.
The full list of configuration options for the "Payload Collection" Rule are:

- `httpRequest` (optional): Collect HTTP request payload data when available. Can be a client (outgoing) request or a server (incoming) request, depending on the instrumentation library
- `mimeTypes` (optional, default is nil - all mime types): Limit payload collection to specific mime types based on the content type header. When not specified, all mime types payloads will be collected. empty array will make the rule ineffective.
Expand All @@ -115,8 +113,8 @@ The full list of configuration options for the "Payload Collection" Rule are:
- `maxPayloadLength` (optional): Maximum length of the payload to collect. If the payload is longer than this value, it will be truncated or dropped, based on the value of `dropPartialPayloads` config option. When not specified (recommended), the instrumentation library will use any reasonable default value
- `dropPartialPayloads` (optional, default is false): If the payload is larger than the MaxPayloadLength, this parameter will determine if the payload should be partially collected up to the allowed length, or not collected at all. This is useful if you require some decoding of the payload (like json) and having it partially is not useful.

## Multiple Rules
## Merging Rules

You can create multiple "Payload Collection" rules to specify different configurations for different workloads or instrumentation libraries.
Rules for specific libraries are considered before rules for all libraries.
In case multiple rules apply to the same workload or library, they will be merged together to create the final configuration.
- Any payload collection rule for a specific instrumentation library will take precedence over a rule that does not specify a library.
- If multiple rules are defined for the same source, the rules will be merged together. Any conflicting options will be resolved by taking the "safest" option.
- Any unspecified options will fallback to a reasonable default value provided by the instrumentation library (recommended).

0 comments on commit 03d7790

Please sign in to comment.