Skip to content

Commit

Permalink
Fix broken links (#804)
Browse files Browse the repository at this point in the history
* some broken links fixed

Signed-off-by: Srinivasan Parthasarathy <[email protected]>

* fixing broken links

Signed-off-by: Srinivasan Parthasarathy <[email protected]>

* seldon tutorial section

Signed-off-by: Srinivasan Parthasarathy <[email protected]>
  • Loading branch information
sriumcp authored Jun 23, 2021
1 parent bf6b6ce commit 43e586d
Show file tree
Hide file tree
Showing 26 changed files with 120 additions and 54 deletions.
10 changes: 5 additions & 5 deletions mkdocs/docs/contributing/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ bug report and let us know!

We welcome many different types of contributions including:

* [Iter8 documentation/tutorials](../tutorials)
* [Iter8 documentation/tutorials](tutorials.md)
* New features
* [New K8s stack/service mesh/ingress](../newk8sstack)
* [New K8s stack/service mesh/ingress](newk8sstack.md)
* Iter8 code samples for OpenShift
* [Analytics](../analytics)
* [Analytics](analytics.md)
* Tasks
* Builds, CI
* Bug fixes
Expand All @@ -35,7 +35,7 @@ We welcome many different types of contributions including:
* Reviewing pull requests

Not everything happens through a GitHub pull request. Please come to our
[community meetings](#come-to-meetings) or [contact us](../../getting-started/help) and let's discuss how we can work together.
[community meetings](#come-to-weekly-community-meetings) or [contact us](../getting-started/help.md) and let's discuss how we can work together.

***

Expand Down Expand Up @@ -77,7 +77,7 @@ The best ways to reach us with a question when contributing is to ask on:

* The original GitHub issue
* `#development` channel in the [Iter8 Slack workspace](https://join.slack.com/t/iter8-tools/shared_invite/zt-awl2se8i-L0pZCpuHntpPejxzLicbmw)
* Bring your questions to our [community meetings](#come-to-meetings)
* Bring your questions to our [community meetings](#come-to-weekly-community-meetings)

## Pull Request Lifecycle

Expand Down
2 changes: 1 addition & 1 deletion mkdocs/docs/contributing/tutorials.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ template: main.html
Iter8 documentation uses [Mkdocs](https://www.mkdocs.org/user-guide/writing-your-docs/). The section on [linking to pages and images](https://www.mkdocs.org/user-guide/writing-your-docs/#writing-with-markdown) is especially useful for Iter8 tutorial authors.

## Test your tutorial
All Iter8 tutorials include e2e tests, either as [part of GitHub Actions workflows](https://github.com/iter8-tools/iter8/blob/master/.github/workflows/e2e-tests.yaml) or as a standalone test script [like this one](https://github.com/iter8-tools/iter8/blob/master/samples/knative/mirroring/e2etest.sh) if they require more resources than what is available in GitHub Actions workflows. When contributing a tutorial, please include relevant e2e tests.
All Iter8 tutorials include e2e tests, either as [part of GitHub Actions workflows](https://github.com/iter8-tools/iter8/blob/master/.github/workflows/e2e-tests.yaml) or as a standalone test script [like this one](https://github.com/iter8-tools/iter8/blob/master/samples/seldon/quickstart/e2etest.sh) if they require more resources than what is available in GitHub Actions workflows. When contributing a tutorial, please include relevant e2e tests.

## Locally serve Iter8 docs
**Pre-requisite:** Python 3+.
Expand Down
8 changes: 4 additions & 4 deletions mkdocs/docs/getting-started/quick-start/kfserving/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@ template: main.html
# A/B Testing

!!! tip "Scenario: A/B testing and progressive rollout of KFServing models"
[A/B testing](../../../../../concepts/buildingblocks/#ab-testing) enables you to compare two versions of an ML model, and select a winner based on a (business) reward metric. In this tutorial, you will:
[A/B testing](../../../concepts/buildingblocks.md#ab-testing) enables you to compare two versions of an ML model, and select a winner based on a (business) reward metric. In this tutorial, you will:

1. Perform A/B testing.
2. Specify *user-engagement* as the reward metric. This metric will be mocked by Iter8 in this tutorial.
3. Combine A/B testing with [progressive traffic shifting](../../../../../concepts/buildingblocks/#progressive-traffic-shift). Iter8 will progressively shift traffic towards the winner and promote it at the end as depicted below.
3. Combine A/B testing with [progressive traffic shifting](../../../concepts/buildingblocks.md#progressive-traffic-shift). Iter8 will progressively shift traffic towards the winner and promote it at the end as depicted below.

![Quickstart KFServing](../../../images/quickstart-ab.png)

Expand All @@ -19,7 +19,7 @@ template: main.html
3. [Go 1.13+](https://golang.org/doc/install).

## 1. Setup
* Setup your K8s cluster with KFServing and Iter8 as described [here](../platform-setup/).
* Setup your K8s cluster with KFServing and Iter8 as described [here](platform-setup.md).
* Ensure that the `ITER8` environment variable is set to the root of your local Iter8 repo.

## 2. Create ML model versions
Expand Down Expand Up @@ -152,7 +152,7 @@ kubectl apply -f $ITER8/samples/kfserving/quickstart/metrics.yaml
??? Note "Metrics in your environment"
You can define and use custom metrics from any database in Iter8 experiments.

For your application, replace the mocked metric used in this tutorial with any custom metric you wish to optimize in the A/B test. Documentation on defining custom metrics is [here](../../../../metrics/custom/).
For your application, replace the mocked metric used in this tutorial with any custom metric you wish to optimize in the A/B test. Documentation on defining custom metrics is [here](../../../metrics/custom.md).

## 5. Launch experiment
Iter8 defines a custom K8s resource called *Experiment* that automates a variety of release engineering and experimentation strategies for K8s applications and ML models. Launch the A/B testing & progressive traffic shift experiment as follows.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ kubectl apply -f $ITER8/samples/knative/quickstart/metrics.yaml
??? Note "Metrics in your environment"
You can define and use custom metrics from any database in Iter8 experiments.

For your application, replace the mocked user-engagement metric used in this tutorial with any custom metric you wish to optimize in the hybrid (A/B + SLOs) test. Documentation on defining custom metrics is [here](../../../../metrics/custom/).
For your application, replace the mocked user-engagement metric used in this tutorial with any custom metric you wish to optimize in the hybrid (A/B + SLOs) test. Documentation on defining custom metrics is [here](../../../metrics/custom.md).

## 5. Launch experiment
Iter8 defines a custom K8s resource called *Experiment* that automates a variety of release engineering and experimentation strategies for K8s applications and ML models. Launch the hybrid (A/B + SLOs) testing & progressive traffic shift experiment as follows.
Expand Down
4 changes: 2 additions & 2 deletions mkdocs/docs/getting-started/quick-start/seldon/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ template: main.html
4. [Helm 3+](https://helm.sh/docs/intro/install/)

## 1. Setup
* Setup your K8s cluster with Seldon and Iter8 as described [here](../platform-setup/).
* Setup your K8s cluster with Seldon and Iter8 as described [here](platform-setup.md).
* Ensure that the `ITER8` environment variable is set to the root of your local Iter8 repo.

## 2. Create ML model versions
Expand Down Expand Up @@ -328,7 +328,7 @@ kubectl apply -f $ITER8/samples/seldon/quickstart/metrics.yaml
??? Note "Metrics in your environment"
You can define and use custom metrics from any database in Iter8 experiments.

For your application, replace the mocked user-engagement metric used in this tutorial with any custom metric you wish to optimize in the hybrid (A/B + SLOs) test. Documentation on defining custom metrics is [here](../../../../metrics/custom/).
For your application, replace the mocked user-engagement metric used in this tutorial with any custom metric you wish to optimize in the hybrid (A/B + SLOs) test. Documentation on defining custom metrics is [here](../../../metrics/custom.md).

## 5. Launch experiment
Iter8 defines a custom K8s resource called *Experiment* that automates a variety of release engineering and experimentation strategies for K8s applications and ML models. Launch the hybrid (A/B + SLOs) testing & progressive traffic shift experiment as follows.
Expand Down
4 changes: 2 additions & 2 deletions mkdocs/docs/metrics/builtin.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The following are the set of builtin Iter8 metrics.
| iter8-system | latency-99th-percentile | Gauge | 99th percentile response latency |

## Collecting builtin metrics
Use the [`metrics/collect` task](../../reference/tasks/metrics/) in an experiment to collect builtin metrics for your app/ML model versions.
Use the [`metrics/collect` task](../reference/tasks/metrics.md) in an experiment to collect builtin metrics for your app/ML model versions.

## Example
For an example of an experiment that uses builtin metrics, look inside the Knative experiment in [this tutorial](../../tutorials/testing-strategies/conformance/#7-launch-experiment).
For an example of an experiment that uses builtin metrics, look inside the Knative experiment in [this tutorial](../../tutorials/testing-strategies/conformance/#5-launch-experiment).
10 changes: 5 additions & 5 deletions mkdocs/docs/metrics/custom.md
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ The examples in this document focus on Prometheus, NewRelic, Sysdig, and Elastic
Iter8 will substitute placeholders in the metric query based on the time elapsed since the start of the experiment, and information associated with each version in the experiment.

Suppose the [metrics defined above](#defining-metrics) are referenced within an experiment as follows. Further, suppose this experiment has started, Iter8 is about to do an iteration of this experiment, and the time elapsed since the start of the experiment is 600 seconds.
Suppose the [metrics defined above](#metrics-withwithout-auth) are referenced within an experiment as follows. Further, suppose this experiment has started, Iter8 is about to do an iteration of this experiment, and the time elapsed since the start of the experiment is 600 seconds.

??? abstract "Look inside sample experiment"
```yaml linenums="1"
Expand Down Expand Up @@ -543,7 +543,7 @@ The metrics provider is expected to respond to Iter8's HTTP request with a JSON
Iter8 uses [jq](https://stedolan.github.io/jq/) to extract the metric value from the JSON response of the provider. The `jqExpression` used by Iter8 is supplied as part of the metric definition. When the `jqExpression` is applied to the JSON response, it is expected to yield a number.

=== "Prometheus"
Consider the `jqExpression` defined in the [sample Prometheus metric](#defining-metrics). Let us apply it to the [sample JSON response from Prometheus](#json-response).
Consider the `jqExpression` defined in the [sample Prometheus metric](#metrics-withwithout-auth). Let us apply it to the [sample JSON response from Prometheus](#json-response).
```shell
echo '{
"status": "success",
Expand All @@ -560,7 +560,7 @@ Iter8 uses [jq](https://stedolan.github.io/jq/) to extract the metric value from
Executing the above command results yields `21.7639`, a number, as required by Iter8.

=== "New Relic"
Consider the `jqExpression` defined in the [sample New Relic metric](#defining-metrics). Let us apply it to the [sample JSON response from New Relic](#json-response).
Consider the `jqExpression` defined in the [sample New Relic metric](#metrics-withwithout-auth). Let us apply it to the [sample JSON response from New Relic](#json-response).
```shell
echo '{
"results": [
Expand Down Expand Up @@ -602,7 +602,7 @@ Iter8 uses [jq](https://stedolan.github.io/jq/) to extract the metric value from
Executing the above command results yields `80275388`, a number, as required by Iter8.

=== "Sysdig"
Consider the `jqExpression` defined in the [sample Sysdig metric](#defining-metrics). Let us apply it to the [sample JSON response from Sysdig](#json-response).
Consider the `jqExpression` defined in the [sample Sysdig metric](#metrics-withwithout-auth). Let us apply it to the [sample JSON response from Sysdig](#json-response).
```shell
echo '{
"data": [
Expand All @@ -620,7 +620,7 @@ Iter8 uses [jq](https://stedolan.github.io/jq/) to extract the metric value from
Executing the above command results yields `6.481`, a number, as required by Iter8.

=== "Elastic"
Consider the `jqExpression` defined in the [sample Elastic metric](#defining-metrics). Let us apply it to the [sample JSON response from Elastic](#json-response).
Consider the `jqExpression` defined in the [sample Elastic metric](#metrics-withwithout-auth). Let us apply it to the [sample JSON response from Elastic](#json-response).
```shell
echo '{
"aggregations": {
Expand Down
6 changes: 3 additions & 3 deletions mkdocs/docs/metrics/using-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ template: main.html
!!! tip "Iter8 metric resources"
Iter8 defines a custom Kubernetes resource (CRD) called **Metric** that makes it easy to define and use metrics in experiments.

Iter8 installation includes a set of pre-defined [builtin metrics](../builtin) that pertain to app/ML model latency/errors. You can also [define custom metrics](../custom) that enable you to utilize data from Prometheus, New Relic, Sysdig, Elastic or any other database of your choice.
Iter8 installation includes a set of pre-defined [builtin metrics](builtin.md) that pertain to app/ML model latency/errors. You can also [define custom metrics](custom.md) that enable you to utilize data from Prometheus, New Relic, Sysdig, Elastic or any other database of your choice.

## List metrics
Find the set Iter8 metrics available in your cluster using `kubectl get`.
Expand All @@ -32,7 +32,7 @@ iter8-system request-count Counter Number of requests (Iter8

## Referencing metrics within experiments

Use metrics in experiments by referencing them in the criteria section of the experiment manifest. Reference metrics using the `namespace/name` or `name` [format](../../../reference/apispec/#criteria).
Use metrics in experiments by referencing them in the criteria section of the experiment manifest. Reference metrics using the `namespace/name` or `name` [format](../reference/experiment.md#criteria).

??? example "Sample experiment illustrating the use of metrics"
```yaml
Expand All @@ -55,4 +55,4 @@ Use metrics in experiments by referencing them in the criteria section of the ex
```

## Observing metric values
During an experiment, Iter8 reports the metric values observed for each version. Use `iter8ctl` to observe these metric values in realtime. See [here](../../../getting-started/quick-start/#a-observe-metrics) for an example.
During an experiment, Iter8 reports the metric values observed for each version. Use `iter8ctl` to observe these metric values in realtime. See [here](../getting-started/quick-start/kfserving/tutorial.md#a-observe-results) for an example.
6 changes: 3 additions & 3 deletions mkdocs/docs/reference/experiment.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ template: main.html
# Experiment Resource

!!! abstract "Experiment resource"
Iter8's **Experiment** resource type enables application developers and service operators to automate A/B, A/B/n, Canary and Conformance experiments for Kubernetes apps/ML models. The controls provided by the experiment resource type encompass [testing, deployment, traffic engineering, and version promotion functions](../../../concepts/buildingblocks/).
Iter8's **Experiment** resource type enables application developers and service operators to automate A/B, A/B/n, Canary and Conformance experiments for Kubernetes apps/ML models. The controls provided by the experiment resource type encompass [testing, deployment, traffic engineering, and version promotion functions](../concepts/buildingblocks.md).

??? info "Sample experiment"
```yaml linenums="1"
Expand Down Expand Up @@ -190,7 +190,7 @@ Standard Kubernetes [meta.v1/ObjectMeta](https://kubernetes.io/docs/reference/ge
| Field name | Field type | Description | Required |
| ----- | ---- | ----------- | -------- |
| name | string | Identifies an Iter8 metric using the [`namespace/name` or `name` format](#criteria). | Yes |
| metric | [][Metric](#metric) | Iter8 metric object referenced by name. | No |
| metric | [][Metric](metrics.md) | Iter8 metric object referenced by name. | No |

### ExperimentCondition

Expand All @@ -209,7 +209,7 @@ Standard Kubernetes [meta.v1/ObjectMeta](https://kubernetes.io/docs/reference/ge

| Field name | Field type | Description | Required |
| ----- | ------------ | ----------- | -------- |
| aggregatedBuiltinHists | [AggregatedBuiltinHists](#aggregatedbuiltinhists) | This field is used to store intermediate results from the [`metrics/collect` task](../tasks/metrics/#metrics-tasks) that enables [builtin metrics](../../metrics/builtin/). Reserved for Iter8 internal use. | No |
| aggregatedBuiltinHists | [AggregatedBuiltinHists](#aggregatedbuiltinhists) | This field is used to store intermediate results from the [`metrics/collect` task](tasks/metrics.md#metrics-tasks) that enables [builtin metrics](../metrics/builtin.md). Reserved for Iter8 internal use. | No |
| aggregatedMetrics | [AggregatedMetricsAnalysis](#aggregatedmetricsanalysis) | Most recently observed metric values for all metrics referenced in the experiment criteria. | No |
| winnerAssessment | [WinnerAssessmentAnalysis](#winnerassessmentanalysis) | Information about the `winner` of the experiment. | No |
| versionAssessments | [VersionAssessmentAnalysis](#versionassessmentanalysis) | For each version, a summary analysis identifying whether or not the version is satisfying the experiment criteria. | No |
Expand Down
2 changes: 1 addition & 1 deletion mkdocs/docs/reference/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ template: main.html
!!! note "Version"
This document describes version `v2alpha2` of Iter8's metric API.

Metrics usage is documented [here](../../metrics/using-metrics/). Iter8 provides a set of builtin metrics that are documented [here](../../metrics/builtin/). Creation of custom metrics is documented [here](../../metrics/custom/). It is also possible to mock metric values; mock metrics are documented [here](../../metrics/mock).
Metrics usage is documented [here](../metrics/using-metrics.md). Iter8 provides a set of builtin metrics that are documented [here](../metrics/builtin.md). Creation of custom metrics is documented [here](../metrics/custom.md). It is also possible to mock metric values; mock metrics are documented [here](../metrics/mock.md).

??? example "Sample metric"
```yaml linenums="1"
Expand Down
4 changes: 2 additions & 2 deletions mkdocs/docs/tutorials/istio/rollout-strategies/fixed-split.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,8 +122,8 @@ kubectl apply -f $ITER8/samples/istio/fixed-split/experiment.yaml
value: https://raw.githubusercontent.com/iter8-tools/iter8/master/samples/istio/quickstart/vs-for-v2.yaml
```

## 6. Observe experiment
Follow [Step 6 of the quick start tutorial](../../../getting-started/quick-start/istio/tutorial.md#6-observe-experiment) to observe metrics, traffic and progress of the experiment. Ensure that you use the correct experiment name (`fixedsplit-exp`) in your `iter8ctl` and `kubectl` commands.
## 6. Understand the experiment
Follow [Step 6 of the quick start tutorial](../../../getting-started/quick-start/istio/tutorial.md#6-understand-the-experiment) to observe metrics, traffic and progress of the experiment. Ensure that you use the correct experiment name (`fixedsplit-exp`) in your `iter8ctl` and `kubectl` commands.

## 7. Cleanup
```shell
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ kubectl apply -f $ITER8/samples/istio/conformance/experiment.yaml
value: bookinfo-iter8
```

## 6. Observe experiment
## 6. Understand the experiment
Follow [Step 6 of the quick start tutorial for Istio](../../../../getting-started/quick-start/istio/tutorial/#6-understand-the-experiment) to observe metrics, traffic and progress of the experiment. Ensure that you use the correct experiment name (`conformance-exp`) in your `iter8ctl` and `kubectl` commands.

## 7. Cleanup
Expand Down
Loading

0 comments on commit 43e586d

Please sign in to comment.