Skip to content

Commit

Permalink
[Rename] kbn-dev-utils to osd-dev-utils (#74)
Browse files Browse the repository at this point in the history
* [Rename] kbn-dev-utils to osd-dev-utils

Signed-off-by: Kawika Avilla <[email protected]>

* Handle PR comments

Signed-off-by: Kawika Avilla <[email protected]>
  • Loading branch information
kavilla authored Mar 6, 2021
1 parent d422508 commit ebb36e5
Show file tree
Hide file tree
Showing 84 changed files with 208 additions and 208 deletions.
8 changes: 0 additions & 8 deletions packages/kbn-dev-utils/README.md

This file was deleted.

62 changes: 0 additions & 62 deletions packages/kbn-dev-utils/certs/README.md

This file was deleted.

8 changes: 8 additions & 0 deletions packages/osd-dev-utils/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# dev utils

## proc runner

For running integration tests in OpenSearch Dashboards

## tooling log

62 changes: 62 additions & 0 deletions packages/osd-dev-utils/certs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Development certificates

OpenSearch Dashboards includes several development certificates to enable easy setup of TLS-encrypted communications with OpenSearch.

_Note: these certificates should **never** be used in production._

## Certificate information

Certificates and keys are provided in multiple formats. These can be used by other packages to set up a new OpenSearch Stack with OpenSearch Dashboards and OpenSearch. The Certificate Authority (CA) private key is intentionally omitted from this package.

### PEM

* `ca.crt` -- A [PEM-formatted](https://tools.ietf.org/html/rfc1421) [X.509](https://tools.ietf.org/html/rfc5280) certificate that is used as a CA.
* `opensearch.crt` -- A PEM-formatted X.509 certificate and public key for OpenSearch.
* `opensearch.key` -- A PEM-formatted [PKCS #1](https://tools.ietf.org/html/rfc8017) private key for OpenSearch.
* `opensearch_dashboards.crt` -- A PEM-formatted X.509 certificate and public key for OpenSearch Dashboards.
* `opensearch_dashboards.key` -- A PEM-formatted PKCS #1 private key for OpenSearch Dashboards.

### PKCS #12

* `opensearch.p12` -- A [PKCS #12](https://tools.ietf.org/html/rfc7292) encrypted key store / trust store that contains `ca.crt`, `opensearch.crt`, and a [PKCS #8](https://tools.ietf.org/html/rfc5208) encrypted version of `opensearch.key`.
* `opensearch_dashboards.p12` -- A PKCS #12 encrypted key store / trust store that contains `ca.crt`, `opensearch_dashboards.crt`, and a PKCS #8 encrypted version of `opensearch_dashboards.key`.

The password used for both of these is "storepass". Other copies are also provided for testing purposes:

* `opensearch_emptypassword.p12` -- The same PKCS #12 key store, encrypted with an empty password.
* `opensearch_nopassword.p12` -- The same PKCS #12 key store, not encrypted with a password.

## Certificate generation

[OpenSearch cert-util](https://www.elastic.co/guide/en/elasticsearch/reference/current/certutil.html) and [OpenSSL](https://www.openssl.org/) were used to generate these certificates. The following commands were used from the root directory of OpenSearch:

```
# Generate the PKCS #12 keystore for a CA, valid for 50 years
bin/opensearch-certutil ca -days 18250 --pass castorepass
# Generate the PKCS #12 keystore for OpenSearch and sign it with the CA
bin/opensearch-certutil cert -days 18250 --ca opensearch-stack-ca.p12 --ca-pass castorepass --name opensearch --dns localhost --pass storepass
# Generate the PKCS #12 keystore for OpenSearch Dashboards and sign it with the CA
bin/opensearch-certutil cert -days 18250 --ca opensearch-stack-ca.p12 --ca-pass castorepass --name opensearch-dashboards --dns localhost --pass storepass
# Copy the PKCS #12 keystore for OpenSearch with an empty password
openssl pkcs12 -in opensearch.p12 -nodes -passin pass:"storepass" -passout pass:"" | openssl pkcs12 -export -out opensearch_emptypassword.p12 -passout pass:""
# Manually create "opensearch_nopassword.p12" -- this can be done on macOS by importing the P12 key store into the Keychain and exporting it again
# Extract the PEM-formatted X.509 certificate for the CA
openssl pkcs12 -in opensearch.p12 -out ca.crt -cacerts -passin pass:"storepass" -passout pass:
# Extract the PEM-formatted PKCS #1 private key for OpenSearch
openssl pkcs12 -in opensearch.p12 -nocerts -passin pass:"storepass" -passout pass:"keypass" | openssl rsa -passin pass:keypass -out opensearch.key
# Extract the PEM-formatted X.509 certificate for OpenSearch
openssl pkcs12 -in opensearch.p12 -out opensearch.crt -clcerts -passin pass:"storepass" -passout pass:
# Extract the PEM-formatted PKCS #1 private key for OpenSearch Dashboards
openssl pkcs12 -in opensearch_dashboards.p12 -nocerts -passin pass:"storepass" -passout pass:"keypass" | openssl rsa -passin pass:keypass -out opensearch_dashboards.key
# Extract the PEM-formatted X.509 certificate for OpenSearch Dashboards
openssl pkcs12 -in opensearch_dashboards.p12 -out opensearch_dashboards.crt -clcerts -passin pass:"storepass" -passout pass:
```
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
Bag Attributes
friendlyName: elasticsearch
friendlyName: opensearch
localKeyID: 54 69 6D 65 20 31 35 37 37 34 36 36 31 39 38 30 33 37
Key Attributes: <No Attributes>
Bag Attributes
friendlyName: ca
2.16.840.1.113894.746875.1.1: <Unsupported tag 6>
subject=/CN=Elastic Certificate Tool Autogenerated CA
issuer=/CN=Elastic Certificate Tool Autogenerated CA
subject=/CN=OpenSearch Certificate Tool Autogenerated CA
issuer=/CN=OpenSearch Certificate Tool Autogenerated CA
-----BEGIN CERTIFICATE-----
MIIDSzCCAjOgAwIBAgIUW0brhEtYK3tUBYlXnUa+AMmAX6kwDQYJKoZIhvcNAQEL
BQAwNDEyMDAGA1UEAxMpRWxhc3RpYyBDZXJ0aWZpY2F0ZSBUb29sIEF1dG9nZW5l
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
Bag Attributes
friendlyName: elasticsearch
friendlyName: opensearch
localKeyID: 54 69 6D 65 20 31 35 37 37 34 36 36 31 39 38 30 33 37
Key Attributes: <No Attributes>
Bag Attributes
friendlyName: elasticsearch
friendlyName: opensearch
localKeyID: 54 69 6D 65 20 31 35 37 37 34 36 36 31 39 38 30 33 37
subject=/CN=elasticsearch
issuer=/CN=Elastic Certificate Tool Autogenerated CA
subject=/CN=opensearch
issuer=/CN=OpenSearch Certificate Tool Autogenerated CA
-----BEGIN CERTIFICATE-----
MIIDQDCCAiigAwIBAgIVAI93OQE6tZffPyzenSg3ljE3JJBzMA0GCSqGSIb3DQEB
CwUAMDQxMjAwBgNVBAMTKUVsYXN0aWMgQ2VydGlmaWNhdGUgVG9vbCBBdXRvZ2Vu
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
Bag Attributes
friendlyName: kibana
friendlyName: opensearchDashboards
localKeyID: 54 69 6D 65 20 31 35 37 37 34 36 36 32 32 33 30 33 39
Key Attributes: <No Attributes>
Bag Attributes
friendlyName: kibana
friendlyName: opensearchDashboards
localKeyID: 54 69 6D 65 20 31 35 37 37 34 36 36 32 32 33 30 33 39
subject=/CN=kibana
issuer=/CN=Elastic Certificate Tool Autogenerated CA
subject=/CN=opensearchDashboards
issuer=/CN=OpenSearch Certificate Tool Autogenerated CA
-----BEGIN CERTIFICATE-----
MIIDOTCCAiGgAwIBAgIVANNWkg9lzNiLqNkMFhFKHcXyaZmqMA0GCSqGSIb3DQEB
CwUAMDQxMjAwBgNVBAMTKUVsYXN0aWMgQ2VydGlmaWNhdGUgVG9vbCBBdXRvZ2Vu
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
{
"name": "@kbn/dev-utils",
"name": "@osd/dev-utils",
"version": "1.0.0",
"private": true,
"license": "Apache-2.0",
"main": "./target/index.js",
"scripts": {
"build": "tsc",
"kbn:bootstrap": "yarn build",
"kbn:watch": "yarn build --watch"
"osd:bootstrap": "yarn build",
"osd:watch": "yarn build --watch"
},
"kibana": {
"opensearchDashboards": {
"devOnly": true
},
"dependencies": {
"@babel/core": "^7.11.6",
"@kbn/utils": "1.0.0",
"@osd/utils": "1.0.0",
"axios": "^0.19.2",
"chalk": "^4.1.0",
"cheerio": "0.22.0",
Expand All @@ -33,8 +33,8 @@
"vinyl": "^2.2.0"
},
"devDependencies": {
"@kbn/babel-preset": "1.0.0",
"@kbn/expect": "1.0.0",
"@osd/babel-preset": "1.0.0",
"@osd/expect": "1.0.0",
"@types/vinyl": "^2.0.4",
"chance": "1.0.18",
"typescript": "4.0.2"
Expand Down
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ export async function transformFileWithBabel(file: File) {
configFile: false,
sourceMaps: 'inline',
filename: file.path,
presets: [require.resolve('@kbn/babel-preset/node_preset')],
presets: [require.resolve('@osd/babel-preset/node_preset')],
});

if (!result || typeof result.code !== 'string') {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,16 @@
import { resolve } from 'path';

export const CA_CERT_PATH = resolve(__dirname, '../certs/ca.crt');
export const ES_KEY_PATH = resolve(__dirname, '../certs/elasticsearch.key');
export const ES_CERT_PATH = resolve(__dirname, '../certs/elasticsearch.crt');
export const ES_P12_PATH = resolve(__dirname, '../certs/elasticsearch.p12');
export const ES_P12_PASSWORD = 'storepass';
export const ES_EMPTYPASSWORD_P12_PATH = resolve(
export const OPENSEARCH_KEY_PATH = resolve(__dirname, '../certs/opensearch.key');
export const OPENSEARCH_CERT_PATH = resolve(__dirname, '../certs/opensearch.crt');
export const OPENSEARCH_P12_PATH = resolve(__dirname, '../certs/opensearch.p12');
export const OPENSEARCH_P12_PASSWORD = 'storepass';
export const OPENSEARCH_EMPTYPASSWORD_P12_PATH = resolve(
__dirname,
'../certs/elasticsearch_emptypassword.p12'
'../certs/opensearch_emptypassword.p12'
);
export const ES_NOPASSWORD_P12_PATH = resolve(__dirname, '../certs/elasticsearch_nopassword.p12');
export const KBN_KEY_PATH = resolve(__dirname, '../certs/kibana.key');
export const KBN_CERT_PATH = resolve(__dirname, '../certs/kibana.crt');
export const KBN_P12_PATH = resolve(__dirname, '../certs/kibana.p12');
export const KBN_P12_PASSWORD = 'storepass';
export const OPENSEARCH_NOPASSWORD_P12_PATH = resolve(__dirname, '../certs/opensearch_nopassword.p12');
export const OPENSEARCH_DASHBOARDS_KEY_PATH = resolve(__dirname, '../certs/opensearch_dashboards.key');
export const OPENSEARCH_DASHBOARDS_CERT_PATH = resolve(__dirname, '../certs/opensearch_dashboards.crt');
export const OPENSEARCH_DASHBOARDS_P12_PATH = resolve(__dirname, '../certs/opensearch_dashboards.p12');
export const OPENSEARCH_DASHBOARDS_P12_PASSWORD = 'storepass';
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Kibana CI Stats reporter
# OpenSearch Dashboards CI Stats reporter

We're working on building a new service, the Kibana CI Stats service, which will collect information about CI runs and metrics produced while testing Kibana, and then provide tools for reporting on those metrics in specific PRs and overall as a way to spot trends.
We're working on building a new service, the OpenSearch Dashboards CI Stats service, which will collect information about CI runs and metrics produced while testing OpenSearch Dashboards, and then provide tools for reporting on those metrics in specific PRs and overall as a way to spot trends.

### `CiStatsReporter`

This class integrates with the `ciStats.trackBuild {}` Jenkins Pipeline function, consuming the `KIBANA_CI_STATS_CONFIG` variable produced by that wrapper, and then allowing test code to report stats to the service.
This class integrates with the `ciStats.trackBuild {}` Jenkins Pipeline function, consuming the `OPENSEARCH_DASHBOARDS_CI_STATS_CONFIG` variable produced by that wrapper, and then allowing test code to report stats to the service.

To create an instance of the reporter, import the class and call `CiStatsReporter.fromEnv(log)` (passing it a tooling log).

#### `CiStatsReporter#metrics(metrics: Metric[])`

Use this method to record metrics in the Kibana CI Stats service.
Use this method to record metrics in the OpenSearch Dashboards CI Stats service.

```ts
interface Metric {
Expand All @@ -28,7 +28,7 @@ interface Metric {
Example:

```ts
import { CiStatsReporter, ToolingLog } from '@kbn/dev-utils';
import { CiStatsReporter, ToolingLog } from '@osd/dev-utils';

const log = new ToolingLog(...);
const reporter = CiStatsReporter.fromEnv(log)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,9 @@ export type CiStatsMetrics = Array<{
}>;

function parseConfig(log: ToolingLog) {
const configJson = process.env.KIBANA_CI_STATS_CONFIG;
const configJson = process.env.OPENSEARCH_DASHBOARDS_CI_STATS_CONFIG;
if (!configJson) {
log.debug('KIBANA_CI_STATS_CONFIG environment variable not found, disabling CiStatsReporter');
log.debug('OPENSEARCH_DASHBOARDS_CI_STATS_CONFIG environment variable not found, disabling CiStatsReporter');
return;
}

Expand All @@ -55,26 +55,26 @@ function parseConfig(log: ToolingLog) {
return validateConfig(log, config as { [k in keyof Config]: unknown });
}

log.warning('KIBANA_CI_STATS_CONFIG is invalid, stats will not be reported');
log.warning('OPENSEARCH_DASHBOARDS_CI_STATS_CONFIG is invalid, stats will not be reported');
return;
}

function validateConfig(log: ToolingLog, config: { [k in keyof Config]: unknown }) {
const validApiUrl = typeof config.apiUrl === 'string' && config.apiUrl.length !== 0;
if (!validApiUrl) {
log.warning('KIBANA_CI_STATS_CONFIG is missing a valid api url, stats will not be reported');
log.warning('OPENSEARCH_DASHBOARDS_CI_STATS_CONFIG is missing a valid api url, stats will not be reported');
return;
}

const validApiToken = typeof config.apiToken === 'string' && config.apiToken.length !== 0;
if (!validApiToken) {
log.warning('KIBANA_CI_STATS_CONFIG is missing a valid api token, stats will not be reported');
log.warning('OPENSEARCH_DASHBOARDS_CI_STATS_CONFIG is missing a valid api token, stats will not be reported');
return;
}

const validId = typeof config.buildId === 'string' && config.buildId.length !== 0;
if (!validId) {
log.warning('KIBANA_CI_STATS_CONFIG is missing a valid build id, stats will not be reported');
log.warning('OPENSEARCH_DASHBOARDS_CI_STATS_CONFIG is missing a valid build id, stats will not be reported');
return;
}

Expand Down Expand Up @@ -139,7 +139,7 @@ export class CiStatsReporter {

if (attempt === maxAttempts) {
this.log.warning(
`failed to reach kibana-ci-stats service too many times, unable to record metric ${bodySummary}`
`failed to reach opensearch-dashboards-ci-stats service too many times, unable to record metric ${bodySummary}`
);
return;
}
Expand All @@ -150,7 +150,7 @@ export class CiStatsReporter {
: 'no response';

this.log.warning(
`failed to reach kibana-ci-stats service [reason=${reason}], retrying in ${attempt} seconds`
`failed to reach opensearch-dashboards-ci-stats service [reason=${reason}], retrying in ${attempt} seconds`
);

await new Promise((resolve) => setTimeout(resolve, attempt * 1000));
Expand Down
Loading

0 comments on commit ebb36e5

Please sign in to comment.