Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Feast admin quickstart guide #155

Merged
merged 2 commits into from
Mar 17, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ __Discovery__: Feast allows users to easily explore and discover features and th
* [Components](docs/components.md)
* [Concepts](docs/concepts.md)

For Feast administrators:
* [Installation quickstart](docs/install.md}
* [Helm charts](charts/README.md} details
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo here "}"


## Notice

Feast is still under active development. Your feedback and contributions are important to us.
Expand Down
87 changes: 87 additions & 0 deletions docs/install.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Feast Installation Quickstart Guide

This is a quickstart guide for Feast administrators setting up a Feast
deployment for the first time.

Feast is meant to run on Kubernetes, and currently requires managed
services from GCP for certain operations
* Dataflow (for loading feature data into Feast)
* BigQuery (to act as feature warehouse)
* Pub/Sub (for event data)
* Cloud Storage (for staging data and logs)

This guide will assume that users are using GKE (Google Container
Engine), and that these managed services are available. In addition,
Redis will be used for the feature serving database.

## Prerequisites

* Kubernetes cluster
* The user should have a GKE cluster provisioned, with `kubectl` set
up to access this cluster
* This cluster should have the right scopes to start jobs on
Dataflow and to modify BigQuery datasets and tables. The simplest
way to set this up is by setting the scope to `cloud-platform` when
provisioning the Kubernetes cluster.
* [Helm](https://helm.sh/)
* Helm should be installed locally and Tiller should be installed
within this cluster. As noted
[here](https://medium.com/google-cloud/helm-on-gke-cluster-quick-hands-on-guide-ecffad94b0),
make sure you have the cluster-admin role attached to the Helm
service account.
* Feast repository
* You have cloned the [Feast
repository](https://github.com/gojek/feast/) and your command line
is active in the root of the repository

## Set up the environment

Set the following environmental variables before beginning the installation

```sh
GCP_PROJECT=my-feast-project
FEAST_CLUSTER=feast
FEAST_REPO=$(pwd)
FEAST_VERSION=0.1.0
FEAST_STORAGE_BUCKET=gs://${PROJECT}-feast
```

Ensure that your `kubectl` context is set to the correct cluster

```sh
gcloud container clusters get-credentials "${FEAST_CLUSTER}" --project "${GCP_PROJECT}"
```

Create a storage bucket for Feast to stage data

```sh
gsutil mb "${FEAST_STORAGE_BUCKET}"
```

## Install Feast using Helm

Create a copy of the Helm `values.yaml` file. This file will need to
be configured with the specifics of your environment.

```sh
cp charts/feast/values.yaml .
```

Keys that likely need to be changed based on the user’s environment are
* `core.projectId` - your GCP project
* `core.image.tag` - the current Feast version
* `serving.image.tag` - the current Feast version
* `core.jobs.*`
* `DataflowRunner` or `DirectRunner` or `FlinkRunner` settings for Beam
* Note: `DirectRunner` should not be used in production it is for testing only.

See the [documentation page](#) for the complete list of options.

You can update `values.yaml` directly, or specify overrides on the Helm command line:

```sh
helm install --name feast charts/dist/feast-${FEAST_VERSION}.tgz \
--set core.projectId=${GCP_PROJECT} \
--set core.image.tag=${FEAST_VERSION} \
--set serving.image.tag=${FEAST_VERSION}
```