Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DHFPROD-496 update tutorial documentation for 3.x #911

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions _assets/css/bundle.scss
Original file line number Diff line number Diff line change
Expand Up @@ -520,3 +520,7 @@ h2 {
}
}
}

.screenshot {
margin-top: 10px;
}
3 changes: 2 additions & 1 deletion _data/global.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
title: 'Data Hub Framework'
description: 'A data hub framework for MarkLogic'
odh_icon: '/images/odh.svg'
hub_version: '2.0.2'
hub_version: '3.0.0' # latest
hub_version_2x: '2.0.4' # for 2.x documentation
File renamed without changes.
34 changes: 17 additions & 17 deletions _pages/tutorial/2x.md → _pages/tutorial/2x/2x.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
layout: inner
title: Getting Started Tutorial 2.x
lead_text: ''
permalink: /tutorial/
permalink: /tutorial/2x/
---

## Intro
Before you get started you might want to check out our high level introductions:

- [What is it?](../understanding/concepts.md)
- [How does it work?](../understanding/how.md)
- [What is it?]({{site.baseurl}}/understanding/concepts/)
- [How does it work?]({{site.baseurl}}/understanding/how-it-works/)

## Building an Online Shopping Hub
This tutorial will walk you through setting up a very simple hub for harmonizing online shopping data.
Expand Down Expand Up @@ -55,21 +55,21 @@ Before you can run the hub, you will need to have some some software installed.


## Table of Contents
1. [Install the Data Hub Framework](install.md)
1. [Install the Data Hub Framework](./install/)
1. Loading Products
1. [Create the Product Entity](./create-product-entity.md)
1. [Create the Product Input Flow](./create-product-input-flow.md)
1. [Load the Product Data As-Is](./load-products-as-is.md)
1. [Create the Product Entity](./create-product-entity/)
1. [Create the Product Input Flow](./create-product-input-flow/)
1. [Load the Product Data As-Is](./load-products-as-is/)
1. Harmonizing Products
1. [Browse and Understand the Product Data](./browse-understand-product-data.md)
1. [Modeling the Product Entity](./modeling-product-entity.md)
1. [Harmonizing the Product Data](./harmonizing-product-data.md)
1. [Browse and Understand the Product Data](./browse-understand-product-data/)
1. [Modeling the Product Entity](./modeling-product-entity/)
1. [Harmonizing the Product Data](./harmonizing-product-data/)
1. Loading Orders
1. [Create the Order Entity](./create-order-entity.md)
1. [Create the Order Input Flow](./create-order-input-flow.md)
1. [Load the Orders As-Is](./load-orders-as-is.md)
1. [Create the Order Entity](./create-order-entity/)
1. [Create the Order Input Flow](./create-order-input-flow/)
1. [Load the Orders As-Is](./load-orders-as-is/)
1. Harmonizing Orders
1. [Modeling the Order Entity](./modeling-order-entity.md)
1. [Harmonizing the Order Data](./harmonizing-order-data.md)
1. [Serve the Data Out of MarkLogic](./serve-data.md)
1. [Wrapping Up](./wrapping-up.md)
1. [Modeling the Order Entity](./modeling-order-entity/)
1. [Harmonizing the Order Data](./harmonizing-order-data/)
1. [Serve the Data Out of MarkLogic](./serve-data/)
1. [Wrapping Up](./wrapping-up/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Browse and Understand the Product Data
lead_text: ''
permalink: /tutorial/browse-understand-product-data/
permalink: /tutorial/2x/browse-understand-product-data/
---

<i class="fa fa-hand-pointer-o"></i> Click on the **Browse** tab in the top navigation bar.
Expand All @@ -25,4 +25,4 @@ You will be shown the raw data.

## Up Next

[Modeling the Product Entity](modeling-product-entity.md)
[Modeling the Product Entity](../modeling-product-entity/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Create the Order Entity
lead_text: ''
permalink: /tutorial/create-order-entity/
permalink: /tutorial/2x/create-order-entity/
---

Just like we did for the Product data, we model an empty Order then load the raw data.
Expand All @@ -25,4 +25,4 @@ You have just created your second Entity.

## Up Next

[Create the Order Input Flow](create-order-input-flow.md)
[Create the Order Input Flow](../create-order-input-flow/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Create the Order Input Flow
lead_text: ''
permalink: /tutorial/create-order-input-flow/
permalink: /tutorial/2x/create-order-input-flow/
---

Let's create an input flow for Orders so that we can load data as-is.
Expand All @@ -20,4 +20,4 @@ Let's create an input flow for Orders so that we can load data as-is.

## Up Next

[Load the Order Data As-Is](load-orders-as-is.md)
[Load the Order Data As-Is](../load-orders-as-is/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Create the Product Entity
lead_text: ''
permalink: /tutorial/create-product-entity/
permalink: /tutorial/2x/create-product-entity/
---

Entities are the business objects that you will be working with in the hub. For this tutorial we be using MarkLogic 9's [Entity Services feature](https://docs.marklogic.com/guide/entity-services). Entity Services allows you to model your business entities as json. Using these data models you can then generate code scaffolding, database configurations, index settings, and validations. The Data Hub Framework handles many of these for you. Later in this tutorial we will use the index and scaffolding generation.
Expand All @@ -21,4 +21,4 @@ You have just created an Entity, albeit not very useful since we didn't define a
![Product Entity]({{site.baseurl}}/images/2x/first-entity.png)

## Next Up
[Create the Product Input Flow](create-product-input-flow.md)
[Create the Product Input Flow](../create-product-input-flow/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Create the Product Input Flow
lead_text: ''
permalink: /tutorial/create-product-input-flow/
permalink: /tutorial/2x/create-product-input-flow/
---

> **Input Flows** are responsible for getting data into the Hub staging area. These flows wrap incoming data in Envelopes. It's useful to wrap incoming data in Envelopes to track lineage and provenance. _Who loaded this data? When was it loaded? Where did it come from?_{:.smaller}
Expand All @@ -23,4 +23,4 @@ We start by creating an input flow so that we can load data as-is. This gives us
![New Load Products Flows]({{site.baseurl}}/images/2x/create-load-product-flow.png)

## Next Up
[Load the Product Data As-Is](load-products-as-is.md)
[Load the Product Data As-Is](../load-products-as-is/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Harmonizing Order Data
lead_text: ''
permalink: /tutorial/harmonizing-order-data/
permalink: /tutorial/2x/harmonizing-order-data/
---

## Harmonizing the Order Data
Expand Down Expand Up @@ -105,4 +105,4 @@ You should see harmonized documents in the search results.

## Up Next

[Serve the Data Out of MarkLogic](serve-data.md)
[Serve the Data Out of MarkLogic](../serve-data/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Browse and Understand the Product Data
lead_text: ''
permalink: /tutorial/harmonizing-product-data/
permalink: /tutorial/2x/harmonizing-product-data/
---

## Harmonizing the Product Data
Expand Down Expand Up @@ -120,4 +120,4 @@ You should see harmonized documents in the search results.

Congratulations! You just loaded and harmonized your Product data. Up next is doing the same thing for the Order data.

[Loading Orders > Create the Order Entity](create-order-entity.md)
[Loading Orders > Create the Order Entity](../create-order-entity/)
10 changes: 5 additions & 5 deletions _pages/tutorial/install.md → _pages/tutorial/2x/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Install the Framework
lead_text: ''
permalink: /tutorial/install/
permalink: /tutorial/2x/install/
---

## 1 - Download and Install MarkLogic
Expand Down Expand Up @@ -37,7 +37,7 @@ Your directory should look like this:
<pre class="cmdline">
data-hub
+-- input
+-- quick-start-{{ site.data.global.hub_version }}.war
+-- quick-start-{{ site.data.global.hub_version_2x }}.war
</pre>

## 4 - Run the QuickStart
Expand All @@ -50,13 +50,13 @@ Let's run the Quickstart:
1. Run the War

<pre class="cmdline">
java -jar quick-start-{{ site.data.global.hub_version }}.war
java -jar quick-start-{{ site.data.global.hub_version_2x }}.war
</pre>

*If you need to run on a different port then add the --server.port option*

<pre class="cmdline">
java -jar quick-start-{{ site.data.global.hub_version }}.war --server.port=9000
java -jar quick-start-{{ site.data.global.hub_version_2x }}.war --server.port=9000
</pre>

![Start QuickStart]({{site.baseurl}}/images/2x/start-quickstart.png)
Expand Down Expand Up @@ -105,4 +105,4 @@ The four databases are:

## Up Next

[Loading Products > Create the Product Entity](create-product-entity.md)
[Loading Products > Create the Product Entity](../create-product-entity/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Load the Orders As-Is
lead_text: ''
permalink: /tutorial/load-orders-as-is/
permalink: /tutorial/2x/load-orders-as-is/
---

## Ingest Order Data As-Is
Expand Down Expand Up @@ -39,4 +39,4 @@ On the Jobs tab you will see a list of previously run jobs. This interface is se

## Up Next

[Harmonizing Orders > Modeling the Order Entity](modeling-order-entity.md)
[Harmonizing Orders > Modeling the Order Entity](../modeling-order-entity/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Load the Product data As-Is
lead_text: ''
permalink: /tutorial/load-products-as-is/
permalink: /tutorial/2x/load-products-as-is/
---

Now you will use the QuickStart Input Flow wizard to load the product data into MarkLogic. The QuickStart application will use the settings you are about to specify to run [MarkLogic Content Pump MLCP](https://docs.marklogic.com/guide/mlcp){:target="_blank"} for you. MLCP is a command line tool for loading large amounts of data into MarkLogic. This Input Flow Wizard is quite awesome because you don't have to learn the intricacies of the MLCP command line to get started.
Expand Down Expand Up @@ -73,4 +73,4 @@ The Trace details view allows you to click on each plugin in the Flow to see the

## Up Next

[Harmonizing Products > Browse and Understand the Product Data](browse-understand-product-data.md)
[Harmonizing Products > Browse and Understand the Product Data](../browse-understand-product-data/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Modeling the Order Entity
lead_text: ''
permalink: /tutorial/modeling-order-entity/
permalink: /tutorial/2x/modeling-order-entity/
---

In real life we would browse the loaded data to try and better understand it. Let's skip ahead and pretend that we have spoken to our Data Guru again.
Expand Down Expand Up @@ -67,4 +67,4 @@ One of the benefits of modeling your data with Entity Services is that you can u

## Up Next

[Harmonizing the Order Data](harmonizing-order-data.md)
[Harmonizing the Order Data](../harmonizing-order-data/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Modeling the Product Entity
lead_text: ''
permalink: /tutorial/modeling-product-entity
permalink: /tutorial/2x/modeling-product-entity
---

## Modeling the Product Entity
Expand Down Expand Up @@ -44,4 +44,4 @@ One of the benefits of modeling your data with Entity Services is that you can u

## Up Next

[Harmonizing the Product Data](harmonizing-product-data.md)
[Harmonizing the Product Data](../harmonizing-product-data/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Serve the Data
lead_text: ''
permalink: /tutorial/serve-data/
permalink: /tutorial/2x/serve-data/
---

You have just successfully loaded two data sources and harmonized them both.
Expand All @@ -19,4 +19,4 @@ Open the [Final Search Endpoint](http://localhost:8011/v1/search?format=json){:t

## Up Next

[Wrapping Up](wrapping-up.md)
[Wrapping Up](../wrapping-up/)
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: inner
title: Getting Started Tutorial 2.x<br>Wrapping Up
lead_text: ''
permalink: /tutorial/wrapping-up
permalink: /tutorial/2x/wrapping-up
---

## Congratulations!
Expand Down
96 changes: 96 additions & 0 deletions _pages/tutorial/3x/3x.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
layout: inner
title: Getting Started Tutorial 3.x
lead_text: ''
permalink: /tutorial/
---

## Before You Start
You might want to check out our high-level introductions before starting this tutorial:

- [What is an Operational Data Hub?]({{site.baseurl}}/understanding/concepts/)
- [How does the Data Hub Framework work?]({{site.baseurl}}/understanding/how-it-works/)

## Build an Online Shopping Hub
This tutorial will walk you through setting up a simple data hub for harmonizing online shopping data.

Imagine you're a company that sells board games and board game accessories. You've been tasked with creating a data hub on top of MarkLogic. You must load all of your product and order data into MarkLogic and harmonize it for use in a new application.

You will take the following approach:

1. Load product data as-is
1. Harmonize product data
1. Load order data as-is
1. Harmonize order data
1. Serve the data to downstream clients

### In a Hurry?
The finished version of this tutorial is available for you to download and play with: [Finished Online Shopping Hub Example](https://github.com/marklogic-community/marklogic-data-hub/tree/develop/examples/online-store){:target="_blank"}

### Introducing QuickStart
This tutorial uses QuickStart, an easy-to-use user interface that you can run locally to start working with the Data Hub Framework quickly. With QuickStart, you will have a working data hub in a matter of minutes. No need to worry about deployment strategies or configuration details. Simply run the QuickStart .war (Java web application archive) and point it at your MarkLogic installation.

_QuickStart is a DevOps tool. It is meant to be run on your development machine to aid you in quickly deploying your hub._

## Prerequisites

Before you can begin this tutorial and work with the Data Hub Framework, you need to have some software installed.

- **[Oracle's Java 8 JRE](http://www.oracle.com/technetwork/java/javase/downloads/index.html)**

Java versions later than 8 are currently not supported. We have not tested with OpenJDK. Not sure which Java version you have? Run the following from the command line:
<pre class="cmdline">
java -version
</pre>
If you have version 8, you will see something like the following:
<pre class="cmdline">
java version "1.8.0_40"
</pre>
The second number denotes the version.

- **[MarkLogic 9](https://developer.marklogic.com/products)**

Version 9.0-5 or later, installed and initialized. You can view the [official instructions](https://docs.marklogic.com/guide/installation){:target="_blank"} for installing MarkLogic.

Not sure which MarkLogic version you have? Open your web browser to [http://localhost:8001](http://localhost:8001). After logging in, look at the top-left corner for the version info:

![Check ML Version]({{site.baseurl}}/images/3x/ml-version-check.png)

The following video describes how to install MarkLogic on Windows 10:
<iframe width="420" height="315" src="https://www.youtube.com/embed/WaRi9HMtz5Q" frameborder="0" allowfullscreen></iframe>

- **A modern web browser**

Chrome or Firefox works best. Use IE at your own risk.

## Common Concerns
**I have a MarkLogic instance but it already has awesome stuff in it. Will this tutorial mess that up?**
No. The Data Hub Framework is installed on isolated databases and application servers. It is possible that the default DHF ports (8010, 8011, 8012, 8013) may already be in use. In that case you will be warned about the conflicts and given the opportunity to change them. The DHF will not harm any existing settings.

**How difficult is it to remove this tutorial when I am finished?**
Easy. Just click Settings at the top of QuickStart and then click Uninstall.


## Table of Contents
1. [Install the Data Hub Framework](./install/)
1. Loading Products
1. [Create the Product Entity](./create-product-entity/)
1. [Create the Product Input Flow](./create-product-input-flow/)
1. [Load the Product Data As-Is](./load-products-as-is/)
1. Harmonizing Products
1. [Browse and Understand the Product Data](./browse-understand-product-data/)
1. [Modeling the Product Entity](./modeling-product-entity/)
1. [Harmonizing the Product Data](./harmonizing-product-data/)
1. Loading Orders
1. [Create the Order Entity](./create-order-entity/)
1. [Create the Order Input Flow](./create-order-input-flow/)
1. [Load the Orders As-Is](./load-orders-as-is/)
1. Harmonizing Orders
1. [Modeling the Order Entity](./modeling-order-entity/)
1. [Harmonizing the Order Data](./harmonizing-order-data/)
1. [Serve the Data Out of MarkLogic](./serve-data/)
1. [Wrapping Up](./wrapping-up/)

## Previous Tutorial Versions
- [Getting Started Tutorial 2.x](./2x/)
- [Getting Started Tutorial 1.x](./1x/)
Loading