Skip to content

Commit

Permalink
migrate site to docusaurus
Browse files Browse the repository at this point in the history
  • Loading branch information
ibarrajo committed Nov 16, 2024
1 parent 3a5ebc4 commit 70be568
Show file tree
Hide file tree
Showing 224 changed files with 12,811 additions and 34,210 deletions.
28 changes: 20 additions & 8 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,30 @@ cypress/plugins
cypress/support
cypress/videos

# node
dist
dist-blog
node_modules
npm-debug.log
# dependencies
/node_modules

# package-lock file is required for cypress github action to work.
# See https://github.com/marketplace/actions/cypress-io#installation for more information.
# package-lock.json
yarn-error.log

# Docusaurus cache and generated files
.docusaurus

### VisualStudioCode ###
.vscode/*

# Production
/build

# Generated files
.docusaurus
.cache-loader

# Misc
.env.local
.env.development.local
.env.test.local
.env.production.local

npm-debug.log*
yarn-debug.log*
yarn-error.log*
Empty file added CHANGELOG.md
Empty file.
Empty file added CONTRIBUTING.md
Empty file.
62 changes: 49 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,54 @@
# [Cadence docs](https://cadenceworkflow.io) · ![Build and Deploy](https://img.shields.io/github/actions/workflow/status/uber/Cadence-Docs/publish-to-gh-pages.yml?label=Build%20and%20Deploy&link=https%3A%2F%2Fgithub.com%2Fuber%2FCadence-Docs%2Factions%2Fworkflows%2Fpublish-to-gh-pages.yml) ![Nightly integration test](https://img.shields.io/github/actions/workflow/status/uber/Cadence-Docs/nightly-integration-test.yml?label=Nightly%20integration%20test&link=https%3A%2F%2Fgithub.com%2Fuber%2FCadence-Docs%2Factions%2Fworkflows%2Fnightly-integration-test.yml)

## Setting up for local development
This will start a local server and can be accessed at http://localhost:8080/
1. Run `npm install`
2. Run `npm run start`

### Adding pages to docs
1. Add the page under `Cadence-Docs/src/docs` in the correct place in the hierarchy
2. Add the page to `Cadence-Docs/src/.vuepress/config.js`

## Setting up for local development for blog pages
This will start a local server and can be accessed at http://localhost:8080/blog
1. Run `npm install`
2. Run `npm run start:blog`


# cadenceworkflow.io

[Cadence docs](https://cadenceworkflow.io) is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.

### Installation

```
$ yarn
```

### Local Development

```
$ yarn start
```

This command starts a local development server and opens up a browser window at http://localhost:3000/. Most changes are reflected live without having to restart the server.

### Build

```
$ yarn build
```

This command generates static content into the `build` directory and can be served using any static contents hosting service.

### Deployment

Using SSH:

```
$ USE_SSH=true yarn deploy
```

Not using SSH:

```
$ GIT_USER=<Your GitHub username> yarn deploy
```

If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.


# NPM Registry

Ensure you have a `.npmrc` [file](https://docs.npmjs.com/cli/v9/configuring-npm/npmrc/) configured with `registry=https://registry.npmjs.org/`.
This will ensure the dependencies are pulled from the correct source and to prevent internal npm registries from being pushed onto the package-lock.json

## License

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ The next step - client migration. We have more users for the Go SDK at Uber, tha
We plan to support TChannel for a few more releases and then eventually drop it in a future.

## System overview
![gRPC migration overview](/img/grpc-migration.svg)
![gRPC migration overview](./grpc-migration.svg)

1. The frontend of [Cadence Server](https://github.com/uber/cadence) exposes two inbounds for both gRPC and TChannel starting [v0.21.0 release](https://github.com/uber/cadence/releases/tag/v0.21.0). gRPC traffic is being served on a different port that can be configured [here](https://github.com/uber/cadence/blob/v0.21.0/config/development.yaml#L25). For gRPC API we introduced [proto IDL](https://github.com/uber/cadence-idl/tree/master/proto/uber/cadence/api/v1) definitions. We will keep TChannel open on frontend for some time to allow gradual client migration.
2. Starting with [v0.21.0](https://github.com/uber/cadence/releases/tag/v0.21.0) internal components of Cadence Server (history & matching) also started accepting gRPC traffic. Sending traffic via gRPC is off by default and could be enabled with a flag in [dynamic config](https://github.com/uber/cadence/blob/v0.21.0/config/dynamicconfig/development.yaml#L10). Planned for v0.24.0 it will be enabled by default, with an option to opt-out.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,40 +11,40 @@ We released a user survey earlier this year to learn about who our users are, ho

Here are some highlights in text and you can check out the visuals to get more details:

![using.png](../../shared/img/2023-survey-results/using.png)
![using.png](./2023-survey-results/using.png)

![job_role.png](../../shared/img/2023-survey-results/job_role.png)
![job_role.png](./2023-survey-results/job_role.png)

Most of the people who replied to our survey were engineers who were already using Cadence, actively evaluating, or migrating from a similar technology. This was exciting to hear! Some of you have contacted us to learn more about benchmarks, scale, and ideal use cases. We will share more guidelines about this but until then, feel free to contact us over our Slack workspace for guidance.

![scale.png](../../shared/img/2023-survey-results/scale.png)
![scale.png](./2023-survey-results/scale.png)

The scale our users operating Cadence varies from thousands to billions of workflows per month. It was exciting to see it being used in both small and large scale companies.

![time_zone.png](../../shared/img/2023-survey-results/time_zone.png)
![time_zone.png](./2023-survey-results/time_zone.png)

Most survey responders were from Europe compared to any other place. This is in-line with the Cadence team growing its presence in Europe. Users from different places also contacted us to contribute to Cadence as a follow up to the survey. We will start putting up-for-grabs and new-starter tasks on Github. Several of them wanted to meet with a Zoom call and to discuss their use cases and best practices. As the Cadence team has presence in both the EU and the US, we welcome all our users to contact us anytime. Slack is the fastest way to reach us.

![following.png](../../shared/img/2023-survey-results/following.png)
![following.png](./2023-survey-results/following.png)

![channels.png](../../shared/img/2023-survey-results/channels.png)
![channels.png](./2023-survey-results/channels.png)

Cadence is followed in [Slack](https://uber-cadence.slack.com/) the most, then [Github](https://github.com/uber/cadence) and [LinkedIn](https://www.linkedin.com/company/cadenceworkflow/). We are the most active in Slack and we plan to be more active in other mediums as well.

![scenarios.png](../../shared/img/2023-survey-results/scenarios.png)
![scenarios.png](./2023-survey-results/scenarios.png)
All of our main use cases were used across the board. While we mentioned the most common cases, several others were mentioned as a comment: enhanced timers, leader election etc.

We found out that Cadence has been used in several science communities. Some of them were using community built clients and were asking if we are going to support more languages. We are planning to take ownership of the Python and Javascript/Typescript clients and support them officially.

![improvement.png](../../shared/img/2023-survey-results/improvement.png)
![improvement.png](./2023-survey-results/improvement.png)

Documentation is by far what our users wanted improvements on. We are revamping our documentation soon and there will be major changes on our website soon.

![help_stage.png](../../shared/img/2023-survey-results/help_stage.png)
![help_stage.png](./2023-survey-results/help_stage.png)

Other requests were about observability, debuggability, operability, and usability. These areas have been our main focus this year and we are planning to release updates and blogs about them.

![support.png](../../shared/img/2023-survey-results/support.png)
![support.png](./2023-survey-results/support.png)

We noticed most of our users need help once a month or more. While we welcome questions and discussions over the mediums mentioned above, we plan to make more public posts about the common issues using our blog, StackOverflow, LinkedIn, or Twitter.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,10 +70,10 @@ Congratulations, you have successfully run your very first Cadence workflow.

For a bonus point, the Cadence team has also developed a demonstrative web dashboard to visualize the history of all workflows you have run when you start the Cadence server. Check http://localhost:8088 to see the dashboard like this.

![cadencde-ui](../../shared/img/cadence_ui.png)
![cadencde-ui](./cadence_ui.png)

This web portal persists all historical workflow you have run recently. Search for the domain you used for this tutorial. In our case, type `test-domain` and hit enter. You may see a list of workflows with detailed information. Feel free to explore the web UI and raise your suggestions to our [Github repo](https://github.com/uber/cadence-web).

![cadence-ui-detailed](../../shared/img/cadence_ui_detailed.png)
![cadence-ui-detailed](./cadence_ui_detailed.png)

For the incoming blogs, we will cover more advanced topics and use cases with Cadence.
For the incoming blogs, we will cover more advanced topics and use cases with Cadence.
File renamed without changes
File renamed without changes
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ If you haven’t heard about Cadence, this section is for you. In a short descri

### What is a Workflow?

![workflow.png](../../shared/img/2024-yearly-update/workflow.png)
![workflow.png](./2024-yearly-update/workflow.png)

In the simplest definition, it is “a multi-step execution”. Step here represents individual operations that are a little heavier than small in-process function calls. Although they are not limited to those: it could be a separate service call, processing a large dataset, map-reduce, thread sleep, scheduling next run, waiting for an external input, starting a sub workflow etc. It’s anything a user thinks as a single unit of logic in their code. Those steps often have dependencies among themselves. Some steps, including the very first step, might require external triggers (e.g. button click) or schedules. In the more broader meaning, any multi-step function or service is a workflow in principle.

Expand All @@ -31,7 +31,7 @@ Given this natural evolution of workflow requirements, it’s better to have a c

### Benefits

![cadence-benefits.png](../../shared/img/2024-yearly-update/cadence-benefits.png)
![cadence-benefits.png](./2024-yearly-update/cadence-benefits.png)

With Cadence, many overheads that need to be built for any well-supported service come for free. Here are some highlights (see [cadenceworkflow.io](http://cadenceworkflow.io)):

Expand Down Expand Up @@ -213,4 +213,4 @@ Github: [https://github.com/uber/cadence](https://github.com/uber/cadence)

Since last year, we have been contacted by various companies to take on bigger projects on the Cadence project. As we have been investing in code coverage and refactoring Cadence for a cleaner codebase, this will be a lot easier now. Let us know if you have project ideas to contribute or if you’d like to pick something we already planned.

Our monthly community meetings are still ongoing, too. That is the best place to get heard and be involved in our decision-making process. Let us know so we can send you an invite. We are also working on a broader governing model to open up this project to more people. Stay tuned for updates on this topic!
Our monthly community meetings are still ongoing, too. That is the best place to get heard and be involved in our decision-making process. Let us know so we can send you an invite. We are also working on a broader governing model to open up this project to more people. Stay tuned for updates on this topic!
99 changes: 99 additions & 0 deletions blog/authors.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
chopincode:
name: Chris Qin
title: Applications Developer @ Uber
url: https://www.linkedin.com/in/chrisqin0610/
image_url: https://github.com/chopincode.png
page: true
socials:
linkedin: https://www.linkedin.com/in/chrisqin0610/
github: chopincode

enderdemirkaya:
name: Ender Demirkaya
title: Senior Manager at Uber, Cadence. Author of the Software Engineering Handbook
url: https://www.linkedin.com/in/enderdemirkaya/
image_url: https://github.com/demirkayaender.png
page: true
socials:
linkedin: https://www.linkedin.com/in/enderdemirkaya/
github: demirkayaender

jakobht:
name: Jakob Haahr Taankvist
title: Software Engineer II @ Uber
url: https://www.linkedin.com/in/jakob-taankvist/
image_url: https://github.com/jakobht.png
page: true
socials:
linkedin: https://www.linkedin.com/in/jakob-taankvist/
github: jakobht

ibarrajo:
name: Josué Alexander Ibarra
title: Developer Advocate @ Uber
url: https://github.com/ibarrajo/
image_url: https://github.com/ibarrajo.png
page: true
socials:
github: ibarrajo
linkedin: https://www.linkedin.com/in/elninja/

kcorbett-netapp:
name: Kevin Corbett
title: Open Source Community Manager | @instaclustr @NetApp
url: https://github.com/kcorbett-netapp
image_url: https://github.com/kcorbett-netapp.png
page: true
socials:
linkedin: https://www.linkedin.com/in/kcorb95/
github: kcorbett-netapp

meiliang86:
name: Liang Mei
title: Engineering Manager @ Uber
url: https://www.linkedin.com/in/meiliang86/
image_url: https://github.com/meiliang86.png
page: true
socials:
linkedin: https://www.linkedin.com/in/meiliang86/
github: meiliang86

sharanf:
name: Sharan Foga
title: Director of Operations & Customer Success @ Encube Technologies
url: https://www.linkedin.com/in/sfoga/
image_url: https://github.com/sharanf.png
page: true
socials:
linkedin: https://www.linkedin.com/in/sfoga/
github: sharanf

taylanisikdemir:
name: Taylan Isikdemir
title: Sr. Staff Software Engineer @ Uber
url: https://www.linkedin.com/in/taylan-isikdemir
image_url: https://github.com/taylanisikdemir.png
page: true
socials:
linkedin: https://www.linkedin.com/in/taylan-isikdemir
github: taylanisikdemir

vytautas-karpavicius:
name: Vytautas Karpavicius
title: Software Engineer @ Uber
url: https://www.linkedin.com/in/vytautas-karpavicius
image_url: https://github.com/vytautas-karpavicius.png
page: true
socials:
linkedin: https://www.linkedin.com/in/vytautas-karpavicius
github: vytautas-karpavicius

Shaddoll:
name: Zijian Chen
title: Software Engineer @ Uber
url: https://www.linkedin.com/in/zijian-chen-5868938b/
image_url: https://github.com/Shaddoll.png
page: true
socials:
linkedin: https://www.linkedin.com/in/zijian-chen-5868938b/
github: Shaddoll
14 changes: 14 additions & 0 deletions blog/tags.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
community-spotlight:
label: Community Spotlight
permalink: /community-spotlights
description: Community Spotlight tag description

announcement:
label: Announcement
permalink: /announcements
description: Announcement tag description

deep-dive:
label: Deep Dives
permalink: /deep-dives
description: Deep Dives tag description
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: "Zonal Isolation for Cadence Workflows"
subtitle: test

date: 2024-10-14
author: Zijian Chen
author: Zijian Chen
authorlink: https://www.linkedin.com/in/zijian-chen-5868938b/
---
At Uber, we want to achieve regional resilience such that losing a zone within a region can be tolerated without requiring a cross-region failover. We also want to make sure that losing a zone only affects a subset of workload, at most, rather than everything. However, in Cadence-based systems, the workload in a region is distributed randomly across all workers in the region at a “task-level granularity”, which means a workflow may be worked on by any worker in the region where the domain is active. To achieve this goal, we introduced Zonal Isolation for Cadence Workflows - a feature designed to pin workflows to the zone they are started in, so that zonal isolation can be achieved at a workflow-level.
Expand All @@ -19,13 +19,14 @@ Infrastructure-level isolation is quite challenging to implement as it requires

As a result, the focus remains on achieving task-level zonal isolation outside the Cadence server, which offers a more practical and immediate way to improve system resilience. It provides the capability of ensuring that an unhealthy zone (i.e. bad deployment of workers) only affect a subset of workflows (started from a certain zone) rather than every workflow in a Cadence domain.

<!-- truncate -->

## How Zonal Isolation Works in Cadence?

### Architecture
Here is what the architecture of a zonally isolated Cadence-based system looks like:
![zonal isolation overview](/img/zone-isolation.png)
*<p style="text-align:center;">Fig: Workflows started in one zone are only dispatched to workers from the same zone. Colors to emphasize pinning.</p>*
![zonal isolation overview](./zone-isolation.png)
*<p>Fig: Workflows started in one zone are only dispatched to workers from the same zone. Colors to emphasize pinning.</p>*
### Implementation

#### Determine the zone of a workflow and workers
Expand Down Expand Up @@ -122,4 +123,4 @@ This metric can be used in the following ways:
## Next Step
Currently, Zonal Isolation is implemented as **hard isolation**, where tasks are strictly limited to the same zone as the originating workflow. To address the traffic skewness issue, we are working on iterating this feature to introduce **soft isolation**. This relaxed version will allow for some task leakage between zones if traffic skewness is detected, ensuring better worker utilization and reducing latency.

This enhancement is one of our major ongoing projects, and we plan to share more details in a future blog post once it is launched.
This enhancement is one of our major ongoing projects, and we plan to share more details in a future blog post once it is launched.
File renamed without changes
1 change: 1 addition & 0 deletions changelog/source/authors.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{}
28 changes: 28 additions & 0 deletions community/0-support.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Support

On this page we've listed some Cadence-related communities that you can be a part of.
You can look up other resources in this section for additional learning materials.

Before participating in Cadence' communities, [please read our Code of Conduct](https://github.com/uber/cadence/tree/master?tab=coc-ov-file#readme). We have adopted the [Contributor Covenant](https://www.contributor-covenant.org/) and we expect that all community members adhere to the guidelines within.

## Stack Overflow {#stack-overflow}

Stack Overflow is a popular forum to ask code-level questions or if you're stuck with a specific error. Read through the [existing questions](https://stackoverflow.com/questions/tagged/cadence-workflow+uber-cadence) tagged with **cadence** or [ask your own](https://stackoverflow.com/questions/ask?tags=cadence-workflow)!

## Discussion forums {#discussion-forums}

There are many online forums for discussion about best practices and application architecture as well as the future of Cadence. If you have an answerable code-level question, Stack Overflow is usually a better fit.

- [Cadence Community Slack](http://t.uber.com/cadence-slack)

## Feature requests {#feature-requests}

- [Github Discussions](https://github.com/uber/cadence/discussions)

## Bug Reports {#bug-reports}

You can report Cadence bugs by opening an issue on the most relevant repository, the primary being [uber/cadence](https://github.com/uber/cadence/issues)

## News {#news}

For the latest news about Cadence, follow the [official Cadence blog](/blog) on this website, [**Cadence Workflow** on LinkedIn](https://www.linkedin.com/company/cadenceworkflow) and [**@cadenceworkflow** on X](https://x.com/cadenceworkflow)
Loading

0 comments on commit 70be568

Please sign in to comment.