Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.10.2 #413

Merged
merged 32 commits into from
Mar 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
ca6e820
Improve parameter input modal (#412)
yohamta Mar 18, 2023
49e8dbe
Fix dead links in README
yohamta Mar 18, 2023
21f42e3
Fix dead links in README
yohamta Mar 18, 2023
c5087de
fix bug
yohamta Mar 18, 2023
73497be
remove unused code
yohamta Mar 18, 2023
c8d4e51
fix bug
yohamta Mar 18, 2023
13002c6
Update Makefile
yohamta Mar 18, 2023
9901de8
fix bug
yohamta Mar 18, 2023
cd6635e
fix bug
yohamta Mar 18, 2023
d45c2be
refactor scheduler function
yohamta Mar 19, 2023
f7c1cf5
refactor builder
yohamta Mar 19, 2023
4ef1868
refactor DAG builder
yohamta Mar 19, 2023
db45b19
Refactor DAGBuilder
yohamta Mar 19, 2023
5592c6b
fix bug
yohamta Mar 19, 2023
9a605df
Refactor DAG struct
yohamta Mar 19, 2023
6cc24ea
fix bug
yohamta Mar 19, 2023
f7c1a31
Refactor DAG struct
yohamta Mar 19, 2023
156f671
Refactor DAG file loader
yohamta Mar 19, 2023
8f64c8e
Refactor DAG builder
yohamta Mar 19, 2023
079bfc8
Refactor DAG loader
yohamta Mar 19, 2023
db66955
Rename unclear variable names
yohamta Mar 19, 2023
50f958e
Refactor Condition struct code
yohamta Mar 19, 2023
63912e4
Refactor step code
yohamta Mar 19, 2023
1e98dd0
Refactor context code
yohamta Mar 19, 2023
5ee99f4
Add errors package
yohamta Mar 19, 2023
1756cb6
Add errors package
yohamta Mar 19, 2023
7a5a596
Refactor cmd package
yohamta Mar 19, 2023
5341ab4
Refactor cmd package
yohamta Mar 19, 2023
ace8be6
Refactor agent package
yohamta Mar 19, 2023
c0cca71
update chatgpt example
yohamta Mar 19, 2023
1ead71f
fix bug
yohamta Mar 19, 2023
5d9c0ea
replace new lines in mail body
yohamta Mar 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ scheduler: build-dir
./bin/dagu scheduler

.PHONY: test
test: build
test:
go test -v ./...

.PHONY: test-clean
Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ Dagu is a tool for scheduling and running tasks based on a directed acyclic grap
- [FAQ](#faq)
- [How to Contribute?](#how-to-contribute)
- [How Long Will the History Data be Stored?](#how-long-will-the-history-data-be-stored)
- [How to Use Specific Host and Port for `dagu server`?](#how-to-use-specific-host-and-port-for-dagu-server)
- [How to Use Specific Host and Port eor `dagu server`?](#how-to-use-specific-host-and-port-eor-dagu-server)
- [How to Specify the DAGs Directory for `dagu server` and `dagu scheduler`?](#how-to-specify-the-dags-directory-for-dagu-server-and-dagu-scheduler)
- [How Can I Retry a DAG from a Specific Task?](#how-can-i-retry-a-dag-from-a-specific-task)
- [How Does It Track Running Processes Without DBMS?](#how-does-it-track-running-processes-without-dbms)
Expand Down Expand Up @@ -160,7 +160,7 @@ Start the server with `dagu server` and browse to `http://127.0.0.1:8080` to exp

Create a DAG by clicking the `New DAG` button on the top page of the web UI. Input `example` in the dialog.

*Note: DAG (YAML) files will be placed in `~/.dagu/dags` by default. See [Admin Configuration](#admin-configuration) for more details.*
*Note: DAG (YAML) files will be placed in `~/.dagu/dags` by default. See [Configuration Options](#configuration-options) for more details.*

### 3. Edit the DAG

Expand All @@ -186,7 +186,7 @@ You can execute the example by pressing the `Start` button.
- `dagu scheduler [--dags=<path/to/the DAGs directory>]` - Starts the scheduler process
- `dagu version` - Shows the current binary version

The `--config=<config>` option is available to all commands. It allows to specify different dagu configuration for the commands. Which enables you to manage multiple dagu process in a single instance. See [Admin Configuration](#admin-configuration) for more details.
The `--config=<config>` option is available to all commands. It allows to specify different dagu configuration for the commands. Which enables you to manage multiple dagu process in a single instance. See [Configuration Options](#configuration-options) for more details.

For example:

Expand Down Expand Up @@ -618,7 +618,7 @@ output:
}
```

The `jq` result can be used in following steps via [Output](#output) or [Stdout Redirection](#stdout-and-stderr-redirection).
The `jq` result can be used in following steps via [Output](#setting-environment-variables-with-standard-output) or [Stdout Redirection](#redirecting-stdout-and-stderr).

### Command Execution over SSH

Expand Down Expand Up @@ -693,7 +693,7 @@ If you want to use the same settings for all DAGs, set them to the [base configu

## Base Configuration for all DAGs

Creating a base configuration (default path: `~/.dagu/config.yaml`) is a convenient way to organize shared settings among all DAGs. The path to the base configuration file can be configured. See [Admin Configuration](#admin-configuration) for more details.
Creating a base configuration (default path: `~/.dagu/config.yaml`) is a convenient way to organize shared settings among all DAGs. The path to the base configuration file can be configured. See [Configuration Options](#configuration-options) for more details.

```yaml
# directory path to save logs from standard output
Expand Down Expand Up @@ -930,7 +930,7 @@ Feel free to contribute in any way you want. Share ideas, questions, submit issu

By default, the execution history data is retained for 30 days. However, you can customize this setting by modifying the `histRetentionDays` field in a YAML file.

### How to Use Specific Host and Port for `dagu server`?
### How to Use Specific Host and Port eor `dagu server`?

To configure the host and port for `dagu server`, you can set the environment variables `DAGU_HOST` and `DAGU_PORT`. Refer to the [Configuration Options](#configuration-options) for more details.

Expand Down
1 change: 0 additions & 1 deletion admin/src/components/molecules/DAGActions.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,6 @@ function DAGActions({
</Stack>
</ConfirmModal>
<StartDAGModal
defaultParams={dag.DefaultParams}
dag={dag}
visible={isStartModal}
onSubmit={(params) => {
Expand Down
122 changes: 87 additions & 35 deletions admin/src/components/molecules/StartDAGModal.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ import {
Typography,
} from '@mui/material';
import React from 'react';
import { Parameter, parseParams, stringifyParams } from '../../lib/parseParams';
import { DAG } from '../../models';
import LabeledItem from '../atoms/LabeledItem';

type Props = {
visible: boolean;
defaultParams: string;
dag: DAG;
dismissModal: () => void;
onSubmit: (params: string) => void;
Expand All @@ -30,13 +30,7 @@ const style = {
p: 4,
};

function StartDAGModal({
visible,
defaultParams,
dag,
dismissModal,
onSubmit,
}: Props) {
function StartDAGModal({ visible, dag, dismissModal, onSubmit }: Props) {
React.useEffect(() => {
const callback = (event: KeyboardEvent) => {
const e = event || window.event;
Expand All @@ -52,11 +46,18 @@ function StartDAGModal({

const ref = React.useRef<HTMLInputElement>(null);

const [params, setParams] = React.useState<string>(dag.DefaultParams);
const parsedParams = React.useMemo(() => {
if (dag.DefaultParams.trim() == '') {
return [];
}
return parseParams(dag.DefaultParams);
}, [dag.DefaultParams]);

const [params, setParams] = React.useState<Parameter[]>([]);

React.useEffect(() => {
ref.current?.focus();
}, [ref.current]);
setParams(parsedParams);
}, [parsedParams]);

return (
<Modal open={visible} onClose={dismissModal}>
Expand All @@ -71,30 +72,81 @@ function StartDAGModal({
spacing={2}
mt={2}
>
{dag.DefaultParams != '' ? (
<>
<Stack direction={'column'}>
<LabeledItem label="Default parameters">{null}</LabeledItem>
<Box sx={{ backgroundColor: '#eee' }}>{dag.DefaultParams}</Box>
</Stack>
<TextField
label="parameters"
multiline
variant="outlined"
style={{
flex: 0.5,
}}
inputRef={ref}
InputProps={{
value: params,
onChange: (e) => {
setParams(e.target.value);
},
}}
/>
</>
) : null}
<Button variant="contained" onClick={() => onSubmit(params)}>
{parsedParams.map((p, i) => {
if (p.Name != undefined) {
return (
<React.Fragment key={i}>
<TextField
label={p.Name}
multiline
placeholder={p.Value}
variant="outlined"
style={{
flex: 0.5,
}}
inputRef={ref}
InputProps={{
value: params.find((pp) => pp.Name == p.Name)?.Value,
onChange: (e) => {
if (p.Name) {
setParams(
params.map((pp) => {
if (pp.Name == p.Name) {
return {
...pp,
Value: e.target.value,
};
} else {
return pp;
}
})
);
}
},
}}
/>
</React.Fragment>
);
} else {
return (
<React.Fragment key={i}>
<TextField
label={`Parameter ${i + 1}`}
multiline
placeholder={p.Value}
variant="outlined"
style={{
flex: 0.5,
}}
inputRef={ref}
InputProps={{
value: params.find((_, j) => i == j)?.Value,
onChange: (e) => {
setParams(
params.map((pp, j) => {
if (j == i) {
return {
...pp,
Value: e.target.value,
};
} else {
return pp;
}
})
);
},
}}
/>
</React.Fragment>
);
}
})}
<Button
variant="contained"
onClick={() => {
onSubmit(stringifyParams(params));
}}
>
Start
</Button>
<Button variant="contained" color="error" onClick={dismissModal}>
Expand Down
39 changes: 39 additions & 0 deletions admin/src/lib/parseParams.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
export type Parameter = {
Name?: string;
Value: string;
};

export function parseParams(input: string): Parameter[] {
const paramRegex = /(?:([^\s=]+)=)?("(?:\\"|[^"])*"|[^"\s]+)/g;
const params: Parameter[] = [];

let match;
while ((match = paramRegex.exec(input)) !== null) {
const [, name, value] = match;

const param: Parameter = {
Value: value.startsWith('"')
? value.slice(1, -1).replace(/\\"/g, '"')
: value,
};

if (name) {
param.Name = name;
}

params.push(param);
}

return params;
}

export function stringifyParams(params: Parameter[]): string {
return params
.map((param) => {
const escapedValue = param.Value.replace(/"/g, '\\"');
const quotedValue = `"${escapedValue}"`;

return param.Name ? `${param.Name}=${quotedValue}` : quotedValue;
})
.join(' ');
}
3 changes: 1 addition & 2 deletions admin/src/pages/dags/dag/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,7 @@ function DAGDetails() {
const { mutate } = useSWRConfig();

const refreshFn = React.useCallback(() => {
mutate(`${baseUrl}/*`);
return;
mutate(`${baseUrl}/`);
}, [mutate, baseUrl]);

React.useEffect(() => {
Expand Down
6 changes: 3 additions & 3 deletions admin/src/pages/dags/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,14 @@ function DAGs() {
const group = query.get('group') || '';
const appBarContext = React.useContext(AppBarContext);

const { mutate } = useSWRConfig();
const { cache, mutate } = useSWRConfig();
const { data } = useSWR<GetDAGsResponse>('/', null, {
refreshInterval: 10000,
});

const refreshFn = React.useCallback(() => {
mutate('*');
}, [mutate]);
setTimeout(() => mutate('/'), 500);
}, [mutate, cache]);

React.useEffect(() => {
appBarContext.setTitle('DAGs');
Expand Down
Loading