Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add @mermaid-js/parser package and info langium parser #4727

Merged
merged 28 commits into from
Aug 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
6141722
feat: create `parser` package in `packages` directory
Yokozuna59 Aug 20, 2023
1559c2c
feat(parser): create `common` directory for langium parsers
Yokozuna59 Aug 20, 2023
1c24617
feat(parser): create `info` parser with exporting parser internals
Yokozuna59 Aug 20, 2023
963dd75
chore(parser): build `parser` package using `esbuild` and `vite`
Yokozuna59 Aug 20, 2023
82054bf
chore: make `parser` as optional in `ParserDefinition`
Yokozuna59 Aug 20, 2023
cbe9490
feat!: integrate `info` parser into `mermaid` package
Yokozuna59 Aug 20, 2023
6c0ef54
Merge branch 'next' into add-info-langium-parser
Yokozuna59 Aug 20, 2023
6502496
Merge remote-tracking branch 'upstream/next' into add-info-langium-pa…
Yokozuna59 Aug 20, 2023
4ae361b
reorder `packages/parser` after `packages/mermaid/src/vitepress`
Yokozuna59 Aug 20, 2023
1d64549
fix(mermaid): mark `mermaid-parser` dependecy with `^`
Yokozuna59 Aug 20, 2023
83e47a7
fix: use `execFileSync` instead of `execSync` in `generateLangium`
Yokozuna59 Aug 20, 2023
cc6f896
chore: remove `./*` part from `exports` in `parser/package.json`
Yokozuna59 Aug 21, 2023
eb63568
chore: refactore `&&` into `if` in `populateCommonDb`
Yokozuna59 Aug 21, 2023
984a0e6
chore: add a comment illustrate why we build packages sequentially
Yokozuna59 Aug 21, 2023
205360c
fix: fix if statment logic checks if `parser` is not `undefined`
Yokozuna59 Aug 21, 2023
3943301
Merge remote-tracking branch 'upstream/next' into add-info-langium-pa…
Yokozuna59 Aug 22, 2023
fc96ebe
build: update `langium` and `langium-cli` to `v2.0.1`
Yokozuna59 Aug 22, 2023
8d0ca2c
build: build `.langium` file using `generate` from `langium-cli`
Yokozuna59 Aug 22, 2023
24c8e57
docs(parser): create `packages/parser` README.md file
Yokozuna59 Aug 22, 2023
8186a54
chore: export `InfoModule` from `infoModule.ts`
Yokozuna59 Aug 26, 2023
91785b8
Merge branch 'next' into add-info-langium-parser
Yokozuna59 Aug 26, 2023
21539df
create `noErrorsOrAlternatives` parser helper function
Yokozuna59 Aug 26, 2023
dd284c0
Merge branch 'add-info-langium-parser' of https://github.com/Yokozuna…
Yokozuna59 Aug 26, 2023
458b90c
chore: run `pnpm lint:fix`
Yokozuna59 Aug 26, 2023
2b6a34e
chore: add `vitest` imports to `test-util.ts`
Yokozuna59 Aug 26, 2023
7ea3c64
chore: increase `test-util.ts` converage by returning `undefined`
Yokozuna59 Aug 26, 2023
cd19829
Merge branch 'next' into pr/Yokozuna59/4727
sidharthv96 Aug 28, 2023
4d53136
chore: Add comment for `yy`.
sidharthv96 Aug 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .build/common.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
* Shared common options for both ESBuild and Vite
*/
export const packageOptions = {
parser: {
name: 'mermaid-parser',
packageName: 'parser',
file: 'index.ts',
},
mermaid: {
name: 'mermaid',
packageName: 'mermaid',
Expand Down
5 changes: 5 additions & 0 deletions .build/generateLangium.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { generate } from 'langium-cli';

export async function generateLangium() {
await generate({ file: `./packages/parser/langium-config.json` });
}
9 changes: 9 additions & 0 deletions .build/langium-cli.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
declare module 'langium-cli' {
export interface GenerateOptions {
file?: string;
mode?: 'development' | 'production';
watch?: boolean;
}

export function generate(options: GenerateOptions): Promise<boolean>;
}
7 changes: 6 additions & 1 deletion .esbuild/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { build } from 'esbuild';
import { mkdir, writeFile } from 'node:fs/promises';
import { MermaidBuildOptions, defaultOptions, getBuildConfig } from './util.js';
import { packageOptions } from '../.build/common.js';
import { generateLangium } from '../.build/generateLangium.js';

const shouldVisualize = process.argv.includes('--visualize');

Expand Down Expand Up @@ -52,9 +53,13 @@ const handler = (e) => {
};

const main = async () => {
await generateLangium();
await mkdir('stats').catch(() => {});
const packageNames = Object.keys(packageOptions) as (keyof typeof packageOptions)[];
await Promise.allSettled(packageNames.map((pkg) => buildPackage(pkg).catch(handler)));
// it should build `parser` before `mermaid` because it's a dependecy
for (const pkg of packageNames) {
await buildPackage(pkg).catch(handler);
}
Yokozuna59 marked this conversation as resolved.
Show resolved Hide resolved
};

void main();
13 changes: 11 additions & 2 deletions .esbuild/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@ import cors from 'cors';
import { getBuildConfig, defaultOptions } from './util.js';
import { context } from 'esbuild';
import chokidar from 'chokidar';
import { generateLangium } from '../.build/generateLangium.js';

const parserCtx = await context(
getBuildConfig({ ...defaultOptions, minify: false, core: false, entryName: 'parser' })
);
const mermaidCtx = await context(
getBuildConfig({ ...defaultOptions, minify: false, core: false, entryName: 'mermaid' })
);
Expand All @@ -28,7 +32,7 @@ const externalCtx = await context(
const zenumlCtx = await context(
getBuildConfig({ ...defaultOptions, minify: false, core: false, entryName: 'mermaid-zenuml' })
);
const contexts = [mermaidCtx, mermaidIIFECtx, externalCtx, zenumlCtx];
const contexts = [parserCtx, mermaidCtx, mermaidIIFECtx, externalCtx, zenumlCtx];

const rebuildAll = async () => {
console.time('Rebuild time');
Expand Down Expand Up @@ -75,10 +79,11 @@ function sendEventsToAll() {
}

async function createServer() {
await generateLangium();
handleFileChange();
const app = express();
chokidar
.watch('**/src/**/*.{js,ts,yaml,json}', {
.watch('**/src/**/*.{js,ts,langium,yaml,json}', {
ignoreInitial: true,
ignored: [/node_modules/, /dist/, /docs/, /coverage/],
})
Expand All @@ -87,12 +92,16 @@ async function createServer() {
if (!['add', 'change'].includes(event)) {
return;
}
if (/\.langium$/.test(path)) {
await generateLangium();
}
console.log(`${path} changed. Rebuilding...`);
handleFileChange();
});

app.use(cors());
app.get('/events', eventsHandler);
app.use(express.static('./packages/parser/dist'));
app.use(express.static('./packages/mermaid/dist'));
app.use(express.static('./packages/mermaid-zenuml/dist'));
app.use(express.static('./packages/mermaid-example-diagram/dist'));
Expand Down
3 changes: 3 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,6 @@ cypress/plugins/index.js
coverage
*.json
node_modules

# autogenereated by langium-cli
generated/
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,6 @@ stats/
demos/dev/**
!/demos/dev/example.html
!/demos/dev/reload.js

# autogenereated by langium-cli
generated/
3 changes: 3 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,6 @@ stats
.nyc_output
# Autogenerated by `pnpm run --filter mermaid types:build-config`
packages/mermaid/src/config.type.ts

# autogenereated by langium-cli
generated/
11 changes: 9 additions & 2 deletions .vite/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { visualizer } from 'rollup-plugin-visualizer';
import type { TemplateType } from 'rollup-plugin-visualizer/dist/plugin/template-types.js';
import istanbul from 'vite-plugin-istanbul';
import { packageOptions } from '../.build/common.js';
import { generateLangium } from '../.build/generateLangium.js';

const visualize = process.argv.includes('--visualize');
const watch = process.argv.includes('--watch');
Expand Down Expand Up @@ -82,7 +83,7 @@ export const getBuildConfig = ({ minify, core, watch, entryName }: BuildOptions)
// @ts-expect-error According to the type definitions, rollup plugins are incompatible with vite
typescript({ compilerOptions: { declaration: false } }),
istanbul({
exclude: ['node_modules', 'test/', '__mocks__'],
exclude: ['node_modules', 'test/', '__mocks__', 'generated'],
extension: ['.js', '.ts'],
requireEnv: true,
forceBuildInstrument: coverage,
Expand All @@ -106,18 +107,24 @@ const buildPackage = async (entryName: keyof typeof packageOptions) => {

const main = async () => {
const packageNames = Object.keys(packageOptions) as (keyof typeof packageOptions)[];
for (const pkg of packageNames.filter((pkg) => !mermaidOnly || pkg === 'mermaid')) {
for (const pkg of packageNames.filter(
(pkg) => !mermaidOnly || pkg === 'mermaid' || pkg === 'parser'
)) {
await buildPackage(pkg);
}
};

await generateLangium();

if (watch) {
await build(getBuildConfig({ minify: false, watch, core: false, entryName: 'parser' }));
build(getBuildConfig({ minify: false, watch, core: false, entryName: 'mermaid' }));
if (!mermaidOnly) {
build(getBuildConfig({ minify: false, watch, entryName: 'mermaid-example-diagram' }));
build(getBuildConfig({ minify: false, watch, entryName: 'mermaid-zenuml' }));
}
} else if (visualize) {
await build(getBuildConfig({ minify: false, watch, core: false, entryName: 'parser' }));
await build(getBuildConfig({ minify: false, core: true, entryName: 'mermaid' }));
await build(getBuildConfig({ minify: false, core: false, entryName: 'mermaid' }));
} else {
Expand Down
1 change: 1 addition & 0 deletions .vite/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ async function createServer() {
});

app.use(cors());
app.use(express.static('./packages/parser/dist'));
app.use(express.static('./packages/mermaid/dist'));
app.use(express.static('./packages/mermaid-zenuml/dist'));
app.use(express.static('./packages/mermaid-example-diagram/dist'));
Expand Down
1 change: 1 addition & 0 deletions cSpell.json
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@
"knut",
"knutsveidqvist",
"laganeckas",
"langium",
"linetype",
"lintstagedrc",
"logmsg",
Expand Down
5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@
"git graph"
],
"scripts": {
"build": "pnpm run -r clean && pnpm build:esbuild && pnpm build:types",
"build": "pnpm build:esbuild && pnpm build:types",
"build:esbuild": "pnpm run -r clean && ts-node-esm --transpileOnly .esbuild/build.ts",
"build:mermaid": "pnpm build:esbuild --mermaid",
"build:viz": "pnpm build:esbuild --visualize",
"build:types": "tsc -p ./packages/mermaid/tsconfig.json --emitDeclarationOnly && tsc -p ./packages/mermaid-zenuml/tsconfig.json --emitDeclarationOnly && tsc -p ./packages/mermaid-example-diagram/tsconfig.json --emitDeclarationOnly",
"build:types": "tsc -p ./packages/parser/tsconfig.json --emitDeclarationOnly && tsc -p ./packages/mermaid/tsconfig.json --emitDeclarationOnly && tsc -p ./packages/mermaid-zenuml/tsconfig.json --emitDeclarationOnly && tsc -p ./packages/mermaid-example-diagram/tsconfig.json --emitDeclarationOnly",
"dev": "ts-node-esm --transpileOnly .esbuild/server.ts",
"dev:vite": "ts-node-esm --transpileOnly .vite/server.ts",
"dev:coverage": "pnpm coverage:cypress:clean && VITE_COVERAGE=true pnpm dev:vite",
Expand Down Expand Up @@ -107,6 +107,7 @@
"jison": "^0.4.18",
"js-yaml": "^4.1.0",
"jsdom": "^22.0.0",
"langium-cli": "2.0.1",
"lint-staged": "^13.2.1",
"nyc": "^15.1.0",
"path-browserify": "^1.0.1",
Expand Down
1 change: 0 additions & 1 deletion packages/mermaid-zenuml/src/parser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
* This is a dummy parser that satisfies the mermaid API logic.
*/
export default {
parser: { yy: {} },
parse: () => {
// no op
},
Expand Down
1 change: 1 addition & 0 deletions packages/mermaid/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
"khroma": "^2.0.0",
"lodash-es": "^4.17.21",
"mdast-util-from-markdown": "^1.3.0",
"mermaid-parser": "workspace:^",
"stylis": "^4.1.3",
"ts-dedent": "^2.2.0",
"uuid": "^9.0.0"
Expand Down
5 changes: 4 additions & 1 deletion packages/mermaid/src/Diagram.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,10 @@ export class Diagram {
this.parser.parse = (text: string) =>
originalParse(cleanupComments(extractFrontMatter(text, this.db, configApi.addDirective)));

this.parser.parser.yy = this.db;
if (this.parser.parser !== undefined) {
// The parser.parser.yy is only present in JISON parsers. So, we'll only set if required.
this.parser.parser.yy = this.db;
}
this.init = diagram.init;
this.parse();
}
Expand Down
1 change: 0 additions & 1 deletion packages/mermaid/src/diagram-api/diagram-orchestration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,6 @@ export const addDiagrams = () => {
styles: {}, // should never be used
renderer: {}, // should never be used
parser: {
parser: { yy: {} },
parse: () => {
throw new Error(
'Diagrams beginning with --- are not valid. ' +
Expand Down
1 change: 0 additions & 1 deletion packages/mermaid/src/diagram-api/diagramAPI.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ describe('DiagramAPI', () => {
parse: (_text) => {
return;
},
parser: { yy: {} },
},
renderer: {},
styles: {},
Expand Down
2 changes: 1 addition & 1 deletion packages/mermaid/src/diagram-api/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ export type DrawDefinition = (

export interface ParserDefinition {
parse: (text: string) => void;
parser: { yy: DiagramDB };
parser?: { yy: DiagramDB };
}

/**
Expand Down
3 changes: 0 additions & 3 deletions packages/mermaid/src/diagram.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,6 @@ describe('diagram detection', () => {
parse: () => {
// no-op
},
parser: {
yy: {},
},
},
renderer: {},
styles: {},
Expand Down
15 changes: 15 additions & 0 deletions packages/mermaid/src/diagrams/common/populateCommonDb.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
import type { DiagramAST } from 'mermaid-parser';

import type { DiagramDB } from '../../diagram-api/types.js';

export function populateCommonDb(ast: DiagramAST, db: DiagramDB) {
if (ast.accDescr) {
db.setAccDescription?.(ast.accDescr);
}
if (ast.accTitle) {
db.setAccTitle?.(ast.accTitle);
}
if (ast.title) {
db.setDiagramTitle?.(ast.title);
}
}
1 change: 0 additions & 1 deletion packages/mermaid/src/diagrams/error/errorDiagram.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ const diagram: DiagramDefinition = {
db: {},
renderer,
parser: {
parser: { yy: {} },
parse: (): void => {
return;
},
Expand Down
35 changes: 21 additions & 14 deletions packages/mermaid/src/diagrams/info/info.spec.ts
Original file line number Diff line number Diff line change
@@ -1,24 +1,31 @@
// @ts-ignore - jison doesn't export types
import { parser } from './parser/info.jison';
import { db } from './infoDb.js';

describe('info diagram', () => {
beforeEach(() => {
parser.yy = db;
parser.yy.clear();
});
import { parser } from './infoParser.js';

describe('info', () => {
it('should handle an info definition', () => {
const str = `info`;
parser.parse(str);

expect(db.getInfo()).toBeFalsy();
expect(() => {
parser.parse(str);
}).not.toThrow();
});

it('should handle an info definition with showInfo', () => {
const str = `info showInfo`;
Yokozuna59 marked this conversation as resolved.
Show resolved Hide resolved
parser.parse(str);
expect(() => {
parser.parse(str);
}).not.toThrow();
});

it('should throw because of unsupported info grammar', () => {
const str = `info unsupported`;
expect(() => {
parser.parse(str);
}).toThrow('Parsing failed: unexpected character: ->u<- at offset: 5, skipped 11 characters.');
});

expect(db.getInfo()).toBeTruthy();
it('should throw because of unsupported info grammar', () => {
const str = `info unsupported`;
expect(() => {
parser.parse(str);
}).toThrow('Parsing failed: unexpected character: ->u<- at offset: 5, skipped 11 characters.');
});
});
21 changes: 4 additions & 17 deletions packages/mermaid/src/diagrams/info/infoDb.ts
Original file line number Diff line number Diff line change
@@ -1,23 +1,10 @@
import type { InfoFields, InfoDB } from './infoTypes.js';
import { version } from '../../../package.json';

export const DEFAULT_INFO_DB: InfoFields = {
info: false,
} as const;
export const DEFAULT_INFO_DB: InfoFields = { version } as const;

let info: boolean = DEFAULT_INFO_DB.info;

export const setInfo = (toggle: boolean): void => {
info = toggle;
};

export const getInfo = (): boolean => info;

const clear = (): void => {
info = DEFAULT_INFO_DB.info;
};
export const getVersion = (): string => DEFAULT_INFO_DB.version;

export const db: InfoDB = {
clear,
setInfo,
getInfo,
getVersion,
};
3 changes: 1 addition & 2 deletions packages/mermaid/src/diagrams/info/infoDiagram.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
import type { DiagramDefinition } from '../../diagram-api/types.js';
// @ts-ignore - jison doesn't export types
import parser from './parser/info.jison';
import { parser } from './infoParser.js';
import { db } from './infoDb.js';
import { renderer } from './infoRenderer.js';

Expand Down
12 changes: 12 additions & 0 deletions packages/mermaid/src/diagrams/info/infoParser.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import type { Info } from 'mermaid-parser';
import { parse } from 'mermaid-parser';

import { log } from '../../logger.js';
import type { ParserDefinition } from '../../diagram-api/types.js';

export const parser: ParserDefinition = {
parse: (input: string): void => {
const ast: Info = parse('info', input);
log.debug(ast);
},
};
6 changes: 2 additions & 4 deletions packages/mermaid/src/diagrams/info/infoTypes.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
import type { DiagramDB } from '../../diagram-api/types.js';

export interface InfoFields {
info: boolean;
version: string;
}

export interface InfoDB extends DiagramDB {
clear: () => void;
setInfo: (info: boolean) => void;
getInfo: () => boolean;
getVersion: () => string;
}
Loading
Loading