🚦 Change severity of the error messages (#1104)

This adds a `error_rules` to the project configuration, which can be used to disable logging rules in the CLI.

.changeset/sixty-ducks-hug.md

@@ -0,0 +1,6 @@
+---
+"myst-config": patch
+"myst-cli": patch
+---
+
+Add error_rules to turn off warnings and errors in the CLI.

CONTRIBUTING.md

@@ -41,7 +41,7 @@ Below is a list of relevant repositories and a brief description of each.
 - [MyST Templates](https://github.com/myst-templates): Repositories that contain templates for rendering MyST documents into various outputs like LaTeX, JATS, Typst, and Docx.
 
 > [!NOTE]
-> There are many repositories with similar functionality in the `executablebooks/` organization. Many of these are based around the [Sphinx documentation ecosystem](https://sphinx-doc.org). For example, the [MyST-NB repository](https://github.com/executablebooks/myst-nb) is a Sphinx extension for Jupyter notebooks, and the [MyST Parser repository](https://github.com/executablebooks/myst-parser) is a MyST markdown parser for Sphinx.
+> There are many repositories with similar functionality in the `executablebooks/` organization. Many of these are based around the [Sphinx documentation ecosystem](https://www.sphinx-doc.org). For example, the [MyST-NB repository](https://github.com/executablebooks/myst-nb) is a Sphinx extension for Jupyter notebooks, and the [MyST Parser repository](https://github.com/executablebooks/myst-parser) is a MyST markdown parser for Sphinx.
 
 ## Contribution workflow
 

docs/creating-pdf-documents.md

@@ -35,7 +35,7 @@ exports:
 ---
 ```
 
-To build the exports, use the `myst build` command, which will work with your [project structure](./project-overview.md) if it exists and create a document in the output path that you specify.
+To build the exports, use the `myst build` command, which will work with your project structure if it exists and create a document in the output path that you specify.
 
 ```bash
 myst build my-document.md --pdf
@@ -54,7 +54,7 @@ The default PDF renderer uses $\LaTeX$ to create PDFs, which means that to work
 As an alternative, for faster PDF builds, you may use [Typst](#rendering-pdfs-with-typst) instead.
 ```
 
-The rendering process for scientific PDFs uses $\LaTeX$ and makes use of the [`jtex`](myst:jtex) templating library, to convert to $\LaTeX$ the [`myst-to-tex`](myst:myst-to-tex) packages is used. The libraries work together for sharing information about [frontmatter](./frontmatter.md) (e.g. title, keywords, authors, and affiliations).
+The rendering process for scientific PDFs uses $\LaTeX$ and makes use of the [`jtex`](myst:jtex) templating library, to convert to $\LaTeX$ the `myst-to-tex` packages is used. The libraries work together for sharing information about [frontmatter](./frontmatter.md) (e.g. title, keywords, authors, and affiliations).
 
 ```{mermaid}
 flowchart LR

docs/creating-word-documents.md

@@ -29,7 +29,7 @@ exports:
 ---
 ```
 
-To build the exports, use the `myst build` command, which will work with your [project structure](./project-overview.md) if it exists and create a document in the output path that you specify.
+To build the exports, use the `myst build` command, which will work with your project structure if it exists and create a document in the output path that you specify.
 
 ```bash
 myst build my-document.md --docx
@@ -69,7 +69,7 @@ To fix equations in Word, use the equation toolbar to select `LaTeX` and from th
 
 ## Rendering Word with `myst-to-docx`
 
-The rendering process for word documents uses the [`myst-to-docx`](myst:myst-to-docx) package. The library works together with `mystmd` for sharing information about [frontmatter](./frontmatter.md) (e.g. title, keywords, authors, and affiliations).
+The rendering process for word documents uses the `myst-to-docx` package. The library works together with `mystmd` for sharing information about [frontmatter](./frontmatter.md) (e.g. title, keywords, authors, and affiliations).
 
 ```{mermaid}
 flowchart LR

docs/execute-notebooks.md

@@ -8,7 +8,7 @@ thumbnail: thumbnails/execute-notebooks.png
 ## Overview
 
 :::{warning} MyST Execution is in Beta
-We are adding support for executing markdown notebooks and ipynb files, including inline execution. As we are adding this functionality we appreciate any feedback from the community on how it is working in your environments. Please add [issues](https://github.com/executablebooks/MyST/issues/new) or join [Discord](https://discord.MyST.org/) to give feedback.
+We are adding support for executing markdown notebooks and ipynb files, including inline execution. As we are adding this functionality we appreciate any feedback from the community on how it is working in your environments. Please add [issues](https://github.com/executablebooks/mystmd/issues/new) or join [Discord](https://discord.mystmd.org/) to give feedback.
 :::
 
 The MyST CLI can execute your notebooks and markdown files by passing the `--execute` flag to the `start` and `build` commands, i.e.:
@@ -18,15 +18,18 @@ myst start --execute
 myst build --execute
 ```
 
-If the flag is passed, notebook cells and inline execution will be executed and the original notebook values ignored. By default, execution is disabled and will use the notebook outputs already saved in the notebook (for text-based notebooks, there are no outputs). The notebook outputs are cached based on the source to allow for speedy editing workflows of text without having to re-execute your notebooks. 
+If the flag is passed, notebook cells and inline execution will be executed and the original notebook values ignored. By default, execution is disabled and will use the notebook outputs already saved in the notebook (for text-based notebooks, there are no outputs). The notebook outputs are cached based on the source to allow for speedy editing workflows of text without having to re-execute your notebooks.
 
 ## Launching a Server
 
 MyST performs execution of notebooks by communication with a Jupyter Server. Jupyter Server is distributed as a Python package, which can be installed from PyPI or conda-forge, e.g.
+
 ```bash
 pip install jupyter-server
 ```
+
 Jupyter Server is only responsible for orchestrating execution of your code. To actually perform execution, you must also install a kernel. For Python, this might be `ipykernel`, e.g.
+
 ```bash
 pip install ipykernel
 ```
@@ -35,6 +38,7 @@ If Jupyter Server is installed and the `--execute` flag is passed to `myst start
 
 :::{note}
 Advanced users may wish to connect to non-local Jupyter Servers, e.g. those running on a remote server. It is possible to instruct MyST to connect to a remote server by setting the `JUPYTER_BASE_URL` and `JUPYTER_TOKEN` environment variables, e.g.
+
 ```bash
 # Set local environment variable
 port="8888"
@@ -52,11 +56,13 @@ myst build --execute
 # Stop server!
 kill %1
 ```
+
 :::
 
 ## Caching Outputs
 
 By default MyST caches the execution of a notebook according to its executable content. In simple terms, this means that MyST avoids re-running a kernel over a notebook if the notebook's code and inline-expressions do not change. This may not always be correct; if the execution environment (e.g. installed Python packages) changes, you may wish to re-run the notebook. For now, you can clear the execution cache with
+
 ```bash
 myst clean --execute
 ```

docs/frontmatter.md

@@ -39,7 +39,7 @@ Without the extension installed, remember to format the contents of the section
 
 :::{note} Using `jupytext` or a Markdown-based notebook?
 :class: dropdown
-If your Jupyter Notebook is described as a markdown file (e.g. using [jupytext](https://jupytext.readthedocs.io/en/latest/formats.html), or [MyST](https://jupyterbook.org/en/stable/file-types/myst-notebooks.html)), then this should be included in the frontmatter section as usual in addition to the `jupyter` key that defines the kernel and jupytext metadata.
+If your Jupyter Notebook is described as a markdown file (e.g. using [jupytext](https://jupytext.readthedocs.io/en/latest/formats-markdown.html), or [MyST](https://jupyterbook.org/en/stable/file-types/myst-notebooks.html)), then this should be included in the frontmatter section as usual in addition to the `jupyter` key that defines the kernel and jupytext metadata.
 :::
 
 ### In a `myst.yml` file

docs/integrating-jupyter.md

@@ -256,7 +256,7 @@ For more on working locally see [](#start-a-local-jupyter-server).
 
 ```{danger} On securing a Jupyter server
 :class: dropdown
-If you intend to run a dedicate single user Jupyter server accessible over a network please carefully read and follow [the advice provided by the Jupyter server team here](https://jupyter-notebook.readthedocs.io/en/stable/public_server.html).
+If you intend to run a dedicate single user Jupyter server accessible over a network please carefully read and follow [the advice provided by the Jupyter server team here](https://jupyter-server.readthedocs.io/en/stable/operators/security.html).
 
 MyST Websites will work best, be safer and be more robust when backed by Jupyter services such as BinderHub or JupyterHub.
 ```

docs/myst.yml

@@ -48,6 +48,17 @@ project:
     - directives.mjs
     - unsplash.mjs
     - latex.mjs
+  error_rules:
+    - rule: link-resolves
+      severity: ignore
+      keys:
+        - /robots.txt
+        - /sitemap.xml
+        - /jtex/template-yml
+        - /jtex
+        - /jtex/create-a-latex-template
+        - /jtex/create-a-beamer-template
+        - /jtex/contribute-a-template
 site:
   title: MyST Markdown Guide
   domains:

docs/proofs-and-theorems.md

@@ -85,7 +85,7 @@ You can refer to a proof using the standard link syntax:
 :::{tip} Compatibility with Sphinx Proof
 :class: dropdown
 
-You may also use the the `{prf:ref}` role like: `` {prf:ref}`my-theorem` ``, which will replace the reference with the theorem number like so: {prf:ref}`my-theorem`. When an explicit text is provided, this caption will serve as the title of the reference. For example, `` {prf:ref}`Orthogonal-Projection-Theorem <my-theorem>`  `` will produce: {prf:ref}`Orthogonal-Projection-Theorem <my-theorem>`.
+You may also use the the `{prf:ref}` role like: `` {prf:ref}`my-theorem` ``, which will replace the reference with the theorem number like so: {prf:ref}`my-theorem`. When an explicit text is provided, this caption will serve as the title of the reference. For example, ``{prf:ref}`Orthogonal-Projection-Theorem <my-theorem>` `` will produce: {prf:ref}`Orthogonal-Projection-Theorem <my-theorem>`.
 :::
 
 ## Hiding Proof Content
@@ -198,7 +198,7 @@ $$
 :::
 ```
 
-_Source:_ [QuantEcon](https://python-advanced.quantecon.org/von_neumann_model.html#Duality)
+_Source:_ [QuantEcon](https://python-advanced.quantecon.org)
 
 ### Criteria
 

docs/settings.md

@@ -52,3 +52,22 @@ beamer
 
     - `true`: Add `\begin{frame}` environment for each block, delimited by `+++`, and enable presentation outline with block metadata `+++ {"outline":true}`
     - `false` (default): No extra `\begin{frame}` environment will be used
+
+## Error Rules
+
+The `error_rules` list in the project configuration can be used to disable or modify logging rules in the CLI:
+
+```{code-block} yaml
+:filename: myst.yml
+project:
+  error_rules:
+    - rule: math-eqnarray-replaced
+      severity: ignore
+    - rule: link-resolves
+      severity: ignore
+      keys:
+        - /known-internal-link
+        - https://flaky-connection.com
+```
+
+The `severity` of each rule can be set to `ignore`, `warn`, or `error`. If the rule is triggered, then the severity listed will be adopted rather than the default log message severity. The default severity for rules included in the list is `ignore`, which means that simply listing the rule IDs as strings will ignore those rules. To discover the rule ID, run myst in debug mode to get the error (and optional key) printed to the console. For example, the above configuration updates will no longer warn on `math-eqnarray-replaced` and will also ignore the two links when running `myst build --check-links --strict` in continuous integration.

packages/myst-cli/src/process/site.ts

@@ -415,9 +415,10 @@ export async function processSite(session: ISession, opts?: ProcessOptions): Pro
     if (hasWarnings[0] > 0) {
       const pluralE = hasWarnings[0] > 1 ? 's' : '';
       const pluralW = hasWarnings[1] > 1 ? 's' : '';
-      throw new Error(
+      session.log.error(
         `Site has ${hasWarnings[0]} error${pluralE} and ${hasWarnings[1]} warning${pluralW}, stopping build.`,
       );
+      process.exit(1);
     }
   }
   if (opts?.writeFiles ?? true) {

packages/myst-cli/src/transforms/links.ts

@@ -206,6 +206,7 @@ export async function checkLinksTransform(
         addWarningForFile(session, file, `Link for "${url}" did not resolve.${status}`, 'error', {
           position,
           ruleId: RuleId.linkResolves,
+          key: url,
         });
         return url as string;
       }),

packages/myst-cli/src/utils/addWarningForFile.ts

@@ -3,17 +3,20 @@ import type { VFileMessage } from 'vfile-message';
 import type { ISession } from '../session/types.js';
 import { warnings } from '../store/reducers.js';
 import type { WarningKind } from '../store/types.js';
+import { selectCurrentProjectConfig } from '../store/selectors.js';
 
 export function addWarningForFile(
   session: ISession,
   file: string | undefined | null,
   message: string,
-  kind: WarningKind = 'warn',
+  severity: WarningKind = 'warn',
   opts?: {
     note?: string | null;
     url?: string | null;
     position?: VFileMessage['position'];
     ruleId?: string | null;
+    /** This key can be combined with the ruleId to suppress a warning */
+    key?: string | null;
   },
 ) {
   const line = opts?.position?.start.line ? `:${opts?.position.start.line}` : '';
@@ -26,7 +29,23 @@ export function addWarningForFile(
   const url = opts?.url ? chalk.reset.dim(`\n   See also: ${opts.url}\n`) : '';
   const prefix = file ? `${file}${line}${column} ` : '';
   const formatted = `${message}${note}${url}`;
-  switch (kind) {
+  if (opts?.ruleId) {
+    const config = selectCurrentProjectConfig(session.store.getState());
+    const handler = config?.error_rules?.find((rule) => {
+      if (rule.key) {
+        return rule.id === opts.ruleId && rule.key === opts.key;
+      }
+      return rule.id === opts.ruleId;
+    });
+    if (handler) {
+      if (handler.severity === 'ignore') {
+        session.log.debug(`${prefix}${formatted}`);
+        return;
+      }
+      severity = (handler.severity as WarningKind) ?? severity;
+    }
+  }
+  switch (severity) {
     case 'info':
       session.log.info(`ℹ️  ${prefix}${formatted}`);
       break;
@@ -38,12 +57,17 @@ export function addWarningForFile(
       session.log.warn(`⚠️  ${prefix}${formatted}`);
       break;
   }
+  if (opts?.ruleId) {
+    session.log.debug(
+      `To suppress this message, add rule: "${opts.ruleId}"${opts.key ? ` with key: "${opts.key}"` : ''} to "error_rules" in your project config`,
+    );
+  }
   if (file) {
     session.store.dispatch(
       warnings.actions.addWarning({
         file,
         message,
-        kind,
+        kind: severity,
         url: opts?.url,
         note: opts?.note,
         position: opts?.position,

packages/myst-config/src/errorRules/errorRules.yml

@@ -0,0 +1,34 @@
+title: Error Rules
+kind: config
+cases:
+  - title: error rules
+    raw:
+      error_rules:
+        - math-eqnarray-replaced
+    normalized:
+      error_rules:
+        - id: math-eqnarray-replaced
+          severity: ignore
+  - title: error rules - incorrect severity
+    raw:
+      error_rules:
+        - rule: math-eqnarray-replaced
+          severity: nope
+    normalized:
+      error_rules: []
+    errors: 1
+  - title: error rules with params
+    raw:
+      error_rules:
+        - rule: a-rule
+          keys:
+            - one
+            - two
+    normalized:
+      error_rules:
+        - id: a-rule
+          severity: ignore
+          key: one
+        - id: a-rule
+          severity: ignore
+          key: two

packages/myst-config/src/errorRules/index.ts

@@ -0,0 +1,2 @@
+export * from './types.js';
+export * from './validators.js';

packages/myst-config/src/errorRules/types.ts

@@ -0,0 +1,5 @@
+export type ErrorRule = {
+  id: string;
+  severity: 'ignore' | 'warn' | 'error';
+  key?: string;
+} & Record<string, any>;

packages/myst-config/src/errorRules/validators.ts

@@ -0,0 +1,64 @@
+import type { ValidationOptions } from 'simple-validators';
+import {
+  defined,
+  incrementOptions,
+  validateChoice,
+  validateList,
+  validateObjectKeys,
+  validateString,
+} from 'simple-validators';
+import type { ErrorRule } from './types.js';
+
+const ERROR_RULE_KEY_OBJECT = {
+  required: ['id'],
+  optional: ['severity', 'keys'],
+  alias: {
+    rule: 'id',
+    key: 'keys',
+  },
+};
+
+export function validateErrorRule(input: any, opts: ValidationOptions): ErrorRule[] | undefined {
+  if (typeof input === 'string') {
+    input = { id: input };
+  }
+  const value = validateObjectKeys(input, ERROR_RULE_KEY_OBJECT, {
+    ...opts,
+  });
+  if (value === undefined) return undefined;
+  const id = validateString(value.id, incrementOptions('id', opts));
+  const severity = validateChoice(value.severity || 'ignore', {
+    ...incrementOptions('severity', opts),
+    choices: ['ignore', 'warn', 'error'],
+  }) as ErrorRule['severity'];
+  if (!id || !severity) return undefined;
+  const output: ErrorRule = { id, severity };
+  if (!defined(value.keys)) return [output];
+  // We now have either a list of keys or a single key
+  // validate and unpack to a separate error rule
+  const keyList = validateList(
+    value.keys,
+    { ...incrementOptions('keys', opts), coerce: true },
+    (key, ind) => {
+      return validateString(key, incrementOptions(`keys.${ind}`, opts));
+    },
+  );
+  if (!keyList) return undefined;
+  return keyList.map((key) => ({ ...output, key }));
+}
+
+export function validateErrorRuleList(
+  input: any,
+  opts: ValidationOptions,
+): ErrorRule[] | undefined {
+  if (input === undefined) return undefined;
+  const output = validateList(
+    input,
+    { coerce: true, ...incrementOptions('error_rules', opts) },
+    (exp, ind) => {
+      return validateErrorRule(exp, incrementOptions(`error_rules.${ind}`, opts));
+    },
+  );
+  if (!output) return undefined;
+  return output.flat();
+}

packages/myst-config/src/project/types.ts

@@ -1,4 +1,5 @@
 import type { ProjectFrontmatter } from 'myst-frontmatter';
+import type { ErrorRule } from '../errorRules/types.js';
 
 export const VERSION = 1;
 
@@ -7,4 +8,5 @@ export type ProjectConfig = ProjectFrontmatter & {
   index?: string;
   exclude?: string[];
   plugins?: string[];
+  error_rules?: ErrorRule[];
 };