Configuring mkdocs plugins from toml

[sindrehan]

I am trying to configure a plugin for mkdocs, but I'm running into some issues in translating the yaml over to toml. Basically, what I want is to specify something similar to what this yaml snippet does:

plugins:
    - foo:
    - bar
    - search:
        lang: en

But as far as I can tell that won't work, since TOML does not support mixed type arrays.

I also tried the approach used in configuring the nav-bar (ie. nested tables):

[[tool.portray.mkdocs.plugins.foo]]

[[tool.portray.mkdocs.plugins.bar]]

[[tool.portray.mkdocs.plugins.search]]
lang = "en"

but this failed, since mkdocs appears to expect an array and not a table:

MkDocs[('plugins', ValidationError('Invalid Plugins configuration. Expected a list of plugins'))]
Traceback (most recent call last):
  File "~/.pyenv/versions/3.7.4/bin/portray", line 10, in <module>
    sys.exit(__hug__.cli())
  File "~/.pyenv/versions/3.7.4/lib/python3.7/site-packages/hug/api.py", line 441, in __call__
    result = self.commands.get(command)()
  File "~/.pyenv/versions/3.7.4/lib/python3.7/site-packages/hug/interface.py", line 649, in __call__
    raise exception
  File "~/.pyenv/versions/3.7.4/lib/python3.7/site-packages/hug/interface.py", line 645, in __call__
    result = self.output(self.interface(**pass_to_function), context)
  File "~/.pyenv/versions/3.7.4/lib/python3.7/site-packages/hug/interface.py", line 129, in __call__
    return __hug_internal_self._function(*args, **kwargs)
  File "~/.pyenv/versions/3.7.4/lib/python3.7/site-packages/portray/api.py", line 88, in server
    with render.documentation_in_temp_folder(project_config) as doc_folder:
  File "~/.pyenv/versions/3.7.4ontextlib.py", line 112, in __enter__
    return next(self.gen)
  File "~/.pyenv/versions/3.7.4/lib/python3.7/site-packages/portray/render.py", line 156, in documentation_in_temp_folder
    mkdocs(config["mkdocs"])
  File "~/.pyenv/versions/3.7.4/lib/python3.7/site-packages/portray/render.py", line 70, in mkdocs
    config_instance = _mkdocs_config(config)
  File "~/.pyenv/versions/3.7.4/lib/python3.7/site-packages/portray/render.py", line 169, in _mkdocs_config
    f"Aborted with {len(errors)} Configuration Errors!"
mkdocs.exceptions.ConfigurationError: Aborted with 1 Configuration Errors!```

[rtbs-dev]

running into this myself now, trying to add the mkdocs-bibtex plugin.

this doesn't work:

[tool.portray.mkdocs.plugins]
bibtex = { bib_file = "docs/biblio.bib", cite_style = "pandoc" }
[tool.portray.mkdocs.plugins.search]

nor does this:

[[tool.portray.mkdocs.plugins]]
    [[tool.portray.mkdocs.plugins.bibtex]]
    bib_file = "docs/biblio.bib"
    [[tool.portray.mkdocs.plugins.bibtex]]
    cite_style = "pandoc"
    [[tool.portray.mkdocs.plugins.search]]

the error for the first looks like this:

⠋ Rendering complete website from Markdown using MkDocs[('plugins', ValidationError('Invalid Plugins configuration. Expected a list of plugins'))]

[andrew-schweitzer-analog]

Would it be possible to directly use mkdocs.yml directly if it is there, rather than trying to pass it through pyproject.toml?

[KyleKing]

I have a similiar problem trying to migrate the markdown_extensions settings from yaml to toml:

markdown_extensions:
  - admonition
  - pymdownx.emoji:
      emoji_index: !!python/name:materialx.emoji.twemoji
      emoji_generator: !!python/name:materialx.emoji.to_svg
  - pymdownx.tasklist:
      custom_checkbox: true
      clickable_checkbox: false
  - toc:
      permalink: ⚓︎
      toc_depth: 5

I would like to use portray, but these settings are a blocker for me. It doesn't look like there is a way for portray to use an existing mkdocs.yml file?

[SamGuay]

Maybe this solution will help others. I was able to configure many markdown extensions following this (not-so) convenient way.
My pyproject.toml looks like something like this (I kept only the relevant bits):

[tool.portray.mkdocs]
plugins = ["git-revision-date", "bibtex"]

markdown_extensions = ["toc",
                       "pymdownx.emoji",
                       "pymdownx.highlight",
                       "list",
                       "all",
                       "your",
                       "markdown.plugins"]

# Additional settings for markdown plugins
[[tool.portray.mkdocs.markdown_extensions]]
[tool.portray.mkdocs.markdown_extensions."pymdownx.highlight"]
linenums = true # Always add line numbers in code blocks

[[tool.portray.mkdocs.markdown_extensions]]
[tool.portray.mkdocs.markdown_extensions.toc]
permalink = "⚓︎"

[[tool.portray.mkdocs.plugins]]
[tool.portray.mkdocs.plugins.bibtex]
bib_file = "docs/biblio.bib"
cite_style = "pandoc"

[[tool.portray.mkdocs.plugins]]
[tool.portray.mkdocs.plugins.git-revision-date]
enabled_if_env = "CI"

Note that you have to wrap extensions with a . in their name with ".

However, I am getting different errors when trying to set up the bibtex and the git-revision-date plugins..
Both errors seem related path, which makes me wonder if it could be a similar issue to #74:

for bibtext:

line 62, in on_config raise Exception("Must supply a bibtex file or directory for bibtex files")
Exception: Must supply a bibtex file or directory for bibtex files

for git-revision-date:

    raise GitCommandError(command, status, stderr_value, stdout_value)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git log --date=short --format=%ad -n 1 /tmp/tmpnqwapqt0/input/README.md
  stderr: 'fatal: /tmp/tmpnqwapqt0/input/README.md: '/tmp/tmpnqwapqt0/input/README.md' is outside repository'

To sum up, @KyleKing, you can set all the markdown extensions this way. However, I have no clue how to fix the problems we encounter for other plugins. I did try to set full path but I have been unsuccessful so far..

[KyleKing]

Thanks @SamGuay! The snippet is helpful

One thing to point out is that the sample declares markdown_extensions twice and will fail

❯ poetry run portray project_configuration

  TOMLError

  Invalid TOML file /Users/kyleking/Developer/calcipy/pyproject.toml: Key "markdown_extensions" already exists.

  at ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/toml/file.py:28 in read
      24│     def read(self):
      25│         try:
      26│             return super(TOMLFile, self).read()
      27│         except (ValueError, TOMLKitError) as e:
    → 28│             raise TOMLError("Invalid TOML file {}: {}".format(self.path.as_posix(), e))
      29│ 
      30│     def __getattr__(self, item):
      31│         return getattr(self.__path, item)
      32│ 

But it seems you may be able to list them without options

# Additional settings for markdown plugins
[[tool.portray.mkdocs.markdown_extensions]]
[tool.portray.mkdocs.markdown_extensions."pymdownx.emoji"]   # extension without options 
[tool.portray.mkdocs.markdown_extensions."pymdownx.highlight"]
linenums = true # Always add line numbers in code blocks
[tool.portray.mkdocs.markdown_extensions.toc]
permalink = "⚓︎"

❯ poetry run portray project_configuration
{'append_directory_to_python_path': True,
 'directory': '/Users/kyleking/Developer/calcipy',
 'docs_dir': 'docs',
 'extra_dirs': ['art', 'images', 'media'],
 'extra_markdown_extensions': [],
 'file': '/Users/kyleking/Developer/calcipy/pyproject.toml',
 'host': '127.0.0.1',
 'include_reference_documentation': True,
 'labels': {'Api': 'API', 'Cli': 'CLI', 'Http': 'HTTP', 'Pypi': 'PyPI'},
 'mkdocs': {'config_file_path': '/Users/kyleking/Developer/calcipy',
            'edit_uri': 'edit/master/',
            'markdown_extensions': [{'pymdownx.emoji': {},
                                     'pymdownx.highlight': {'linenums': True},
                                     'toc': {'permalink': '⚓︎'}}],
            'repo_name': 'calcipy',
            'repo_url': 'https://github.com/KyleKing/calcipy',
            'site_name': 'calcipy',
            'theme': {'custom_dir': '/Users/kyleking/Developer/calcipy/.venv/lib/python3.8/site-packages/portray/mkdocs_templates',
                      'name': 'material',
                      'palette': {'accent': 'lightgreen', 'primary': 'green'}}},
 'modules': ['calcipy'],
 'output_dir': 'site',
 'pdocs': {'exclude_source': False, 'modules': ['calcipy'], 'overwrite': True},
 'port': 8000}

[KyleKing]

Although I'm running into issues trying to port over my markdown extension settings

[[tool.portray.mkdocs.markdown_extensions]]
[tool.portray.mkdocs.markdown_extensions.admonition]
[tool.portray.mkdocs.markdown_extensions.attr_list]
[tool.portray.mkdocs.markdown_extensions.def_list]
[tool.portray.mkdocs.markdown_extensions."markdown_include.include"]
[tool.portray.mkdocs.markdown_extensions."pymdownx.emoji"]
emoji_index = "!!python/name:materialx.emoji.twemoji"
emoji_generator = "!!python/name:materialx.emoji.to_svg"
[tool.portray.mkdocs.markdown_extensions."pymdownx.details"]
[tool.portray.mkdocs.markdown_extensions."pymdownx.superfences"]
[tool.portray.mkdocs.markdown_extensions."pymdownx.tabbed"]
[tool.portray.mkdocs.markdown_extensions."pymdownx.tasklist"]
custom_checkbox = true
clickable_checkbox = false
[tool.portray.mkdocs.markdown_extensions.toc]
permalink = "⚓︎"
toc_depth = "5"

❯ poetry run portray project_configuration
{'append_directory_to_python_path': True,
 'directory': '/Users/kyleking/Developer/calcipy',
 'docs_dir': 'docs',
 'extra_dirs': ['art', 'images', 'media'],
 'extra_markdown_extensions': [],
 'file': '/Users/kyleking/Developer/calcipy/pyproject.toml',
 'host': '127.0.0.1',
 'include_reference_documentation': True,
 'labels': {'Api': 'API', 'Cli': 'CLI', 'Http': 'HTTP', 'Pypi': 'PyPI'},
 'mkdocs': {'config_file_path': '/Users/kyleking/Developer/calcipy',
            'edit_uri': 'edit/master/',
            'markdown_extensions': [{'admonition': {},
                                     'attr_list': {},
                                     'def_list': {},
                                     'markdown_include.include': {},
                                     'pymdownx.details': {},
                                     'pymdownx.emoji': {'emoji_generator': '!!python/name:materialx.emoji.to_svg',
                                                        'emoji_index': '!!python/name:materialx.emoji.twemoji'},
                                     'pymdownx.superfences': {},
                                     'pymdownx.tabbed': {},
                                     'pymdownx.tasklist': {'clickable_checkbox': False,
                                                           'custom_checkbox': True},
                                     'toc': {'permalink': '⚓︎',
                                             'toc_depth': '5'}}],
            'repo_name': 'calcipy',
            'repo_url': 'https://github.com/KyleKing/calcipy',
            'site_name': 'calcipy',
            'theme': {'custom_dir': '/Users/kyleking/Developer/calcipy/.venv/lib/python3.8/site-packages/portray/mkdocs_templates',
                      'name': 'material',
                      'palette': {'accent': 'lightgreen', 'primary': 'green'}}},
 'modules': ['calcipy'],
 'output_dir': 'site',
 'pdocs': {'exclude_source': False, 'modules': ['calcipy'], 'overwrite': True},
 'port': 8000}

When trying to run, this error is reported from the validation step

portray/portray/render.py

Line 175 in ecac89e

errors, warnings = config_instance.validate()

❯ poetry run portray in_browser
Done Copying source documentation to temporary compilation directory
Done Auto generating reference documentation using pdocs
⠴ Rendering complete website from Markdown using MkDocs[('markdown_extensions', ValidationError('Invalid Markdown Extensions configuration'))]
Traceback (most recent call last):
  File "/Users/kyleking/Developer/calcipy/.venv/bin/portray", line 8, in <module>
    sys.exit(__hug__.cli())
  File "/Users/kyleking/Developer/calcipy/.venv/lib/python3.8/site-packages/hug/api.py", line 441, in __call__
    result = self.commands.get(command)()
....
  File "/Users/kyleking/Developer/calcipy/.venv/lib/python3.8/site-packages/portray/render.py", line 178, in _mkdocs_config
    raise _mkdocs_exceptions.ConfigurationError(
mkdocs.exceptions.ConfigurationError: Aborted with 1 Configuration Errors!

Update: I'm experimenting with using pdocs and mkdocs without portray. I think this type of customization might be out of scope of portray?

[SamGuay]

Hmm, weird, it works perfectly with the snippet I pasted above 🤔. I had tried to remove the multiple instance of [[tool.portray.mkdocs.markdown_extensions]] and it gave me the same error you get. However, when I set settings for each extension individually, it works 🤷. I admit I'm not really knowledgeable about TOML, but was happy I could make it work.

But it seems you may be able to list them without options

You are right, I can comment out the whole list of extensions, and it does work. However, I like to see all of them at once. On my side, it doesn't break either way when I run portray.

markdown_extensions = ["toc",
                       "admonition",
                       "footnotes",
                       "pymdownx.emoji",
                       "pymdownx.caret",
                       "pymdownx.mark",
                       "pymdownx.tilde",
                        ...

To test it, I just added your setting for pymdownx.tasklist in addition to the others and commented out the list above and it worked out.

[[tool.portray.mkdocs.markdown_extensions]]
[tool.portray.mkdocs.markdown_extensions."pymdownx.tasklist"]
custom_checkbox = true
clickable_checkbox = false

Feel free to peak into my pyproject.toml, maybe the order or the way it is set up will help you.

(Sidenote, I had never seen poetry before, I had to google it to understand how you ran portray 😄)

[dawsonbooth]

@KyleKing I just ran into the same issue. It turns out that TOML doesn't usually allow multiple separate types and you can't have

[tool.portray.mkdocs]
markdown_extensions = [
    ...
]

and also

[[tool.portray.mkdocs.markdown_extensions]]
[tool.portray.mkdocs.markdown_extensions."pymdownx.tasklist"]
custom_checkbox = true
clickable_checkbox = false

which is why the poetry check fails. Merging them as you did would work, but portray does not (yet?) handle markdown_extensions with empty properties.

However, portray does have the extra_markdown_extensions property, which I assume is to avoid the multiple type error. I did the following, which passes the poetry check and works for portray!

[tool.portray]
modules = ["MODULE_NAME"]

[[tool.portray.extra_markdown_extensions]]
[tool.portray.extra_markdown_extensions.toc]
permalink = true

[[tool.portray.extra_markdown_extensions]]
[tool.portray.extra_markdown_extensions."pymdownx.highlight"]
linenums = true

[tool.portray.mkdocs]
site_name = "PACKAGE_NAME"
site_url = "https://dawsonbooth.github.io/REPO_NAME/"
edit_uri = "blob/master/"

markdown_extensions = [
    "admonition",
    "codehilite",
    "extra",
    "pymdownx.details",
    "pymdownx.superfences",
]

nav = [{ Overview = "README.md" }]

[tool.portray.mkdocs.theme]
name = "material"
palette = { primary = "blue grey", accent = "red" }

(there's some extra stuff in there, just pay attention to the use of tool.portray.mkdocs.markdown_extensions and tool.portray.extra_markdown_extensions)

[JesseWebDotCom]

Any idea on how to do a custom stylesheet?

[SamGuay]

Hi @JesseWebDotCom,

I implemented custom slate/light styles in the dcm2bids repo based on whether the person has light vs dark OS. Feel free to peek into the repo.

In short the pyproject.toml looks like this:

[tool.portray.mkdocs]
# Left navigation menu
nav = [{  Home = "README.md"},
       {  "Code of conduct" = "CODE_OF_CONDUCT.md" },
       {  "Contributing" = "CONTRIBUTING.md"},
       {  "1. Usage" = "docs/1-usage.md"},
       {  "2. Tutorial" = "docs/2-tutorial.md"},
       {  "3. Configuration" = "docs/3-configuration.md"},
       {  "4. Advance" = "docs/4-advance.md"},
       {  "Changelog" = "CHANGELOG.md"}]

extra_css = ["docs/stylesheets/extra.css"] # To fix some colors in light vs slate mode.