Unable to configure logging in 2.8.11 using TOML.

[hirolau]

In 2.8.11 the TOML parser freezes the config dict, this makes it no longer possible to load logging using that config.

Traceback (most recent call last):
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/logging/config.py", line 538, in configure
    formatters[name])
TypeError: 'FrozenOrderedDict' object does not support item assignment

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/app-root/bin/luigid", line 11, in <module>
    load_entry_point('luigi==2.8.11', 'console_scripts', 'luigid')()
  File "/opt/app-root/lib/python3.6/site-packages/luigi/cmdline.py", line 31, in luigid
    DaemonLogging.setup(opts)
  File "/opt/app-root/lib/python3.6/site-packages/luigi/setup_logging.py", line 83, in setup
    configured = cls._section(opts)
  File "/opt/app-root/lib/python3.6/site-packages/luigi/setup_logging.py", line 48, in _section
    logging.config.dictConfig(logging_config)
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/logging/config.py", line 795, in dictConfig
    dictConfigClass(config).configure()
  File "/opt/rh/rh-python36/root/usr/lib64/python3.6/logging/config.py", line 541, in configure
    'formatter %r: %s' % (name, e))
ValueError: Unable to configure formatter 'simple': 'FrozenOrderedDict' object does not support item assignment

I have not had time to look into possible solutions for now.

[hirolau]

One way to fix this would be to not freeze the logging section in the parser:

In luigi/luigi/configuration/toml_parser.py instead of:

        # freeze dict params
        for section, content in self.data.items():
            for key, value in content.items():
                if isinstance(value, dict):
                    self.data[section][key] = recursively_freeze(value)

It would be possible to do:

        # freeze dict params
        for section, content in self.data.items():
            if key.lower() == 'logging':
                 continue
            for key, value in content.items():
                if isinstance(value, dict):
                    self.data[section][key] = recursively_freeze(value)

Another solution, that is probably better, is to let the config parser make a mutable copy. That would however require us to write a more complicated "recursively_unfreeze" function.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions.

[DVlahovic]

Also experiencing this issue when attempting to configure logging via toml.

I tried editing my toml_parser.py as suggested by @hirolau and can confirm the solution works (I assume you meant section.lower())

# freeze dict params
        for section, content in self.data.items():
            if section.lower() == 'logging':
                 continue
            for key, value in content.items():
                if isinstance(value, dict):
                    self.data[section][key] = recursively_freeze(value)

[mzpqnxow]

I'm confused about why this is WONTFIX. I have a fork with the change (and one other change that was blocking toml config processing) that I will send, will they be accepted? I don't see a way to work around this in the toml file (maybe I'm missing something here?)

EDIT: Is the (non-code-change) "solution" to not include the logging section in the main TOML file? If so, the example toml configuration file in the repository should probably be updated- I had issues using that file verbatim