Skip to content

Commit

Permalink
yaml: Dump tuple objects as !!python/tuple nodes
Browse files Browse the repository at this point in the history
Before, `!!python/tuple` was only used for `OrderedDumper`.  Now it is
also used for `SafeOrderedDumper` and `IndentedSafeOrderedDumper`.

This is a behavior change: Any users that depended on a `tuple`
becoming a plain YAML sequence (which would be read in as a `list`
object) must first convert the `tuple` to `list`.

This change preserves the semantics of the object, and it preserves
round-trip identity.  Preserving round-trip identity is particularly
important if the tuple is used as a key in a `dict` because `list`
objects are not hashable.

Behavior before:

```pycon
>>> import salt.utils.yaml as y
>>> print(y.dump(("foo", "bar"), default_flow_style=False))
!!python/tuple
- foo
- bar

>>> print(y.safe_dump(("foo", "bar"), default_flow_style=False))
- foo
- bar

```

Behavior after:

```pycon
>>> import salt.utils.yaml as y
>>> print(y.dump(("foo", "bar"), default_flow_style=False))
!!python/tuple
- foo
- bar

>>> print(y.safe_dump(("foo", "bar"), default_flow_style=False))
!!python/tuple
- foo
- bar

```
  • Loading branch information
rhansen committed Dec 19, 2022
1 parent 6c7dd34 commit bd2b756
Show file tree
Hide file tree
Showing 5 changed files with 39 additions and 4 deletions.
3 changes: 3 additions & 0 deletions changelog/62932.fixed
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ but can be previewed now by setting the option `yaml_compatibility` to `3007`:
`datetime.datetime` object instead of a string.
* Dumping a `datetime.datetime` object will explicitly tag the node with
`!!timestamp`. (Currently the nodes are untagged.)
* Dumping a `tuple` object will consistently produce a sequence node
explicitly tagged with `!!python/tuple`. (Currently `safe_dump()` omits the
tag and `dump()` includes it.)
* `salt.utils.yaml.dump()` will default to `salt.utils.yaml.OrderedDumper`
instead of `yaml.Dumper`.
* Dumping a YAML sequence with `salt.utils.yaml.IndentedSafeOrderedDumper`
Expand Down
13 changes: 9 additions & 4 deletions doc/topics/troubleshooting/yaml_idiosyncrasies.rst
Original file line number Diff line number Diff line change
Expand Up @@ -498,6 +498,13 @@ Tuples

Loading a YAML ``!!python/tuple`` node is now supported.

.. versionchanged:: 3007.0

Dumping a ``tuple`` object to YAML now always produces a sequence node
tagged with ``!!python/tuple``. Previously, ``salt.utils.yaml.safe_dump()``
did not tag the node. Set the ``yaml_compatibility`` option to 3006 to
revert to the previous behavior.

The YAML ``!!python/tuple`` type can be used to produce a Python ``tuple``
object when loaded:

Expand All @@ -507,10 +514,8 @@ object when loaded:
- first item
- second item
When dumped to YAML with ``salt.utils.yaml.dump()``, a ``tuple`` object produces
a ``!!python/tuple`` node. When dumped to YAML with
``salt.utils.yaml.safe_dump()``, a ``tuple`` object produces a plain sequence
node (which will be loaded as a ``list`` object).
When dumped to YAML, a ``tuple`` object produces a sequence node tagged with
``!!python/tuple``.

Beware that Salt currently serializes ``tuple`` objects the same way it
serializes ``list`` objects, so they become ``list`` objects when deserialized
Expand Down
4 changes: 4 additions & 0 deletions salt/utils/yamldumper.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,10 @@ def _rep_default(self, data):
salt.utils.context.NamespacedDictWrapper,
yaml.representer.SafeRepresenter.represent_dict,
)
# SafeDumper represents tuples as lists, but Dumper's behavior (sequence
# tagged with `!!python/tuple`) is safe, so use it for all dumpers.
_CommonMixin.add_multi_representer(tuple, Dumper.yaml_representers[tuple])
_CommonMixin.add_representer(tuple, Dumper.yaml_representers[tuple])


class SafeOrderedDumper(_CommonMixin, SafeDumper):
Expand Down
1 change: 1 addition & 0 deletions tests/pytests/unit/utils/jinja/test_custom_extensions.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ def test_serialize_yaml():
"baz": [1, 2, 3],
"qux": 2.0,
"spam": OrderedDict([("foo", OrderedDict([("bar", "baz"), ("qux", 42)]))]),
"tuple": ("foo", "bar"),
}
env = Environment(extensions=[SerializerExtension])
rendered = env.from_string("{{ dataset|yaml }}").render(dataset=dataset)
Expand Down
22 changes: 22 additions & 0 deletions tests/pytests/unit/utils/test_yaml.py
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,28 @@ def test_dump_timestamp(yaml_compatibility, want_tag, dumpercls):
assert re.fullmatch(want_re, got)


@pytest.mark.parametrize(
"mktuple",
[
lambda *args: tuple(args),
collections.namedtuple("TestTuple", "a b"),
],
)
@pytest.mark.parametrize(
"dumpercls",
[
salt_yaml.OrderedDumper,
salt_yaml.SafeOrderedDumper,
salt_yaml.IndentedSafeOrderedDumper,
],
)
def test_dump_tuple(mktuple, dumpercls):
data = mktuple("foo", "bar")
want = "!!python/tuple [foo, bar]\n"
got = salt_yaml.dump(data, Dumper=dumpercls)
assert got == want


def render_yaml(data):
"""
Takes a YAML string, puts it into a mock file, passes that to the YAML
Expand Down

0 comments on commit bd2b756

Please sign in to comment.