Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unwanted newlines being added to output. #48

Open
epage opened this issue Apr 11, 2019 · 5 comments
Open

Unwanted newlines being added to output. #48

epage opened this issue Apr 11, 2019 · 5 comments

Comments

@epage
Copy link

epage commented Apr 11, 2019

I want to group my tables:

[first]
foo = 5
bar = 20
[first.a.b.c]
alice = 10
bob = 30

[second]       
foo = 3
bar = 21
[second.a.b.c]
alice = 15
bob = 3543

When I try to manually construct this, I instead get

[first]
foo = 5
bar = 20

[first.a.b.c]
alice = 10
bob = 30


[second]       
foo = 3
bar = 21

[second.a.b.c]
alice = 15
bob = 3543

I expected no newlines to be added for me and explicitly called out a newline between [first.a.b.c] table and [second]. Newlines being added surprised me and made made me wonder if tomlkit was properly preserving the lack of newlines but it seems to.

So I created the following test case to experiment with how newlines are dealt with

import tomlkit

t = tomlkit.loads("""[first]
foo = 5
bar = 20
[first.a.b.c]
alice = 10
bob = 30
[second]
foo = 3
bar = 21
[second.a.b.c]
alice = 15
bob = 3543""")

print("Round-trip")
print("```toml")
print(tomlkit.dumps(t))
print("```")
print()

t["second"].add("extra", tomlkit.table())
t["second"].add("extra1", tomlkit.table())
three = tomlkit.table()
three["foo"] = 2
three["bar"] = 1
child = tomlkit.table()
child["alice"] = 3
child["bob"] = 10
three["child"] = child
t["three"] = three


print("Changed")
print("```toml")
print(tomlkit.dumps(t))
print("```")
print()

The output is:
Round-trip

[first]
foo = 5
bar = 20
[first.a.b.c]
alice = 10
bob = 30
[second]
foo = 3
bar = 21
[second.a.b.c]
alice = 15
bob = 3543

Changed

[first]
foo = 5
bar = 20
[first.a.b.c]
alice = 10
bob = 30
[second]
foo = 3
bar = 21
[second.a.b.c]
alice = 15
bob = 3543
[second.extra]

[second.extra1]

[three]
foo = 2
bar = 1

[three.child]
alice = 3
bob = 10

I was surprised that second.extra didn't have a newline before it but second.extra1 did. I imagine this just shows how the newlines are being auto-added but it is still surprising.

@epage
Copy link
Author

epage commented Apr 11, 2019

My ideal would be for no extra newlines to be added but it'd be nice if the API was clarified to at least help someone know how to prevent this from happening.

@ofek
Copy link

ofek commented Oct 25, 2020

I handle this with:

if new_config.__class__.__name__ == 'Table':
    table_body = getattr(new_config.value, 'body', [])
    possible_whitespace = table_body[-2:]
    if len(possible_whitespace) == 2:
        for key, item in possible_whitespace:
            if key is not None:
                break
            if item.__class__.__name__ != 'Whitespace':
                break
        else:
            del table_body[-2]

@hockeybuggy
Copy link

ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Feb 25, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

In my opinion this makes the transition easy enough that we no longer
need to document bugwarriorrc at all so I went ahead and dropped what
little remained.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Mar 3, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

In my opinion this makes the transition easy enough that we no longer
need to document bugwarriorrc at all so I went ahead and dropped what
little remained.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Mar 12, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

In my opinion this makes the transition easy enough that we no longer
need to document bugwarriorrc at all so I went ahead and dropped what
little remained.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Mar 12, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

In my opinion this makes the transition easy enough that we no longer
need to document bugwarriorrc at all so I went ahead and dropped what
little remained.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Mar 18, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

In my opinion this makes the transition easy enough that we no longer
need to document bugwarriorrc at all so I went ahead and dropped what
little remained.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Mar 19, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

In my opinion this makes the transition easy enough that we no longer
need to document bugwarriorrc at all so I went ahead and dropped what
little remained.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Mar 19, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Mar 20, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

In my opinion this makes the transition easy enough that we no longer
need to document bugwarriorrc at all so I went ahead and dropped what
little remained.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Apr 10, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Apr 10, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Apr 10, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Apr 10, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Jul 14, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to ryneeverett/bugwarrior that referenced this issue Jul 14, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
ryneeverett added a commit to GothenburgBitFactory/bugwarrior that referenced this issue Jul 17, 2023
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
@voronind
Copy link

voronind commented Jan 8, 2024

Annoying bug

NexAdn pushed a commit to NexAdn/bugwarrior that referenced this issue Sep 19, 2024
Add a new subcommand to automatically convert bugwarriorrc to
bugwarrior.toml.

This commit adds the ini2toml package as a dependency and implements a
custom profile to handle the idiosyncrasies of our bugwarriorrc format.

The only "bug" I've noticed so far with ini2toml is that empty lines are
not preserved. This is the standard behavior of ini2toml due to the
`normalise_newlines` postprocessor, which removes all empty lines and
adds one before each section. However, this seems to be due to the issue
that upstream tomlkit adds lots of arbitrary empty lines (aka
`Whitespace()` and leaving them all would be an even worse result. See
python-poetry/tomlkit#48. This is about as far as
I care to chase this bug down the rabbit hole at the moment.
@ofek
Copy link

ofek commented Sep 26, 2024

I can no longer reproduce this bug on the latest minor series.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants