Twitter: "content" metadata information does not include emoticons

[Defrost4528]

First of all, thank you very much for the fast implementation of the other request! Much appreciated.

I've been testing out the latest version, and it seems to work almost perfectly. Just some things I noticed...

(can be observed using https://twitter.com/yumi_san0112/status/1151144618936823808)

1. Emoticons are not properly copied and are replaced with a space. Looking at the HTML code, they seem to be represented by an img element, with the proper unicode equivalent in the "alt" attribute, so maybe that can be utilized.
2. Line breaks are replaced by a space. This isn't a big deal, and honestly I'm not well versed enough in JSON to know if there's an elegant representation for this, so it's just an observation.

Once again, thank you! With the addition to the content text, this program is the perfect solution to backing up Twitter artists, especially important since some accounts seem to come and go. This has made my life so much easier.

[mikf]

Emoticons and newlines are now preserved, and it also removes the pic.twitter.com URL at the end. Since the content parsing has become a bit more complicated and is probably not used by the majority of users, I've put it behind the twitter.content option.

Please test the current implementation and see if this better suits your expectations.
Also thank you for providing an example, makes this a lot easier for me.

And since you are talking about Twitter backup: make sure you are (also) using twitter.com/<user>/media as input and remember that gallery-dl can't necessarily get all tweets of a timeline thanks to Twitter's own restrictions (see #186 and #314). Projects like twint and whatever you find for twitter backup on GitHub might be better suited at preserving Twitter timelines.

[Defrost4528]

I've tested it thoroughly, and it seems to work great now! Since newlines were added too, I also tested the backslash escaping just to be safe, and it works fine. Thanks!

As for twint, I'll definitely look at that. However, at first glance it seems like a complex method for my simple need (and probably takes some tweaking to get it to download images). My focus is on batch archiving artist's images in an organized way, which your program already practically perfectly.