What happened to pandoc-discuss?

[jgm]

Practically since this project began, we've had a mailing list on google groups (pandoc-discuss).
As of yesterday, it is no longer accessible. What happened is that it got overrun with spam, and Google flagged the group with a "banned content warning." I was able to log in as administrator, delete all the spam, ban the spammers, and turn on moderation to avoid the problem in the future. I have requested review and reinstatement from Google, but who knows how long that will take.

I'm hoping the group will be back soon, and at this point the purpose of this issue is just to let people know what is up. If the group is not back soon, I will look into alternatives.

[p3palazzo]

GitHub now has a discussions feature built right into the repository, if Google fails.

[jgm]

I know. That may be the best backup plan, and I know many people would prefer that interface. But I'd rather not interrupt the continuity of a searchable discussion forum that goes back 15 years. Google Groups also has the advantage that you don't need a GitHub account to ask a question.

Still, it irks me that Google has not yet responded. I tried to use Google's "download your data" feature to download the full archive in mbox format, but it did not give me pandoc-discuss, even though I'm the owner.

[iandol]

It is crazy, that spam started to increase can be attributed to Google's failure to filter the spam in the first place, and then in response, it shuts down the whole group! Lets hope you can get the messages out. I checked Github discussions but it seems there is no mechanism to import messages from another source?

[nandac]

@jgm Do we have a way to access past messages from pandoc-discuss?

I would like to get access to this message: https://groups.google.com/g/pandoc-discuss/c/fi_utmqU-mQ
Regarding Typst highlighting.

In addition, do we have an alternative or should we just use GitHub issues for now?

[jgm]

No, the messages are not accessible.
If Google doesn't respond soon, I plan to enable GitHub discussions as an alternative.

[jgm]

I'm going to open up GitHub discussions. If Google comes through and resurrects our old group, we'll need to decide whether to go back to it or continue with discussions, but knowing that this kind of thing can happen inclines me not to rely on Google.

[reagle]

I suspect Google is neglecting Google Groups and will ignore our query. (If the old messages are of value to folks, perhaps someone has a local archive that could be made available as static HTML?)

[tarleb]

My archives go back only to mid March 2018. Are there any good tools to publish this kind of archive?

[simonmichael]

I just discovered this via mastodon. I'm sorry this happened @jgm. Surely someone at google will eventually repair the pandoc googlegroup, but who knows when that might be.

You can recover a partial archive, at least, by pointing a newsreader like Thunderbird at NNTP server news.gmane.io, and subscribing to newsgroup gmane.text.pandoc. It seems to have all messages from 2010-05-20. I believe you can get them out of Thunderbird in mbox format, eg.

[badumont]

It seems to be possible to import archives in mbox format into Mailman. It documents importing Mailman 2.1 archives into Mailman 3, but I hope that it would work with any mbox files.

[tarleb]

Information provided by @pamoroso, posted on Mastodon:

Have you already tried posting to the Google Groups Help Community?
https://support.google.com/groups/community

For some products it's possible to escalate issues to or get further assistance from Google. This is done by posting to the product's official help community so that some of the badged volunteers who can make escalations (any type of badge except for Bronze) notices.

The badged volunteers (Product Experts) may not be able to do much. The only way to find out is to post to the relevant help community.

[leahneukirchen]

I scraped an archive from gmane and put it online at https://inbox.vuxu.org/pandoc-discuss for now. This goes back to 2010-05-20.

I noticed some threads are broken due to missing Message-IDs, I'm not sure how complete the archive at gmane was.

If someone else has old messages to backfill, I'll gladly add them.

[simonmichael]

https://public-inbox.org/README.html eh.. interesting.

[jgm]

@leahneukirchen I have 68000 messages going back to 2007. Let me know how to get them to you.

[leahneukirchen]

@jgm Upload them somewhere and tell me at [email protected] how to fetch them?

[jgm]

@tarleb I posted on the google help forum, we'll see if that helps.

[jgm]

I have found messages from 2007-2010 and @leahneukirchen has added them to the searchable archive here, which should now be pretty complete: https://inbox.vuxu.org/pandoc-discuss

[simonmichael]

Very cool. And one can point Thunderbird at inbox.vuxu.org and subscribe to vuxu.archive.pandoc-discuss.

I believe you could also add the earlier messages to gmane (and update other metadata) at https://admin.gmane.io . Gmane is NNTP-only unlike inbox.vuxu.org, but it is a (relatively) well known place for finding FOSS mail lists.

[simonmichael]

(Gmane admins might remove the recent spam too.)

If you don't mind me asking I'm curious how this happened - was the list left too open (new subscribers/non subscribers allowed to post unmoderated) for too long ?

[jgm]

New users were always allowed to post unmoderated. In the past, Google's spam filter would almost always catch spam and block it -- and often block a lot of genuine messages too, which I'd have to moderate myself. I almost suspect they have decided to give up on Groups and turned off the spam filter intentionally. The spam in this case was obviously spam, easily detectable -- the sort of thing Google's email filter blocks all the time.

[simonmichael]

Interesting, thanks. I started seeing too much spam and had to turn on moderation a few years back.

[leahneukirchen]

@simonmichael note that inbox.vuxu.org is currently not subscribed (because I'm not sure the list is working?) but this can be done to make it receive the new messages too.

[leahneukirchen]

I can't mail [email protected] at least, ...

[hg2581]

What is the current status of this? Would it be possible to move to groups.io instead of googlegroups.com? Perhaps also upload old messages from googlegroups to groups.io?

[jgm]

I think the GitHub discussions approach is working well. I'm not sure what the point of another change would be, and groups.io would cost $20/month, with the number of members we have. We have an archive of the old messages on pandoc-discuss (see above).

[tarleb]

My one worry here is archiving, it would be nice if we could export the discussions if we ever need to.

On the other hand, pandoc using GitHub discussions is very convenient for projects like Quarto, which are using the same.

[jgm]

There is no official export API from Discussions, but there are some hacked up solutions here: https://github.com/orgs/community/discussions/3315
I figure we could export if we needed to. (And maybe we should set up a script that does this on a regular basis.)

[hg2581]

I find a mailing list much more convenient since it allows you to see all questions and replies thus allowing the casual user to expand his/her knowledge of - in this case - how to efficiently use pandoc. If the questions or the replies are not of interest to you, you simply delete the emails.

A forum, or similar, such as github, forces you to take the time to make the effort to visit the forum and thus makes information /less/ accessible for anyone - except those who have a strong interest in the project and thus probably visit the forum/github every day/often.

An RSS feed for the github discussions would ameliorate the above but I do not see that one exists? I would love to be proven wrong though.

[ant-222]

I think that mailing lists are ligher, simpler, cleaner, more convenient, more traditional, and more accessible means of communication than the bloated github web interface. I wish the wonderful Pandoc maling lists were resurrected on a sounder basis than Google Groups, such as freelists.org or another proper maling-list host.

[simonmichael]

Longevity of such hosts is a problem. They tend to disappear within about 10 years. The alternative is to take on a costly long term list hosting maintenance project which takes time away from other things. Google and Github have the appeal of being relatively long lived and easy to export from when the time comes.

[simonmichael]

(Now I've clicked past the giant cookie banner and seen freelists' featured 20-year age... You still can't be sure they'll be around in 10 years, but that's a good track record.)

[jgm]

Google and Github have the appeal of being relatively long lived and easy to export from when the time comes.

This is what I'd always assumed, but it turned out not to be true. When Google froze our mailing list (and didn't reply to any of our requests to unfreeze it), it also made all the content inaccessible, even to the maintainers.

freelists does look like an interesting possibility. But now that discussions have gotten going on GitHub, I'm reluctant to change yet again...

[ant-222]

@simonmichael

(Now I've clicked past the giant cookie banner and seen freelists' featured 20-year age... You still can't be sure they'll be around in 10 years, but that's a good track record.)

Yes, the age of a service is a good predictor of its longetifity. The Oracle mailing list has been hosted there since 2004. Plus, the archives may be exported and reused on another host.

[hg2581]

On June 27, 2024 1:18:15 PM EDT, Anton Shepelev ***@***.***> wrote: I think that mailing lists are ligher, simpler, cleaner, more convenient, more traditional, and more accessible means of communication than the bloated github web interface. It would be a great if the wonderful *Pandoc* maling lists were resurrected on a sounder basis than *Google Groups*, such as [`freelists.org`](https://freelists.org/) or another proper maling-list host. -- Reply to this email directly or view it on GitHub: #9250 (comment) You are receiving this because you commented. Message ID: ***@***.***>

Absolutely agree!

[ant-222]

@jgm , if you register and host your project on GNU Savannah, their reliable mailing list service (via mailman will be at your disposal. It is very reliable, and will probably exist as long as GNU itself exists. See if Pandoc satisfies their hosting requirements.

[iandol]

Just to add a voice of dissent here. Pandoc is developed on Github, issues are managed here, 3rd party pandoc filters are mostly developed here, many templates are found here. Keeping discussions here makes a lot of sense. Some people prefer an email interface, but others don't; and the arguments above seem to be based on personal preferences which are not a good justification for uprooting just the discussions from github IMO...