XWiki formats support

[surli]

Is your feature request related to a problem? Please describe.
As part of XWiki development we manipulate three different kind of translation formats:

• standard java properties
• XWiki XML properties
• XWiki full pages

If the first format is a standard one, the two others are really specific to XWiki, and we need to keep them for backward compatibility.

Describe the solution you'd like
It would be nice to have an API to define a transformation, from a monolingual source file, to its translation. To be able to define custom translation formats.

Describe alternatives you've considered
Right now we manipulate our different formats by using dedicated addons.
Basically our workflow is the following:

• our XML files are automatically transformed to properties on update events
• the translations are performed on those properties file
• on pre commit event the properties are transformed back to the right format
• on post commit event, we clean the commit to remove the files we don't want on our VCS

This solution works, but it's costly for us since we have to maintain our scripts, and it prevents us to use some nice features of Weblate such as the discovery addon.

Additional context

To give you an hint of our specific translation format, here's some link:

• XWiki XML property base translation: https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwiki-platform-menu/xwiki-platform-menu-ui/src/main/resources/Menu/MenuTranslations.xml

• XWiki XML property translation: https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwiki-platform-menu/xwiki-platform-menu-ui/src/main/resources/Menu/MenuTranslations.es.xml

• XWiki Full page base translation: https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwiki-platform-sandbox/src/main/resources/Sandbox/WebHome.xml

• XWiki Full page translation: https://github.com/xwiki/xwiki-platform/blob/master/xwiki-platform-core/xwiki-platform-sandbox/src/main/resources/Sandbox/WebHome.es.xml

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[nijel]

Related issues #2978 (though regexp is probably not best approach for parsing XML) and #1897

As usual my recommendation would be to stick with standard formats (there is already plenty of them) instead of inventing own ones ;-).

[surli]

As usual my recommendation would be to stick with standard formats (there is already plenty of them) instead of inventing own ones ;-).

I usually agree with that, but here it's a bit complicated: XWiki is a 15 years old wiki platform that supports translations and which can be used as a development platform. So lots of applications of XWiki are directly developed inside XWiki and needs to use the wiki pages for translations.

A bit difficult to change now the way it works, especially since we try to be very careful with backward compatibility :)

[vmassol]

Definitely +1 for the need! That would be a big help.

It would make the Weblate project a lot more extensible and cater for various extra needs. This is not about asking Weblate devs to support more formats but to allow users to do the work and provide new formats :)

Please consider it, that would make weblate even more awesome and extensible.

Thx for a great product.

[nijel]

That sounds like an interesting feature. However, I currently don't see how such formats would be defined. I don't think that regexp (see #2978) is a good way to do this. In the end it might turn out to be easier to implement support properly in Python...

[surli]

In the end it might turn out to be easier to implement support properly in Python...

I agree. For a first implem, I'd be really happy to "just" have an interface to implement in which I could define

1. the extraction of the translation keys from the translation source, and
2. the transformation from the saved translations to the final file.

[nijel]

For the XML properties stuff you can use similar approach like we do for Excel files. It does parse the XLSX file and feeds resulting CSV to the CSV format handler. See https://github.com/WeblateOrg/weblate/blob/master/weblate/formats/external.py

[surli]

Interesting, I'll check that! Actually it might even work for the other format we have: our current approach is to consider only two tags of the XML (content and title) and to create fake properties for them, so looks like I could use the same approach here.

I guess I'd need to change the list of formats in https://github.com/WeblateOrg/weblate/blob/master/weblate/formats/models.py is that it?

[nijel]

You can change that in the settings.py, no need to modify Weblate code, see https://docs.weblate.org/en/latest/admin/config.html#weblate-formats

[surli]

ok, thanks for all the info, I'll do some experiment on it

[surli]

Hi @nijel

I started a while ago to implement the needed changes to support XWiki specific translations formats in Weblate, the work is not entirely finished, mainly because I had more urgent things to do and I had to adapt the changes I started to the evolutions of Weblate (for example the introduction of proppluralunit changed a bit of things :) ).
So now I'm wondering what's the best way for us to use those formats and still be able to follow the upgrades on Weblate. So far I see three solutions:

1. to merge the changes directly on the official project. Pros: for us, it's the best it would mean a direct integration of XWiki formats in Weblate. Cons: it means in the long term a support of it in the project to not break the test, so more work for you for a format you don't really want, so I'm pretty sure you're against this solution :)
2. to support a fork of Weblate with the specific formats, and to deploy/maintain the fork. Pros: we keep having the same infra of tests than for Weblate. Cons: we need to maintain it manually by rebasing regularly the work and deploying it.
3. to encapsulate the changes in a dedicated module that we could install with pip. Pros: we only maintain what we need (we'd just need to bump a Weblate version in case of release). Cons: I tried to do that, but it sounds like an heavy task to be able to perform the tests, to deploy on pip and to install in one command. Maybe you'd have some advice for doing that properly?

So maybe you have some other suggestions on how to extend Weblate properly?

[nijel]

The additional format maintenance is pretty low, so I'd go with option 1. If there are some low-level things, the code should preferably end up in the translate-toolkit and Weblate having just thin layer on top of that.

[github-actions]

Thank you for your report, the issue you have reported has just been fixed.

• In case you see a problem with the fix, please comment on this issue.
• In case you see a similar problem, please open a separate issue.
• If you are happy with the outcome, consider supporting Weblate by donating.