Migrate Python code to a dedicated package

[stefan6419846]

This is my attempt to migrate the existing code for working with artificial training data to a dedicated Python package, as proposed in #308 and #307. This includes some additional refactoring to the module structure to better encapsulate specific functionality.

I have used version number 0.1 for now, although I am up to changing this.

When migrating, I had two parameters which I am not sure about:

• overwrite, defaulting to False, does not seem to be used at all.

• It has not been clear enough for me what extract_font_properties really means and therefore misses documentation. text2image --help did not really help me in this case as well:

  --only_extract_font_properties Assumes that the input file contains a list of ngrams. Renders each ngram, extracts spacing properties and records them in output_base/[font_name].fontinfo file. (type:bool default:false)

  What would be an appropriate documentation of the parameter?

If there is anything unclear or you want to see anything changed about this, feel free to ask or report.

[Shreeshrii]

Thanks. A dedicated package for training from fonts is a good idea.

You may want to look at the original bash scripts that were sought to be replicated in these python scripts in older versions eg. https://github.com/tesseract-ocr/tesseract/tree/4.0/src/training

'overwrite' if I recall correctly was used for the legacy training offered in the bash scripts.

[stefan6419846]

• linedata = False is a legacy-only parameter which is unsupported, so we might be able to drop it in this process as well.
• extract_font_properties has always been without any documentation, see usage in https://github.com/tesseract-ocr/tesseract/blob/4.1/src/training/tesstrain.sh#L18-L51 for example.
• overwrite has been used in make__traineddata only (see https://github.com/tesseract-ocr/tesseract/blob/4.1/src/training/tesstrain_utils.sh#L622-L624). As this method is not available any more, we can probably drop it.

[stweil]

Suggested change

'Programming Language :: Python :: 3.6',

[stweil]

Suggested change

	python_requires='>=3.6',
	python_requires='>=3.7',

[stweil]

Font properties (bold, italic, ...) are also from the legacy training and still unsupported with the LSTM recognizer. That's one of the reasons why there remains a certain need for legacy models. The old tesstrain.sh supported training of legacy models, and I think that it would be good to support it in the Python code, too. That should be done separately, not in this pull request here, but maybe you can keep the corresponding parameters with appropriate TODO comments.

[stefan6419846]

@stweil Do you mean that we should keep the existing parameters for now when you are talking about the legacy support? Or does this refer to the linedata parameter only?

[stweil]

@stweil Do you mean that we should keep the existing parameters for now when you are talking about the legacy support? Or does this refer to the linedata parameter only?

I'd keep all existing parameters for now (with comments).

[stefan6419846]

I have updated the requirements inside the README and fixed the parameters for tesstrain.wrapper.run().

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stefan6419846]

Is there anything to be changed here to get this PR merged?

[stweil]

This whole file looks like a copy from somewhere else. Which parts are really required? Why can those parts not be included in the root .gitignore?

[kba]

I think this is the standard GitHub Python .gitignore

[stefan6419846]

Yes, this is the default Python .gitignore template which usually is recommended for Python projects.

Only keeping the relevant entries might be an option, mainly concerning specific libraries which are currently unused. Nevertheless, I usually prefer to keep the default .gitignore file as it usually is some type of common sense.

I did not add this to the root .gitignore file to avoid unintended side effects for now, as the whole repository consists of two more or less independent parts, while not being clearly separated for now (see #307).

[stefan6419846]

@stweil I have reduced the .gitignore to the entries which actually make sense in this use case (at least from my point of view).

Do you still want me to migrate the corresponding entries to the root .gitignore file?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stefan6419846]

Any further update on this?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[zdenop]

Can this be merged?

[stefan6419846]

At least from my side, yes. @stweil had some objections about the .gitignore file, but I did not yet hear back after my latest request for further changes which might be required.

[stweil]

I'd remove any reference to Python 3.6 (see my two comments). .gitignore still contains lots of entries which are not strictly necessary, but that is not critical for merging.

[stefan6419846]

I have removed the references to Python 3.6 and updated the README to make clear that Tesseract version 5 is supported as well.

[stweil]

Let's try the new code. Thank you @stefan6419846 for your patience.

[zdenop]

@stweil: what about tagging the previous code/commit as version 1.0?
So we can maybe do more reorganization of code without breaking somebody's workflow.
I am interested to make training on windows just by using python. If python is required then Auxiliaries are really not needed.