[FIX] Enable preprocessing in reg_cocktails

[ravinkohli]

Types of changes

• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Note that a Pull Request should only contain one of refactoring, new features or documentation changes.
Please separate these changes and send us individual PRs for each.
For more information on how to create a good pull request, please refer to The anatomy of a perfect pull request.

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.

• Have you checked to ensure there aren't other open Pull Requests for the same update/change?
• Have you added an explanation of what your changes do and why you'd like us to include them?
• Have you written new tests for your core changes, as applicable?
• Have you successfully ran tests with your changes locally?

Description

Following the discussion in last weeks meeting, we would like to have preprocessing as part of the pipeline in regularisation cocktails. Therefore this PR makes the necessary changes to achieve that. These include uncommenting the various preprocessor transforms as well as code for learning embeddings. Moreover, this PR also removes the is_small_preprocess attribute from BaseDataset as we will always preprocess early in the EarlyPreprocessing node.

Motivation and Context

It is required to allow searching for preprocessing hyperparameters in the reg_cocktails branch. Also, to avoid redundant preprocessing of the dataset for each epoch, we would like to preprocess the whole data all at once.

How has this been tested?

Previous tests for preprocessing and early preprocessing have been enabled. Also tests for embedding module have been enabled to check if embedding is compatible with the preprocessing approach

[nabenabe0928]

Thanks for the PR, I reviewed all the files except tabular_feature_validator.py related (two) files.

[nabenabe0928]

Add some comments on tabular_feature_validator.

[nabenabe0928]

I checked the tabular_feature_validator

[nabenabe0928]

The lines below will look better by this (len(enc_columns) > 0 for data containing categoricals, right?):

            num_numericals, num_categoricals = self.feat_type.count('numerical'), self.feat_type.count('categorical')
            if num_numericals + num_categoricals != len(self.feat_type):
                raise ValueError("Elements of feat_type must be either ['numerical', 'categorical']")

            self.categorical_columns = list(range(num_categoricals))
            self.numerical_columns = list(range(num_categoricals, num_categoricals + num_numericals))

[nabenabe0928]

Why do not we need this anymore?

[nabenabe0928]

Suggested change

	if len(self.dtypes) != 0:
	# when train data has no object dtype, but test does
	# we prioritise the datatype given in training data
	for column, data_type in zip(X.columns, self.dtypes):
	X[column] = X[column].astype(data_type)
	else:
	# Calling for the first time to infer the categories
	X = X.infer_objects()
	for column, data_type in zip(X.columns, X.dtypes):
	if not is_numeric_dtype(data_type):
	X[column] = X[column].astype('category')
	elif len(self.dtypes) != 0: # when train data has no object dtype, but test does
	# we prioritise the datatype given in training data
	for column, data_type in zip(X.columns, self.dtypes):
	X[column] = X[column].astype(data_type)
	else: # Calling for the first time to infer the categories
	X = X.infer_objects()
	for column, data_type in zip(X.columns, X.dtypes):
	if not is_numeric_dtype(data_type):
	X[column] = X[column].astype('category')

[ravinkohli]

I think these are just preferences on where to start the comment.

[nabenabe0928]

Nono, it actually removed an indent level.

[ravinkohli]

actually, if you notice we are also saving the dtypes in self.object_dtype_mapping which should be done for both of the two conditions you moved back an indent level. So, I think its fine the way it is.

[nabenabe0928]

Oh yeah, I did not notice, but I also did not notice that we still have the same issue (which happens when we have a huge number of features) in this method.
Could you fix it?

        if hasattr(self, 'object_dtype_mapping'):
            # Mypy does not process the has attr. This dict is defined below
            try:
                X = X.astype(self.object_dtype_mapping)
            except Exception as e:
                self.logger.warning(f'Casting test data to data type in train data caused the exception {e}')
                pass
            return

        if len(self.dtypes) != 0:
            # when train data has no object dtype, but test does.  Prioritise the datatype given in training data
            dtype_dict = {col: dtype for col, dtype in zip(X.columns, self.dtypes)}
            X = X.astype(dtype_dict)
        else:
            # Calling for the first time to infer the categories
            X = X.infer_objects()
            dtype_dict = {col: 'category' for col, dtype in zip(X.columns, X.dtypes) if not is_numeric_dtype(dtype)}
            X = X.astype(dtype_dict)

        # only numerical attributes and categories
        self.object_dtype_mapping = {col: dtype for col, dtype in zip(X.columns, X.dtypes)}

[nabenabe0928]

Hey, thanks for your effort.
I think we can merge once you addressed this.

[nabenabe0928]

Hi, thanks for the changes and sorry for the late response.
I will approve the changes.

[nabenabe0928]

Looks good to me now:)

[ArlindKadra]

Thanks for the PR :). It looks good to me too now. I only have one last minor question.