Make operator TreeEnsemble 5x faster for batches of size 100.000

[xadupre]

Description:

The fix only improves the operator. It parallelizes by trees instead of observations.

1 target or 1 class

BEFORE

AFTER

multiregression or multiclass

BEFORE

AFTER

Motivation and Context
This change makes onnxruntime as fast as scikit-learn on big batches.

[yuslepukhin]

          std::lock_guard<OrtMutex> lock(merge_mutex);

Do we need a mutex here since each thread has its own block? Can we we make code take advantage of it and rid of the mutex? #Resolved

---

Refers to: onnxruntime/core/providers/cpu/ml/tree_ensemble_common.h:384 in ad600af. [](commit_id = ad600af, deletion_comment = False)

[xadupre]

          std::lock_guard<OrtMutex> lock(merge_mutex);

Do we need a mutex here since each thread has its own block? Can we we make code take advantage of it and rid of the mutex?

Refers to: onnxruntime/core/providers/cpu/ml/tree_ensemble_common.h:384 in ad600af. [](commit_id = ad600af, deletion_comment = False)

Removed.

[yuslepukhin]