Creation of Exact Quantile Check #512
Conversation
|
@jmilis2000 thanks for the PR! We will be having a look shortly. Can you please check in the meantime why the build is failing? |
@rdsharma26 I fixed the issue. The build should work now. It failed because there was no amazon copyright at the top of the new analyzer i created. |
There was a problem hiding this comment.
LGTM. To confirm, you are using Spark's percentile to calculate the exact percentile rather than using the existing hasApproxQuantile check with error = 0.0. I assume this is because the latter uses Spark's approx quantile method.
When merging, please use the Squash and merge option.
| import com.amazon.deequ.analyzers.Preconditions.{hasColumn, isNumeric} | ||
| import com.amazon.deequ.analyzers.Analyzers.{conditionalSelection, ifNoNullsIn} | ||
| import com.amazon.deequ.metrics.FullColumn | ||
| import org.apache.curator.shaded.com.google.common.annotations.VisibleForTesting |
There was a problem hiding this comment.
We use import com.google.common.annotations.VisibleForTesting in other places.
* Creation of Exact Quantile Check * Fix build issue --------- Co-authored-by: ZDQ870 <[email protected]>
* Creation of Exact Quantile Check * Fix build issue --------- Co-authored-by: ZDQ870 <[email protected]>
* Creation of Exact Quantile Check * Fix build issue --------- Co-authored-by: ZDQ870 <[email protected]>
* Creation of Exact Quantile Check * Fix build issue --------- Co-authored-by: ZDQ870 <[email protected]>
* Creation of Exact Quantile Check * Fix build issue --------- Co-authored-by: ZDQ870 <[email protected]>
* Creation of Exact Quantile Check * Fix build issue --------- Co-authored-by: ZDQ870 <[email protected]>
* Creation of Exact Quantile Check * Fix build issue --------- Co-authored-by: ZDQ870 <[email protected]>
Issue #, if available:
Description of changes:
Added hasExactQuantile Check. Similar to hasApproxQuantile but finds more precise quantile value.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.