Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] test_ceil_scale_zero may fail with different datagen_seed #9846

Closed
thirtiseven opened this issue Nov 24, 2023 · 1 comment · Fixed by #9874
Closed

[BUG] test_ceil_scale_zero may fail with different datagen_seed #9846

thirtiseven opened this issue Nov 24, 2023 · 1 comment · Fixed by #9874
Assignees
Labels
bug Something isn't working

Comments

@thirtiseven
Copy link
Collaborator

Describe the bug

Similar to #9722, ceil scale zero will also failed with a very low chance.

I just tested with a large datagen length and it failed 21/500000 cases.

Steps/Code to reproduce bug

@pytest.mark.skipif(is_before_spark_330(), reason='scale parameter in Ceil function is not supported before Spark 3.3.0')
@pytest.mark.parametrize('data_gen', [double_gen], ids=idfn)
def test_ceil_scale_zero(data_gen):
    assert_gpu_and_cpu_are_equal_collect(
            lambda spark : unary_op_df(spark, data_gen, length=500000).selectExpr('ceil(a, 0)'),
            conf={'spark.rapids.sql.castFloatToDecimal.enabled':'true'})
SPARK_RAPIDS_TEST_DATAGEN_SEED=0 ./integration_tests/run_pyspark_from_build.sh -k test_ceil_scale_zero
--- CPU OUTPUT
+++ GPU OUTPUT
@@ -27201,7 +27201,7 @@
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('914785406793038'))
+Row(ceil(a, 0)=Decimal('914785406793039'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
@@ -83333,7 +83333,7 @@
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('-268780119644888'))
+Row(ceil(a, 0)=Decimal('-268780119644887'))
 Row(ceil(a, 0)=Decimal('765'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
@@ -145522,7 +145522,7 @@
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('386821058338736'))
+Row(ceil(a, 0)=Decimal('386821058338737'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
@@ -151592,7 +151592,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('347468926523874'))
+Row(ceil(a, 0)=Decimal('347468926523875'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
@@ -174909,7 +174909,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('-648213542067262'))
+Row(ceil(a, 0)=Decimal('-648213542067261'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
@@ -184110,7 +184110,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('34715321583059'))
+Row(ceil(a, 0)=Decimal('34715321583060'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
@@ -248673,7 +248673,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('-336439502593597'))
+Row(ceil(a, 0)=Decimal('-336439502593596'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
@@ -275566,7 +275566,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('726035573992915'))
+Row(ceil(a, 0)=Decimal('726035573992916'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
@@ -286220,7 +286220,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('613482001616293'))
+Row(ceil(a, 0)=Decimal('613482001616294'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
@@ -299785,7 +299785,7 @@
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('155367056917850'))
+Row(ceil(a, 0)=Decimal('155367056917851'))
 Row(ceil(a, 0)=Decimal('-1'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
@@ -309533,7 +309533,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('63236347372494'))
+Row(ceil(a, 0)=Decimal('63236347372495'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
@@ -322107,7 +322107,7 @@
 Row(ceil(a, 0)=Decimal('279481813468'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('-509538434436340'))
+Row(ceil(a, 0)=Decimal('-509538434436339'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
@@ -341983,7 +341983,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('71028686951699'))
+Row(ceil(a, 0)=Decimal('71028686951700'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
@@ -361213,7 +361213,7 @@
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('-774584842740091'))
+Row(ceil(a, 0)=Decimal('-774584842740090'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
@@ -361946,7 +361946,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('-281282663258196'))
+Row(ceil(a, 0)=Decimal('-281282663258195'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
@@ -378437,7 +378437,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('820944638453204'))
+Row(ceil(a, 0)=Decimal('820944638453205'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
@@ -386232,7 +386232,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('-956904927710798'))
+Row(ceil(a, 0)=Decimal('-956904927710797'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
@@ -429971,7 +429971,7 @@
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('-198186798685224'))
+Row(ceil(a, 0)=Decimal('-198186798685223'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('-1'))
 Row(ceil(a, 0)=None)
@@ -479864,7 +479864,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('783836022861805'))
+Row(ceil(a, 0)=Decimal('783836022861806'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
@@ -482327,7 +482327,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
-Row(ceil(a, 0)=Decimal('-403434742584967'))
+Row(ceil(a, 0)=Decimal('-403434742584966'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
@@ -494891,7 +494891,7 @@
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=None)
-Row(ceil(a, 0)=Decimal('646272999448835'))
+Row(ceil(a, 0)=Decimal('646272999448836'))
 Row(ceil(a, 0)=None)
 Row(ceil(a, 0)=Decimal('0'))
 Row(ceil(a, 0)=None)
@thirtiseven thirtiseven added bug Something isn't working ? - Needs Triage Need team to review and classify labels Nov 24, 2023
@thirtiseven thirtiseven changed the title [BUG] test_ceil_scale_zero may fail with different seed [BUG] test_ceil_scale_zero may fail with different datagen_seed Nov 24, 2023
@jlowe jlowe self-assigned this Nov 28, 2023
@jlowe
Copy link
Member

jlowe commented Nov 28, 2023

The failures have the same root cause as in #9722. The problem isn't with the ceiling operation but rather the casting of floating point to decimal which is already a known issue (and disabled by default).

@jlowe jlowe linked a pull request Nov 28, 2023 that will close this issue
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Nov 28, 2023
@jlowe jlowe closed this as completed Nov 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants