Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests: certain tests are locale-dependent #8557

Open
darrenwee opened this issue Feb 28, 2018 · 0 comments · May be fixed by #13162
Open

Tests: certain tests are locale-dependent #8557

darrenwee opened this issue Feb 28, 2018 · 0 comments · May be fixed by #13162
Labels
a-Testing Testing-related traits such as efficiency, robustness, coverage

Comments

@darrenwee
Copy link
Member

darrenwee commented Feb 28, 2018 •
edited
Loading

Environment

  • current master or newer
  • I reproduced this on Ubuntu 16.04, and the original issue was in Arch Linux, but it should happen on any OS with the same locale settings.

Steps to Reproduce

  1. set system locale to a locale that uses different date/decimal formats, like a German locale (language code de). On Ubuntu, I set LC_TIME and LC_NUMERIC to de_DE.UTF-8.
  2. run the following tests:
./gradlew test --tests teammates.test.cases.util.TimeHelperTest --tests teammates.test.cases.datatransfer.FeedbackResponseDetailsTest --tests teammates.test.cases.action.InstructorFeedbackQuestionEditActionTest

(there are more tests failing due to locale, but I have kept it brief for this issue)

Expected Behavior

Tests will pass.

Actual Behavior

Some tests will fail.

Gradle suite > Gradle test > teammates.test.cases.datatransfer.FeedbackResponseDetailsTest.testCreateResponseDetails FAILED
    java.lang.AssertionError: expected:<-3.5> but was:<-3,5>
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101)
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108)
        at teammates.test.cases.BaseTestCase.assertEquals(BaseTestCase.java:143)
        at teammates.test.cases.datatransfer.FeedbackResponseDetailsTest.testCreateResponseDetails(FeedbackResponseDetailsTest.java:179)

Gradle suite > Gradle test > teammates.test.cases.action.InstructorFeedbackQuestionEditActionTest.testExecuteAndPostProcessNumScale FAILED
    java.lang.NumberFormatException: For input string: "0,5"
        at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
        at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
        at java.lang.Double.parseDouble(Double.java:538)
        at teammates.common.datatransfer.questions.FeedbackNumericalScaleQuestionDetails.extractQuestionDetails(FeedbackNumericalScaleQuestionDetails.java:65)
        at teammates.common.datatransfer.questions.FeedbackQuestionType.getFeedbackQuestionDetailsInstance(FeedbackQuestionType.java:86)
        at teammates.common.datatransfer.questions.FeedbackQuestionDetails.createQuestionDetails(FeedbackQuestionDetails.java:178)
        at teammates.ui.controller.InstructorFeedbackQuestionEditAction.extractFeedbackQuestionData(InstructorFeedbackQuestionEditAction.java:213)
        at teammates.ui.controller.InstructorFeedbackQuestionEditAction.execute(InstructorFeedbackQuestionEditAction.java:45)
        at teammates.ui.controller.Action.executeAndPostProcess(Action.java:476)
        at teammates.test.cases.action.BaseActionTest.getRedirectResult(BaseActionTest.java:76)
        at teammates.test.cases.action.InstructorFeedbackQuestionEditActionTest.testExecuteAndPostProcessNumScale(InstructorFeedbackQuestionEditActionTest.java:812)

Gradle suite > Gradle test > teammates.test.cases.browsertests.InstructorFeedbackSessionsPageUiTest.testMiscellaneous FAILED
    java.lang.AssertionError: expected:<Mi Apr 16 2014> but was:<Wed Apr 16 2014>
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101)
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108)
        at teammates.test.cases.BaseTestCase.assertEquals(BaseTestCase.java:143)
        at teammates.test.cases.browsertests.InstructorFeedbackSessionsPageUiTest.testDatePickerScripts(InstructorFeedbackSessionsPageUiTest.java:707)
        at teammates.test.cases.browsertests.InstructorFeedbackSessionsPageUiTest.testJScripts(InstructorFeedbackSessionsPageUiTest.java:657)
        at teammates.test.cases.browsertests.InstructorFeedbackSessionsPageUiTest.testMiscellaneous(InstructorFeedbackSessionsPageUiTest.java:87)

java.lang.AssertionError: expected:<Wed, 30 Dec 2015, 12:00 NOON> but was:<Mi, 30 Dez 2015, 12:00 NOON>
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101)
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108)
        at teammates.test.cases.BaseTestCase.assertEquals(BaseTestCase.java:143)
        at teammates.test.cases.util.TimeHelperTest.testEndOfYearDates(TimeHelperTest.java:205)
        ...

Gradle suite > Gradle test > teammates.test.cases.util.TimeHelperTest.testEndOfYearDates FAILED
    java.lang.AssertionError: expected:<Wed, 30 Dec 2015, 12:00 NOON> but was:<Mi, 30 Dez 2015, 12:00 NOON>
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101)
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108)
        at teammates.test.cases.BaseTestCase.assertEquals(BaseTestCase.java:143)
        at teammates.test.cases.util.TimeHelperTest.testEndOfYearDates(TimeHelperTest.java:205)

java.lang.AssertionError: expected:<Mon, 30 Nov 2015, 12:00 NOON UTC+0000> but was:<Mo, 30 Nov 2015, 12:00 NOON UTC+0000>
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101)
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108)
        at teammates.test.cases.BaseTestCase.assertEquals(BaseTestCase.java:143)
        at teammates.test.cases.util.TimeHelperTest.testFormatDateTimeForSessions(TimeHelperTest.java:216)
        ...

Gradle suite > Gradle test > teammates.test.cases.util.TimeHelperTest.testFormatDateTimeForSessions FAILED
    java.lang.AssertionError: expected:<Mon, 30 Nov 2015, 12:00 NOON UTC+0000> but was:<Mo, 30 Nov 2015, 12:00 NOON UTC+0000>
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:101)
        at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:108)
        at teammates.test.cases.BaseTestCase.assertEquals(BaseTestCase.java:143)
        at teammates.test.cases.util.TimeHelperTest.testFormatDateTimeForSessions(TimeHelperTest.java:216)

Investigation

Certain tests are locale-dependent. In the case of #8444, the contributor's locale was set to de_DE.utf8 which uses commas as a decimal separator instead of periods. He was unable to complete setup due to these failing tests which expected 3.5 but the method tested would print 3,5 instead.

They are usually related to string formatting functions where the locale is not explicitly specified.

Potential Solutions

Using the failing tests above as an example, this works by overriding JVM settings:

./gradlew test -Duser.language=en --tests teammates.test.cases.util.TimeHelperTest --tests teammates.test.cases.datatransfer.FeedbackResponseDetailsTest --tests teammates.test.cases.action.InstructorFeedbackQuestionEditActionTest

Alternatively, you can change your system locale like in #8444, where the contributor used (on his unix-like system)

LANG=en_US.UTF-8 ./eclipse

to start his IDE using the en_US locale, but this requires user intervention and only for certain systems.

We can do the first solution and add the argument for user.language into gradle.properties by specifying org.gradle.jvmargs=-Duser.language=en. Any opinions?
@whipermr5 whipermr5 added the a-Testing Testing-related traits such as efficiency, robustness, coverage label Mar 3, 2018
@tshradheya tshradheya mentioned this issue May 23, 2018
Closed
@tshradheya tshradheya mentioned this issue Jun 11, 2018
Closed
This was referenced Jul 20, 2018
Closed
Closed
Closed
Closed
@igorpenhaa igorpenhaa mentioned this issue Jun 17, 2023
Closed
@itstrueitstrueitsrealitsreal itstrueitstrueitsrealitsreal linked a pull request Aug 6, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
a-Testing Testing-related traits such as efficiency, robustness, coverage
Projects
None yet
Milestone
No milestone
2 participants
@whipermr5 @darrenwee