Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update pr-data.csv, test renamed #1127

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Update pr-data.csv, test renamed #1127

wants to merge 3 commits into from

Conversation

yijut2
Copy link
Contributor

@yijut2 yijut2 commented Nov 30, 2023

The ClassHelperTest.java was renamed as ClassUtilsTest.java. Here is the link to the related commit: apache/dubbo#3806

@yijut2
Update pr-data.csv
6e3d78e 
Test renamed
@darko-marinov
Copy link
Contributor

Did you check if it remained OD after the rename?

@darko-marinov
Copy link
Contributor

Please update the PR with the commit where the test was moved or renamed. Otherwise, we'll take -2 "points" if the PR remains unresolved past the deadline.

@yijut2
Copy link
Contributor Author

yijut2 commented Dec 9, 2023

Hi professor,

I cannot successfully build this project, so I can't tell whether it is still OD or not. Should I close this PR?

@darko-marinov
Copy link
Contributor

Just add in the Notes a link to the commit that moved or renamed the test. Separate the new link from the existing link with a ;.

@darko-marinov
Copy link
Contributor

Please put a link to the commit not a PR. Also squash your commits (without closing this PR).

@alexbadia1
Copy link
Contributor

alexbadia1 commented Dec 22, 2023 •
edited
Loading

Following up on the unresolved PR.

Hi professor,

I cannot successfully build this project, so I can't tell whether it is still OD or not. Should I close this PR?

Developers sometimes commit partial changes that would obviously fail to build. This commit d7e11f08a6bfc85ce47c4db8dffe8c307aafc54d, with the renaming of the test case, fails to build:

...
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.0:compile (default-compile) on project dubbo-config-api: Compilation failure
[ERROR] /root/dubbo/dubbo-config/dubbo-config-api/src/main/java/org/apache/dubbo/config/AbstractConfig.java:[646,26] error: cannot find symbol
[ERROR]   symbol:   method isGetter(Method)
[ERROR]   location: class ClassUtils
...

The next closest commit in the PR that builds is: 65a9227f379f2278192b4c2d141a958d287dde55

Commit With Rename

This apache/dubbo#3806 is correct where the rename occurred here: apache/dubbo@d7e11f0.

However, the diff isn't a simple renaming of a Java Class name. Instead the maintainer removed
dubbo-common/src/test/java/org/apache/dubbo/common/utils/ClassHelperTest.java and replaced it with dubbo-common/src/test/java/org/apache/dubbo/common/utils/ClassUtilsTest.java.

Did you check if it remained OD after the rename?

After 89 rounds of running iDFlakies against the PR branch and 270 rounds running against the latest (3.2) branch, it does not appear org.apache.dubbo.common.utils.ClassUtilsTest.testForName3 is still OD Flaky.

However, the latest branch has over 1,000 tests alone in the dubbo-common submodule and I cannot check all 1,000! permutations. I would attempt git bisect, but combined with iDFlakies randomness, doing so would be a complete waste of time.

I tried to look into the source of the flakiness, but it is unclear what is causing the flakiness. I'm not even sure if this is really flaky or not...

So whether or not this is still flaky or not is inconclusive.

Steps To Reproduce

###############################################################################
# Setup
# 
# Windows WSL2 Ubuntu 18.04.6 LTS:
###############################################################################

# Maven Version
root@LAPTOP-MR10D0GP:~/dubbo# mvn -v
Apache Maven 3.9.2 (c9616018c7a021c1c39be70fb2843d6f5f9b8a1c)
Maven home: /root/apache-maven-3.9.2
Java version: 1.8.0_202, vendor: Oracle Corporation, runtime: /root/jdk1.8.0_202/jre
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.10.16.3-microsoft-standard-wsl2", arch: "amd64", family: "unix"

# Java Version
root@LAPTOP-MR10D0GP:~/dubbo# java -version
java version "1.8.0_202"
Java(TM) SE Runtime Environment (build 1.8.0_202-b08)
Java HotSpot(TM) 64-Bit Server VM (build 25.202-b08, mixed mode)

###############################################################################
# iDFlakies
###############################################################################

# Sanity Check IDoft
git checkout 737f7a7ea67832d7f17517326fb2491d0a086dd7
mvn clean install -DskipTests -Dgpg.skip=true -Dcheckstyle.skip=true -U
mvn -pl dubbo-common edu.illinois.cs:idflakies-maven-plugin:2.0.1-SNAPSHOT:detect -Ddetector.detector_type=random-class-method -Ddt.randomize.rounds=100 -Ddt.detector.original_order.all_must_pass=false > idflakies--100_rounds--dubbo_common--idoft--737f7a7ea67832d7f17517326fb2491d0a086dd7.log

# Check Commit With Rename
git fetch origin 65a9227f379f2278192b4c2d141a958d287dde55
git checkout 65a9227f379f2278192b4c2d141a958d287dde55
mvn clean install -DskipTests -Dgpg.skip=true -Dcheckstyle.skip=true -U
mvn -pl dubbo-common edu.illinois.cs:idflakies-maven-plugin:2.0.1-SNAPSHOT:detect -Ddetector.detector_type=random-class-method -Ddt.randomize.rounds=100 -Ddt.detector.original_order.all_must_pass=false > idflakies--100_rounds--dubbo_common--commit_with_test_rename--65a9227f379f2278192b4c2d141a958d287dde55.log

# 
mvn -pl dubbo-common edu.illinois.cs:idflakies-maven-plugin:2.0.1-SNAPSHOT:detect -Ddetector.detector_type=random-class-method -Ddt.randomize.rounds=10 -Ddt.detector.original_order.all_must_pass=false > idflakies--10_rounds--dubbo-all--latest--f7a32be429e815e9bc70f0dec31d67461e5c695e.log

###############################################################################
# iDFlakies (Experimental Approach)
# 
# Targeting just the test class fails to reproduce the flakiness
###############################################################################

#  Try Targeting org.apache.dubbo.common.utils.ClassHelperTest
mvn clean install -DskipTests -Dcheckstyle.skip=true -U
mvn -pl dubbo-common edu.illinois.cs:idflakies-maven-plugin:2.0.1-SNAPSHOT:detect -Ddetector.detector_type=random-class-method -Ddt.randomize.rounds=10000 -Ddt.detector.original_order.all_must_pass=false -Ddt.original.order=./input.txt > idflakies--10000_rounds--dubbo_common_utils_ClassHelperTest--idoft--737f7a7ea67832d7f17517326fb2491d0a086dd7.log

# input.txt
org.apache.dubbo.common.utils.ClassHelperTest.testForName1
org.apache.dubbo.common.utils.ClassHelperTest.testForName2
org.apache.dubbo.common.utils.ClassHelperTest.testForName3
org.apache.dubbo.common.utils.ClassHelperTest.testForNameWithThreadContextClassLoader
org.apache.dubbo.common.utils.ClassHelperTest.testGetCallerClassLoader
org.apache.dubbo.common.utils.ClassHelperTest.testGetClassLoader1
org.apache.dubbo.common.utils.ClassHelperTest.testGetClassLoader2
org.apache.dubbo.common.utils.ClassHelperTest.testResolvePrimitiveClassName
org.apache.dubbo.common.utils.ClassHelperTest.testToShortString
org.apache.dubbo.common.utils.ClassHelperTest.tetForNameWithCallerClassLoader

I also tried to run iDFlakies against the latest (Apache Dubbo 3.2 branch) and get this error:

root@LAPTOP-MR10D0GP:~/dubbo# mvn edu.illinois.cs:idflakies-maven-plugin:2.0.1-SNAPSHOT:detect -Ddetector.detector_type=random-class-method -Ddt.randomize.rounds=10 -Ddt.detector.original_order.all_must_pass=false > idflakies--10_rounds--dubbo-all--latest--f7a32be429e815e9bc70f0dec31d67461e5c695e.log
java.lang.NullPointerException
        at edu.illinois.cs.dt.tools.plugin.DetectorMojo.detectorExecute(DetectorMojo.java:287)
        at edu.illinois.cs.dt.tools.plugin.DetectorMojo.lambda$execute$5(DetectorMojo.java:211)
        at edu.illinois.cs.dt.tools.utility.ErrorLogger.runAndLogError(ErrorLogger.java:17)
        at edu.illinois.cs.dt.tools.plugin.DetectorMojo.execute(DetectorMojo.java:211)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:126)
        at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:342)
        at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:330)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:175)
        at org.apache.maven.lifecycle.internal.MojoExecutor.access$000(MojoExecutor.java:76)
        at org.apache.maven.lifecycle.internal.MojoExecutor$1.run(MojoExecutor.java:163)
        at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute(DefaultMojosExecutionStrategy.java:39)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:160)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:105)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:73)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:53)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:118)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:261)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:173)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:101)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:910)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:283)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:206)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:283)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:226)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:407)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:348)
java.lang.NullPointerException
        at edu.illinois.cs.dt.tools.plugin.DetectorMojo.detectorExecute(DetectorMojo.java:287)
        at edu.illinois.cs.dt.tools.plugin.DetectorMojo.lambda$execute$5(DetectorMojo.java:211)
        at edu.illinois.cs.dt.tools.utility.ErrorLogger.runAndLogError(ErrorLogger.java:17)
        at edu.illinois.cs.dt.tools.plugin.DetectorMojo.execute(DetectorMojo.java:211)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:126)
        at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2(MojoExecutor.java:342)
        at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:330)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:175)
        at org.apache.maven.lifecycle.internal.MojoExecutor.access$000(MojoExecutor.java:76)
        at org.apache.maven.lifecycle.internal.MojoExecutor$1.run(MojoExecutor.java:163)
        at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute(DefaultMojosExecutionStrategy.java:39)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:160)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:105)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:73)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:53)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:118)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:261)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:173)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:101)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:910)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:283)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:206)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:283)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:226)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:407)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:348)

The logs are here: https://gitlab.engr.illinois.edu/abadia2/cs527/-/tree/master/incomplete/finalize-pull-1127?ref_type=heads

Is this enough to close the PR?

If so, logistically, how do you want me to "resolve" this PR as I am not the author?

@darko-marinov
Copy link
Contributor

In the PR changes, what should be the link instead of apache/dubbo#3806 ? For this test itself, we can stop at MovedOrRenamed and not inspect further when/if was not OD.

For iDFlakies, it seems you found some bug. It'd be good to report it at https://github.com/UT-SE-Research/iDFlakies

@alexbadia1
Copy link
Contributor

alexbadia1 commented Jan 4, 2024 •
edited
Loading

In the PR changes, what should be the link instead of apache/dubbo#3806 ? For this test itself, we can stop at MovedOrRenamed and not inspect further when/if was not OD.

The specific commit to be listed in idoft from that PR apache/dubbo#3806 should be: d7e11f08a6bfc85ce47c4db8dffe8c307aafc54d

@darko-marinov
Copy link
Contributor

@yijut2 can you change the last column per the previous comment?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@yijut2 @darko-marinov @alexbadia1