Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[11.0.21.18.9][lilliput]release版本x64平台vmTestbase/nsk/monitoring/stress/lowmem/lowmem007/TestDescription.java小概率随机crash：G1ParEvacuateFollowersClosure::do_void() #762

Open
sendaoYan opened this issue Jan 2, 2024 · 8 comments
Open

[11.0.21.18.9][lilliput]release版本x64平台vmTestbase/nsk/monitoring/stress/lowmem/lowmem007/TestDescription.java小概率随机crash：G1ParEvacuateFollowersClosure::do_void() #762

sendaoYan opened this issue Jan 2, 2024 · 8 comments
Assignees
@mmyxym

Comments

@sendaoYan
Copy link
Collaborator

sendaoYan commented Jan 2, 2024 •
edited
Loading

job：https://tone.aliyun-inc.com/ws/xesljfzh/test_result/263375?tab=1

Description
release版本x64平台vmTestbase/nsk/monitoring/stress/lowmem/lowmem007/TestDescription.java小概率随机crash：G1ParEvacuateFollowersClosure::do_void()

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f6f227a53df, pid=2198740, tid=2198848
#
# JRE version: OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (11.0.21.18+9) (build 11.0.21.18+9)
# Java VM: OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (11.0.21.18+9, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0x7a53df]  G1ParEvacuateFollowersClosure::do_void()+0x47f
#
# Core dump will be written. Default location: /tmp/tone/run/jtreg/jt-work/hotspot_jtreg/vmTestbase/nsk/monitoring/stress/lowmem/lowmem007/TestDescription/core.2198740
#
# If you would like to submit a bug report, please visit:
#   mailto:[email protected]

Steps to Reproduce
Steps to reproduce the behavior:

export test=test/hotspot/jtreg/vmTestbase/nsk/monitoring/stress/lowmem/lowmem007/TestDescription.java
function runJtreg() { jtreg -ea -esa -timeoutFactor:4 -v:fail,error,time,nopass -nr -w $dir/index-$1 $test &> $dir/$1.log ; if [[ 0 -ne $? ]] ; then echo -n "$1 " ; else rm -rf $dir/index-$1 $dir/$1.log ; fi ; } ; export -f runJtreg ; export dir="tmp-jtreg-"`basename ${test##* } .java` ; rm -rf $dir ; mkdir -p $dir ; time seq 10000 | xargs -i -n 1 -P 1 bash -c "
runJtreg {}" ; echo total fail number: `ls $dir/*.log 2> /dev/null | wc | awk '{print $1}'`

Expected behavior
all passed

tone.log

hs_err_pid2198740.log

JDK version

# uname -a ; cat /etc/os-release ; free -h ; lscpu | head -n 25 ; java -version ; java -Xinternalversion
Linux iZbp1go2piwffcpjzwyemxZ 5.10.134-15.an8.x86_64 #1 SMP Thu Jul 20 00:35:47 CST 2023 x86_64 x86_64 x86_64 GNU/Linux
NAME="Anolis OS"
VERSION="8.8"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.8"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.8"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"

              total        used        free      shared  buff/cache   available
Mem:           60Gi        15Gi        43Gi       2.0Mi       1.7Gi        44Gi
Swap:            0B          0B          0B
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
BIOS Model name:     pc-i440fx-2.1
Stepping:            6
CPU MHz:             3498.388
BogoMIPS:            5399.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-31
openjdk version "11.0.21.18" 2023-10-17
OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9)
OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9, mixed mode)
OpenJDK 64-Bit Server VM (11.0.21.18+9) for linux-amd64 JRE (11.0.21.18+9), built on Dec 27 2023 07:48:26 by "" with gcc 7.5.0
@sendaoYan sendaoYan mentioned this issue Jan 3, 2024
Closed
@sendaoYan
Copy link
Collaborator Author

二进制链接：

BINARY_URL=https://dragonwell.oss-cn-shanghai.aliyuncs.com/11.0.21.18.9/Alibaba_Dragonwell_Extended_11.0.21.18.9_x64_linux.tar.gz
TEST_IMAGE=https://dragonwell.oss-cn-shanghai.aliyuncs.com/11.0.21.18.9/Alibaba_Dragonwell_Extended_11.0.21.18.9_x64_linux-testimage.tar.gz

ip-list-temurin.txt
ip-list-Xmixed.txt
ip-list-UseCompactObjectHeaders.txt

从昨天晚上到现在，开启和关闭UseCompactObjectHeaders总共分别运行了1.5w该用例，均未复现该问题

@sendaoYan
Copy link
Collaborator Author

sendaoYan commented Jan 3, 2024 •
edited
Loading

2024年01月03日14:30:44复现一次（总共大概运行了2w次）：

887.log
hs_err_pid472879.log

115.29.214.196

# uname -a ; cat /etc/os-release ; free -h ; lscpu | head -n 25 ; java -version ; java -Xinternalversion
Linux iZbp1igy5idwfm60r53sb7Z 5.10.134-15.an8.x86_64 #1 SMP Thu Jul 20 00:35:47 CST 2023 x86_64 x86_64 x86_64 GNU/Linux
NAME="Anolis OS"
VERSION="8.8"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.8"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.8"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"

              total        used        free      shared  buff/cache   available
Mem:           60Gi       980Mi        54Gi       2.0Mi       4.9Gi        58Gi
Swap:            0B          0B          0B
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
BIOS Model name:     pc-i440fx-2.1
Stepping:            6
CPU MHz:             3499.993
BogoMIPS:            5399.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-31
openjdk version "11.0.21.18" 2023-10-17
OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9)
OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9, mixed mode)
OpenJDK 64-Bit Server VM (11.0.21.18+9) for linux-amd64 JRE (11.0.21.18+9), built on Dec 27 2023 07:48:26 by "" with gcc 7.5.0

@sendaoYan
Copy link
Collaborator Author

yitian总共跑了大概1.5w，没有复现
x64 alinux3总共跑了大概2.5w次，复现了3次

# uname -a ; cat /etc/os-release ; free -h ; lscpu | head -n 25 ; java -version ; java -Xinternalversion
Linux iZbp1hsfceg13a47ez2dazZ 5.10.134-15.al8.x86_64 #1 SMP Thu Jul 20 00:44:04 CST 2023 x86_64 x86_64 x86_64 GNU/Linux
NAME="Alibaba Cloud Linux"
VERSION="3 (Soaring Falcon)"
ID="alinux"
ID_LIKE="rhel fedora centos anolis"
VERSION_ID="3"
PLATFORM_ID="platform:al8"
PRETTY_NAME="Alibaba Cloud Linux 3 (Soaring Falcon)"
ANSI_COLOR="0;31"
HOME_URL="https://www.aliyun.com/"

              total        used        free      shared  buff/cache   available
Mem:          247Gi        16Gi       208Gi       2.0Mi        21Gi       228Gi
Swap:            0B          0B          0B
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
BIOS Model name:     pc-i440fx-2.1
Stepping:            6
CPU MHz:             3484.701
BogoMIPS:            5399.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-31
openjdk version "11.0.21.18" 2023-10-17
OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9)
OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9, mixed mode)
OpenJDK 64-Bit Server VM (11.0.21.18+9) for linux-amd64 JRE (11.0.21.18+9), built on Dec 27 2023 07:48:26 by "" with gcc 7.5.0

hs_err_pid46383.log

@sendaoYan
Copy link
Collaborator Author

sendaoYan commented Jan 5, 2024 •
edited
Loading

倚天运行大概3.7w次，未复现

@sendaoYan
Copy link
Collaborator Author

centos运行2.3w次复现一次：[email protected]

# uname -a ; cat /etc/os-release ; free -h ; lscpu | head -n 25 ; java -version ; java -Xinternalversion
Linux iZbp1aflbhgkeypurnjg2kZ 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Wed Dec 22 13:25:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
NAME="CentOS Linux"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"
              total        used        free      shared  buff/cache   available
Mem:          247Gi        24Gi       191Gi       2.0Mi        31Gi       221Gi
Swap:            0B          0B          0B
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              64
On-line CPU(s) list: 0-63
Thread(s) per core:  2
Core(s) per socket:  32
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
BIOS Model name:     pc-i440fx-2.1
Stepping:            6
CPU MHz:             3021.284
BogoMIPS:            5399.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-63
openjdk version "11.0.21.18" 2023-10-17
OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9)
OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9, mixed mode)
OpenJDK 64-Bit Server VM (11.0.21.18+9) for linux-amd64 JRE (11.0.21.18+9), built on Dec 27 2023 07:48:26 by "" with gcc 7.5.0

hs_err_pid403367.log

573.log

@sendaoYan
Copy link
Collaborator Author

ecs.hfg8i.16xlarge这个规格的ecs，跑了大概12w次，目前没有复现

@sendaoYan
Copy link
Collaborator Author

sendaoYan commented Jan 8, 2024 •
edited
Loading

（intel 8代）ecs.hfg8i.16xlarge规格 alinux3运行12w次未复现
（intel）ecs.g7.16xlarge规格，centos、alinux3、anolis8复现概率大概为1/1w
（倚天）ecs.c8y.8xlarge规格，alinux3运行10w次未复现

@mmyxym
Copy link
Collaborator

mmyxym commented Jan 8, 2024

目前只有g7 icelake小概率复现，并且sig handler和core显示的error address和上下指令寄存器不一致

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mmyxym mmyxym

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sendaoYan @mmyxym