runtime error #1129

wwfcnu · 2022-12-09T01:16:01Z

Traceback (most recent call last):
File "tools/k2/icefall/egs/aishell/ASR/local/compile_hlg.py", line 166, in
main()
File "tools/k2/icefall/egs/aishell/ASR/local/compile_hlg.py", line 156, in main
HLG = compile_HLG(lang_dir, args.lm)
File "tools/k2/icefall/egs/aishell/ASR/local/compile_hlg.py", line 94, in compile_HLG
LG = k2.compose(L, G)
File "/usr/local/python3.7.10/lib/python3.7/site-packages/k2/fsa_algo.py", line 488, in compose
treat_epsilons_specially=treat_epsilons_specially)
File "/usr/local/python3.7.10/lib/python3.7/site-packages/k2/fsa_algo.py", line 405, in intersect
treat_epsilons_specially, need_arc_map)
RuntimeError:
Some bad things happened. Please read the above error messages and stack
trace. If you are using Python, the following command may be helpful:

  gdb --args python /path/to/your/code.py

(You can use `gdb` to debug the code. Please consider compiling
a debug version of k2.).

If you are unable to fix it, please open an issue at:

  https://github.com/k2-fsa/k2/issues/new

Compile decoding graph HLG.pt succeeded

The text was updated successfully, but these errors were encountered:

csukuangfj · 2022-12-09T01:26:12Z

@wwfcnu
Please post all the logs.

wwfcnu · 2022-12-09T01:53:51Z

@wwfcnu Please post all the logs.

/usr/local/python3.7.10/lib/python3.7/site-packages/torchaudio/_internal/module_utils.py:99: UserWarning: Failed to import soundfile. 'soundfile' backend
is not available.
warnings.warn("Failed to import soundfile. 'soundfile' backend is not available.")
Compile lexicon L.pt L_disambig.pt succeeded
/tmp/pip-install-w9q6xu_b/kaldilm_80542c8aeb934a60b2460688478f9031/kaldilm/csrc/arpa_file_parser.cc:void kaldilm::ArpaFileParser::Read(std::istream&):79
[I] Reading \data\ section.
/tmp/pip-install-w9q6xu_b/kaldilm_80542c8aeb934a60b2460688478f9031/kaldilm/csrc/arpa_file_parser.cc:void kaldilm::ArpaFileParser::Read(std::istream&):140
[I] Reading \1-grams: section.
/tmp/pip-install-w9q6xu_b/kaldilm_80542c8aeb934a60b2460688478f9031/kaldilm/csrc/arpa_file_parser.cc:void kaldilm::ArpaFileParser::Read(std::istream&):213
[W] line 7 [-8.6712055 0] skipped: word '' not in symbol table
/tmp/pip-install-w9q6xu_b/kaldilm_80542c8aeb934a60b2460688478f9031/kaldilm/csrc/arpa_file_parser.cc:void kaldilm::ArpaFileParser::Read(std::istream&):140
[I] Reading \2-grams: section.
/tmp/pip-install-w9q6xu_b/kaldilm_80542c8aeb934a60b2460688478f9031/kaldilm/csrc/arpa_file_parser.cc:void kaldilm::ArpaFileParser::Read(std::istream&):140
[I] Reading \3-grams: section.
/usr/local/python3.7.10/lib/python3.7/site-packages/torchaudio/_internal/module_utils.py:99: UserWarning: Failed to import soundfile. 'soundfile' backend
is not available.
warnings.warn("Failed to import soundfile. 'soundfile' backend is not available.")
2022-12-08 10:04:06,974 INFO [compile_hlg.py:154] Processing /home/wangweifei/data/local/hlg
2022-12-08 10:04:07,840 INFO [lexicon.py:168] Loading pre-compiled /home/wangweifei/data/local/hlg/Linv.pt
2022-12-08 10:04:07,910 INFO [compile_hlg.py:73] Building ctc_topo. max_token_id: 4232
2022-12-08 10:04:08,680 INFO [compile_hlg.py:78] Loading pre-compiled G_3_gram
2022-12-08 10:07:15,573 INFO [compile_hlg.py:93] Intersecting L and G
[F] /home/runner/work/k2/k2/k2/csrc/array.h:501:void k2::Array1::Init(k2::ContextPtr, int32_t, k2::Dtype) [with T = k2::Arc; k2::ContextPtr = std::sha
red_ptrk2::Context; int32_t = int] Check failed: size >= 0 (-2043020552 vs. 0) Array size MUST be greater than or equal to 0, given :-2043020552

[ Stack-Trace: ]
/usr/local/python3.7.10/lib/python3.7/site-packages/k2/lib/libk2_log.so(k2::internal::GetStackTrace()+0x47) [0x7f7d83be9f37]
/usr/local/python3.7.10/lib/python3.7/site-packages/k2/lib/libk2context.so(k2::internal::Logger::~Logger()+0x5a) [0x7f7d83f1eaaa]
/usr/local/python3.7.10/lib/python3.7/site-packages/k2/lib/libk2context.so(k2::FsaVecCreator::Init(std::vector<k2host::Array2Size, std::allocator<k2
host::Array2Size > > const&)+0x58b) [0x7f7d84059eeb]
/usr/local/python3.7.10/lib/python3.7/site-packages/k2/lib/libk2context.so(k2::Intersect(k2::Raggedk2::Arc&, int, k2::Raggedk2::Arc&, int, bool, k2::
Raggedk2::Arc, k2::Array1, k2::Array1)+0x920) [0x7f7d83fab2c0]
/usr/local/python3.7.10/lib/python3.7/site-packages/k2/lib/libk2context.so(k2::Intersect(k2::Raggedk2::Arc&, int, k2::Raggedk2::Arc&, int, bool, k2::
Raggedk2::Arc, k2::Array1, k2::Array1)+0x11cd) [0x7f7d83fabb6d]
/usr/local/python3.7.10/lib/python3.7/site-packages/k2/lib/libk2context.so(k2::Intersect(k2::Raggedk2::Arc&, int, k2::Raggedk2::Arc&, int, bool, k2::
Raggedk2::Arc, k2::Array1, k2::Array1*)+0x1246) [0x7f7d83fabbe6]
/usr/local/python3.7.10/lib/python3.7/site-packages/_k2.cpython-37m-x86_64-linux-gnu.so(+0x651f8) [0x7f7d8b1561f8]
/usr/local/python3.7.10/lib/python3.7/site-packages/_k2.cpython-37m-x86_64-linux-gnu.so(+0x203d7) [0x7f7d8b1113d7]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyMethodDef_RawFastCallKeywords+0x30b) [0x7f7ea2de2bcb]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyCFunction_FastCallKeywords+0x25) [0x7f7ea2de3945]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyEval_EvalFrameDefault+0x8ba0) [0x7f7ea2be5420]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyEval_EvalCodeWithName+0x996) [0x7f7ea2d0c086]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyFunction_FastCallKeywords+0x93) [0x7f7ea2de3063]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyEval_EvalFrameDefault+0x62f7) [0x7f7ea2be2b77]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyEval_EvalCodeWithName+0x996) [0x7f7ea2d0c086]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyFunction_FastCallKeywords+0x93) [0x7f7ea2de3063]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyEval_EvalFrameDefault+0x8445) [0x7f7ea2be4cc5]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyEval_EvalCodeWithName+0x996) [0x7f7ea2d0c086]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyFunction_FastCallKeywords+0x93) [0x7f7ea2de3063]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyEval_EvalFrameDefault+0x5d18) [0x7f7ea2be2598]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(+0x77f43) [0x7f7ea2be6f43]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyEval_EvalFrameDefault+0x5d18) [0x7f7ea2be2598]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_PyEval_EvalCodeWithName+0x996) [0x7f7ea2d0c086]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(PyEval_EvalCodeEx+0x3e) [0x7f7ea2d0c30e]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(PyEval_EvalCode+0x1b) [0x7f7ea2d0d0db]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(PyRun_FileExFlags+0xb2) [0x7f7ea2cdabf2]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(PyRun_SimpleFileExFlags+0xed) [0x7f7ea2cdbffd]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(+0x159a72) [0x7f7ea2cc8a72]
/usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0(_Py_UnixMain+0x39) [0x7f7ea2cac049]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7f7ea279fc87]
python(_start+0x2a) [0x55649a3de75a]
Traceback (most recent call last):
File "tools/k2/icefall/egs/aishell/ASR/local/compile_hlg.py", line 166, in
main()
File "tools/k2/icefall/egs/aishell/ASR/local/compile_hlg.py", line 156, in main
HLG = compile_HLG(lang_dir, args.lm)
File "tools/k2/icefall/egs/aishell/ASR/local/compile_hlg.py", line 94, in compile_HLG
LG = k2.compose(L, G)
File "/usr/local/python3.7.10/lib/python3.7/site-packages/k2/fsa_algo.py", line 488, in compose
treat_epsilons_specially=treat_epsilons_specially)
File "/usr/local/python3.7.10/lib/python3.7/site-packages/k2/fsa_algo.py", line 405, in intersect
treat_epsilons_specially, need_arc_map)
RuntimeError:
Some bad things happened. Please read the above error messages and stack
trace. If you are using Python, the following command may be helpful:

  gdb --args python /path/to/your/code.py

(You can use `gdb` to debug the code. Please consider compiling
a debug version of k2.).

If you are unable to fix it, please open an issue at:

  https://github.com/k2-fsa/k2/issues/new

Compile decoding graph HLG.pt succeeded

csukuangfj · 2022-12-09T01:57:11Z

[F] /home/runner/work/k2/k2/k2/csrc/array.h:501:void k2::Array1::Init(k2::ContextPtr, int32_t, k2::Dtype) [with T = k2::Arc; k2::ContextPtr = std::sha
red_ptrk2::Context; int32_t = int] Check failed: size >= 0 (-2043020552 vs. 0) Array size MUST be greater than or equal to 0, given :-2043020552

How large is your G? (e.g., its file size).

Also, are you using the latest k2?

python3 -m k2.version

shall give you the version information.

wwfcnu · 2022-12-09T02:08:04Z

[F] /home/runner/work/k2/k2/k2/csrc/array.h:501:void k2::Array1::Init(k2::ContextPtr, int32_t, k2::Dtype) [with T = k2::Arc; k2::ContextPtr = std::sha
red_ptrk2::Context; int32_t = int] Check failed: size >= 0 (-2043020552 vs. 0) Array size MUST be greater than or equal to 0, given :-2043020552

How large is your G? (e.g., its file size).

Also, are you using the latest k2?

python3 -m k2.version

shall give you the version information.

Collecting environment information...

k2 version: 1.21
Build type: Release
Git SHA1: f1ae355
Git date: Sun Oct 30 16:37:07 2022
Cuda used to build k2: 11.3
cuDNN used to build k2: 8.2.0
Python version used to build k2: 3.7
OS used to build k2: Ubuntu 18.04.6 LTS
CMake version: 3.24.2
GCC version: 7.5.0
CMAKE_CUDA_FLAGS: -Wno-deprecated-gpu-targets -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_35,code=sm_35 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_50,code=sm_50 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_60,code=sm_60 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_61,code=sm_61 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_70,code=sm_70 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_75,code=sm_75 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_80,code=sm_80 -lineinfo --expt-extended-lambda -use_fast_math -Xptxas=-w --expt-extended-lambda -gencode arch=compute_86,code=sm_86 -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_86,code=sm_86 -gencode arch=compute_86,code=compute_86 -Xcudafe --diag_suppress=cc_clobber_ignored,--diag_suppress=integer_sign_change,--diag_suppress=useless_using_declaration,--diag_suppress=set_but_not_used,--diag_suppress=field_without_dll_interface,--diag_suppress=base_class_has_different_dll_interface,--diag_suppress=dll_interface_conflict_none_assumed,--diag_suppress=dll_interface_conflict_dllexport_assumed,--diag_suppress=implicit_return_from_non_void_function,--diag_suppress=unsigned_compare_with_zero,--diag_suppress=declared_but_not_referenced,--diag_suppress=bad_friend_decl --expt-relaxed-constexpr --expt-extended-lambda -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options -Wall --compiler-options -Wno-strict-overflow --compiler-options -Wno-unknown-pragmas
CMAKE_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=0 -Wno-unused-variable -Wno-strict-overflow
PyTorch version used to build k2: 1.12.1+cu113
PyTorch is using Cuda: 11.3
NVTX enabled: True
With CUDA: True
Disable debug: True
Sync kernels : False
Disable checks: False
Max cpu memory allocate: 214748364800 bytes (or 200.0 GB)
k2 abort: False
file: /usr/local/python3.7.10/lib/python3.7/site-packages/k2/version/version.py
_k2.file: /usr/local/python3.7.10/lib/python3.7/site-packages/_k2.cpython-37m-x86_64-linux-gnu.so

csukuangfj · 2022-12-09T02:15:04Z

Could you try the following pull request?
k2-fsa/icefall#606

It uses OpenFst to build HLG and is able to support a much larger G.

csukuangfj · 2022-12-09T02:15:42Z

Git date: Sun Oct 30 16:37:07 2022

Also, your k2 is somewhat outdated. please update it to the latest one.

wwfcnu · 2022-12-09T02:19:54Z

Git date: Sun Oct 30 16:37:07 2022

Also, your k2 is somewhat outdated. please update it to the latest one.

okay, I'll give it a try

csukuangfj · 2022-12-09T02:24:52Z

Also, your G is so large that I am afraid the resulting HLG won't fit into the GPU memory.

wwfcnu · 2022-12-09T03:36:22Z

Also, your G is so large that I am afraid the resulting HLG won't fit into the GPU memory.
my lm.arpa is 50G ,so do i need to prune arpa?

csukuangfj · 2022-12-09T03:38:20Z

Yes, I think so.

You may use
kaldi-asr/kaldi#4594
to prune it.

wwfcnu · 2022-12-09T08:33:21Z

egs/librispeech/ASR/local/compile_hlg_using_openfst.py路径下没找到这个文件啊

csukuangfj · 2022-12-09T08:47:17Z

k2-fsa/icefall#606

egs/librispeech/ASR/local/compile_hlg_using_openfst.py路径下没找到这个文件啊

I just merged k2-fsa/icefall#606

Please check again. You should find it.

wwfcnu · 2022-12-09T08:59:52Z

k2-fsa/icefall#606

egs/librispeech/ASR/local/compile_hlg_using_openfst.py路径下没找到这个文件啊

I just merged k2-fsa/icefall#606

Please check again. You should find it
找到了

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime error #1129

runtime error #1129

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

runtime error #1129

runtime error #1129

Comments

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022

csukuangfj commented Dec 9, 2022

wwfcnu commented Dec 9, 2022