Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Testsuite fails on several archs #252

Closed
sthibaul opened this issue Jun 9, 2022 · 16 comments · Fixed by #253
Closed

Testsuite fails on several archs #252

sthibaul opened this issue Jun 9, 2022 · 16 comments · Fixed by #253

Comments

@sthibaul
Copy link

sthibaul commented Jun 9, 2022 •
edited
Loading

cmusphinx/sphinxbase#77 was archived, so copying it here.

The sphinxbase testsuite fails on some architectures in Debian :

ppc64el: https://buildd.debian.org/status/fetch.php?pkg=sphinxbase&arch=ppc64el&ver=0.8%2B5prealpha%2B1-8&stamp=1582414870&raw=0

alpha: https://buildd.debian.org/status/fetch.php?pkg=sphinxbase&arch=alpha&ver=0.8%2B5prealpha%2B1-8&stamp=1583125429&raw=0

Here is the test.log on s390x (on commit 4f07168):

test.log
@dhdaines
Copy link
Contributor

dhdaines commented Jun 9, 2022

Thanks! Looks like we still have some issues. It looks like IBM provides free access to s390 servers for development so I'll sign up for that as it seems that I need to debug things on an actual big-endian machine.

@dhdaines
Copy link
Contributor

dhdaines commented Jun 9, 2022

There are a number of bad x86/amd64 assumptions, not just endianness but also packing and alignment, in the trie LM code (can't remember who wrote it but I'm sure it wasn't me). Hopefully fixing for s390 will help other architectures as well. Working on it in this branch: https://github.com/cmusphinx/pocketsphinx/tree/bigendian_bigiron_fixes

@dhdaines dhdaines mentioned this issue Jun 9, 2022
Merged
@dhdaines
Copy link
Contributor

dhdaines commented Jun 9, 2022

Please reopen if you see problems on Alpha and PPCEL still, or open a separate bug as those will be memory access rather than byteorder issues.

@sthibaul
Copy link
Author

100% tests passed, 0 tests failed out of 93 on ppcel indeed

@sthibaul
Copy link
Author

I don't have access to an alpha machine

@sthibaul
Copy link
Author

It also went fine on hppa

@sthibaul
Copy link
Author

On mipsel it failed:

Test project /home/sthibault/pocketsphinx/build
      Start  1: test-cards.sh
 1/93 Test  #1: test-cards.sh ....................   Passed    6.48 sec
      Start  2: test-lm.sh
 2/93 Test  #2: test-lm.sh .......................***Failed  130.39 sec
Test: test-lm
run PASSED
INFO: pocketsphinx_batch.c(868): AVERAGE 5.25 xRT (CPU), 5.25 xRT (elapsed)
TOTAL Percent correct = 66.20% Error = 36.62% Accuracy = 63.38%
match FAILED
Sub-tests failed: match

      Start  3: test-align.sh
 3/93 Test  #3: test-align.sh ....................   Passed   20.29 sec
      Start  4: test-tidigits-fsg.sh
 4/93 Test  #4: test-tidigits-fsg.sh .............   Passed    5.54 sec
      Start  5: test-tidigits-simple.sh
 5/93 Test  #5: test-tidigits-simple.sh ..........   Passed    7.57 sec
      Start  6: test_acmod
 6/93 Test  #6: test_acmod .......................   Passed   16.47 sec
      Start  7: test_acmod_grow
 7/93 Test  #7: test_acmod_grow ..................   Passed    7.45 sec
      Start  8: test_alignment
 8/93 Test  #8: test_alignment ...................   Passed    1.22 sec
      Start  9: test_allphone
 9/93 Test  #9: test_allphone ....................***Failed   37.19 sec
FAIL: (0) == (strcmp(hyp, expected))
ALLPHONE: SIL G OW F UW R W ER D K AE N M IY T ER Z +SPN+ (-7864, 0)

      Start 10: test_bitvec
10/93 Test #10: test_bitvec ......................   Passed    2.46 sec
      Start 11: test_dict2pid
11/93 Test #11: test_dict2pid ....................   Passed    1.21 sec
      Start 12: test_dict
12/93 Test #12: test_dict ........................   Passed    1.85 sec
      Start 13: test_fe
13/93 Test #13: test_fe ..........................   Passed    0.01 sec
      Start 14: test_fwdflat
14/93 Test #14: test_fwdflat .....................   Passed  314.14 sec
      Start 15: test_fwdtree_bestpath
15/93 Test #15: test_fwdtree_bestpath ............   Passed   34.84 sec
      Start 16: test_fwdtree
16/93 Test #16: test_fwdtree .....................   Passed   33.99 sec
      Start 17: test_init
17/93 Test #17: test_init ........................   Passed    4.07 sec
      Start 18: test_jsgf
18/93 Test #18: test_jsgf ........................   Passed    3.78 sec
      Start 19: test_keyphrase
19/93 Test #19: test_keyphrase ...................   Passed    7.38 sec
      Start 20: test_lattice
20/93 Test #20: test_lattice .....................   Passed   16.53 sec
      Start 21: test_ngram_model_read
21/93 Test #21: test_ngram_model_read ............***Failed    7.21 sec
FAIL: (0) == (strcmp(hyp, "go forward ten degrees"))
go forward ten years (-7617)
go forward ten (-6517)

      Start 22: test_log_shifted
22/93 Test #22: test_log_shifted .................   Passed    0.02 sec
      Start 23: test_log_int8
23/93 Test #23: test_log_int8 ....................   Passed    0.01 sec
      Start 24: test_log_int16
24/93 Test #24: test_log_int16 ...................   Passed    0.04 sec
      Start 25: test_mllr
25/93 Test #25: test_mllr ........................   Passed    0.34 sec
      Start 26: test_nbest
26/93 Test #26: test_nbest .......................   Passed   18.57 sec
      Start 27: test_pitch
27/93 Test #27: test_pitch .......................   Passed    1.40 sec
      Start 28: test_posterior
28/93 Test #28: test_posterior ...................   Passed   24.85 sec
      Start 29: test_ptm_mgau
29/93 Test #29: test_ptm_mgau ....................   Passed    3.81 sec
      Start 30: test_reinit
30/93 Test #30: test_reinit ......................   Passed    2.68 sec
      Start 31: test_senfh
31/93 Test #31: test_senfh .......................   Passed   41.23 sec
      Start 32: test_set_search
32/93 Test #32: test_set_search ..................   Passed   53.78 sec
      Start 33: test_simple
33/93 Test #33: test_simple ......................   Passed   41.25 sec
      Start 34: test_state_align
34/93 Test #34: test_state_align .................   Passed    8.80 sec
      Start 35: test_ckd_alloc
35/93 Test #35: test_ckd_alloc ...................   Passed    0.01 sec
      Start 36: test_listelem_alloc
36/93 Test #36: test_listelem_alloc ..............   Passed    0.01 sec
      Start 37: test_ckd_alloc_catch
37/93 Test #37: test_ckd_alloc_catch .............   Passed    0.01 sec
      Start 38: test_ckd_alloc_fail
38/93 Test #38: test_ckd_alloc_fail ..............   Passed    0.01 sec
      Start 39: test_ckd_alloc_abort
39/93 Test #39: test_ckd_alloc_abort .............   Passed    0.01 sec
      Start 40: _lcase1.test
40/93 Test #40: _lcase1.test .....................   Passed    0.01 sec
      Start 41: _lcase2.test
41/93 Test #41: _lcase2.test .....................   Passed    0.01 sec
      Start 42: _lcase3.test
42/93 Test #42: _lcase3.test .....................   Passed    0.01 sec
      Start 43: _strcmp1.test
43/93 Test #43: _strcmp1.test ....................   Passed    0.01 sec
      Start 44: _strcmp2.test
44/93 Test #44: _strcmp2.test ....................   Passed    0.01 sec
      Start 45: _strcmp3.test
45/93 Test #45: _strcmp3.test ....................   Passed    0.01 sec
      Start 46: _ucase1.test
46/93 Test #46: _ucase1.test .....................   Passed    0.01 sec
      Start 47: _ucase2.test
47/93 Test #47: _ucase2.test .....................   Passed    0.01 sec
      Start 48: _ucase3.test
48/93 Test #48: _ucase3.test .....................   Passed    0.01 sec
      Start 49: _test_parse_badargs.test
49/93 Test #49: _test_parse_badargs.test .........   Passed    0.05 sec
      Start 50: _test_parse_defaults_r.test
50/93 Test #50: _test_parse_defaults_r.test ......   Passed    0.03 sec
      Start 51: _test_parse_defaults.test
51/93 Test #51: _test_parse_defaults.test ........   Passed    0.02 sec
      Start 52: _test_parse_goodargs.test
52/93 Test #52: _test_parse_goodargs.test ........   Passed    0.03 sec
      Start 53: _test_parse_multiple.test
53/93 Test #53: _test_parse_multiple.test ........   Passed    0.03 sec
      Start 54: _test_feat.test
54/93 Test #54: _test_feat.test ..................   Passed    0.01 sec
      Start 55: test_feat_live
55/93 Test #55: test_feat_live ...................   Passed    0.01 sec
      Start 56: test_feat_fe
56/93 Test #56: test_feat_fe .....................   Passed    0.52 sec
      Start 57: test_subvq
57/93 Test #57: test_subvq .......................   Passed    0.01 sec
      Start 58: test_fsg_read
58/93 Test #58: test_fsg_read ....................   Passed    0.01 sec
      Start 59: test_fsg_jsgf
59/93 Test #59: test_fsg_jsgf ....................   Passed    0.01 sec
      Start 60: test_fsg_write_fsm
60/93 Test #60: test_fsg_write_fsm ...............   Passed    0.01 sec
      Start 61: test_hash_iter
61/93 Test #61: test_hash_iter ...................   Passed    0.01 sec
      Start 62: _hash_delete1.test
62/93 Test #62: _hash_delete1.test ...............   Passed    0.01 sec
      Start 63: _hash_delete2.test
63/93 Test #63: _hash_delete2.test ...............   Passed    0.01 sec
      Start 64: _hash_delete3.test
64/93 Test #64: _hash_delete3.test ...............   Passed    0.01 sec
      Start 65: _hash_delete4.test
65/93 Test #65: _hash_delete4.test ...............   Passed    0.01 sec
      Start 66: _hash_delete5.test
66/93 Test #66: _hash_delete5.test ...............   Passed    0.01 sec
      Start 67: test_lineiter
67/93 Test #67: test_lineiter ....................   Passed    0.01 sec
      Start 68: _test_determinant.test
68/93 Test #68: _test_determinant.test ...........   Passed    0.03 sec
      Start 69: _test_invert.test
69/93 Test #69: _test_invert.test ................   Passed    0.03 sec
      Start 70: _test_solve.test
70/93 Test #70: _test_solve.test .................   Passed    0.03 sec
      Start 71: test_lm_read
71/93 Test #71: test_lm_read .....................   Passed    0.17 sec
      Start 72: test_lm_score
72/93 Test #72: test_lm_score ....................   Passed    0.06 sec
      Start 73: test_lm_add
73/93 Test #73: test_lm_add ......................   Passed    0.10 sec
      Start 74: test_lm_casefold
74/93 Test #74: test_lm_casefold .................   Passed    0.04 sec
      Start 75: test_lm_class
75/93 Test #75: test_lm_class ....................   Passed    0.10 sec
      Start 76: test_lm_set
76/93 Test #76: test_lm_set ......................   Passed    0.33 sec
      Start 77: test_lm_write
77/93 Test #77: test_lm_write ....................   Passed    0.19 sec
      Start 78: _fread_line.test
78/93 Test #78: _fread_line.test .................   Passed    0.01 sec
      Start 79: _nextword.test
79/93 Test #79: _nextword.test ...................   Passed    0.01 sec
      Start 80: _str2words.test
80/93 Test #80: _str2words.test ..................   Passed    0.01 sec
      Start 81: _string_join.test
81/93 Test #81: _string_join.test ................   Passed    0.01 sec
      Start 82: _string_trim.test
82/93 Test #82: _string_trim.test ................   Passed    0.01 sec
      Start 83: test_atof
83/93 Test #83: test_atof ........................   Passed    0.01 sec
      Start 84: test_thread
84/93 Test #84: test_thread ......................   Passed    1.28 sec
      Start 85: test_event
85/93 Test #85: test_event .......................   Passed    1.51 sec
      Start 86: test_msgq
86/93 Test #86: test_msgq ........................   Passed    0.01 sec
      Start 87: test_fopen
87/93 Test #87: test_fopen .......................   Passed    0.04 sec
      Start 88: test_bitarr
88/93 Test #88: test_bitarr ......................   Passed    0.01 sec
      Start 89: test_bit_encode
89/93 Test #89: test_bit_encode ..................   Passed    0.01 sec
      Start 90: test_build_directory
90/93 Test #90: test_build_directory .............   Passed    0.01 sec
      Start 91: test_heap
91/93 Test #91: test_heap ........................   Passed    0.01 sec
      Start 92: test_filename
92/93 Test #92: test_filename ....................   Passed    0.01 sec
      Start 93: test_readfile
93/93 Test #93: test_readfile ....................   Passed    0.01 sec

97% tests passed, 3 tests failed out of 93

Total Test time (real) = 861.90 sec

The following tests FAILED:
	  2 - test-lm.sh (Failed)
	  9 - test_allphone (Failed)
	 21 - test_ngram_model_read (Failed)

Here is the config.h:
config.h.txt

@sthibaul
Copy link
Author

(it seems I cannot reopen the bug)

@dhdaines dhdaines reopened this Jun 12, 2022
@dhdaines
Copy link
Contributor

Ah okay, reopened. The mipsel failures may be harder to fix, it looks like there is a numerical issue of some kind, is floating-point not standard on that platform?

@sthibaul
Copy link
Author

is floating-point not standard on that platform?

I'm not aware of standardization issue on the platform, but the numerical details can vary depending on the arch, yes, for instance glibc has its expected libm test results accuracy depending on the arch

@dotmain
Copy link

dotmain commented Jun 13, 2022

I don't have access to an alpha machine

Hi there, @sthibaul

Such a delight to see Egypt getting some much welcome attention.
And regarding machines though, don't be thinking unlicensed, as that turns autoimmune deficient when the causality is corrupt. For binary intelligence license requests, please contact me or mainvolume, github.com/mainvolume
@dhdaines

and if there is anything I or mainvolume can help with, please don't be shy.

Thank you!

@dhdaines
Copy link
Contributor

I'm sorry, is this last comment spam? Was it passed through bad machine translation? It make no sense.

@dhdaines dhdaines mentioned this issue Sep 28, 2022
Closed
@dhdaines
Copy link
Contributor

Just an update, I'll set up a test harness with QEMU for s390x, alpha, and mipsel, hopefully it will find the problem.

dhdaines added a commit that referenced this issue Sep 28, 2022
@dhdaines
Merge pull request #303 from cmusphinx/199-pocketsphinx_continuous-is…
080aa8c 
…-unable-to-convert-speech-to-text

Fix float-to-int underflows (#199, #252)
@dhdaines
Copy link
Contributor

Won't do alpha, because it apparently hasn't been supported by debian for a long time and I don't want to have to compile CMake for it. But the same fix for mipsel should help everybody.

@sthibaul
Copy link
Author

Confirmed!
https://buildd.debian.org/status/package.php?p=pocketsphinx
It also fixed ia64 \o/
We'll see about alpha when it gets to it (there are about 50 packages to build in the queue before it)

@sthibaul
Copy link
Author

Yup, alpha fixed too :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix endianness issues in binary trie LM code cmusphinx/pocketsphinx
3 participants
@dhdaines @sthibaul @dotmain