unexpectedly reached end of fileSIGILL: illegal instruction #288
Comments
|
Hi, @CaioLuppo Can you try other models from https://github.com/go-skynet/model-gallery/blob/3d2c9e259f6b51aa618b42a9f891521bbd253f79/gpt4all-j.yaml#LL22C20-L22C52. And please share more detail about your environment like:
Which way do you run |
|
Hi, Aisuko! All I've done so far is take the README.md file and try to run the "Use chatgpt4all" example. I'm using LocalAI 1.13.0, the latest in this repository via docker, as far as I know. |
|
I've just tried the gpt model you suggested, but nearly the same error happened:
|
|
Just tried with "Bert embeddings" too, same error. It crashes ass soon as I send the post request. |
|
I wonder what is your input, can you follow this example? It should work: https://github.com/go-skynet/LocalAI/blob/master/README.md#example-use-gpt4all-j-model |
|
I have the same problem. I googled a bit and there are a few things that point to the cpu not having the avx2 instruction (flashlight/wav2letter#604). I don't know enough about cpus or the interals of the code to be sure, but it makes sense to me. Probably a cflag in the build would fix it, or to buy a new computer (pre 2013 cpus seem to have the problem). Obviously I would like it much more to have the software-solution, but... well... |
In fact, I strictly followed this example as I mentioned at the first reply. Every time I use curl to post, it crashes. |
|
I also tried with postman, but the same error happens. |
|
I just tested with the new version of LocalAI, but the error persists. |
|
same here, but i investigated a bit more and i am quite sure right now that its the instructionset that is the problem. compiling it all for ourselfs (no docker, i mean everything that is used for localAI) should fix it, but i didnt do that as of now - i have had better/more important things to do. my guess is that its a binary that is loaded from somewhere that requires this instruction. In my case its probably the vzeroupper instruction, as "seen" in my error, which is slightly different from that of @CaioLuppo (i get " instruction bytes: 0xc5 0xf1 0xef 0xc9 0xc5 0xfd 0x6f 0x5 0x18 0xab 0x24 0x0 0x49 0xc7 0x86 0x80"; the first one should be: 0xc5 =>11000101 => vzeroupper => AVX instruction set). |
|
Yeah, I think AVX is really the problem here. I tried the "AVX only" version of the Gpt4All software and it worked, but I don't think there is such an option in LocalAI... @Aisuko, is there any fix we can do to make it work with sorta like Gpt4All AVX only? |
|
confirming that issue occurs when there is no avx2 support. i have tried to run vicuna based model on different server configurations:
I think there is a problem while go-llama linkings building process. |
Can you try building LocalAI from source with: That should build LocalAI without avx2 on llama.cpp |
That didn't work. Still getting the same error. |
|
possible, might be other CPU instruction flag missing. You can try disabling more instructions set like this (although, will be quite slow): |
|
Hey, I have something to add to this. I did run into this issue with an AVX only CPU. I wasn't able to get docker to work but I could build the LocalAI binary using Is this an issue with the go docker container? |
|
I have the same issue. My CPU is based on Sandy Bridge microarchitecture which doesn't support F16C extension. Based on the error I'm getting the instruction set is 'c4 e2 79 13 c9 vcvtph2ps xmm1,xmm1'. CMAKE_ARGS=-DLLAMA_F16C=OFF has no effect, because there are plenty of dependencies which always build with "-mf16c" flag regardless whether CPU supports that extension or not. I'm currently trying to figure out how to remove those flags from makefiles and rebuild. |
|
This is fixed for me in the latest release as long as I do docker compose with build vs pull. I didn't have to put any flags and those didn't seem to help before anyway. |
|
I think the issue is the change to llama.cpp, which introduced support for AVX only, with new cmake flag added to control this feature. I'm running local-ai on X5550, and it was failing until I used the following options: Note -DLLAMA_AVX=OFF, this option is not mentioned anywhere on local-ai doco, or github. I also used -DLLAMA_F16C=OFF for good measure, as my CPU doesn't support it, and it looks like it always on by default. This two changes fixed local-ai and now I can run it with no problem. |
|
Namely issue is this PR - ggerganov/llama.cpp#617, which added AVX support, along with new compile options to control it, with a default ON. Those new options are not part of REBUILD=true machinery, as such you still get llama compiled with AVX support, not AVX2, or AVX512, but plain AVX. |
Greetings! Please check the very beginning of the variable value ^^ |
|
After comparing results on two different computers, one of which has avx2 whereas the other doesn't, I also came to the conclusion that it had to have something to do with these CPU flags (I wish I had found this thread earlier but I was searching for the output I got on the command line "error reading from server: EOF" rather than the one I later found in the debug info which could have led me here sooner...). So I tried to rebuild local-ai in the docker but it complained about not finding something (some libraries?). So instead of investing more time in this container, knowing all would be gone with its next recreation, I decided to build local-ai directly in my VM, without docker. With avx2 support switched off, the build eventually worked and the error is gone now. |
|
Just in case, GitHub supports spoilers for huge pages of wonderful content. |
Could you help to please show me which file i could put "`CMAKE_ARGS=...' into to fix the problem? THank you. |
|
It is documented here: https://localai.io/basics/build/#build-locally |
Thank you. I tried but still getting error |
|
Trying this on yet another machine. This one has avx, avx2, f16c and fma. But I am still getting this error when using this docker image quay.io/go-skynet/local-ai:v2.2.0-cublas-cuda12-ffmpeg Do I also need AVX512 to run the image? (It would be great, if someone could point me to where the requirements for the docker images are documented.) Thanks! |
@stereotypy Could you share your compose yaml? It would be greatly appreciated. |
turns out we were not building the fallbacks when building the container iamges. This of course would lead to SIGILL issues again in case the host does not support the CPU instruction sets. See: #288 Signed-off-by: Ettore Di Giacinto <[email protected]>
turns out we were not building the fallbacks when building the container iamges. This of course would lead to SIGILL issues again in case the host does not support the CPU instruction sets. See: #288 Signed-off-by: Ettore Di Giacinto <[email protected]>
this should be fixed - can you share the full logs? |
I'm trying to use many models, but none of them are working when I use the /chat/completions page. Here's what I'm getting with gpt4all:
I'm using Postman with this query:
{ "model": "gpt", "messages": [ { "role": "user", "content": "How are you?" } ], "temperature": 0.9 }The text was updated successfully, but these errors were encountered: