Can distserve change block_size ?

[WANGHanshuo1220]

The default block size is 16 and i want to change it to another value like 8, 4, 1... and it crashes.
Does anyone have the same issue? Or any code modification suggestions are welcome and appreciated.

The crash info is like this. It seems that ray actor is died but i have no idea what's happenning

After some debugging, i found that it crashes when decoding, not prefilling (context).
This just makes me more confused, because prefill code and decode code is nearly the same. And crash or not highly depends on the output length (if output_length is smaller, everything is fine; but if it's larger, ray actor crashes)