Canceled bbrf process keeps running on CouchDB

[pdelteil]

I have tested this situation many times:

1. Running a heavy process (deleting over 100k urls or domains)
2. Press Control + C
3. Processes on the CouchDB server keep running. Sometimes overloading the server.

I often need to restart the DB in order to get rid of the left over processes.
This might be the cause for the 'unknown error' described in another issue.

[pdelteil]

I'm having better results removing data using chunks:

This oneliner will remove the urls in steps of 5000 urls
chunk=5000; size=$(bbrf urls|wc -l); urls="1,${chunk}p" parts=$((size%chunk?size/chunk+1:size/chunk)) ; echo $parts ; for i in $(seq 1 $parts) ; do echo "try $i"; bbrf urls|sed -n "$urls"|bbrf url remove - ; done

[honoki]

Thanks, Philippe. This sounds like it should help me improve performance for large input data.

[pdelteil]

Your welcome @honoki.

After more testing I can confirm that using chunks works faster and it's more reliable than the regular way.