Failures not detected when web server is shutdown

[bfinlay]

When nginx is shutdown on the server, locust reports all requests as successful when in fact they are all 503 errors - connection not available.

The documentation mentions that:

Safe mode:
The HTTP client is configured to run in safe_mode. What this does is that any request that fails due to a connection error, timeout, or similar will not raise an exception, but rather return an empty dummy Response object. The request will be reported as a failure in Locust’s statistics. The returned dummy Response’s content attribute will be set to None, and it’s status_code will be 0.

This seems like it is related to how "connection errors" are handled. What it seems to be saying is that the exception is caught and a dummy response is returned that is supposed to be counted as a failure. The problem is that it is not being counted as a failure.

Thinking forward, this concerns me that in high load conditions timeout/connection failures due to an unresponsive server due to high load will not be reported as failures and will seem like the server is performing acceptably when it is not.

my environment is Python 2.7.6 on Mac and locust-0.7.2

[bfinlay]

The cause of this issue turned out to be that unknown to me I was behind a transparent caching http proxy which actually provided cached content to some tools (like browsers and locust) but not others (like curl). So it would seem that locust was receiving 200 responses and the actual content from the proxy cache, and so was properly reporting successes.

To verify, I moved locust to a cloud server and retested, and it did report failures for 503 errors when I shut down the nginx server.