Add support for filtering results with a unique size for each status code #118
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When attempting to brute force web servers, they may return similar responses for different URLs.
For example, a web server may redirect us to its homepage, giving us the same final result each time for multiple URLs. If we pipeline gobuster's output into other tools, there may be cases where we only care about URLs that point us to unique content.
For example, brute-forcing a web server with the following command:
./gobuster -m dir -q -k -v -l -w wordlist.txt -u https://example.com
And wordlist:
Will yield the following:
Notice that the paths ../../bar and ../foo (likely) result in the same response since they have content of the same size.
There may be cases where I would like to have the following output instead:
Since I only care about URLs that will give me a unique response.
I have implemented a patch that allows gobuster to filter results with a unique size for each status code to addresses the issue above.
If there is anything I have missed, I will be happy to address it in subsequent commits.