[Issue] Github stats errors

[Bundy01]

Describe the bug
During a scan of a file containing 122 domains, I noticed that the github source gave me a negative result three times.

I should point out that I compiled SubFinder from the fix_github_perf_issue branch. #1421

I was able to note an affected domain > facebook.com.br. The error is not displayed every time.

Despite five attempts, I was unable to reproduce this error with relase v2.6.7.

Subfinder version
Current Version: v2.6.7 (fix_github_perf_issue branch)

Complete command you used to reproduce this
subfinder -s github -d facebook.com.br -stats

Screenshots

[INF] Enumerating subdomains for facebook.com.br
www.facebook.com.br
graph.facebook.com.br
pt.facebook.com.br
[INF] Found 3 subdomains for facebook.com.br in 1 minute 12 seconds
[INF] Printing source statistics for facebook.com.br

 Source               Duration      Results     Errors
────────────────────────────────────────────────────────
 github               1m12.712s          -3          0

[INF] Enumerating subdomains for facebook.com.br
www.facebook.com.br
graph.facebook.com.br
pt.facebook.com.br
[INF] Found 3 subdomains for facebook.com.br in 1 minute 10 seconds
[INF] Printing source statistics for facebook.com.br

 Source               Duration      Results     Errors
────────────────────────────────────────────────────────
 github               1m10.297s          -4          0

or

[INF] Enumerating subdomains for facebook.com.br
www.facebook.com.br
graph.facebook.com.br
pt.facebook.com.br
[INF] Found 3 subdomains for facebook.com.br in 1 minute 17 seconds
[INF] Printing source statistics for facebook.com.br

 Source               Duration      Results     Errors
────────────────────────────────────────────────────────
 github               1m17.418s           2          0

Edit: I also noticed the problem of negative results with the sitedossier source when adding the -all option.

[dogancanbakir]

I couldn't repro the issue,

#1 attempt

$ o run . -s github -d facebook.com.br -stats

               __    _____           __         
   _______  __/ /_  / __(_)___  ____/ /__  _____
  / ___/ / / / __ \/ /_/ / __ \/ __  / _ \/ ___/
 (__  ) /_/ / /_/ / __/ / / / / /_/ /  __/ /    
/____/\__,_/_.___/_/ /_/_/ /_/\__,_/\___/_/

                projectdiscovery.io

[INF] Current subfinder version v2.6.7 (latest)
[INF] Loading provider config from /Users/dogancanbakir/Library/Application Support/subfinder/provider-config.yaml
[INF] Enumerating subdomains for facebook.com.br
pt.facebook.com.br
www.facebook.com.br
graph.facebook.com.br
[INF] Found 3 subdomains for facebook.com.br in 1 minute 19 seconds
[INF] Printing source statistics for facebook.com.br

 Source               Duration      Results     Errors
────────────────────────────────────────────────────────
 github               1m19.61s            3          0

...

#10 attempt

$ ```console
$ o run . -s github -d facebook.com.br -stats

               __    _____           __         
   _______  __/ /_  / __(_)___  ____/ /__  _____
  / ___/ / / / __ \/ /_/ / __ \/ __  / _ \/ ___/
 (__  ) /_/ / /_/ / __/ / / / / /_/ /  __/ /    
/____/\__,_/_.___/_/ /_/_/ /_/\__,_/\___/_/

                projectdiscovery.io

[INF] Current subfinder version v2.6.7 (latest)
[INF] Loading provider config from /Users/dogancanbakir/Library/Application Support/subfinder/provider-config.yaml
[INF] Enumerating subdomains for facebook.com.br
pt.facebook.com.br
www.facebook.com.br
graph.facebook.com.br
[INF] Found 3 subdomains for facebook.com.br in 1 minute 10 seconds
[INF] Printing source statistics for facebook.com.br

 Source               Duration      Results     Errors
────────────────────────────────────────────────────────
 github               1m19.61s            3          0

But, fixed sitedossier negative stats #1432

[Bundy01]

Hi,

for github:
I compiled the branch mentioned by moving and renaming the file created so as not to conflict with /usr/bin/subfinder.

cd subfinder/v2/cmd/subfinder; \

go build; \

mv subfinder ~/.local/bin/subtest

~/.local/bin is of course in my PATH.

In ~/.config, a config.yaml file for subfinder and another for subtest. I share the same provider-config.yaml file for both.

Did you make several successive attempts?

for sitedossier:
Thanks, even if I can't use it because I'm on a VPN (blocked IPs). It should be useful for other users :)

[dogancanbakir]

Did you make several successive attempts?

Yes, I didn't see any negative results.

[Bundy01]

I don't know what additional infos to add.
I just restarted the command and I still have inconsistencies in the stats :/

I've run the command three times, the first was correct and here are the other two:

[INF] Enumerating subdomains for facebook.com.br
www.facebook.com.br
graph.facebook.com.br
pt.facebook.com.br
[INF] Found 3 subdomains for facebook.com.br in 1 minute 13 seconds
[INF] Printing source statistics for facebook.com.br

 Source               Duration      Results     Errors
────────────────────────────────────────────────────────
 github               1m13.857s           0          0

[INF] Enumerating subdomains for facebook.com.br
pt.facebook.com.br
www.facebook.com.br
graph.facebook.com.br
[INF] Found 3 subdomains for facebook.com.br in 1 minute 13 seconds
[INF] Printing source statistics for facebook.com.br

 Source               Duration      Results     Errors
────────────────────────────────────────────────────────
 github               1m13.251s           2          0

[dogancanbakir]

Do you get these inconsistent results in the latest version?

[Bundy01]

Despite five attempts, I was unable to reproduce this error with relase v2.6.7.

No, I couldn't reproduce this problem with the release.