-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Calculate max allowance #45
Open
navFooh
wants to merge
32
commits into
primary
Choose a base branch
from
verifier-score
base: primary
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 20 commits
Commits
Show all changes
32 commits
Select commit
Hold shift + click to select a range
8381a1e
Update .envrc file name in README
navFooh caba68c
Add FilplusApiKey environment variable
navFooh 27d58c6
Create getVerifiedDealCount.go
navFooh fa1eb18
Create getLinkHeaderURI.go
navFooh 775f831
Create getGitHubEventDates.go
navFooh 7263e11
Update getGitHubEventDates.go
navFooh 59c1562
Update getGitHubEventDates.go
navFooh bf6ca68
Create getVerifierScore.go
navFooh 6b99d16
Apply formatter fixes
navFooh ef0f476
Make JSON bindings required
navFooh f610e8f
Rename MaxAllowanceBytes to BaseAllowanceBytes
navFooh ba501e1
Update getVerifierScore.go
navFooh e6c0b2e
getFilecoinDealsMultiplier is separate func
navFooh 4539f87
Return zero on score calculation fail
navFooh 42a335a
Rename getVerifierScore to getMaxAllowance
navFooh 0398a3a
Add comments to getMaxAllowance
navFooh 6585eda
Add GetMaxAllowance to User
navFooh dbb40b9
Add /max-allowance server endpoint
navFooh cd74a18
Calculate maximum allowance for verifier
navFooh 8882959
Add /max-allowance-github testing route
navFooh a4ae44c
Rename githubMaxDateCount to githubMaxEventCount
navFooh d7fa451
Add getAbsoluteMaxAllowance() method and use for fiftyDataCaps
navFooh 01c686d
Move GetMaxAllowance down in serveVerifyAccount
navFooh 00ef3d7
Return max allowance in verify endpoint
navFooh 01dbf06
Rename allowance variables
navFooh 79cda29
Rename some max allowance endpoints
navFooh 4e898f9
Rename getMaxAllowance to getAllowance
navFooh e479b45
Rename user.GetMaxAllowance to user.GetAllowance
navFooh f5c63db
Implement Go logger (#46)
navFooh c0612b2
Merge branch 'primary' into verifier-score
navFooh 39387e5
restore indent
navFooh cc231aa
Merge branch 'primary' into verifier-score
navFooh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
package main | ||
|
||
import ( | ||
"encoding/json" | ||
"fmt" | ||
"io" | ||
"net/http" | ||
"time" | ||
) | ||
|
||
type GithubEvent struct { | ||
CreatedAt string `json:"created_at" binding:"required"` | ||
} | ||
|
||
/* | ||
* Get all "created_at" dates for GitHub events belonging to the provided GitHub account | ||
*/ | ||
func getGitHubEventDates(account string) ([]time.Time, error) { | ||
dates := []time.Time{} | ||
url := fmt.Sprintf("https://api.github.com/users/%v/events?per_page=100", account) | ||
for url != "" { | ||
pageDates, next, err := getGitHubEventPageDates(url) | ||
if err != nil { | ||
return nil, err | ||
} | ||
dates = append(dates, pageDates...) | ||
url = next | ||
} | ||
return dates, nil | ||
} | ||
|
||
/* | ||
* Get all "created_at" dates from the provided GitHub event page URL. | ||
* Also returns the URL for the next GitHub event page, if it exists. | ||
*/ | ||
func getGitHubEventPageDates(url string) ([]time.Time, string, error) { | ||
// Create HTTP client and request | ||
client := &http.Client{} | ||
req, err := http.NewRequest("GET", url, nil) | ||
if err != nil { | ||
return nil, "", err | ||
} | ||
|
||
// Add headers and perform request | ||
req.Header.Add("accept", "application/vnd.github.v3+json") | ||
resp, err := client.Do(req) | ||
if err != nil { | ||
return nil, "", err | ||
} | ||
|
||
// Read the response body | ||
defer resp.Body.Close() | ||
body, err := io.ReadAll(resp.Body) | ||
if err != nil { | ||
return nil, "", err | ||
} | ||
|
||
// Get events from response body | ||
var githubEvents []GithubEvent | ||
err = json.Unmarshal(body, &githubEvents) | ||
if err != nil { | ||
return nil, "", err | ||
} | ||
|
||
// Extract "created_at" dates | ||
dates := []time.Time{} | ||
for _, event := range githubEvents { | ||
date, err := time.Parse("2006-01-02T15:04:05Z", event.CreatedAt) | ||
if err != nil { | ||
return nil, "", err | ||
} | ||
dates = append(dates, date) | ||
} | ||
|
||
// Retrieve the next page URL | ||
link := resp.Header.Get("link") | ||
next := getLinkHeaderURI(link, "next") | ||
return dates, next, nil | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
package main | ||
|
||
import ( | ||
"fmt" | ||
"regexp" | ||
) | ||
|
||
/* | ||
* Returns the URI in a link header with the specified "rel" attribute | ||
* or an empty string when the "rel" attribute is not found. For example: | ||
* link = `<https://example.com>; rel="test"` | ||
* rel = `test` | ||
* returns `https://example.com` | ||
*/ | ||
func getLinkHeaderURI(link string, rel string) string { | ||
reStr := fmt.Sprintf(`<([^,]*)>; rel="%v"`, rel) | ||
re := regexp.MustCompile(reStr) | ||
match := re.FindStringSubmatch(link) | ||
if len(match) == 2 { | ||
return match[1] | ||
} | ||
return "" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,127 @@ | ||
package main | ||
|
||
import ( | ||
"time" | ||
|
||
"github.com/filecoin-project/go-state-types/big" | ||
) | ||
|
||
func getMaxAllowanceForGithub(githubAccount string, filecoinAddress string) (big.Int, error) { | ||
// Check GitHub account activity | ||
activityCheck, err := checkGithubAccountActivity(githubAccount, 3) | ||
if err != nil { | ||
return big.Zero(), err | ||
} | ||
|
||
// Get Filecoin deals multiplier | ||
dealsMultiplier, err := getFilecoinDealsMultiplier(filecoinAddress) | ||
if err != nil { | ||
return big.Zero(), err | ||
} | ||
|
||
// Calculate allowance | ||
score := env.BaseAllowanceBytes | ||
if activityCheck { | ||
score = big.Mul(score, big.NewInt(2)) | ||
} | ||
if dealsMultiplier != 1 { | ||
score = big.Mul(score, big.NewInt(int64(dealsMultiplier))) | ||
} | ||
return score, nil | ||
} | ||
|
||
/* | ||
* Returns true when there was public activity in the GitHub account for the past X months. | ||
* Also returns true when there was too much recent activity and the returned events from | ||
* GitHub do not reach back far enought to give a proper evaluation. | ||
*/ | ||
func checkGithubAccountActivity(githubAccount string, months int) (bool, error) { | ||
// Get event dates from the GitHub account | ||
dates, err := getGitHubEventDates(githubAccount) | ||
if err != nil { | ||
return false, err | ||
} | ||
|
||
// Evaluate account activity | ||
dateCount := len(dates) | ||
githubMaxDateCount := 300 | ||
activityCheck, enoughDates := hasDateInEachMonthBefore(months, dates) | ||
historyInsufficient := dateCount == githubMaxDateCount && !enoughDates | ||
|
||
// Github limits the maximum event history. When we have | ||
// the maximum amount of events but not enough data to go | ||
// back far enough, we give the user the benefit of the doubt. | ||
return activityCheck || historyInsufficient, nil | ||
} | ||
|
||
/* | ||
* Returns a multiplier for the max allowance based on the | ||
* verified deal count for the supplied filecoin address | ||
*/ | ||
func getFilecoinDealsMultiplier(filecoinAddress string) (int, error) { | ||
// Get amount or verified deals for Filecoin address | ||
dealCount, err := getVerifiedDealCount(filecoinAddress) | ||
if err != nil { | ||
return 0, err | ||
} | ||
|
||
// Return multiplier | ||
if dealCount > 100 { | ||
return 8, nil | ||
} | ||
if dealCount > 10 { | ||
return 4, nil | ||
} | ||
if dealCount > 0 { | ||
return 2, nil | ||
} | ||
return 1, nil | ||
} | ||
|
||
/* | ||
* Checks for an X amount of months before today, whether the supplied dates | ||
* contain a date between the start- and endtime of each of the months. For example, | ||
* when today is 2022-03-23, a date need to be present in all the following months: | ||
* 2022-02-23 to 2022-03-23, | ||
* 2022-01-23 to 2022-02-23 and | ||
* 2021-12-23 to 2022-01-23 | ||
* The second return value indicates whether the history of the dates went | ||
* back far enough to check for the presence of a date in each of the months. | ||
*/ | ||
func hasDateInEachMonthBefore(months int, dates []time.Time) (bool, bool) { | ||
monthEndTime := time.Now() | ||
for i := 0; i < months; i++ { | ||
// Check whether a date exists in the month before monthEndTime | ||
hasDate, enoughDates := hasDateInMonth(monthEndTime, dates) | ||
// Return instantly when a date is not found and | ||
// indicate whether we had enough dates to evaluate | ||
if !hasDate { | ||
return false, enoughDates | ||
} | ||
// Go back one month before checking again | ||
monthEndTime = monthEndTime.AddDate(0, -1, 0) | ||
} | ||
return true, true | ||
} | ||
|
||
/* | ||
* Checks whether a date in the supplied dates falls in the month ending at the supplied | ||
* endTime. The second return value indicates whether there were enough dates. When none | ||
* of the dates fall in or before the given month, more history might be required. | ||
*/ | ||
func hasDateInMonth(endTime time.Time, dates []time.Time) (bool, bool) { | ||
startTime := endTime.AddDate(0, -1, 0) | ||
enoughDates := false | ||
for _, date := range dates { | ||
// Return instantly when date falls in the month | ||
if date.After(startTime) && date.Before(endTime) { | ||
return true, true | ||
} | ||
// If a date is before the start time | ||
// the dataset should have been big enough | ||
if date.Before(startTime) { | ||
enoughDates = true | ||
} | ||
} | ||
return false, enoughDates | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
package main | ||
|
||
import ( | ||
"encoding/json" | ||
"fmt" | ||
"io" | ||
"net/http" | ||
"strconv" | ||
) | ||
|
||
type VerifiedDeals struct { | ||
Count string `json:"count" binding:"required"` | ||
} | ||
|
||
/* | ||
* Returns the amount of verified deals for a Filecoin address | ||
*/ | ||
func getVerifiedDealCount(address string) (int, error) { | ||
// Create HTTP client and request | ||
client := &http.Client{} | ||
url := fmt.Sprintf("https://api.filplus.d.interplanetary.one/public/api/getVerifiedDeals/%v?limit=1&page=1", address) | ||
req, err := http.NewRequest("GET", url, nil) | ||
if err != nil { | ||
return 0, err | ||
} | ||
|
||
// Add headers and perform request | ||
req.Header.Add("x-api-key", env.FilplusApiKey) | ||
resp, err := client.Do(req) | ||
if err != nil { | ||
return 0, err | ||
} | ||
|
||
// Read the response body | ||
defer resp.Body.Close() | ||
body, err := io.ReadAll(resp.Body) | ||
if err != nil { | ||
return 0, err | ||
} | ||
|
||
// Get count from response body | ||
var verifiedDeals VerifiedDeals | ||
err = json.Unmarshal(body, &verifiedDeals) | ||
if err != nil { | ||
return 0, err | ||
} | ||
|
||
// Convert count to integer | ||
count, err := strconv.Atoi(verifiedDeals.Count) | ||
if err != nil { | ||
return 0, err | ||
} | ||
|
||
return count, nil | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so this basically just formats the URL so we can paginate through results properly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whenever you make a paginated request to the API, it returns a link header that looks as follows:
Baiscally this is one big string, so getting the URL that belongs to
rel="next"
requires something like a parser or regex, which I wrote for this. It's very convenient to use this header, since you'll know you've reached the last page whenrel="next"
is not present anymore (getLinkHeaderURI()
returns an empty string)