Skip to content

Commit

Permalink
scraper: Fix for SLR Scraper Error Fixes #1779 & #1815 (#1820)
Browse files Browse the repository at this point in the history
* Fix for SLR Scraper Error

The Head request to check to ensure the image link was good didn't handle error. I have now added a switch that if it error out because of E.G DNS Resolution error it doesn't crash XBVR. Needs more testing has the conditions to cause the error are difficult to reproduce

* Format

* Delete slrstudios.go Woops

* Format placed in the correct dir
  • Loading branch information
pops64 committed Aug 28, 2024
1 parent 4204175 commit e36e375
Showing 1 changed file with 9 additions and 5 deletions.
14 changes: 9 additions & 5 deletions pkg/scrape/slrstudios.go
Original file line number Diff line number Diff line change
Expand Up @@ -151,21 +151,25 @@ func SexLikeReal(wg *sync.WaitGroup, updateSite bool, knownScenes []string, out
if !isTransScene {
appCover := gjson.Get(JsonMetadataA, "thumbnailUrl").String()
desktopCover := strings.Replace(gjson.Get(JsonMetadataA, "thumbnailUrl").String(), "app", "desktop", -1)
desktopCresp, _ := http.Head(desktopCover)
if desktopCresp.StatusCode == 200 {
desktopCresp, err := http.Head(desktopCover)
if err != nil {
log.Errorf("Method Head Failed on desktopCover %s with error %s", desktopCover, err)
} else if desktopCresp.StatusCode == 200 {
coverURL := desktopCover
sc.Covers = append(sc.Covers, coverURL)
} else {
appCresp, _ := http.Head(appCover)
if appCresp.StatusCode == 200 {
appCresp, err := http.Head(appCover)
if err != nil {
log.Errorf("Method Head Failed on appCover %s with error %s", appCover, err)
} else if appCresp.StatusCode == 200 {
coverURL := appCover
sc.Covers = append(sc.Covers, coverURL)
defer appCresp.Body.Close()
} else {
e.ForEach(`link[as="image"]`, func(id int, e *colly.HTMLElement) {
sc.Covers = append(sc.Covers, e.Request.AbsoluteURL(e.Attr("href")))
})
}
defer appCresp.Body.Close()
}
defer desktopCresp.Body.Close()
} else {
Expand Down

0 comments on commit e36e375

Please sign in to comment.