Use both query1 and query2.

[gadicc]

As noted by @PythonCreator27 in #8 (comment), both hostnames above may be used interchangeably.

I took a quick look at https://finance.yahoo.com/, and in the source we have:

root.App.main = {
  "context": {
    // ...
    "plugins": {
      // ...
      "ServicePlugin": {
        "configs": {
          "tachyon": {
            "defaults": {
              "withCredentials": true,
              "timeout": 4000,
              "protocol": "http",
              "host": "iquery.finance.yahoo.com:4080",
              "proxyFallback": "\u002F__query",
              "client": {
                "protocol": "https",
                "host": ["query1.finance.yahoo.com", "query2.finance.yahoo.com"]
              }
            }
          }
        }
      }
      // ...
    }
    // ...
  }
}

My current thinking:

• Super short term: not urgent at all, everything works as is (and has for like 10 years I guess)
• Shortish term: could assume both hostnames above and randomly pick one at query time
• Longer term: if request for one host name fails, could try the other
• Longest term: if both fail, check the main html above if there are any updates (I definitely wouldn't do this on every request, and even "on first request" is not ideal on serverless setups, but could be behind an option - still, don't think it's worth the wasted time on the extra request until we have good reason).

[advaiyalad]

@gadicc This is going to require a good bit of breaking changes and refactoring...

1. Options will have to be added to _moduleExec so that it can tell whether the module uses the query# endpoints or not.
2. Hosts will have to be changed dynamically, calling for a _moduleExec wrapper, made for modules using the query# endpoints.
3. We are going to have to check whether errors are coming from validation or from the API and handle them accordingly.
4. A new major version will have to be released, as we are changing the behavior of the library (trying again instead of failing).

Through all of it, we are going to have to make sure that we don't mess up modules like autoc that don't use the query# endpoints.

My suggestion:

• Short term: Unify what we use in our code at first (probably query1), so that we have some sort of unity in our code.
• Long term:
  • Add logic for checking if the first failed and trying again with query2 and iquery.
  • Also send an error report to some server.
    Example format:

    {
        "host": "query1",
        "module": "quote",
        "symbol": "AMZN",
        "queryOptions": {
            // Query options here
        }
        "result": {
            // Will have real result/response here
        },
        "errorStatus": "401"
    }

  • If everything fails, throw an error and file an issue through the GitHub API.

We probably don't want to be too dynamic and start checking the HTML. Then we would check the HTML on every request, and that could pile up fast and slow down programs. Even if we did request and cache the new host(s), we would have to cache it somewhere. In node, we would probably use the filesystem, but in the browser, we would use localStorage. Changing between them, preventing ReferenceErrors, etc. would not be worth the cost. Ideally, people should just file issues and the problem should fixed like normal.

In the end, it is quite unlikely that yahoo will change the domain endpoints. So overall, I think the approach stated above should work well. Again, @gadicc, as you said, things have been working the way that they are for 10 years, so I would take this as a low priority issue.

[gadicc]

Agreed. Low priority, let people open an issue. As an easy first step we could just randomly pick one of the hosts, I can think of a fairly straightforward way to do it and will give it a shot when I have a chance. Watch this space :) And thanks for the feedback.

[gadicc]

🎉 This issue has been resolved in version 2.3.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀