Infinite network requests/refresh loop while accessing docs #5055

idontknowjs · 2021-06-24T17:04:13Z

🐛 Bug Report

Infinite refresh & network request loops are made while accessing any other page except the homepage.

This issue does not occurs when a doc page is navigated from homepage.
This issue only occurs when the website is deployed. Not when the build is locally served and tested.

Prerequisites

I'm using the latest version of Docusaurus.
I have tried the npm run clear or yarn clear command.
I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
I have tried creating a repro with https://new.docusaurus.io
I have read the console error message carefully (if applicable)

Description

Have you read the Contributing Guidelines on issues?

Yes

Steps to reproduce

Visit website deployed using https://60d4d94678faef62fcfa902f--zowe-docs-testing.netlify.app/stable/user-guide/install-zos (Page falls in an infinite refresh loop)

OR

Visit https://60d4d94678faef62fcfa902f--zowe-docs-testing.netlify.app/
Navigate to any other section from navbar and refresh the page.

Steps to build the website locally

Clone the specific branch: git clone --branch zowe-docs-v2 https://github.com/zowe/docs-site.git
Install dependencies: npm install
Build locally: npm run build and deploy at a URL

Expected behavior

The doc page is expected to load up normally.

Actual behavior

The doc page does not loads up.

Your environment

Public source code: https://github.com/zowe/docs-site/tree/zowe-docs-v2
Public site URL: https://60d4d94678faef62fcfa902f--zowe-docs-testing.netlify.app/
Docusaurus version used: 2.0.0-beta.1 (Also exists on 2.0.0-beta.2)
Environment name and version (Chrome 78.0.3904.108, Node.js v16.3.0)
Operating system and version: MacOS bigSur v11.4

The text was updated successfully, but these errors were encountered:

slorber · 2021-06-24T17:17:22Z

If you look at the served page, it is serving a redirect file:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <meta http-equiv="refresh" content="0; url=/stable/getting-started/overview">
    <link rel="canonical" href="/stable/getting-started/overview" />
  </head>
  <script>
    window.location.href = '/stable/getting-started/overview';
  </script>
</html>

I suspect your usage of redirect plugin creates an edge case where the original file gets overriden by the redirect file:

    [
      "@docusaurus/plugin-client-redirects",
      {
        redirects: [
          {
            to: "/",
            from: ["/stable"],
          },
        ],
        fromExtensions: ["html"],
        //For eg. redirects "v1-22-x" to "v1.22.x";
        createRedirects: function (existingPath) {
          for (var i = 0; i < versionsArray.length; i++) {
            var x = versionsArray[i];
            if (existingPath.includes(x)) {
              return [
                existingPath.replace(x, x.replace(".", "-").replace(".", "-")),
              ];
            }
          }
        },
      },
    ],

Normally there's a security that prevents this from happening (build is supposed to fail)

Maybe you can investigate why the redirect plugin is overriding the file at website/build/stable/getting-started/overview/index.html, and try adjusting the config?

If there's additional security to add to the redirect plugin to prevent this, a smaller repro would be helpful.

idontknowjs · 2021-06-24T19:44:34Z

Hi, FYI while building locally. This error is shown in console. This error exists even after npm run clear & rm -rf node_modules yarn.lock package-lock.json. Not sure if this error is related to this issue:

Error: read ENOTCONN
    at tryReadStart (node:net:566:20)
    at Socket._read (node:net:577:5)
    at Socket.Readable.read (node:internal/streams/readable:483:10)
    at Socket.read (node:net:618:39)
    at new Socket (node:net:375:12)
    at Object.Socket (node:net:273:41)
    at createSocket (node:internal/child_process:327:14)
    at ChildProcess.spawn (node:internal/child_process:447:23)
    at Object.spawn (node:child_process:609:9)
    at Object.execa [as default] (/Users/Desktop/docs-site/node_modules/execa/index.js:83:26)
    at Object.getFileLastUpdate (/Users/Desktop/docs-site/node_modules/@docusaurus/plugin-content-docs/lib/lastUpdate.js:41:49)
    at readLastUpdateData (/Users/Desktop/docs-site/node_modules/@docusaurus/plugin-content-docs/lib/docs.js:27:34)
    at readDocFile (/Users/Desktop/docs-site/node_modules/@docusaurus/plugin-content-docs/lib/docs.js:47:9)
    at async Promise.all (index 86)
    at async loadVersionDocsBase (/Users/Desktop/docs-site/node_modules/@docusaurus/plugin-content-docs/lib/index.js:87:34)
    at async doLoadVersion (/Users/Desktop/docs-site/node_modules/@docusaurus/plugin-content-docs/lib/index.js:103:34) {
  errno: -57,
  code: 'ENOTCONN',
  syscall: 'read',
  originalMessage: 'read ENOTCONN',
  shortMessage: 'Command failed with ENOTCONN: git log -1 --format=%ct, %an zosmf-cli.md\n' +
    'read ENOTCONN',
  command: 'git log -1 --format=%ct, %an zosmf-cli.md',
  escapedCommand: 'git log -1 "--format=%ct, %an" zosmf-cli.md',
  exitCode: undefined,
  signal: undefined,
  signalDescription: undefined,
  stdout: '',
  stderr: '',

The redirect plugin function above just does three things:

Redirects https://zowe-docs-testing.netlfiy.app/stable to https://zowe-docs-testing.netlfiy.app/
Redirects https://zowe-docs-testing.netlfiy.app/xyz.html to https://zowe-docs-testing.netlfiy.app/xyz
Changes version links like v1-22-x to v1.22.x

The same redirect configuration was used from past week and was working as expected. Also the function works fine when local build is served at port 3000.

It successfully:

Redirects http://localhost:3000/stable to http://localhost:3000/
Redirects http://localhost:3000/xyz.html to http://localhost:3000/xyz
Changes version links like v1-22-x to v1.22.x

idontknowjs · 2021-06-24T20:34:52Z

Update: Checked the website with same redirect plugin functions using v2.0.0-beta.0. The issue does not exist there.
The issue only occurs while switching to v2.0.0-beta.1 & v2.0.0-beta.2.

Site Deployed link using v2.0.0-beta.0: https://60d4e68d5c0ad27129391330--zowe-docs-testing.netlify.app/

All the above redirect functions work properly.

slorber · 2021-06-25T17:55:25Z

@CovalentBond debugging this is likely extremely time-consuming for me. Just cloning your repo takes forever 😅

Please try to debug this and provide all the relevant infos.
I see a lot of links above and it's a bit complicated to understand.

If you can show me 2 PR with deploy previews, one on beta.0, one on beta.2, I can inspect the code that belongs to a given deployment, being sure no mistake was done.

Also please try to remove the redirects until you are able to get a /stable/getting-started/overview/index.html that contains the real content using beta.2. We may find out that it's not even a problem related to the redirect plugin.

This might also be related to the new trailingSlash config (which affects the redirect plugin)

If you can create me 2 zip of a representative folder in build with the beta.0 / beta.2, we can compare the site output and see how it is different from before

idontknowjs · 2021-06-25T23:19:49Z

Deploy preview

[1] Using beta.0 with redirect plugin: https://60d62fb82094240af9dbbbd2--zowe-docs-testing.netlify.app/
[2] Using beta.2 with redirect plugin: https://60d63bc58803cf1395630167--zowe-docs-testing.netlify.app/
[3] Using beta.2 without redirect plugin: https://60d6461875267924fe17f74e--zowe-docs-testing.netlify.app/

The first beta.0 deploy [1] has been based on latest commit on the repo. The is using beta.0 and this deploy works fine.

For the other two beta.2 [2] & [3] deploy links. I can confirm that no other code change were there, except updating the version in package.json and removing the redirect functions for generating the [3] link.

Builds

All the builds for each preview can be find on Google Drive. Each build size ~ 400MB seem to exceed GitHub's file size limit.

Another Issue❗️

The deploy made using v2.0.0-beta.2 shows archieved banner saying "This is unreleased documentation for Zowe Docs v1.22.x LTS version." on website's latest version v1.22.x. Interestingly no unreleased documentation is maintained in our case.

Most probably the banner shown is also for the website which are falling under the First Versioning usecase.

Our versioning configuration is like

{
  "lastVersion": "current",
  "versions": {
    "current": {
      "label": "1.0.0",
      "path": "1.0.0"
    }
  }
}

achimnol · 2021-06-27T04:14:00Z

I got a "freeze" of the webpage after navigating to any other page from the first page in beta2. The first page itself is fine.
Reproduced in Safari, Chrome, Firefox + yarn serve and yarn start. (even before deploying to netlify and/or applying any webserver-specific configs)
My website is a just simple single-page blog site with the default theme.
Downgrading to beta0 with resetting node_modules has resolved the issue.

Firefox has shown a warning that "this webpage is slowing down your computer" and a JS console log that says "maximum update depth exceeded" with the following log:

Warning: Maximum update depth exceeded. This can happen when a component calls setState inside useEffect, but useEffect either doesn't have a dependency array, or one of the dependencies changes on every render.
    in DocsPreferredVersionContextProviderUnsafe (created by DocsPreferredVersionContextProvider)
    in DocsPreferredVersionContextProvider (created by LayoutProviders)
    in UserPreferencesProvider (created by LayoutProviders)
    in AnnouncementBarProvider (created by LayoutProviders)
    in ThemeProvider (created by LayoutProviders)
    in LayoutProviders (created by Layout)
    in Layout (created by BlogListPage)
    in BlogListPage (created by LoadableComponent)
    in LoadableComponent (created by Context.Consumer)
    in Route (created by App)
    in Switch (created by App)
    in Route (created by PendingNavigation)
    in PendingNavigation (created by Context.Consumer)
    in withRouter(PendingNavigation) (created by App)
    in Root (created by App)
    in App
    in Router (created by BrowserRouter)
    in BrowserRouter 198 index.js:1

slorber · 2021-06-29T14:02:53Z

Thanks @CovalentBond , investigating now

@achimnol That looks unrelated to the current issue. Can you please open another bug report linking with more info and a repro/public repo? That looks like a regression we also need to fix, but I don't know how to reproduce it yet.

slorber · 2021-06-29T16:09:51Z

@CovalentBond on beta.2 we have some breaking changes regarding the version banner you talk about.
The DocVersionBanner has been renamed in #5052
You must update your swizzled DocItem code (see slorber/zowe-test@1e2263f), this will fix the issue you have. In general it's worth checking breaking changes on swizzled components when such UI bugs happen.

Unexpectedly, your site still builds fine despite this breaking change. This is because I publish docusaurus official releases from my local computer and didn't run any cleanup before the publish, so the dist folder still contains the old component (will fix that).

Not upgrading this code for beta.3 and canary releases would give you an error such as:

Module not found: Error: Can't resolve '@theme/DocVersionSuggestions' in '/Users/sebastienlorber/Desktop/docs-site-zowe-docs-v2/src/theme/DocItem'

Now let's go back to our redirect problem.

Here's what has changed:

beta.0

Real file: build/v1.21.x/getting-started/overview/index.html
Redirect file: build/v1.21.x/getting-started/overview.html/index.html
Redirect file: build/v1-21-x/getting-started/overview/index.html

beta.2

Real file: build/v1.21.x/getting-started/overview/index.html
Redirect file: build/v1.21.x/getting-started/overview.html ❌
Redirect file: build/v1-21-x/getting-started/overview/index.html

The overview.html/index.html filename became overview.html.

I'm thinking about reverting this behavior change of the redirect plugin for better retro-compatibility.

However, some important things to consider:

Netlify already redirects your /x/y.html to /x/y, why do you need the fromExtensions: ["html"]? (this is the source of problem)
We now recommend using trailingSlash: true/false instead of undefined, particularly for Netlify, as it adds by default a trailing slash to URLs with a server redirect, which impacts SEO/perf/UX. See this URL: https://zowe-docs-testing.netlify.app/stable/getting-started/overview => Netlify adds a / at the end

In general, I created https://github.com/slorber/trailing-slash-guide to hep you figure out these problems by exposing the differences between different static hosts

You should be able to fix this problem today by using one of those 2 potential solutions, but I'd recommend both of them.

idontknowjs · 2021-06-29T17:52:15Z

Thank you so much @slorber the banner issue has been solved. But I don't think Netlify redirects /x/y.html to /x/y by default.

Check this https://60db55aebb09341de57576a1--zowe-docs-testing.netlify.app/v1.21.x/user-guide/installandconfig.html (Created with trailingSlash: false, removing fromExtensions: ["html"] with beta.2). This gives a Page Not Found.

This could also be reproduced with the Preview link generated of PR #5085 on Netlify.
Preview Link: https://deploy-preview-5085--docusaurus-2.netlify.app/docs/typescript-support.html

It gives Page Not Found.

idontknowjs · 2021-06-29T22:08:08Z

Update: Checked with trailingSlash undefined & using fromExtensions: ["html"] based on the latest canary release works fine. Page redirect works!

But with trailingSlash: false & not using fronExtensions: ["html"] gives "Page Not Found". Check here: https://60db93eee46b29138797921e--zowe-docs-testing.netlify.app/stable/getting-started/overview.html

If any link is /x/y.html/ (with a trailing slash), then it redirects on /x/y. But gives "Page Not Found" on /x/y.html links

achimnol · 2021-06-30T05:10:19Z

@achimnol That looks unrelated to the current issue. Can you please open another bug report linking with more info and a repro/public repo? That looks like a regression we also need to fix, but I don't know how to reproduce it yet.

I've created #5089.

slorber · 2021-06-30T08:26:56Z

Ah yes I agree, Netlify is able to serve /file.html but it does not redirect to /file when doing so.

Also highlighted by on my table here: https://github.com/slorber/trailing-slash-guide

I've reverted the behavior of emitting /file.html/index.html, it seems a bit weird filename pattern but seems to actually work fine so let's keep it this way.

My test deployment (https://github.com/slorber/zowe-test) seems to work better now:

The issue affecting our deploy preview should not affect you, but I'll fix it (it's only when using trailingSlash: true): #5093

slorber · 2021-06-30T08:47:40Z

Alright, so it looks like all the reported issues have been fixed, let me know if you still find anything weird.

I think you'll have to keep using fromExtensions: ["html"] if you want the redirect to happen, as Netlify does not do that redirect automatically and just serve the file, but /xyz.html is not a route the client-side router (React Router) know about. It could also be fixed by having some client-side code remove the .html extension from url, but we can't do that automatically because using slug: /xyz.html is a valid thing to do, even if it's quite unusual

idontknowjs · 2021-07-01T11:24:03Z

Not related to this issue, but is there any way we can optimize/minify the static files.

The size of our static f/older case is more than 400MB. The size probably seem to increase with every version added. It massively increases our build size and time. Since Docusaurus does not optimize static files by default. Can you tell me how can we optimize it

PS. It would be really great if Docusaurus by default could optimize the static files. Related issue: #568

slorber · 2021-07-01T13:35:38Z

Not related to this issue, but is there any way we can optimize/minify the static files.

It's not clear what you mean. Which static file type, and which optimization?

We don't optimize an image in the same way we optimize a JS/CSS file.

JS/CSS files already already optimized in Terser / cssnano

HTML files are also optimized by Webpack (afaik)

The size of our static f/older case is more than 400MB. The size probably seem to increase with every version added. It massively increases our build size and time.

If you have 10 versions, you have 10x more HTML files, and having increased build times and deployment size is totally expected, this is one of the drawbacks of Jamstack / static site generators

That's also why other big players in this field work on solving this issue:

Gatsby has incremental build
Next.js / Vercel has Incremental static regeneration
Netlify has On-demand builders

We don't offer such a solution for now (it is not something easy provide), apart Webpack 5 caching that improves rebuild-time if you persist node_modules/.cache/webpack across builds

Note on versioning:

we encourage users to handle multiple versions in the same git branch by default
nothing prevents you to move back to the more traditional approach of having one branch per version, each deployed independently
hybrid approaches are possible

I plan to write some performance/scalability guide.

If you have doc versions that you do not maintain anymore, you'd rather move them out of your site, so that they don't impact your build time anymore.
Are you still working on docs of this "old version", or can it eventually be archived? https://zowe-docs-testing.netlify.app/v1.0.x/getting-started/overview/

Look at what we do on the Docusaurus site itself: not all versions are on the site, and older versions can remain accessible by using archived standalone deployments, external link added to the end of the dropdown:

slorber · 2021-07-01T13:37:26Z

Note, as you are using Netlify, to leverage Webpack 5 caching (much faster rebuild times), you can use the Netlify cache plugin, as we do on the Docussaurus site:

https://github.com/facebook/docusaurus/blob/master/website/netlify.toml#L22

idontknowjs · 2021-07-01T14:34:37Z

It's not clear what you mean. Which static file type, and which optimization?

In the static folder, we are directly serving various HTML, CSS & JS files in every version which are being rendered like this. As mentioned in the Docusaurus Documentation, these files in the static folder are not post-processed or minified.

I was asking if we can further optimize these files and save some more time.

Look at what we do on the Docusaurus site itself: not all versions are on the site, and older versions can remain accessible by using archived standalone deployments, external link added to the end of the dropdown:

Yes, we have been planning to archive old doc versions and seperate it as done on the Docusaurus website. It might significantly improve build time!

Note, as you are using Netlify, to leverage Webpack 5 caching (much faster rebuild times), you can use the Netlify cache plugin, as we do on the Docussaurus site.

Thanks I will take a look!

slorber · 2021-07-01T14:45:23Z

I was asking if we can further optimize these files and save some more time.

Everything inside /static should be optimized by your own means. I won't do anything to these files

idontknowjs added bug An error in the Docusaurus core causing instability or issues with its execution status: needs triage This issue has not been triaged by maintainers labels Jun 24, 2021

slorber mentioned this issue Jun 26, 2021

Feedback on trailingSlash option + GH Pages slorber/trailing-slash-guide#2

Closed

slorber mentioned this issue Jun 29, 2021

fix(v2): redirect plugin should emit redirect files with lower precedence than redirect target #5085

Merged

slorber closed this as completed in #5085 Jun 29, 2021

achimnol mentioned this issue Jun 30, 2021

Freeze of page after navigating to any other page from the first page in beta2 #5089

Closed

slorber mentioned this issue Jun 30, 2021

fix(v2): redirect from should work with trailingSlash: true #5093

Merged

Josh-Cena removed the status: needs triage This issue has not been triaged by maintainers label Feb 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Infinite network requests/refresh loop while accessing docs #5055

Infinite network requests/refresh loop while accessing docs #5055

idontknowjs commented Jun 24, 2021 •

edited

Loading

slorber commented Jun 24, 2021

idontknowjs commented Jun 24, 2021

idontknowjs commented Jun 24, 2021

slorber commented Jun 25, 2021

idontknowjs commented Jun 25, 2021

achimnol commented Jun 27, 2021 •

edited

Loading

slorber commented Jun 29, 2021

slorber commented Jun 29, 2021

idontknowjs commented Jun 29, 2021

idontknowjs commented Jun 29, 2021

achimnol commented Jun 30, 2021

slorber commented Jun 30, 2021 •

edited

Loading

slorber commented Jun 30, 2021 •

edited

Loading

idontknowjs commented Jul 1, 2021

slorber commented Jul 1, 2021

slorber commented Jul 1, 2021

idontknowjs commented Jul 1, 2021

slorber commented Jul 1, 2021

Infinite network requests/refresh loop while accessing docs #5055

Infinite network requests/refresh loop while accessing docs #5055

Comments

idontknowjs commented Jun 24, 2021 • edited Loading

🐛 Bug Report

Prerequisites

Description

Have you read the Contributing Guidelines on issues?

Steps to reproduce

Steps to build the website locally

Expected behavior

Actual behavior

Your environment

slorber commented Jun 24, 2021

idontknowjs commented Jun 24, 2021

idontknowjs commented Jun 24, 2021

slorber commented Jun 25, 2021

idontknowjs commented Jun 25, 2021

Deploy preview

Builds

Another Issue❗️

achimnol commented Jun 27, 2021 • edited Loading

slorber commented Jun 29, 2021

slorber commented Jun 29, 2021

beta.0

beta.2

idontknowjs commented Jun 29, 2021

idontknowjs commented Jun 29, 2021

achimnol commented Jun 30, 2021

slorber commented Jun 30, 2021 • edited Loading

slorber commented Jun 30, 2021 • edited Loading

idontknowjs commented Jul 1, 2021

slorber commented Jul 1, 2021

slorber commented Jul 1, 2021

idontknowjs commented Jul 1, 2021

slorber commented Jul 1, 2021

idontknowjs commented Jun 24, 2021 •

edited

Loading

achimnol commented Jun 27, 2021 •

edited

Loading

slorber commented Jun 30, 2021 •

edited

Loading

slorber commented Jun 30, 2021 •

edited

Loading