V2 API — Async snapshots #30
Conversation
|
First off, this is wild, and you are wild for building it. I like the use of HOC, and I think your assumptions are sound. One area I think could be tweaked are the method names for
(Or any mix of the first two.) One other thing—should I just clocked (lol) that with This is so great, @geelen. This is so great. |
Agree, this seems like the best default behaviour. In what cases would you not want this? That'd allow for CDN caching, as well as something that would work pretty well with a local serviceworker too, right?
Going on that idea that it could 'rehydrate' by default — The method could be componentWillMount() {
- snapshot(() => (
+ snapshot.hydrate(() => (
fetch('/api/quotes')
.then(response => response.json())
))
.then(quotes => {
this.setState({ quotes })
})
} |
|
Hmm, I'm not quite sure I follow. "Rehydration" is where there's a big chunk of data inline in the HTML that gets used to build up the exact React VDOM tree to match what's already there: That happens regardless. The question is whether, on the client, with all the data already there, do you go back to the API and see if anything's changed? If something has changed, suddenly data just appears out of nowhere. That's why it's a repetition. I've just updated the DB behind the current deployment: http://react-snapshot-demo.pithy.af/. If you load that up now you should see a Roger Ebert quote quickly replaced by a Kurt Vonnegut one. It's jarring, and would require adding some extra code for a component to deal with the fact that new data might arrive. On another page: http://react-snapshot-demo.pithy.af/authors/roger-ebert-2, you just get a single quote, even though I've added another quote into the DB. If you navigate (click "Review of the film North" then hit back), the fresh data is retrieved, but refreshing the page you'll go back to just one. So it's not ideal either, but it's certainly going to come up less often. In each case it's made worse by just how simple the demo app is. If you were using any kind of centralised data store like Redux you'd have very different ways of managing data & API access. And, if you dropped in something like react-flip-move you could make sudden appearances of data look a lot better. But the real problem is letting snapshots and the DB get out of sync, so, naturally you need to consider how often your DB changes. You'd never use this for a twitter homepage but it would certainly work for a blog with updates every once in a while or a news site with multiple changes a day. You just need to regenerate snapshots when new data is there to be served (and if you do that, I don't think |
Isn't that the 'rehydration' process? Something is loaded, then rehydrated with fresh content? At least that's how I'd understood it previously anyway.
Hmm, that brings up a UI question—is there a way to get a handle on that update cycle? So customers can be shown a 'Loading fresh content' message? |
|
Nope rehydrate doesn't hit the API at all, it's about booting the app with the data that's already present: https://stackoverflow.com/a/29826133 |
|
I like the terminology that @superhighfives used around this then. |
|
In that case, perhaps Also, wa-hey, I did not realise that was the definition of rehydrate, @geelen. That makes sense, for sure. I suppose our difference is that we're dealing with static pages, so the server doesn't have the opportunity to pull the latest state from the API for the initial render. That said, you're hitting the API the same amount of times—once—given it's a static page vs getting state from the server. componentWillMount() {
- snapshot(() => (
+ snapshot.resync(() => (
fetch('/api/quotes')
.then(response => response.json())
))
.then(quotes => {
this.setState({ quotes })
})
}Alternatively have it as a setting option? componentWillMount() {
- snapshot.resync(() => (
+ snapshot(() => (
fetch('/api/quotes')
.then(response => response.json())
- ))
+ ), {resync: true})
.then(quotes => {
this.setState({ quotes })
})
}That's a little less clear though, x 1,000,000. I still lean toward automatically hitting the API / |
Doesn't the server hit the API? That's semantics, though. |
|
Nah when the server/build hits the API that's just normal execution. Then it takes the result & dehydrates it (serialises it into the HTML). When the app boots on the client, it sees the data in the HTML and rehydrates (parses). Basically it's like you've started the program running on one computer, paused it, then resumed it on another. I think |
|
This has been very educational. 👍 |
|
@geelen, async rendering works great so far! I only faced one issue related to webpack's code splitting but it can be related to the base version, not only this branch. Whenever I use Webpack uses While I was writing all this text, I've found the original issue: jsdom/jsdom#1564. I cannot confirm that latest version of |
|
I've tried to think about additional cases related to async modules loading and it seems that the better approach would be to remove those scripts from |
|
@alexeyraspopov very interesting. I've been thinking about making react-snapshot use chrome headless instead, because these kinds of bugs are irritating (and potentially endless) But that's cool, glad it's (mostly) working!! |
|
@geelen, I'm going to build a reference app using |
|
@alexeyraspopov OMG thank you that sounds amazing. I'm off on holidays the next 3 weeks so I won't be able to make core updates but I'll be around on emails to bounce ideas around ❤️ |
|
Hey @geelen, really liking this so far! We're testing it out with API calls via redux, and have hit a bit of an issue when there are multiple requests. From what I can tell, snapshot is recording duplicate requests - for example when there are three requests made, the window.react_snapshot_state = {
{0: {response_one}},
{1: {response_one}},
{2: {response_two}},
{3: {response_two}},
{4: {response_three}},
{5: {response_three}}
}Inside the app it appears that snapshot expects the correct number of requests, and as a result it loads Happy to put together a simple example if it'd help, but here's an idea of how we've structured our async actions: const asyncFetch = () => dispatch => {
dispatch(asyncRequest());
const url = 'api-url-goes-here';
const headers = new Headers();
headers.append('Content-Type', 'application/json');
snapshot(() => (
fetch(url, { headers, method: 'get' })
.then(response => response.json())
.then(json => parseData(json))
.catch(error => ({ error }))
))
.then(response => {
if (response.error) {
dispatch(asyncError(response.error));
} else {
dispatch(asyncSuccess(response.payload));
}
});
};Is there something wrong about what we're doing which might be causing the bug? |
|
Hmm, nothing looks wrong at first glance... Unless somehow you're calling |
|
The updated version didn't fix it, so I looked through the logic again and you're right - one of the components was looking for a flag in the wrong place, and sent a duplicate fetch request in some situations. Fixed that, and it all works perfectly now, so thanks for the help! 😄 |
|
I guess that means the updated version didn't break anything new either, so 🎉 Looking at how things might change with React 16+. It's quite likely react-snapshot 2 will require 16 and up because of big changes to the SSR stuff. Would that be an issue for you? |
|
Yeah true - fairly sure it was a little faster too! 😁 I don't know actually, though from what I remember there aren't any breaking changes being introduced? Doubt there'll be any issues, since we're able to be pretty flexible on stuff like that; as long as it doesn't require a big refactor we'll probably test it out pretty soon. 🙂 |
|
how canI use snapshot when rehydrating persistent redux sttore using redux-persist. |
|
Hey @geelen this is really great. But I am having trouble getting this to work with my redux actions. Basically I have a route that loads md files with text dynamically based on the prop passed to it (e.g. domain.com/page/test will fetch the test.md file from my public static dir and display it in the component. I am trying to figure out if there is some way to render the page. Note I am hosting everything on surge as a frontend site. Code for reference In actions.js: export function fetchPageContent(path) {
return dispatch => {
return snapshot(() => (
fetch(`/content/${path}.md`)
.then(res => dispatch(receiveContent(res))
)
)
}
}in Page.js class Page extends Component {
constructor(props) { super(props) }
componentWillMount() {
const { dispatch } = this.props;
// i store the path in this.props.location.state in React Router
dispatch(fetchPageContent(this.props.location.state))
}
render() {
return ( <div>{ this.props.content }</div> )
}
}
function mapStateToProps(state) { return state }
export default withRouter(connect(mapStateToProps)(Page))Currently absolutely nothing shows up in the window.react_snapshot_state (other than an empty object) for pages on that component. Is there something I am doing wrong here? Another edit: When I move all the fetch logic into the component itself, everything works as expected; however, I do need all the logic to be in a redux store. |
|
Hello @geelen, I am very excited that I found this pull request. Are there any updates on this? If you need any, I am willing to help. |
|
@geelen there is a simpler approach. We can repeat what puppeteer does. Puppeteer waits 0.5 seconds after last network request made. All we need is to catch all network requests, store each as a promise into an array, wait for all, wait for 0.5 seconds more. That's all! const requests = [];
window.fetch = (path, options) => {
path = url.resolve(window.location.toString(), path);
const response = fetch(path, options);
requests.push(response);
return response;
};
...
if (requests.length > 0) {
Promise.all(requests).then(...)
}Trying to do something similar stereobooster/react-snap#71 |
|
Rendertron approach /**
* Listens for changes to the 'renderComplete' flag.
*/
function listenToCompletionFlag() {
Object.defineProperty(window, 'renderComplete', {
set: function(value) {
if (value == false) {
console.log('Rendertron: Waiting for rendering flag');
} else if (value == true) {
console.log('Rendertron: Rendering complete');
}
}
});
}
// Check if an explicit `renderComplete` flag is being used.
let waitForFlag = false;
Console.messageAdded((event) => {
if (event.message.text === LOG_WAITING_FOR_FLAG) {
waitForFlag = true;
} else if (event.message.text === LOG_RENDER_COMPLETE) {
pageReady();
}
}); |
|
This is great, and precisely what we need for a project we'll be launching soon. Although, we are using Apollo for GraphQL connectivity. I'm quite new to Apollo, so unsure what its internals are like and how I could translate its HOC pattern to the If there's any examples available that I haven't been able to find, it would be super duper appreciated! |
|
Is this still active? |
|
@zomars, I believe https://github.com/stereobooster/react-snap is a more actively maintained take on the core principles of react-snapshot. cc @geelen |
Hey friends! I have some FANCY NEW IDEAS™ about this library but would love some REAL HUMAN FEEDBACK™ from smart, good-looking people such as yourself.
Some backstory: React-snapshot started life as basically a zero-configuration static site generator, where you have a file system full of 'pages' or 'posts' or something, that it then gets crawled and generates all the static pages for. What I wanted was a version of Gatsby or Phenomic or static-site-generator-webpack-plugin or static-react except it was a three-line change from default Create React App install:
It worked pretty well, and after I fixed a couple of silly bugs that had been sitting around for several months it works even better. If you want a bit more context read the project Readme
However, I pretty quickly wanted to try this out on a non-static site, to see if there was a way to have the crawler snapshot a DB-driven application. This makes it a lot more similar to something like next.js, but again, I wanted the minimal possible changes from a vanilla CRA.
This is what I've been able to build so far:
During the initial app rendering at build time (using JSDOM), if any
snapshotmethods are called, we start tracking it, cache the result, then serialise the result alongside the static HTML that we produce. When the app gets booted on the client, that samesnapshotmethod short-circuits, immediately calling the.thenand the state gets populated before the render method is called. This means there's no flash, the checksum matches, and everything is JUST GREAT™.I have to do some tricks behind the scenes to make this work—I'm not using a Promise internally because they're always asynchronous i.e. the render method would run with an empty state, then on the next tick (potentially after a paint) the
setStatecall would fix things up. I simply stub about thethenandcatchmethods to call them immediately. As far as I can tell, this will work JUST FINE™.Basically, any async part of your app that gets data and returns a promise can be wrapped in a
snapshotcall and it'll get cached. Since there's no way to know whichsnapshotcall is which, I simply assume that it'll happen in precisely the same order during build in JSDOM as in the client. This assumption also seems ok for the moment, but again there might be a case where it breaks down. For now though, things JUST WORK™.I also expanded the API a little bit because these kinds of data-fetching components are super common and Next.JS has it's trendy
async getInitialPropsextension (which is too heavy for my liking) so I made like a hipster and carved out an artisanal HOC for my new library:That
renderingbit means "renderHomeeven if I have no data". The alternative isthenRender, which will wait until all the promises have been resolved.There's also another neat thing I just added while I was writing this PR where, if there's a chance your data will change more often than you'll deploy new snapshots, you might want to run the async code again and re-render when you've got the data hot & fresh off the API. That's easy:
componentWillMount() { - snapshot(() => ( + snapshot.repeat(() => ( fetch('/api/quotes') .then(response => response.json()) )) .then(quotes => { this.setState({ quotes }) }) }Basically, I don't want to try to cover the whole gamut of server-rendering use-cases, I'm just trying to solve the common use cases in the smallest amount of code possible. I want people to build more stuff in React using CRA but then have this super easy transition to hosting pre-rendered, critical-CSS-inlined, cached-on-a-nearby-CDN, reddit-frontpage-proof site.
What'd I'd like from you, smart & good-looking human, is to let me know what you think of the idea & implementation. Is the magic-synchronous-Promise & assumption of deterministic order too magic? Are methods like
renderingandrepeatworse than simple config objects? Or, and maybe this is the most important one, can you see a use case where this approach would break down so horribly that I shouldn't ship the API in its current state.Just FYI, I do have this running on a demo site in production: http://react-snapshot-demo.pithy.af/. During deployment, the current API is crawled and snapshots for each page are taken. You can click around the whole site without JS, but if you have it on, it works like a normal React app, making API requests as it needs to. The homepage is using
Snapshot.repeatbut all other pages should not make a single API request if you reload the page. The code's here btw.If you want to try it out on your own site, it's published as
react-snapshot@next. Hope it works! ❤️