Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement client cache of GET urls with fragments #7562

Merged
merged 16 commits into from
Mar 17, 2017

Conversation

cvializ
Copy link
Contributor

@cvializ cvializ commented Feb 15, 2017

Implements #7463

.then(response => response.json());

const isCacheable = (init.method === 'GET' && propertyPath);
if (isCacheable) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I bet we'll add an option to opt out of cache at some point, but this is fine :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged.

return this.fetch(input, init)
.then(response => response.json());

const isCacheable = (init.method === 'GET' && propertyPath);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering: Is it a valid use case that you sometimes want the a sub path and sometimes the whole thing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say it's valid. I'll remove the && propertyPath from the condition.

* @template T
*/
export function sortProperties(obj) {
if (!obj || typeof obj !== 'object') { return obj; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: expand conditional into 3 lines.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I removed this method.

}

/**
* Creates a new object identical to the first with sorted properties.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you are relying on JS' implicit insertion order is read order semantics?

I think I'd prefer a simpler cache key generated that directly writes a string from input. For our effective use case that should just work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are suggesting to just JSON.stringify the opt_init object directly without sorting the properties, is that correct? If so I'll do that and then remove this method.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might misunderstand your code, but what I'm suggesting it to just generate direct string from the traversal instead of generating an object and then stringifying that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I made a simpler serializer that should be sufficient for this use case.

cache.put('d', {foo: 'bar'});

expect(toArray(cache, 'abcd'.split(''))).to.deep.equal(
['abc', 123, ['x', 'y'], {foo: 'bar'}]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: indent +2

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

cache.put('c', ['x', 'y']);
cache.put('d', {foo: 'bar'});

expect(toArray(cache, 'abcd'.split(''))).to.deep.equal(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jsonEqual gives much nicer error messages :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

if (init.method == 'POST' && !isFormData(init.body)) {
const parsedInput = parseUrl(input);
const propertyPath = parsedInput.hash.slice(1); // remove # prefix
const getPropertyAtPath = getPath.bind(null, propertyPath);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't bind until you know you need it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

export function getPath(path, obj) {
const arrayIndexRe = /\[(\d+)\]/g;
const keys = path.replace(arrayIndexRe, '.$1').split('.');
return keys.reduce((acc, key) => acc[key], obj);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CC @choumx who is doing this kind of stuff for amp-bind.

Since JSON returns full objects we need to guard this with hasOwnProperty to avoid exposing stuff like constructor and toString.

Please also think about how to make the error message good if this fails to find an element.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I split it out into a less terse loop with special error messaging.

import {isArray, isObject, isFormData} from '../types';
import {utf8EncodeSync} from '../utils/bytes';
import Cache from '../utils/cache';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dvoytenko How do you feel about this being directly integrated into XHR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved into CachedXhr class

@cramforce
Copy link
Member

Awesome! Thank you!

* @private
*/
getCacheKey_(url, opt_init) {
const urlWithoutHash = url.slice(0, url.indexOf('#'));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Danger! url.indexOf can return -1, which'll make this be something like:

const url = 'https://example.com';
const urlWithoutHash = url.slice(0, url.indexOf('#')); // => https://example.co

We have a removeFragment helper to do this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

.then(response => response.json());

const isCacheable = (init.method === 'GET' && propertyPath);
if (isCacheable) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is the right class for cacheable queries. How about a composed CachedXHR class?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, I'll try that out.

const isCacheable = (init.method === 'GET' && propertyPath);
if (isCacheable) {
const cacheKey = this.getCacheKey_(input, opt_init);
const cachedPromise = this.fragmentCache_.get(cacheKey);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the fetchX methods suffer from fragments causing repeated requests. With the composed class suggestion, we can solve all of them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I'm not sure I understand. The fragment is a path to a property inside a JSON object. How should fetchDocument or fetchText specify a subset of data like the issue specs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fragment is a path to a property inside a JSON object.

We've defined that for fetchJSON only.

But fragments can be added to any request, and they're never sent to the server. So, fecthDoc("#frag1") and fecthDoc("#frag2") are the same request, but won't be cached properly. They should be cached, without doing anything special to the response (no deep JSON property returning, etc).

* @param value {T}
*/
put(key, value) {
if (typeof key !== 'string') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is guaranteed by Closure types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok cool. Is there a way to test this guarantee, or should I not test it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to test, gulp check-types does it for us.

return this.map_[key];
}

get length() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic getters are significantly slower than normal property accesses or methods. This is simple enough that we can update #length after every #put, or we could just make it a method.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll just remove the length getter, I don't see a good use-case for it given it only increases to the size and then remains constant.

export function getPath(path, obj) {
const arrayIndexRe = /\[(\d+)\]/g;
const keys = path.replace(arrayIndexRe, '.$1').split('.');
return keys.reduce((acc, key) => acc[key], obj);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will throw if acc is falsey!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

const keys = Object.keys(obj).sort();
return keys.reduce((acc, key) => {
const value = obj[key];
if (typeof value === 'object') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test that it's not an array either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@cvializ cvializ force-pushed the cv-fragment-cache branch 2 times, most recently from f45fda1 to d61c81f Compare February 28, 2017 00:59
*
* @param {T} obj a map-like value
* @param {string} path a dot-separated list of keys to reference a value
* @return {T}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be *, it's not guaranteed to be an Object (which T has to be).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, even if it was, the shape wouldn't be the same

this.queue_ = [];

/** @private @const {Object<string, T>} */
this.map_ = {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

map, please.

* @param {string=} opt_accept The HTTP Accept header value.
* @return {!FetchInitDef}
*/
setupInit_(opt_init, opt_accept) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to make this an instance method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To use in the first line of CachedXhr#fetchJson. I thought it made sense to share it through subclassing rather than exporting, so that it was clear that it isn't a part of Xhr's public API

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With d61c81f#r103352978, it won't be necessary. You'll just call Xhr#fetchJson, and cache the result if cacheable.

@@ -191,8 +193,8 @@ export class Xhr {
init.headers['Content-Type'] = 'application/json;charset=utf-8';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somehow missed this before. Would you mind deleting this line, and updating the 'application/json' above to be 'application/json;charset=utf-8'?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a test checking that this property is present in the request headers, and the second argument to setupInit sets the Accepts header, not Content-Type.
I can change the test to succeed, but I just want to make sure this is what you really intended.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, I thought the second param was for Content-Type. Thanks for looking into it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, no problem

const init = setupInit(opt_init, 'application/json');
if (init.method == 'POST' && !isFormData(init.body)) {
const init = this.setupInit_(opt_init, 'application/json');
const getResponseJson = response => response.json();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: if we're going to pull this into a variable, hoist it out of the function.

* ampCors: (boolean|undefined)
* }}
*/
let FetchInitDef;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be exported from xhr-impl.js, then you can reference the type in this file.

* @return {!Promise<!JSONType>}
* @override
*/
fetchJson(input, opt_init) {
Copy link
Contributor

@jridgewell jridgewell Feb 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a lot of duplication with Xhr#fetchJson here. I think we can simplify down based on:

  1. Any call with opt_init is un-cacheable
    • Removes crazy simpleSerialize
    • Handles POST case

Actually, that's it. 😉

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple enough.

// Since a fragment is present, cache the full response promise, then
// return a promise with just the value specified by the fragment.
this.fragmentCache_.put(cacheKey, fetchPromise);
return fetchPromise.then(getPropertyAtPath);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason to put this functionality in CachedXhr? Seems like CachedXhr is generally useful, while deep JSON gets are only useful to <amp-list>?

Copy link
Contributor Author

@cvializ cvializ Feb 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I belive the intended use-case for the fragment is e-commerce JSON APIs that return a big user-object with nested data that are displayed on many different parts of the page. For example, a user object with a name that is displayed at the top and bottom, a shopping cart list, etc. and each piece of the UI that references a deep property of the user-object would specify which property through the fragment. Does that answer your question?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. I understand the use case, but I don't think the functionality belongs here. You could just as easily call getPath like so:

// amp-list.js
cachedXhr.fetchJson(endpoint).then(json => getPath(fragment, json))

That keeps CachedXhr generically useful, while giving <amp-list> the special deep JSON gets.

Copy link
Contributor Author

@cvializ cvializ Feb 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's true you could do that, and that would follow the single responsibility principle. But you'd be duplicating that fragment extraction and getPath logic in any component or extension where you'd want to get a sub-object of the overall response, which I think would be 3 or more places, more than just amp-list. IMO it keeps things DRYer if that logic is implicitly triggered in CachedXhr by the presence of a fragment. It also enables any AMP element to access a fragment without explicitly coding it into every fetchJson call site, you'd just swap in the CachedXhr class. If you think the separation will be more beneficial than the deduplication, I'll split it out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The duplication can be handled with a helper, and code size changes should be negligible. For my rationale, I need this to be useable by amp-call-tracking.js and amp-ad-custom.js, both which do not expect this behavior.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. Thank you for explaining your reasoning, I'll do it this way.

*
* @param {T} obj a map-like value
* @param {string} path a dot-separated list of keys to reference a value
* @return {T}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, even if it was, the shape wouldn't be the same



/**
* A service that polyfills Fetch API for use within AMP.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update

src/url.js Outdated
if (index == -1) {
return '';
}
return url.substring(index + 1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: location.hash returns the #. Why don't we do so as well?

Copy link
Contributor Author

@cvializ cvializ Mar 2, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the # here for convenience, but keeping it does follow the POLS

* @return {*}
* @template T
*/
export function getPath(path, obj) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Can we flip the params?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I had path first for to make currying with bind easy to use to map on an array of objects, but I'm not doing that anywhere.

* @return {!Promise<!JSONType>}
* @override
*/
fetchJson_(input, init) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just override the public methods?

fetchJSON(input, init) {
    return this.cached(input) || this.cacheFetch_(input, super.fetchJSON(input, init));
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The public methods accept an optional opt_init value, while the private methods have init guaranteed to be defined by setupInit in the corresponding public method. This lets the cache separately key the edge case where identical URLs serve different documents for different Accepts header values. I could override the public methods, but then I think I'd need to expose setupInit to the subclass. I'll change it if you prefer it implemented that way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for different Accepts header values

Don't we know that by virtue of the public method being called? Ie, we can do:

fetchJSON(input, init) {
    return this.cached(input, 'json') || this.cacheFetch_(input, 'json', super.fetchJSON(input, init));
}

fetch(input, init) {
    return this.cached(input, 'generic') || this.cacheFetch_(input, 'generic', super.fetchJSON(input, init));
}

fetchText(input, init) {
    return this.cached(input, 'text') || this.cacheFetch_(input, 'text', super.fetchJSON(input, init));
}

#getCacheKey can then just concat the URL and the method to generate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh true, thanks I didn't see that

export class Cache {
/**
* Construct a new Cache with the given size
* @param size {number=}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type definitions in this file are backwards. Should be @param {number=} size

* e.g.
* `getPath({a: {b: [{c: 2}]}}, 'a.b[0].c') === 2`
*
* @param {T} obj a map-like value
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{*}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you help me understand why is this different than hasOwn, which returns a boolean but specifies a @param {T} obj?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't have a good answer. @erwinmombay?

xhr.fetchJson('/get?k=v1'),
xhr.fetchJson('/get?k=v1'),
]).then(results => {
expect(fetchStub.calledOnce).to.be.true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

expect(fetchStub).to.be.calledOnce;

import {installCachedXhrService} from '../../src/service/cached-xhr-impl';


describes.realWin('CachedXhr', {}, env => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test that POSTs aren't cached.

import {installCachedXhrService} from '../../src/service/cached-xhr-impl';


describes.realWin('CachedXhr', {}, env => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test that fragments do not affect caching.

return response.text()
.then(result => {
const callCloneText = () => clone.text().then(clonedText => {
expect(clonedText).to.equal(TEST_TEXT);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This expectation will not be validated, since the test will have exited before this is run.

const callCloneText = () => clone.text().then(clonedText => {
expect(clonedText).to.equal(TEST_TEXT);
});
expect(callCloneText, 'should not throw').to.not.throw(Error);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wrapping isn't necessary. If the clone's #text were to throw syncly, it would cause the tests to fail (because the promise block it's inside is waited on by the test).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, makes sense.

expect(clonedText).to.equal(TEST_TEXT);
});
expect(callCloneText, 'should not throw').to.not.throw(Error);
expect(result).to.equal(TEST_TEXT);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return the clone's #text promise here, with its expectation in a then block.

@cvializ cvializ mentioned this pull request Mar 9, 2017
src/runtime.js Outdated
@@ -119,6 +120,7 @@ export function installRuntimeServices(global) {
installTimerService(global);
installVsyncService(global);
installXhrService(global);
installCachedXhrService(global);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you run gulp size and see the impact on the core payload?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly, here are the results.

max min gzip
master 1.11 MB 245.71 kB amp.js
PR #7562 1.11 MB 246.56 kB amp.js

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're gzip is a little large. 😜

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha just copying what I see in my terminal:

screen shot 2017-03-10 at 1 28 04 pm

I think something might be broken with that script

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Run gulp dist

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah that makes sense, I had just run build. What do you think of adding dist as a gulp task dependency for the size task? Are there any cases where we want to run size on files produced by a task other than dist?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here are the correct numbers:

max min gzip file
master 1.11 MB 206.7 kB 61.94 kB v0.js / amp.js
PR #7562 1.11 MB 207.05 kB 62.11 kB v0.js / amp.js

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping @erwinmombay.

@cramforce
Copy link
Member

Sorry, for opening pandoras box. I think we should reconsider the implementation strategy, but would like to get input from others: Delete the app layer cache from this CL. We have to document how to do the fragment stuff anyway, so we might as well document people should do something like set a 1 minute max-age on the response. We just need to test whether Safari does the right thing in that case. They sometimes do redundant fetching of concurrent requests to same resource.

@ampprojectbot
Copy link
Member

Hi, ampproject bot here! Here are a list of the owners that can approve your files.

You may leave an issue comment stating "@ampprojectbot retry!" to force me to re-evaluate this Pull Request's status

/to cramforce dvoytenko jridgewell

  • extensions/amp-list/0.1/amp-list.js
  • extensions/amp-list/0.1/test/test-amp-list.js

/to dvoytenko jridgewell

  • src/service/xhr-impl.js
  • src/url.js
  • src/utils/object.js

/to alanorozco camelburrito chenshay choumx cvializ ericlindley-g erwinmombay gregable honeybadgerdontcare jridgewell kmh287 lannka mkhatib mrjoro muxin newmuis powdercloud zhouyx

  • test/functional/test-object.js
  • test/functional/test-xhr.js

For any issues please file a bug at https://github.com/google/github-owners-bot/issues

@cvializ
Copy link
Contributor Author

cvializ commented Mar 17, 2017

PTAL, removed app layer cache as requested by @cramforce and implemented fetch-batching per @jridgewell's suggestion

Copy link
Member

@cramforce cramforce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, but have one question?

@@ -64,7 +65,11 @@ export class AmpList extends AMP.BaseElement {
if (!opts.credentials) {
opts.requireAmpResponseSourceOrigin = false;
}
return xhrFor(this.win).fetchJson(src, opts);
const fetchPromise = batchedXhrFor(this.win).fetchJson(src, opts);
const fragment = getFragment(src).slice(1);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a feature of BatchedXhr?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Justin and I discussed keeping the fragment-extraction and object property access behavior inside of the new Xhr class, but we decided to keep them separate. Discussion here.

Copy link
Member

@cramforce cramforce left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Important follow up is to support the path based stuff in more places and to document it.

@cvializ
Copy link
Contributor Author

cvializ commented Mar 17, 2017

Wahoo! I created an issue to track the documentation effort. I don't have merge permission yet, so feel free to merge any time.

@cramforce cramforce merged commit cdf1f1c into ampproject:master Mar 17, 2017
@cvializ cvializ deleted the cv-fragment-cache branch March 17, 2017 20:45
if (isBatchable) {
this.fetchPromises_[key] = fetchPromise.then(response => {
delete this.fetchPromises_[key];
return response.clone();
Copy link
Contributor

@jridgewell jridgewell Mar 21, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: This isn't sufficient to prevent double draining. The problem is this is part of the promise chain.
So, first call to fetch returns the clone. I then add to the chain to read the body, and issue a second request to the same endpoint. The second request will see the batched call, and return a clone of that promise (a clone of a clone) Problem is, that clone will happen after the body read. (Because execution chains on a single promise is defined)

This should return the response without cloning. This total promise chain will be stored as the batch. For the return value, we should return the batching promise with a .then((resp) => resp.clone()) added to its chain.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After thinking about it, I'm not sure your assessment is correct. The clone is created, but not returned to the caller, just to the batching mechanism. Your argument assumes that the first fetch returns the clone, which is not what is happening in this code. So nothing can drain the body of the response, so there cannot be a double drain.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, missed that.

@cvializ cvializ mentioned this pull request Mar 21, 2017
* @param {string} path a dot-separated list of keys to reference a value
* @return {*}
*/
export function getPath(obj, path) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cvializ Is this method significantly different from https://github.com/ampproject/amphtml/blob/master/src/json.js#L84 ?

If no, please file a bug to reconcile with P1.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! I did not see that. I'll file the bug

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants