Declarative routing

[jakearchibald]

Here are the requirements I'm working towards:

• Be able to bypass the service worker for particular requests.
• Speed up simple offline-first, online-first routes by avoiding service worker startup time.
• Be polyfillable – do not introduce things that cannot already be done in a fetch event.
• Be extensible – consider what future additions to the API might look like.
• Avoid state on the registration if possible – prefer state on the service worker itself.

I'm going to start with static routes, and provide additional ideas in follow-up posts.

The aim is to allow the developer to declaratively express a series of steps the browser should perform in attempt to get a response.

The rest of this post is superseded by the second draft

Creating a route

WebIDL

// Install currently uses a plain ExtendableEvent, so we'd need something specific
partial interface ServiceWorkerInstallEvent {
  attribute ServiceWorkerRouter router;
}

[Exposed=ServiceWorker]
interface ServiceWorkerRouter {
  void add(ServiceWorkerRouterItem... items);
}

[Exposed=ServiceWorker]
interface ServiceWorkerRouterItem {}

JavaScript

addEventListener('install', (event) => {
  event.router.add(...items);
  event.router.add(...otherItems);
});

The browser will consider routes in the order declared, and will consider route items in the order they're given.

Route items

Route items fall into two categories:

• Conditions – These determine if additional items should be considered.
• Sources – A place to attempt to get a response from.

Sources

WebIDL

[Exposed=ServiceWorker, Constructor(optional RouterSourceNetworkOptions options)]
interface RouterSourceNetwork : ServiceWorkerRouterItem {}

dictionary RouterSourceNetworkOptions {
  // A specific request can be provided, otherwise the current request is used.
  Request request;
}

[Exposed=ServiceWorker, Constructor(optional RouterSourceCacheOptions options)]
interface RouterSourceCache : ServiceWorkerRouterItem {}

RouterSourceCacheOptions : MultiCacheQueryOptions {
  // A specific request can be provided, otherwise the current request is used.
  Request request;
}

[Exposed=ServiceWorker, Constructor(optional RouterSourceFetchEventOptions options)]
interface RouterSourceFetchEvent : ServiceWorkerRouterItem {}

dictionary RouterSourceFetchEventOptions {
  DOMString id = '';
}

These interfaces don't currently have attributes, but they could have attributes that reflect the options/defaults passed into the constructor.

Conditions

WebIDL

[Exposed=ServiceWorker, Constructor(ByteString method)]
interface RouterIfMethod : ServiceWorkerRouterItem {}

[Exposed=ServiceWorker, Constructor(USVString url, optional RouterIfURLOptions options)]
interface RouterIfURL : ServiceWorkerRouterItem {}

dictionary RouterIfURLOptions {
  boolean ignoreSearch = false;
}

[Exposed=ServiceWorker, Constructor(USVString url)]
interface RouterIfURLPrefix : ServiceWorkerRouterItem {}

[Exposed=ServiceWorker, Constructor(USVString url, optional RouterIfURLOptions options)]
interface RouterIfURLSuffix : ServiceWorkerRouterItem {}

[Exposed=ServiceWorker, Constructor(optional RouterIfDateOptions options)]
interface RouterIfDate : ServiceWorkerRouterItem {}

dictionary RouterIfDateOptions {
  // These should accept Date objects too, but I'm not sure how to do that in WebIDL.
  unsigned long long from = 0;
  // I think Infinity is an invalid value here, but you get the point.
  unsigned long long to = Infinity;
}

[Exposed=ServiceWorker, Constructor(optional RouterIfRequestOptions options)]
interface RouterIfRequest : ServiceWorkerRouterItem {}

dictionary RouterIfRequestOptions {
  RequestDestination destination;
  RequestMode mode;
  RequestCredentials credentials;
  RequestCache cache;
  RequestRedirect redirect;
}

Again, these interfaces don't have attributes, but they could reflect the options/defaults passed into the constructor.

Shortcuts

GET requests are the most common type of request to provide specific routing for.

WebIDL

partial interface ServiceWorkerRouter {
  void get(ServiceWorkerRouterItem... items);
}

Where the JavaScript implementation is roughly:

router.get = function(...items) {
  router.add(new RouterIfMethod('GET'), ...items);
};

We may also consider treating strings as URL matchers.

• router.add('/foo/') === router.add(new RouterIfURL('/foo/')).
• router.add('/foo/*') === router.add(new RouterIfURLPrefix('/foo/')).
• router.add('*.png') === router.add(new RouterIfURLSuffix('.png')).

Examples

Bypassing the service worker for particular resources

JavaScript

// Go straight to the network after 25 hrs.
router.add(
  new RouterIfDate({ from: Date.now() + 1000 * 60 * 60 * 25 }),
  new RouterSourceNetwork(),
);

// Go straight to the network for all same-origin URLs starting '/videos/'.
router.add(
  new RouterIfURLPrefix('/videos/'),
  new RouterSourceNetwork(),
);

Offline-first

JavaScript

router.get(
  // If the URL is same-origin and starts '/avatars/'.
  new RouterIfURLPrefix('/avatars/'),
  // Try to get a match for the request from the cache.
  new RouterSourceCache(),
  // Otherwise, try to fetch the request from the network.
  new RouterSourceNetwork(),
  // Otherwise, try to get a match for the request from the cache for '/avatars/fallback.png'.
  new RouterSourceCache({ request: '/avatars/fallback.png' }),
);

Online-first

JavaScript

router.get(
  // If the URL is same-origin and starts '/articles/'.
  new RouterIfURLPrefix('/articles/'),
  // Try to fetch the request from the network.
  new RouterSourceNetwork(),
  // Otherwise, try to match the request in the cache.
  new RouterSourceCache(),
  // Otherwise, if the request destination is 'document'.
  new RouterIfRequest({ destination: 'document' }),
  // Try to match '/articles/offline' in the cache.
  new RouterSourceCache({ request: '/articles/offline' }),
);

Processing

This is very rough prose, but hopefully it explains the order of things.

A service worker has routes. The routes do not belong to the registration, so a new empty service worker will have no defined routes, even if the previous service worker defined many.

A route has items.

To create a new route containing items

1. If the service worker is not "installing", throw. Routes must be created before the service worker has installed.
2. Create a new route with items, and append it to routes.

Handling a fetch

These steps will come before handling navigation preload, meaning no preload will be made if a route handles the request.

request is the request being made.

1. Let routerCallbackId be the empty string.
2. RouterLoop: For each route of this service worker's routes:
   1. For each item of route's items:
      1. If item is a RouterIfMethod, then:
         1. If item's method does not equal request's method, then break.
      2. Otherwise, if item is a RouterIfURL, then:
         1. If item's url does not equal request's url, then break.
      3. Etc etc for other conditions.
      4. Otherwise, if item is a RouterSourceNetwork, then:
         1. Let networkRequest be item's request.
         2. If networkRequest is null, then set networkRequest to request.
         3. Let response be the result of fetching networkRequest.
         4. If response is not an error, return response.
      5. Otherwise, if item is a RouterSourceCache, then:
         1. Let networkRequest be item's request.
         2. If networkRequest is null, then set networkRequest to request.
         3. Let response be the result of looking for a match in the cache, passing in item's options.
         4. If response is not null, return response.
      6. Otherwise, if item is a RouterSourceFetchEvent, then:
         1. Set routerCallbackId to item's id.
         2. Break RouterLoop.
3. Call the fetch event as usual, but with routerCallbackId as one of the event properties.

Extensibility

I can imagine things like:

• RouterOr(...conditionalItems) – True if any of the conditional items are true.
• RouterNot(condition) – Inverts a condition.
• RouterIfResponse(options) – Right now, a response is returned immediately once one is found. However, the route could continue, skipping sources, but processing conditions. This condition could check the response and break the route if it doesn't match. Along with a way to discard any selected response, you could discard responses that didn't have an ok status.
• RouterCacheResponse(cacheName) – If a response has been found, add it to a cache.
• RouterCloneRequest() – It feels like RouterSourceNetwork would consume requests, so if you need to do additional processing, this could clone the request.

But these could arrive much later. Some of the things in the main proposal may also be considered "v2".

[jakearchibald]

Here's an alternative model suggested by @wanderview, which is focused on allowing developers to toggle the fetch event:

partial interface ServiceWorkerGlobalScope {
  attribute ServiceWorkerEventSubscriptions eventSubscriptions;
}

interface ServiceWorkerEventSubscriptions {
  Array<DOMString> get();
  void add(DOMString eventType);
  void remove(DOMString eventType);
}

Where get returns the set of event types to handle.

add adds to the set.

remove removes from the set.

This means the developer would be able to expire their fetch handler after some amount of time (I believe this is Facebook's use-case).

const deployTimestamp = 1543938484103;
const oneDay = 1000 * 60 * 60 * 24;

addEventListener('fetch', (event) => {
  if (Date.now() - deployTimestamp > oneDay) {
    eventSubscriptions.remove('fetch');
    return;
  }

  // …
});

This is a much simpler feature, but it doesn't allow skipping the service worker by route, or going straight to the cache by route.

[jakearchibald]

One more suggestion (again from @wanderview) would be to add a new CSP rule (or something similar) that defines which url-prefixes a service worker would intercept, if any.

This means the setting would be page-by-page. Navigations wouldn't be able to opt-out of service worker, but presumably navigation preload would solve a lot of performance issues there.

[jakearchibald]

cc @n8schloss, @jatindersmann, @aliams, @youennf, @asutherland

[jakearchibald]

There's a gotcha with the static routes: If you put them at top level, they'll execute without issue, but next time the service worker is started (after activating), they'll fail. Also, if the service worker starts multiple times before activation, you'll get duplicate routes.

Might make more sense to throw unless the routes are being added during the install event, so they'd always fail top-level.

Given that, it might make sense to put the API on the install event. Edit: I've moved it to the install event.

[annevk]

How does this relate to https://github.com/domenic/import-maps? Intuitively it feels like these should be the same thing.

[wanderview]

Just to clarify, the two suggestions I had were to support a subset of the use cases where the site might want to stop consulting the fetch handler until an update can occur. Some sites are blocked on these use cases right now and I wondered if there was a way we could unblock them without conflicting with any future possible static routes spec.

[jakearchibald]

@annevk I think they're different enough, and can be used together.

Import maps URLs with scheme import: to one or more built-in modules or HTTP fetches. Those HTTP fetches then go through the service worker.

Then, the service worker's fetch event (or its static routes) can decide how to conduct those fetches.

[jeffposnick]

But these could arrive much later. Some of the things in the main proposal may also be considered "v2".

I can see developers eagerly adopting this approach if browsers started to implement it. Being able to string together complex routing rules with network/cache interactions without the overhead of starting up a service worker or pulling in extra runtime code (except for the polyfill scenario...) would be a nice performance win.

I have something of a meta-question about the longer term, v2+, plans though. There are some common concerns that production web apps need to think about, like cache maintenance/expiration policies for runtime caches. Today, that type of activity is likely to happen inside of a fetch handler. If we move to a model where you can either get

a) fast routing/response generation, but no fetch handler or
b) the ability to run code to deal with housekeeping, but incur the overhead of a fetch handler

developers might feel conflicted about that tradeoff.

One approach could be to create a new type of event that's fired after a response has been generated by the native router, like routingcomplete, that allows developers to run their own bookkeeping code outside of the critical response generation flow.

Looking at it from a slightly different perspective, this proposal ends up creating a native implementation of a subset of things that have up until now been accomplished using service worker runtime libraries. Cache-expiration also falls into the category of things that developers commonly accomplish via service worker runtime libraries. The feeling I got from #863 is that there hasn't been an appetite for natively implementing that as part of the Cache Storage API. If this routing proposal moves forward, does that change the equation around natively implementing higher-level functionality like cache expiration?

[jakearchibald]

That's a good point. I had this in earlier drafts but didn't think we needed it yet. Should be relatively simple to add.

[n8schloss]

This is awesome! @jakearchibald, I really like the proposal you outlined outlined in the first comment, it solves the two big issues that we're seeing 😃

1. There's a good amount of added overhead we are measuring for fetching user generated non-cached resources when a fetch event is enabled.

2. After a period of time the items cached in the service worker and served via the fetch event are invalid, so when starting the service worker we end up skipping the cache and fetching from the network, however we end up paying a large cost of starting the service worker in that case and get no benefit.

@jakearchibald, @wanderview's solution that you outlined in the second comment above gives us a solution to issue 2 but not issue 1. Like @wanderview said in his comment, issue 2 impacts us more right now than issue 1. So with the first proposal, as long as the spec is written in such a way that vendors can quickly implement the experary time condition without having to block on implementing the RouterIfURL conditions then I think this is really really great!

[jakearchibald]

@n8schloss thanks for the feedback! It feels like IfURL is pretty fundamental to other developers, but the prefix/suffix stuff could probably wait.

@jatindersmann, @aliams, @youennf, @asutherland, @mattto, @wanderview: How do you feel about this implementation-wise?

[aaronsn]

@jakearchibald - This is really great! I like the flexibility of the proposal. I’m interested in understanding better if this will extend to more complex scenarios in the future.

• Would it be possible to have something like RouterIfTimeout which activates that route but cancels if the headers aren't received within a certain timeout? This would allow for preferring the network but falling back to cache if the network takes too long.
• Can this API handle racing sources rather than doing them in sequence?
• I think RouterIfResponse is an important feature, to allow for custom handling for error responses. However, things start to get more complicated with options that apply after the request has been made. For example, say you want to prefer network if the response is a 200, then try cache, then use the server response anyway if cached response is missing. Could you accomplish that with add(new RouterSourceNetwork(), new RouterIfResponse()); add(new RouterSourceCache(), new RouterSourceNetwork()) without it issuing multiple network requests? Or, say you want to do the same thing but if either response isn’t a 200 then use custom logic to determine which to use. Would the fetch event be able to access the responses that were already requested via the routes?
• Avoiding controlling certain sub-scopes is something I'd like to see for service workers, but I'm not sure if this API can/should provide this support. I can sort of see how you could do this with a route like add(new RouterIfURLPrefix(), new RouterSourceNetwork()), but you'd have to know which sub-resources will be requested. Could there be a RouterSource that causes the service worker to not control that client? Eg add(new RouterIfURLPrefix(), new RouterSourceUnclaimClient()). Or would that be too much of an abuse of the API?

Also one thought about implementation - if a request is handled via a static route, would the service worker still be started up in the background even though it isn't needed for the request? On the one hand, this would ensure that the service worker is started for future resource requests. On the other hand, if an entire page load (main resource plus sub resources) can be handled by static routes, then it's nice to avoid the performance cost of starting up the service worker.

[jakearchibald]

@aaronsn

Would it be possible to have something like RouterIfTimeout which activates that route but cancels if the headers aren't received within a certain timeout? This would allow for preferring the network but falling back to cache if the network takes too long.

I think I'd make this an option to RouterSourceNetwork, as it's the only one that would benefit from a timeout right now.

Can this API handle racing sources rather than doing them in sequence?

Something like:

router.get(
  new RouterSourceAny([
    ...sources
  ])
);

This could also have a timeout option.

I think RouterIfResponse is an important feature

I think everything you mention here is possible, although we might be hitting edge cases that are easier to read as logic in a fetch event, and uncommon enough that optimising them via a router doesn't bring much of a benefit.

Avoiding controlling certain sub-scopes is something I'd like to see for service workers, but I'm not sure if this API can/should provide this support

I don't think it should. Only things you can currently do in a fetch event are in scope. To do this, we either want a way to exclude routes as part of the call to serviceWorker.register, or a way to do it via a fetch event (after which we could look at adding something to the router).

if a request is handled via a static route, would the service worker still be started up in the background even though it isn't needed for the request? On the one hand, this would ensure that the service worker is started for future resource requests. On the other hand, if an entire page load (main resource plus sub resources) can be handled by static routes, then it's nice to avoid the performance cost of starting up the service worker.

Interesting! The spec is deliberately loose when it comes to when the service worker is started, and how long it stays alive for. Either behaviour would be spec compatible.

If a service worker is started, and not needed, it shouldn't affect page performance as nothing's blocked on it.

[wanderview]

It might also be useful to describe the default routes you get when a service worker is installed. Either route to FetchEvent or no where depending on if there is a fetch handler.

@wanderview: How do you feel about this implementation-wise?

Personal opinion that does not represent any actual implementation priorities:

I guess I'm most interested in how something like this could be incrementally approached. For example, if we started with:

1. ServiceWorkerRouter.add()
2. RouterIfDate
3. RouterSourceNetwork with default options

Or replace (2) with RouterIfURL. This minimal initial set might unblock certain use cases. We could then layer additional items later. I'm not sure how people would feel about a having a partial "router" in the platform that doesn't provide a full routing capability.

To me the RouterIfDate is more compelling at the moment because we don't have a good alternative solution for avoiding service worker startup costs in an expired state. RouterIfURL is more "router-like" but it seems oriented at carving out exceptions for certain subresources which feels like less of a problem since typically service worker startup is not necessary for subresources.

I imagine, though, there is going to be a tension between how many of these options and extensions to implement vs using javascript in FetchEvent. For example, the list of logical combinations in the "extensibility" section seemed like perhaps something that should just be done in js. I'm not sure we want to implement and maintain a complex DSL when we can achieve the same thing with js.

[jakearchibald]

One thing that isn't clear to me yet:

router.add(
  new RouterIfURLPrefix('/avatars/'),
  new RouterSourceCache(),
);

router.add(
  new RouterIfURLSuffix('.jpg'),
  new RouterSourceNetwork(),
);

If /avatars/foo.jpg is requested, but it isn't in the cache, what happens? Does the request fall through to the next route?

[jakearchibald]

Having slept on it, I think it's important that a single route is selected based on conditions. I'll work on a new draft that uses a router.add(conditions, sources) pattern.

This matches other routers like Express, where "continue to other routes" is opt-in.

[jakearchibald]

Ok, here's a second draft:

Creating a route

WebIDL

// Install currently uses a plain ExtendableEvent, so we'd need something specific
partial interface ServiceWorkerInstallEvent {
  attribute ServiceWorkerRouter router;
}

[Exposed=ServiceWorker]
interface ServiceWorkerRouter {
  void add(
    (RouterCondition or sequence<RouterCondition>) conditions,
    (RouterSource or sequence<RouterSource>) sources,
  );
}

[Exposed=ServiceWorker]
interface RouterSource {}

[Exposed=ServiceWorker]
interface RouterCondition {}

JavaScript

addEventListener('install', (event) => {
  event.router.add(conditions, sources);
  event.router.add(otherConditions, otherSources);
});

The browser will consider routes in the order declared, and if all conditions match, each source will be tried in turn.

Conditions

These determine if a particular static route should be used rather than dispatching a fetch event.

WebIDL

[Exposed=ServiceWorker, Constructor(ByteString method)]
interface RouterIfMethod : RouterCondition {}

[Exposed=ServiceWorker, Constructor(USVString url, optional RouterIfURLOptions options)]
interface RouterIfURL : RouterCondition {}

dictionary RouterIfURLOptions {
  boolean ignoreSearch = false;
}

[Exposed=ServiceWorker, Constructor(USVString url)]
interface RouterIfURLStarts : RouterCondition {}

[Exposed=ServiceWorker, Constructor(USVString url, optional RouterIfURLOptions options)]
interface RouterIfURLEnds : RouterCondition {}

[Exposed=ServiceWorker, Constructor(optional RouterIfDateOptions options)]
interface RouterIfDate : RouterCondition {}

dictionary RouterIfDateOptions {
  // These should accept Date objects too, but I'm not sure how to do that in WebIDL.
  unsigned long long from = 0;
  // I think Infinity is an invalid value here, but you get the point.
  unsigned long long to = Infinity;
}

[Exposed=ServiceWorker, Constructor(optional RouterIfRequestOptions options)]
interface RouterIfRequest : RouterCondition {}

dictionary RouterIfRequestOptions {
  RequestDestination destination;
  RequestMode mode;
  RequestCredentials credentials;
  RequestCache cache;
  RequestRedirect redirect;
}

Again, these interfaces don't have attributes, but they could reflect the options/defaults passed into the constructor.

Sources

These determine where the route should try to get a response from.

WebIDL

[Exposed=ServiceWorker, Constructor(optional RouterSourceNetworkOptions options)]
interface RouterSourceNetwork : RouterSource {}

dictionary RouterSourceNetworkOptions {
  // A specific request can be provided, otherwise the current request is used.
  Request request;
  // Reject responses that do not have an ok status.
  boolean requireOkStatus;
}

[Exposed=ServiceWorker, Constructor(optional RouterSourceCacheOptions options)]
interface RouterSourceCache : RouterSource {}

RouterSourceCacheOptions : MultiCacheQueryOptions {
  // A specific request can be provided, otherwise the current request is used.
  Request request;
}

[Exposed=ServiceWorker, Constructor(optional RouterSourceFetchEventOptions options)]
interface RouterSourceFetchEvent : RouterSource {}

dictionary RouterSourceFetchEventOptions {
  DOMString id = '';
}

These interfaces don't currently have attributes, but they could have attributes that reflect the options/defaults passed into the constructor.

Shortcuts

GET requests are the most common type of request to provide specific routing for.

WebIDL

partial interface ServiceWorkerRouter {
  void get(/* same as add */);
}

Where the JavaScript implementation is roughly:

router.get = function(conditions, sources) {
  if (conditions instanceof RouterCondition) {
    conditions = [conditions];
  }
  router.add([new RouterIfMethod('GET'), ...conditions], sources);
};

We may also consider treating strings as URL matchers.

• router.add('/foo/', sources) === router.add(new RouterIfURL('/foo/'), sources).
• router.add('/foo/*', sources) === router.add(new RouterIfURLStarts('/foo/'), sources).
• router.add('*.png', sources) === router.add(new RouterIfURLEnds('.png'), sources).

Examples

Bypassing the service worker for particular resources

JavaScript

// Go straight to the network after 25 hrs.
router.add(
  new RouterIfDate({ from: Date.now() + 1000 * 60 * 60 * 25 }),
  new RouterSourceNetwork(),
);

// Go straight to the network for all same-origin URLs starting '/videos/'.
router.add(
  new RouterIfURLStarts('/videos/'),
  new RouterSourceNetwork(),
);

Offline-first

JavaScript

router.get(
  // If the URL is same-origin and starts '/avatars/'.
  new RouterIfURLStarts('/avatars/'),
  [
    // Try to get a match for the request from the cache.
    new RouterSourceCache(),
    // Otherwise, try to fetch the request from the network.
    new RouterSourceNetwork(),
    // Otherwise, try to get a match for the request from the cache for '/avatars/fallback.png'.
    new RouterSourceCache({ request: '/avatars/fallback.png' }),
  ],
);

Online-first

JavaScript

router.get(
  // If the URL is same-origin and starts '/articles/'.
  new RouterIfURLStarts('/articles/'),
  [
    // Try to fetch the request from the network.
    new RouterSourceNetwork(),
    // Otherwise, try to match the request in the cache.
    new RouterSourceCache(),
    // Otherwise, try to match '/articles/offline' in the cache.
    new RouterSourceCache({ request: '/articles/offline' }),
  ],
);

Processing

This is very rough prose, but hopefully it explains the order of things.

A service worker has routes. The routes do not belong to the registration, so a new empty service worker will have no defined routes, even if the previous service worker defined many.

A route has conditions and sources.

To create a new route containing conditions and sources

1. If the service worker is not "installing", throw. Routes must be created before the service worker has installed.
2. Create a new route with conditions and sources, and append it to routes.

Handling a fetch

These steps will come before handling navigation preload, meaning no preload will be made if a route handles the request.

request is the request being made.

1. RouterLoop: For each route of this service worker's routes:
   1. For each condition of route's conditions:
      1. If condition is a RouterIfMethod, then:
         1. If condition's method does not equal request's method, then continue RouterLoop.
      2. Otherwise, if condition is a RouterIfURL, then:
         1. If condition's url does not equal request's url, then continue RouterLoop.
      3. Etc etc for other conditions.
   2. For each source of route's sources:
      1. If source is a RouterSourceNetwork, then:
         1. Let networkRequest be source's request.
         2. If networkRequest is null, then set networkRequest to request.
         3. Let response be the result of fetching networkRequest.
         4. If response is not an error, return response.
      2. Otherwise, if source is a RouterSourceCache, then:
         1. Let networkRequest be source's request.
         2. If networkRequest is null, then set networkRequest to request.
         3. Let response be the result of looking for a match in the cache, passing in source's options.
         4. If response is not null, return response.
      3. Otherwise, if source is a RouterSourceFetchEvent, then:
         1. Set routerCallbackId to source's id.
         2. Call the fetch event as usual, but with source's id as one of the event properties.
         3. Return.
   3. Return a network error.
2. Call the fetch event as usual.

Extensibility

I can imagine things like:

• RouterOr(...conditionalItems) – True if any of the conditional items are true.
• RouterNot(condition) – Inverts a condition.
• RouterFilterResponse(options) – Right now, a response is returned immediately once one is found. However, the route could continue, skipping sources, but processing filters. This could check the response and discard it if it doesn't match. An example would be discarding responses that don't have an ok status.
• RouterCacheResponse(cacheName) – If a response has been found, add it to a cache.

But these could arrive much later. Some of the things in the main proposal may also be considered "v2".

[jeffposnick]

Here's a couple of things that, based on what we've run into with Workbox's routing, tend to crop up in the real-world. I wanted to draw them to your attention so that you could think about them earlier rather than later.

Cross-origin routing

Developers sometimes want to route cross-origin requests. And sometimes they don't. Coming up with a syntax for conditions that supports both scenarios can be difficult. It looks like the current proposal assumes that the URL prefix/suffix will only work for same-origin requests, so be prepared for folks asking for a cross-origin syntax at some point.

Workbox has a few different ways of specifying routing conditions, but the most common is RegExp-based, and we settled on the following behavior: if the RegExp matches the full URL starting with the first character [the 'h' in 'https://...'] then we assume that it can match cross-origin requests. If the RegExp matches, but the match starts on anything other than the first character in the full URL, then it will only trigger if it's a same-origin request.

I would imagine folks wanting to see at least a RouterIfURLOrigin condition that they could use to customize this behavior, and that might need to support wildcards to deal with CDN origins that don't have fixed naming conventions.

Non-200 OK responses sometimes need to be treated as errors

The proposal for RouterSourceNetwork currently reads If response is not an error, return response. I think some developers are going to find this too limiting. There are folks who will end up needing different behavior based on the status of the response, not just whether a NetworkError occurred.

I'm not sure what the cleanest approach is there in terms of your proposal—maybe adding in a way of setting a list of "acceptable" status codes, including letting folks opt-in or opt-out to status code 0 from an opaque response being considered a success?

[jakearchibald]

I've renamed prefix/suffix to starts/ends to match str.startsWith.

[jakearchibald]

I wrote https://jakearchibald.com/2019/service-worker-declarative-router/ to seek wider feedback.

Also see my tweet about it for replies.

[jakearchibald]

@jeffposnick

Developers sometimes want to route cross-origin requests. And sometimes they don't.

router.add(
  [
    new RouterIfURLStarts('https://photos.example.com/'),
    new RouterIfURLEnds('.jpg', { ignoreSearch: true }),
  ]
  // …
);

The above would match on URLs that start https://photos.example.com/ and end .jpg. It gets trickier if you want to match URLs to all other origins that end .jpg, you'd need RouterNot for that.

RegExp is tricky here as there's no spec for how it could outlive JavaScript. We could try and standardise globbing, but I worry that would take a big chunk of time.

Non-200 OK responses sometimes need to be treated as errors

I've added requireOkStatus as an option.

[nhoizey]

Hi @jakearchibald, this is really interesting!

Reading the post on your blog, I wondered how I could add the RouterSourceNetwork to the cache, because it looks like the fetch event would not be fired. But I see here that you suggest adding RouterCacheResponse to a future "v2" version.

IMHO, this would be great in the "v1" to help build offline experiences without manually preloading.

But I understand there must be some priorities. 😅

[jakearchibald]

Yeah, we want to avoid trying to do everything at once. In terms of fetching and caching, see https://jakearchibald.com/2019/service-worker-declarative-router/#routersourcefetchevent.

[WORMSS]

This is not going to be a popular view. but I am not a massive fan of the 'string' shortcut of the conditions. I understand you want it to be as close to existing apis like express.. but not everyone understands express all the time.
I was really enjoying the very explicit class approach and the everything in the conditional list must eval to true, rather than this or thistoo or that.

I wondered if there could be some optimisation for that, behind the scenes. Since they all have to be true to be considered true, then the order they are executed could be arbitrary? (I am of course assuming they are all stateless operations).

So ones that are a little more expensive to calculate could be done last, and super dirty cheap ones could be done first? Rather than do them in the order the service worker coder wrote them in ?? That way different browsers can optimise in their own way and what might be expensive for one is cheap for the others, and the developer doesn't need to know this when setting up the conditions.

For example,

router.add(
  [
    new RouterIfURLMatchesRegex('lets assume some horrible regex'),
    new RouterIfMethod('GET'),
  ]
  // …
);

I know RouterIfURLMatchesRegex doesn't exist, but you have to believe that it would be far less expensive to do the method condition before a possible addition to the conditions in the future. Rather than try and resolve the regex, just to find it was a POST request anyway, so didn't even need it.

[jakearchibald]

@WORMSS

I am not a massive fan of the 'string' shortcut

I see what you're saying about the string thing. Also, it isn't all that similar to express. It might be better to drop that shortcut until something like globbing can be properly spec'd.

I wondered if there could be some optimisation for that, behind the scenes. Since they all have to be true to be considered true, then the order they are executed could be arbitrary?

In the case of multiple conditions, I'd spec them as a sequence, but if a browser decided to check them in a different order, or in parallel, there'd be no observable difference.

[jakearchibald]

Some feedback I've received offline:

The string shortcut might lead developers to think something like /articles/*.jpg would 'work'. Also, it wouldn't be backwards compatible to add support for that later. It might be better to hold off on that shortcut, and later explore standardising globbing, which would be useful in other parts of the platform.

When conditions and sources are sequences, conditions is an 'and', whereas sources is more like an 'or'. This is weird. It might be better to drop sequences here in favour of explicit grouping like RouterIfAny(...conditions), RouterIfAll(...conditions), RouterSourceFirst(...sources), RouterSourceRace(...sources) etc etc.

[yoavweiss]

My feedback:

• I've seen cases where the request destination would have been extremely helpful as a filter.
• I found the use of get() as a shortcut for GET requests confusing as first, as I expected it to be a way to get the set routes or something similar. Maybe it's just me, but if not, might be worthwhile to rename.

[shortercode]

Really liking the concept of avoiding the overhead of starting the ServiceWorker where possible, but I feel like not adding an additional Source type that utilises a callback is a missed opportunity. It would effectively be the same behaviour as the fetch event fallback, but specific to the given conditions. For all other source types we would still avoid starting up the service worker.

Also I think syntax wise I feel it reads better to have static methods on a RouterCondition/RouterSource class that instantiate the specific type. Although I'm not sure how in keeping that is with other web APIs.

const { router } = e;

router.get(
  RouterCondition.startsWith("/avatars/"),
  [ RouterSource.cache(), RouterSource.network() ]
);

router.get(
  RouterCondition.fileExtension(".mp4"),
  RouterSource.network()
);

router.get(
  RouterCondition.URL("/"),
  RouterSource.cache("/shell.html")
);

router.add(
  RouterCondition.any(),
  RouterSource.custom(fetchEvent => {
    
  })
);

[tomayac]

@yoavweiss: I think the .get() is heavily inspired by Express.js’ routing: http://expressjs.com/en/guide/routing.html.

[domenic]

I apologize that this is not a very substantial contribution, but I think I'm having an allergic reaction to the "Java-esque" (or Dart-esque) nested class constructors. I'd encourage thinking about what this design would look like if done in the opposite direction: a purely JSON format. For example, something like

router.route([
  {
    condition: {
      method: "GET",
      urlStartsWith: "/avatars"
    },
    destination: ["cache", "network"]
  },
  {
    condition: {
      method: "GET",
      urlPathEndsWith: ".mp4"
    }
    destination: ["network"]
  },
  {
    condition: {
      method: "GET",
      urlPath: "/"
    },
    destination: [{ cache: "/shell.html" }]
  }
]);

I don't think this extreme is right either (in particular the [{ cache: "/shell.html" }] seems weird and underdeveloped) but I think it'd be a valuable exercise to see what it would look like to have a purely declarative routing format expressed just in JS objects. Then, you could figure out where it would make sense to strategically convert some POJSOs into class instances, versus where the POJSOs are simpler.

Another way to think about this is, when are these types ever useful? If they're only ever consumed by the router system, and not manipulated, composed, or accessed by users, then perhaps it's better to just pass the data they encapsulate directly to the system. (That data is roughly a "type tag" plus their constructor arguments.)

[mgiuca]

@WORMSS Yeah it's called that in the URL object too. It's called query inside the spec language but now that I re-read it, I realised that the word "query" never appears in the API interface itself. So "search" is fine.

[fallaciousreasoning]

Curious whether we could use @wanderview's URLPattern proposal instead of startsWith/endsWith.

[yoshisatoyanagisawa]

FYI, we are about to implement a subset of the API.
https://github.com/yoshisatoyanagisawa/service-worker-static-routing-api

[webmaxru]

Thanks for your work!
FYI: I tweeted about it to get the developer community ready for experimenting :)
https://twitter.com/webmaxru/status/1664214464999182338

[sisidovski]

Should we allow registerRouter() to be called multiple times?

As commented in #1373 (comment), we are about to implement a subset, as ServiceWorker Static Routing API.
https://github.com/yoshisatoyanagisawa/service-worker-static-routing-api

We (Google Chrome team) are ready for the Origin Trial, it's available from M116. In the meantime, we'd like to hear opinions around how InstallEvent.registerRouter() should work in this API.

Unlike add() or get() in the original Jake's proposal, registerRouter() sets all the routes at once, and it can only be called once. This is for ease of understanding the latest routes. However, we may rethink this limitation because we saw some interests in using this API from 3rd parties. 3rd parties in this context mean the SW scripts which are served from cross origins, and imported by the main SW script via importScripts() or ES modules.

Like Speed Kit, some companies provide ServiceWorker scripts as SDK, and customer websites use their scripts via importScripts(). If both a 3rd party provider and a customer want to use registerRouter(), in our current implementation the browser only registers the routing info called for the first time, and throws type error for the second registerRouter() call.

I personally feel it makes sense that 3rd parties use registerRouter() to handle something, but do you think we should support multiple registerRouter() calls? If so, a naive approach is just adding a new routing rule into existing rules, but do we need additional mechanisms or API surfaces to manage registered routes more smartly?

cc: @ErikWitt

[domenic]

I think allowing multiple calls is fine. In that case I would rename the method from registerRouter() to registerRoutes().

The naive approach makes perfect sense to me. In other words, registerRoutes(a); registerRoutes(b); should be equivalent to registerRoutes([...a, ...b]);. I think that is what everyone would expect.

[jakearchibald]

Given that, is there a benefit to registerRoutes vs calling registerRoute as many times as needed? One route at a time would make it easier to tell which route caused a throw.

[ErikWitt]

Heythere, sorry for being so late to the discussion. My two cents:

1. Calling registerRoute from multiple handlers would be awesome to make combining multiple Service Workers easier. We have that case from time to time and it can be a complex task.

2. Calling registerRoutes outside the install event is off the table right? It's not a deal breaker but would have been convenient. At the moment out Service Worker loads a configuration from indexeddb (the service worker is generic, every customer has a unique config), so it would be convenient to configure the router outside the install event. That said, we are looking into inlining the config in our shipped service worker for every customer which would resolve that issue for us.

3. Would it be possible to make the origin trial a 3rd party origin trial? I.e. could send the origin trial token in the 3rd party script imported into the same origin service worker? If it is not a third part origin trial, our customers would need to implement the header themselves which is a lengthy process for large e-commerce companies.

btw. I love the use of URLPattern in the api :) Looking forward to trying this in production and reporting back on the performance gains

[sisidovski]

Thank you all for the feedback!

@jakearchibald That's a fair point. WDYT @yoshisatoyanagisawa ?

@ErikWitt

Calling registerRoutes outside the install event is off the table right?

At the moment we want to keep it inside the install event. We don't want to make the API dynamically configurable. If you have a workaround please consider doing so.

Would it be possible to make the origin trial a 3rd party origin trial?

OriginTrial for ServiceWorker features is running on a bit different mechanism, and unfortunately 3rd party origin trial is not supported yet. We understood the use case from 3rd party context. Let us consider how to deal with it, but please expect it takes some time to solve.

[sisidovski]

I came up with this scenario, perhaps it can be a problem for some cases.

a.com/sw.js:

importScripts('b.com/imported-sw.js');

addEventListener('install', (event) => {
  event.registerRouter({
    condition: {
     // example.jpg is returned form the fetch handler.
      urlPattern: {pathname: "example.jpg"}
    },
    source: "fetch-event"
  });
})

b.com/imported-sw.js:

addEventListener('install', (event) => {
  event.registerRouter({
    condition: {
     // All jpg resources are fetched from the network.
      urlPattern: {pathname: "*.jpg"}
    },
    source: "network"
  });
})

registerRouter() in imported-sw.js is executed first, and then the one in sw.js is executed. In the current algorithm, the API simply has the list of registered routing info and try to evaluate from the first item. So any requests to jpg resources, including example.jpg are matched with {pathname: "*.jpg"} which is registered in the imported-sw.js and the routing info registered by the main sw.js is never used.

Do you think the API should have a mechanism to address this case which is introduced by allowing multiple registerRoutes or registerRoute calls? I think this is kind of a WAI behavior, but love to hear anyone's thoughts around it.

[domenic]

I think this behavior is WAI. If you call importScripts() on a script before your own install handler, you are giving that script priority to install routes.

If you would like to have your own code take priority, then you should rearrange sw.js like so:

addEventListener('install', (event) => {
  event.registerRouter(/* ... */);
});

importScripts('b.com/imported-sw.js');

[jakearchibald]

It could be addRoutes(...routes).

I usually don't like that pattern, since it prevents adding new parameters in future, but if seems like any options would be per route.

add is shorter than register, and add makes it clearer that it's additive to the previous call. It might not make it clear that previous routes are cleared with the installation of a new service worker, but I'm not sure register makes that clear either.

[yoshisatoyanagisawa]

Sorry for being late to the discussion.
When I wrote the explainer, I did not consider 3rd party SW providers to use the API. However, considering the use case, it makes more sense to allow the routes updated multiple times.

For the naming of the API,
I remember that we called it register instead of add to clarify it is a write once operation. Note that Jake's original proposal called it add and allowed to call it multiple times. If we allow it to be called multiple times, I prefer to rename it add or append instead, which sounds more like the rules can grow.

By the way, I have concerns on calling the API multiple times,

1. Is the order of the API call guaranteed? The current registerRouter is an asynchronous API. When the multiple rule has been installed via the API, how should the order of the routes be? I am not so much familiar with JavaScript, but if JavaScript may not wait for the previous API call, then the order of rules may flip?
2. I also have the same concern as @sisidovski. If somebody sets a rule that has an intersection with other rules, it may interfere unexpectedly. Moreover, if somebody sets a rule like:

{
  condition: {}, // i.e. matches everything.
  source: "fetch-handler"
}

all rule updates after the rule would be just ignored. I guess it would be fine to be WAI if the order of rules is guaranteed.

Calling registerRoutes outside the install event

It is intended to make the rule updated only inside the install event.
We were concerned about the difficulty of understanding rule applications to inflight requests.

Would it be possible to make the origin trial a 3rd party origin trial?

FYI, https://bugs.chromium.org/p/chromium/issues/detail?id=1471005

is there a benefit to registerRoutes vs calling registerRoute as many times as needed?

Current API accepts both sequence of routes and a route. I suppose you suggest only accepting a route.
I feel it is fine to allow both ways of writing a rule. i.e. sequence of rules and a rule. For those who want to understand an error, they can write a rule one by one. For those who want to write a rule at once, it is also allowed.

Will you elaborate more on why you want to prohibit accepting a sequence of rules?

[yoshisatoyanagisawa]

Just FYI, the issues on the static routing API have been filed in https://github.com/WICG/service-worker-static-routing-api/issues. Please take a look.
Also, I created an issue for "Should we allow registerRouter() to be called multiple times?" as WICG/service-worker-static-routing-api#10 for ease of focusing on this discussion.

[yoshisatoyanagisawa]

FYI, we are discussing on the way to handle an empty "not" condition in WICG/service-worker-static-routing-api#22. We are leaning on raising. Please feel free to share your opinions.

[quasi-mod]

FYI, we are currently actively discussing extending the resource timing API to support static routing API.
w3c/resource-timing#389