add createLayout api + multiple layouts + layout queries #1503

craigmulligan · 2017-07-14T15:20:10Z

connects to #1494 && #1255
After discussions with @jbolda, we thought it'd be a good idea to add layouts to the redux store and give em their own api.

This PR adds:

layouts to redux store.
createLayout, deleteLayout actions
createLayouts onCreateLayout node api hooks
component-layout-creator internal plugin to auto create all layouts in /src/layouts.
Ability to add graphql queries to layouts

It doesn't add:

Nested layouts

Behaviour:
If no layouts in src/layouts and pages don't have a page.layout attribute then no layout will be loaded.
If src/layouts/index.js exists then it will be loaded for all pages that don't have a page.layout attribute.
If you create a layout with createLayout and you assign it when creating a page(see example) then it'll load the specified layout.

KyleAMathews · 2017-07-14T15:20:50Z

Deploy preview failed.

Built with commit f520828

https://app.netlify.com/sites/using-styled-components/deploys/597b1461cf321c5521a3a4df

gatsbybot · 2017-07-14T15:26:01Z

Deploy preview failed.

Built with commit f520828

https://app.netlify.com/sites/gatsbyjs/deploys/597b1460cf321c5521a3a4c7

gatsbybot · 2017-07-14T15:26:29Z

Deploy preview ready!

Built with commit 08bd0f3

https://deploy-preview-1503--using-drupal.netlify.com

gatsbybot · 2017-07-14T15:28:15Z

Deploy preview ready!

Built with commit 08bd0f3

https://deploy-preview-1503--gatsbygram.netlify.com

KyleAMathews · 2017-07-14T15:55:52Z

This looks great — this is definitely the next step to get working.

I haven't read through the code in detail but I like the idea of copying the page patterns for layouts. I hadn't really thought about manually adding layouts but yeah, we definitely should expose that capability for more advanced use cases (like say Landr ;-))

craigmulligan · 2017-07-14T16:15:12Z

Okay, but we should change the model slightly? or do you think it's fine to treat them exactly the same?

As I mentioned layout.path doesn't make a lot of sense as an id, but it does make sharing functions between the layouts and pages easier haha.

dardub · 2017-07-14T16:26:04Z

packages/gatsby/src/bin/cli.js

@@ -65,7 +65,7 @@ program
  )
  .action(command => {
    // Set NODE_ENV to 'production'
-    process.env.NODE_ENV = `production`
+    // process.env.NODE_ENV = `production`


Should this stay commented?

Nope I was just testing something, thanks for pointing it out 👍

dardub · 2017-07-14T16:29:59Z

packages/gatsby/src/redux/actions.js

@@ -1,10 +1,12 @@
 // @flow
 import Joi from "joi"
 import chalk from "chalk"
-const _ = require(`lodash`)
+const _ = require(`lodash`
+)


Extra return

KyleAMathews · 2017-07-14T16:58:30Z

We should treat them the same in that they're both first class objects in Gatsby-land but otherwise, they're different. Layouts aren't tied to paths so yeah, that doesn't make any sense.

KyleAMathews · 2017-07-14T17:00:18Z

packages/gatsby/src/cache-dir/production-app.js

@@ -18,7 +18,9 @@ window.___emitter = emitter
 import pages from "./pages.json"
 import ComponentRenderer from "./component-renderer"
 import asyncRequires from "./async-requires"
+import syncRequires from "./sync-requires"


sync-requires is just for development. It creates one giant bundle which is fine for dev but not for prod. asyncRequires has all the layout bundles.

craigmulligan · 2017-07-16T16:15:13Z

packages/gatsby/src/cache-dir/loader.js

@@ -67,7 +68,7 @@ const fetchResource = (resourceName, cb = () => {}) => {
    // Find resource
    const resourceFunction =
      resourceName.slice(0, 6) === `page-c`
-        ? asyncRequires.components[resourceName]
+        ? asyncRequires.components[resourceName] || asyncRequires.layouts[resourceName]


Would there ever be a clash between page chunkNames and layout chunk names? If so we should name them differently.

Layout chunks are at async.layouts

asyncRequires I meant

Hmm nvm, on phone so not entirely sure what this code is doing.

But in any case, no, the two component names are created differently.

Yea, it's created from their filename so we should be good using the same naming func.

KyleAMathews · 2017-07-16T16:27:10Z

Deploy preview failed.

Built with commit 0317f6d

https://app.netlify.com/sites/using-glamor/deploys/5980f6eda700c45c623def12

KyleAMathews · 2017-07-16T16:27:14Z

Deploy preview failed.

Built with commit f520828

https://app.netlify.com/sites/using-postcss-sass/deploys/597b1461cf321c5521a3a4d3

craigmulligan · 2017-07-16T16:55:34Z

Okay I think it's mostly there. @KyleAMathews could you give it a review?

KyleAMathews

Won't have computer time until tomorrow at the earliest but left some notes on my phone.

KyleAMathews · 2017-07-16T17:34:01Z

packages/gatsby/src/internal-plugins/component-layout-creator/gatsby-node.js

+    .on(`unlink`, path => {
+      // Delete the page for the now deleted component.
+      store.getState().layouts.filter(p => p.component === path).forEach(layout => {
+        deletePage({ name: layout.name })


Looks like some copied code needs updated

KyleAMathews · 2017-07-16T17:35:35Z

packages/gatsby/src/joi-schemas/joi.js

+export const layoutSchema = Joi.object()
+  .keys({
+    name: Joi.string().required(),
+    matchPath: Joi.string(),


This is extra

KyleAMathews · 2017-07-16T17:35:58Z

packages/gatsby/src/joi-schemas/joi.js

+    matchPath: Joi.string(),
+    component: Joi.string().required(),
+    componentChunkName: Joi.string().required(),
+    context: Joi.object(),


KyleAMathews · 2017-07-16T17:37:50Z

packages/gatsby/src/redux/actions.js

+ * @param {string} page.component The absolute path to the component for this layout
+ * @example
+ * createLayout({
+ *   path: `myNewLayout`,


Example outdated.

Also, if you're using name as an id, perhaps we should just call it an id.

KyleAMathews · 2017-07-16T17:39:43Z

packages/gatsby/src/redux/actions.js

+  // TODO: We currently don't support context for layouts but left here
+  // as will implement soon.
+  // Ensure the layout has a context object
+  if (!layout.context) {


Ah I see. So you are thinking of supporting context on layouts.

I've removed it for this PR it's easy enough to add in the next PR

craigmulligan · 2017-07-17T11:20:00Z

packages/gatsby/src/cache-dir/component-renderer.js

-        ...this.props,
-        ...this.state.pageResources.json,
-      })
+      return createElement(


@KyleAMathews can you sanity check here, I've moved the layout rendering to the component-renderer, which makes things neater, but I want to make sure I didn't overlook anything when making the move.

Is it cleaner to put layouts in here? I explicitly pulled page rendering into here to separate it from layout rendering as there's some tricky logic around when/how to render pages. What's your reasoning?

so layouts can now change on route changes, so they need to be handled within the pageResources state, I we could create another stateful component but IMO this cleaner.

Ok, we can roll with this for now.

This stuff needs refactored anyways so... :-)

This doesn't work as it turns out because it means layouts are re-rendered on every page change which breaks the contract that they're stateful which breaks among other things gatsbygram.

Also it's less performant.

Thinking about alternatives...

craigmulligan · 2017-07-19T18:41:03Z

@KyleAMathews hold off on reviewing for a little while, I'm close to finishing off changes for supporting queries in layouts so I think we can just merge the two PRs into one since they touch on mostly the same code.

craigmulligan · 2017-07-19T19:36:19Z

I've updated this PR to include layout queries, as well as refactoring some of the redux model, everything except query watching seems to be working. I'll be back tomorrow to try finish it off, but if anyone wants to review in the meantime 🥇.

craigmulligan · 2017-07-19T19:46:05Z

packages/gatsby/src/cache-dir/static-entry.js

+  )
+  let layout
+  if (syncRequires.layouts[page.layoutComponentChunkName]) {
+    layout = syncRequires.layouts[page.layoutComponentChunkName]


Note to self static entry isn't loading json

craigmulligan · 2017-07-19T19:47:02Z

packages/gatsby/src/redux/reducers/components.js

      action.payload.componentPath = normalize(action.payload.componentPath)
+      // console.log(state)


craigmulligan · 2017-07-19T19:47:26Z

packages/gatsby/src/redux/index.js

@@ -55,6 +55,7 @@ const saveState = _.debounce(state => {

 store.subscribe(() => {
  const lastAction = store.getState().lastAction
+  // console.log(lastAction.type)


KyleAMathews · 2017-07-19T19:47:49Z

Deploy preview failed.

Built with commit f520828

https://app.netlify.com/sites/using-remark/deploys/597b1461cf321c5521a3a4e5

craigmulligan · 2017-07-19T19:50:21Z

packages/gatsby/src/utils/api-node-docs.js

+exports.onCreateLayout = true
+
+/**
+ * Called when a new layout is created. This extension API is useful


KyleAMathews

Ugh, made a bunch of review comments on my phone a few days ago that didn't get submitted.

KyleAMathews · 2017-07-18T20:33:34Z

packages/gatsby/src/redux/actions.js

+/**
+ * Create a layout.
+ * @param {Object} layout a layout object
+ * @param {string} page.id Unique id for layout


little thing but this should be layout.id not page.id

KyleAMathews · 2017-07-18T20:39:41Z

examples/multi-layout/src/html.js

@@ -0,0 +1,44 @@
+import React from "react"


html.js isn't required anymore. I think generally people shouldn't have their own html.js and just use the default as it's just more code to understand. Instead people should use plugins and things like gatsby-plugin-react-helmet to manage stuff in their <head>. So if this is just a standard html.js file, please remove it.

KyleAMathews · 2017-07-18T20:40:30Z

examples/multi-layout/gatsby-node.js

+exports.onCreateLayout = ({ layout, boundActionCreators }) => {
+  // Note you may see this logged more times than you'd expect
+  // that's because gatsby automatically creates a layout for each file in /src/layouts/
+  console.log('Whooop! A new layout was created!')


perhaps log out the layout.id as well

KyleAMathews · 2017-07-18T20:41:40Z

examples/multi-layout/gatsby-node.js

@@ -0,0 +1,23 @@
+const path = require('path')
+// Example if you need to programmatically create a layout


add "i.e. the layout component is not in src/layouts/"

KyleAMathews · 2017-07-18T20:48:29Z

packages/gatsby/src/redux/actions.js

@@ -57,6 +58,15 @@ actions.createPage = (page, plugin = ``, traceId) => {
    internalComponentName = `ComponentIndex`
  }

+  // if no layout is set we check if `/src/layouts/index`


Make comments sentences so capitalize the first letter + end with a period.

Also "if it" on second line is extra.

KyleAMathews · 2017-07-18T20:53:17Z

packages/gatsby/src/cache-dir/component-renderer.js

-        ...this.props,
-        ...this.state.pageResources.json,
-      })
+      return createElement(


Is it cleaner to put layouts in here? I explicitly pulled page rendering into here to separate it from layout rendering as there's some tricky logic around when/how to render pages. What's your reasoning?

KyleAMathews · 2017-07-25T23:01:34Z

packages/gatsby/src/internal-plugins/query-runner/query-watcher.js

@@ -19,13 +19,14 @@ const invariant = require(`invariant`)
 const normalize = require(`normalize-path`)

 exports.extractQueries = () => {
-  const pages = store.getState().pages
-  const components = _.uniq(pages.map(p => p.component))
+  // TODO We can just grab the components straight from store here?


👍 yup, the store is always up-to-date.

KyleAMathews · 2017-07-25T23:03:25Z

packages/gatsby/src/redux/actions.js

+ * @example
+ * createLayout({
+ *   id: `myNewLayout`,
+ *   component: path.resolve('./src/templates/myNewLayout.js`)


mis-matched quote types. Just use a back tick since the default in Gatsby.

jbolda · 2017-08-08T00:50:41Z

packages/gatsby/src/redux/actions.js

@@ -48,7 +49,7 @@ const pascalCase = _.flow(_.camelCase, _.upperFirst)
 * })


It doesn't let me comment above, but the following seen above doesn't have matching quotes:
component: path.resolve('./src/templates/my-sweet-new-page.js`),

KyleAMathews · 2017-08-08T00:55:36Z

Hey Craig, I started working on this locally earlier. Almost got it ready to land. Main missing thing is hot reloading for layout queries. Willh changes in morning and merge. Thanks for all your hard work! This is a huge leap forward.

jbolda · 2017-08-08T01:30:30Z

@craig-mulligan I figured it out. I thought it might be a misunderstanding on my end, hah. It would be nice to compose / nest layouts, but this is great even as is.

I have using-javascript-transforms updated and working with this 💯 . @KyleAMathews, I am assuming that it makes sense just to do a separate PR after this lands?

craigmulligan · 2017-08-08T09:15:09Z

Awesome!! I'm also really amped excited for nested layouts too. Happy to lend a hand in the PRs to follow :)

Tom-Bonnike · 2017-08-08T09:18:23Z

Thanks for this PR. I needed the ability to add graphQL queries in layouts to migrate a project to V1. :)

KyleAMathews · 2017-08-08T14:16:39Z

@craig-mulligan me too ❤️ there's a ton of use cases for nested layouts so excited for it.

* When creating layouts, only pass in the layout path and then automatically create the id from the name of the component file. This is safe as unless someone is doing something unusual layouts should all be in the src/layouts directory. If someone is manually creating layouts, they'll need to be careful to not overwrite layouts created elsewhere. * Fix hot re-running of queries in layouts & pages.

…mponent renderer

KyleAMathews · 2017-08-08T21:07:37Z

Ok GatsbyGram is back with latest commit https://598a253c7960b15847092350--gatsbygram.netlify.com/

KyleAMathews · 2017-08-08T21:58:15Z

Ok I think this is ready to go. I really dislike how complicated this code is. Every time I touch it it takes forever to load it into my head which suggests we're using the wrong abstractions or something. Haven't figured out better way yet. Someone please come along and PR a much better implementation :-P

A bit of context for this commit 750d273 Gatsby loads code + data seperately for pages as it's quite often to have a page component power many pages e.g. a blog post component. But since layouts can only have one instance, I changed things so that the code + data are bundled together to avoid making two requests where one would do.

KyleAMathews · 2017-08-08T21:58:59Z

And for the inevitable breakage this will cause, sorry 🙏

We definitely need better abstractions / tests for these parts of the code.

jbolda · 2017-08-08T23:07:53Z

packages/gatsby/src/redux/actions.js

@@ -125,9 +125,16 @@ actions.deleteLayout = (layout, plugin = ``) => {
 */
 actions.createLayout = (layout, plugin = ``, traceId) => {
  layout.id = path.parse(layout.component).name
+  layout.componentWrapperPath = path.join(


path.join should be joinPath as path.join breaks in windows.

jbolda · 2017-08-08T23:11:26Z

packages/gatsby/src/internal-plugins/query-runner/gatsby-node.js

-    if (!fs.existsSync(dest)) {
-      fs.writeFile(dest, `{}`, () => {})
-    }
+  const jsonDest = path.join(


This should also be joinPath, but does it make sense to require('../../utils/path')? Feels odd traversing packages, but that does fix it :)

aeneasr · 2017-09-12T15:12:33Z

Are there any updates on nested layouts? If not, I could take a shot. I'm a capable developer, but I would need some hand-holding in identifying the right components to modify.

KyleAMathews · 2017-09-12T17:52:20Z

@jbolda has an open PR for this that I'd be he'd love some eyes on #1895

dardub reviewed Jul 14, 2017

View reviewed changes

KyleAMathews reviewed Jul 14, 2017

View reviewed changes

craigmulligan commented Jul 16, 2017

View reviewed changes

craigmulligan changed the title ~~[WIP] add createLayout api~~ add createLayout api Jul 16, 2017

KyleAMathews reviewed Jul 16, 2017

View reviewed changes

craigmulligan commented Jul 17, 2017

View reviewed changes

craigmulligan commented Jul 19, 2017

View reviewed changes

craigmulligan force-pushed the multi-layouts branch from a6df654 to 3bfa256 Compare July 22, 2017 14:23

craigmulligan changed the title ~~add createLayout api~~ add createLayout api + multiple layouts + layout queries Jul 24, 2017

craigmulligan mentioned this pull request Jul 25, 2017

layouts with queries product-os/landr#38

Closed

KyleAMathews suggested changes Jul 25, 2017

View reviewed changes

KyleAMathews reviewed Jul 25, 2017

View reviewed changes

jbolda reviewed Aug 8, 2017

View reviewed changes

KyleAMathews added 3 commits August 8, 2017 11:24

Merge branch 'master' into multi-layouts

1bb6a25

Fix tests

58832d0

KyleAMathews approved these changes Aug 8, 2017

View reviewed changes

KyleAMathews added 4 commits August 8, 2017 12:13

Fix comments

35b1a57

Layout code + data should be one bundle as always loaded together

750d273

Restore letting layout override what props passed to component

724ff41

Move layout rendering to a stateful layout renderer similar to the co…

b679f80

…mponent renderer

update snapshots

08bd0f3

KyleAMathews merged commit a38f49e into gatsbyjs:master Aug 8, 2017

jbolda reviewed Aug 8, 2017

View reviewed changes

jbolda mentioned this pull request Aug 8, 2017

fix windows path issues *sigh* #1747

Merged

bskimball mentioned this pull request Aug 9, 2017

TypeError: Data must be a string or a buffer #1659

Closed

craigmulligan mentioned this pull request Aug 10, 2017

Layouts with graphql queries #1589

Closed

jbolda mentioned this pull request Aug 20, 2017

[1.0] Adding graphql query in layouts/index.js is not allowed #1175

Closed

ataylorme mentioned this pull request Dec 20, 2017

Data not passed as data from graphql query in layouts/index.js #3279

Closed

KyleAMathews mentioned this pull request Feb 6, 2018

[RFC] remove our special layout components in Gatsby v2 #3830

Closed

pieh mentioned this pull request May 13, 2020

chore(gatsby): convert component-data-dependencies to typescript #24028

Merged

		action.payload.componentPath = normalize(action.payload.componentPath)
		// console.log(state)

		@@ -0,0 +1,23 @@
		const path = require('path')
		// Example if you need to programmatically create a layout

		@@ -48,7 +49,7 @@ const pascalCase = _.flow(_.camelCase, _.upperFirst)
		* })

add createLayout api + multiple layouts + layout queries #1503

add createLayout api + multiple layouts + layout queries #1503

Conversation

craigmulligan commented Jul 14, 2017 • edited Loading

KyleAMathews commented Jul 14, 2017 • edited Loading

gatsbybot commented Jul 14, 2017 • edited by KyleAMathews Loading

gatsbybot commented Jul 14, 2017 • edited Loading

gatsbybot commented Jul 14, 2017 • edited Loading

KyleAMathews commented Jul 14, 2017

craigmulligan commented Jul 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KyleAMathews commented Jul 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KyleAMathews commented Jul 16, 2017 • edited Loading

KyleAMathews commented Jul 16, 2017 • edited Loading

craigmulligan commented Jul 16, 2017

KyleAMathews left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

craigmulligan commented Jul 19, 2017

craigmulligan commented Jul 19, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KyleAMathews commented Jul 19, 2017 • edited Loading

Choose a reason for hiding this comment

KyleAMathews left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KyleAMathews commented Aug 8, 2017

jbolda commented Aug 8, 2017

craigmulligan commented Aug 8, 2017

Tom-Bonnike commented Aug 8, 2017

KyleAMathews commented Aug 8, 2017

KyleAMathews commented Aug 8, 2017

KyleAMathews commented Aug 8, 2017

KyleAMathews commented Aug 8, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aeneasr commented Sep 12, 2017

KyleAMathews commented Sep 12, 2017

craigmulligan commented Jul 14, 2017 •

edited

Loading

KyleAMathews commented Jul 14, 2017 •

edited

Loading

gatsbybot commented Jul 14, 2017 •

edited by KyleAMathews

Loading

gatsbybot commented Jul 14, 2017 •

edited

Loading

gatsbybot commented Jul 14, 2017 •

edited

Loading

KyleAMathews commented Jul 16, 2017 •

edited

Loading

KyleAMathews commented Jul 16, 2017 •

edited

Loading

KyleAMathews commented Jul 19, 2017 •

edited

Loading