feat(gatsby-transformer-remark): Allow for multiple different remark sources

[alexkirsz]

This PR allows for having more than one gatsby-transformer-remark plugin. With the addition of the filter option, the user can now decide which nodes should be handled by which plugin. For instance, with gatsby-source-filesystem, one could do the following in order to have multiple different markdown sources:

const plugins = [
  {
    resolve: `gatsby-source-filesystem`,
    options: {
      path: `${__dirname}/src/pages`,
      name: `pages`,
    },
  },
  {
    resolve: `gatsby-source-filesystem`,
    options: {
      path: `${__dirname}/src/blog`,
      name: `blog`,
    },
  },
  {
    resolve: `gatsby-transformer-remark`,
    options: {
      filter: node => node.sourceInstanceName === `pages`,
      type: `MarkdownPage`,
    },
  },
  {
    resolve: `gatsby-transformer-remark`,
    options: {
      filter: node => node.sourceInstanceName === `blog`,
      type: `BlogPost`,
    },
  }
]

I've also added a type option so that these different plugins create GraphQL types with different names.

In the above example, we could expect that our markdown pages might have different frontmatter fields than our blog entries, but having them all under the MarkdownRemark type would force us to add filters on fields in order to only retrieve a list of either of them. Having different frontmatter formats would also pollute our GraphiQL documentation.

[alexkirsz]

It could make sense to adopt this kind of API for other transformers as well.

[KyleAMathews]

Deploy preview for using-contentful failed.

Built with commit 59aadf5

https://app.netlify.com/sites/using-contentful/deploys/5b82917d4ed62f7116e8d176

[m-allanson]

Thanks @alexkirsz 👍 this is a really useful feature - so much so, that something like it should be added to Gatsby core instead of this being implemented separately across different plugins.

It'll be particularly useful for the new Gatsby themes functionality that @ChristopherBiscardi is working on. As part of the themes work there's going to be an issue or rfc soon™️ (next week or after) to figure out exactly how this could be added to Gatsby core. Once the rfc has been written I'll add a link here.

You can check out more info on Gatsby themes here and here.

[alexkirsz]

@m-allanson Any news on this?

I'm not quite sure how this would play with other transformers. For instance, the yaml transformer already leverages folder names and/or file names to generate types.

[alexkirsz]

FWIW I've rebased on master and published this plugin as @alexkirsz/[email protected]

[wardpeet]

I removed bluebird and did some tests and seems to work. Nothing breaks so lets ship this!

thank you for your patience 🙏! 💪

[alexkirsz]

Great! Thanks!

[pieh]

this doesn't seem to work

[pieh]

this void here is problematic? after removing it, it seems to work - why is it here? some context is needed

[DSchau]

Also generally - where do we stand on this? We use bluebird pretty heavily--so I'm not sure it was worth refactoring this.

But a bit tangential, if we went that way--just need to ensure we have a working alternative for sequential promises!

[pieh]

I personally don't agree with removing dependency for the sake of removing it (at least for stuff that is only used during the builds and don't end up in production js bundles produced by webpack). If it fixes bugs or improve performance, then that's fine.

I will open PR fixing that and add some tests for this helper.

[pieh]

When utils like bluebird have maintainers with domain knowledge and it was battle-tested by thousands of developers - I will trust that more than hand rolled and not very tested approaches.

[pieh]

In any case - here's hot-fix for issue #12578 (first commit on purpose just added failing test, and second commit "fixes" the problem.

@alexkirsz I would appreciate you taking a look on this, because I imagine there was some reason for that void to be there, but I have no clue what that would be (and it did break that utility)

[alexkirsz]

@pieh I'm not the author of this particular piece of code, @wardpeet is (I only authored the first commit of this PR).

[pieh]

Ah, ok - sorry @alexkirsz. I should pay more attention to individual commits in the PR

[wardpeet]

my bady, sorry about this

[wardpeet]

hey @alexkirsz I'm sorry but reverted this change (#12639) as @m-allanson was right and we should have this implementation in core.

Thanks @alexkirsz 👍 this is a really useful feature - so much so, that something like it should be added to Gatsby core instead of this being implemented separately across different plugins:

It'll be particularly useful for the new Gatsby themes functionality that @ChristopherBiscardi is working on. As part of the themes work there's going to be an issue or rfc soon™️ (next week or after) to figure out exactly how this could be added to Gatsby core. Once the rfc has been written I'll add a link here.

You can check out more info on Gatsby themes here and here.

We want you to know that we certainly want to keep you as a contributor to Gatsby, so we'd encourage you to check out our open issues.

Thanks again, and we look forward to seeing more PRs from you in the future! 💪 💜