Collections as data only

[babycourageous]

Hi there

Curious about a best practice to go about this. I have a section of content that would be iterated over on the front page but doesn't need to be rendered as individual pages and I'm debating between two ways of setting this up in 11ty -

1. a single data file with each service as an item in the JSON object that i can iterate over on the front page

_data/services.json

{
  services: [
    {"name": "name1", "icon":"icon1", "description": MARKDOWN_HERE???},
    {"name": "name2", "icon":"icon2", "description": MARKDOWN_HERE???}
  ]
}

2. A custom collection with each markdown file as a service

services/service1.md

---
name: name1
icon: icon1
---
MARKDOWN INFO HERE

The reason I'm thinking of option 2 is I need the main content of the service to be rendered as html (bold, lists, etc) which I haven't been able to accomplish in a JSON file...

Would it make the most sense to use collections in this way and just add that collection folder to the .eleventyignore file to prevent rendering of each service as a page? Or since it's just data is it preferred to keep it in a global data file and just store the markdown as a string?

Sorry if this has been asked before - i didn't find anything when i searched through the queue

[Ryuno-Ki]

Hi,
I'm using a similar approach on my website.

I'd use JSON, if you need control over the order or some kind of batch processing (doing that with events/conferences/... on my website right now).

On the other hand, dedicated files are easier to reason about separately.
As long as you don't add a layout, they shouldn't render to a file (can double check).

[babycourageous]

Thanks @Ryuno-Ki

I'm currently using the collection technique. Although it does appear that the folder of collection items gets rendered out to the compiled _site folder. Since it is only data and won't require a front-facing end result outside of a listing I think I would prefer a data file...

How would you recommend handling the Markdown "content" data in a JSON file?

I'm going to need a content editor to be able to update that (using Netlify CMS).

Thanks!

[Ryuno-Ki]

Hi @babycourageous,
meh, you're right: They are rendered (for me as empty files, since I only use Frontmatter).

I digged the last two hours into code.
I think, it could be made work, but we would need changes on eleventy (i.e. @zachleat) here.

But let me start from the beginning.

My first idea was to use templateEngineOverride: false to have no template rendering attached. This didn't worked (i.e. file still get generated).

Then I recalled, that you can adjust some overrides.

For example, markdown-it has an option html. So I added it to the .eleventy.js file, but the docs explain it is about rendering HTML tags. Not what we want here (we want no output whatsoever).

Okay, then maybe something else?

I guess, you could go with JavaScript Template Literals, but you would still get an output.

What about using a custom Nunjucks environment. The idea being, that it would have a render method as noop (doing nothing).
However, since my templates are generally written in Nunjucks, I don't want to override them with a noop.

Problem is, that the list of template formats is fix.
#207 set out to introduce extension aliases, but the respective code was „removed” in f2218f8#diff-a503f65ada1cd4808b2eee3e01e9471f (@zachleat why? I couldn't find a reference to an issue on that commit).

If I comment the code in (locally), I can see error messages like this:

> Having trouble rendering noop template ./public/feed/24ways.md (TemplateContentRenderError)
> Template Engine noop does not exist in getEngine (includes dir: public/_includes) (Error):
    Error: Template Engine noop does not exist in getEngine (includes dir: public/_includes)
        at Function.getEngine ($HOME/path/to/project/node_modules/@11ty/eleventy/src/Engines/TemplateEngine.js:136:13)
        at TemplateRender.init ($HOME/path/to/project/node_modules/@11ty/eleventy/src/TemplateRender.js:55:34)
        at TemplateRender.setEngineOverride ($HOME/path/to/project/node_modules/@11ty/eleventy/src/TemplateRender.js:142:10)
        at Template.setupTemplateRender ($HOME/path/to/project/node_modules/@11ty/eleventy/src/TemplateContent.js:129:27)
        at process.internalTickCallback (internal/process/next_tick.js:77:7)

My .eleventy.js included the following for this:

const Nunjucks = require("nunjucks");

module.exports = function (eleventyConfig) {
  const nunjucksEnvironment = new Nunjucks.Environment(
    new Nunjucks.FileSystemLoader("_includes")
  );
  eleventyConfig.setLibrary("noop", nunjucksEnvironment);
  eleventyConfig.addTemplateExtensionAlias("noop", "noop");
  /* … */
};

Here I interrupted my work (took already quite a while).
@zachleat Any chance, to add something like #207 back in again?

[Ryuno-Ki]

How would you recommend handling the Markdown "content" data in a JSON file?

With a JSON (_data/events.json) of

[
 {
    "name": "Towel Day",
    "start": "2019-05-25",
    "end": "2019-05-25",
    "url": "http://www.towelday.org/",
    "location": null
  },
  {
    "name": "E3",
    "start": "2019-06-11",
    "end": "2019-06-14",
    "url": "https://www.e3expo.com/",
    "location": {
      "name": "Los Angeles Convention Center",
      "address": "1201 S Figueroa Street",
      "locality": "Los Angeles",
      "country": "California",
      "postalCode": "90015"
    }
  }
]

I render it via a layout file (_includes/events.njk):

---
layout: h1
---
{% if events %}
  <h2>Future events</h2>
  {% set futureEvents = events | isFutureDate %}
  <ol>
    {% for futureEvent in futureEvents %}
      {% set event = futureEvent | asEventObject %}
      <li>
        {% include "_partials/event.njk" %}
      </li>
    {% endfor %}
   </ol>

  <h2>Past events</h2>
  {% set pastEvents = events | isPastDate | reverse %}
  <ol>
    {% for pastEvent in pastEvents %}
      {% set event = pastEvent | asEventObject %}
      <li>
        {% include "_partials/event.njk" %}
      </li>
    {% endfor %}
  </ol>
{% else %}
  Currently none.
{% endif %}

by referencing it in a Markdown file (events/index.md):

---
layout: events
title: 'Interesting events'
---
Here are some events you might be interested in attending:

[Ryuno-Ki]

Perhaps related:

• Custom File Extension Handlers #117
• Arbitrary JavaScript Template Type #118
• Easier way to run transforms on passthrough copy files #344

[babycourageous]

Thanks @Ryuno-Ki

Instead of trying to square peg a markdown list into the JSON round hole (or use collections as data items), what I might wind up doing is have each list item as an entry in the JSON object. Something like:

{
  services: 
    [
      {
        "name": "name1", 
        "icon":"icon1", 
        "bullets": ["bullet one", "bullet two", "bullet three"]
      }
    ]
}

Then I can loop over the bullets and render each in an <li> that i have control over (classes, attributes, etc.
A little less user friendly on the site-owner side but I think will be a more consolidated and clean route on the development end.

[jevets]

I've faced similar issues when keeping raw markdown in a JSON doc, and I've ended up trying a few different routes. There hasn't been a clear winner yet, for me at least. (subscribing here to stay in the loop on this one, curious how you get on with this over time.)

You could compile markdown source files into the _includes directory, then include/consume those from your actual pages. Just an idea...

A bit of an aside:

Your structure is starting to resemble portable text, which I just started consuming this morning from a sanity.io datastore. I used the portable-text-to-html package, but there's also one to convert to markdown. You can set up custom serializers to handle special tags or, say, convert <em> marks into your own <span class="my-special-em">.

[adamkiss]

Hey, on one of my sites, I use csv for the data, and I use a javascript file to read it, parse it, and feed it to 11ty - 11ty can use .js flies as data files, and I just drop both into data folder.

You could try pairing it with the fact that yaml supports multiple documents and use a filter in your template.

So, yaml with structured data with markdown is parsed by JS file which is read by 11ty and then rendered out.

[Ryuno-Ki]

I think, I had a similiar idea like adam:
Putting the YAML files outside of the root of eleventy (I'm using public as input directory, since I came from harp.js).
I could put my YAML files there and have them read in with a JS file inside _data. This way, they would be available as data file, but not rendered. Will give it a shot and report back here.

@adamkiss Could you share some code? Sounds interesting!

[adamkiss]

@Ryuno-Ki Sure, here you go: https://github.com/adamkiss/11ty-yaml-data

[babycourageous]

Thanks for all the suggestions! I'll try some stuff out and fiddle around with my use case. Sorry it's taken a while to respond!

[Ryuno-Ki]

Ah, that reminds me, that I wanted to share my approach as well!

Setup:

• Directory: feed/zach-leatherman.md with content

---
tags: feed
title: Zach Leatherman
xmlUrl: https://www.zachleat.com/web/feed/
---

• Global data file: public/_data/feeds.js with content

const fs = require('fs')
const path = require('path')

const { promisify } = require('es6-promisify')
const yaml = require('js-yaml')

const readdir = promisify(fs.readdir)
const readFile = promisify(fs.readFile)
const feedPath = path.resolve(__dirname, '..', '..', 'feed')

async function getFeeds () {
  let jsonContent

  const fileNames = await readdir(feedPath)
  const filePaths = fileNames.map((filename) => path.join(feedPath, filename))
  const yamlContent = await Promise.all(filePaths.map(async (filePath) => {
    return readFile(filePath, 'utf8')
  }))
  try {
    jsonContent = await Promise.all(yamlContent.map(async (content) => {
      return new Promise((resolve, reject) => {
        try {
          yaml.safeLoadAll(content, (parsed) => {
            return resolve(parsed)
          })
        } catch (error) {
          return reject(error)
        }
      })
    }))
  } catch (error) {
    console.error('Oops', error)
    return Promise.resolve([])
  }
  return jsonContent
}

module.exports = getFeeds

• Partial rendering the feeds: public/_includes/_partial/blogroll.njk with content:

<ul>
  {% for feed in feeds %}
    {% set feedData = feed.xmlUrl | withFeedData %}
    {% set latestEntry = feedData.items | first %}
    {% if latestEntry %}
      <li>
        {% set entryLink = latestEntry.guid or latestEntry.id or latestEntry.link %}
        {% set feedUrl = feedData.feedUrl or feed.xmlUrl %}
  
        {{ feedData.title }}:
        <a href="{{ entryLink }}">
          {{ latestEntry.title }}
        </a>
        {% if latestEntry.creator %}
          by {{ latestEntry.creator }}
        {% endif %}
        {% if latestEntry.pubDate %}
          on {{ latestEntry.pubDate | dmyDate }}
        {% endif %}
        (<a href="{{ feedUrl }}">Subscribe</a>)
      </li>
    {% endif %}
  {% endfor %}
</ul>

Eleventy render call: eleventy --quiet --input ./public (package.json is sitting next to public and feed directories).

Works fine for me :-)

[zachleat]

Hmm, lemme see if I can summarize the questions here?

To the original post, @babycourageous: I think method 1 is if you’re using a CMS to generate the file (as I believe you stated). Method 2 is preferred if you’re editing the files yourself.

How to prevent a directory of files from appearing in the output?

Use permalink: false https://www.11ty.io/docs/permalinks/#permalink%3A-false You can put this in a directory data file to apply to everything in the directory. https://www.11ty.io/docs/data-template-dir/

Status of extension aliasing

#207 (comment)

I wasn’t happy with it so it got punted. It’ll hopefully make it back in, eventually.

Did I miss any other questions?

Is this issue resolved?

[babycourageous]

@zachleat thanks!

Sorry - other projects took hold and i lost track of responding to the issue.

Original Question

I'm not pulling from a CMS so I will use method 2 since that will give me a little more control over the markdown content (they are basically pricing table style items). Trying to store lists and headings and paragraphs as a single JSON string is the pits, heheh.

Output prevention

Classic facepalm. Totally forgot about disabling permalinks (especially via collection/directory specific data files!)

Those were my questions - solved!

[zachleat]

Belated thanks!

[milahu]

How to prevent a directory of files from appearing in the output?

Use permalink: false https://www.11ty.io/docs/permalinks/#permalink%3A-false You can put this in a directory data file to apply to everything in the directory. https://www.11ty.io/docs/data-template-dir/

example:

src/path/to/pages/pages.11tydata.json

{
  "tags": "pages",
  "tags-pages-comment": "make files available in `collections.pages`",

  "permalink": false,
  "permalint-false-comment": "exclude files from output, only make them available in collections",
  "permalint-false-docs": "https://www.11ty.dev/docs/permalinks/#permalink-false"
} 

src/index.njk

---
layout: layouts/base.njk
---
<div class="page-container">
  {%- for page in collections.pages -%}
    <div class="page">
      {{ page.templateContent | safe }}
    </div>
  {%- endfor -%}
</div>

use case: create single file apps with eleventy