Eleventy collection from external REST API loses default collection item data structure

[paulshryock]

I'm trying to build an Eleventy collection from external data, while keeping the default collection item data structure.

I have a REST API that serves JSON content, which looks like this:

GET https://mysite.example/api/articles:

[
  {
    "tags": [],
    "_id": "5e60083ec3671e003bf7994e",
    "title": "New Article 1",
    "author": "Author Name",
    "status": "draft",
    "date": "2020-03-04T19:57:50.061Z",
    "__v": 0,
    "slug": "new-article-1"
  },
  {
    "tags": [],
    "_id": "5e600845c3671e003bf7994f",
    "title": "New Article 2",
    "author": "Author Name",
    "status": "published",
    "date": "2020-03-04T19:57:57.583Z",
    "__v": 0,
    "slug": "new-article-2"
  },
  {
    "tags": [],
    "_id": "5e60084dc3671e003bf79950",
    "title": "New Article 3",
    "author": "Author Name",
    "status": "scheduled",
    "date": "2020-03-04T19:58:05.267Z",
    "__v": 0,
    "slug": "new-article-3"
  }
]

Here is my Eleventy configuration:

.eleventy.js:

const axios = require('axios')

module.exports = function (eleventyConfig) {
  eleventyConfig.addCollection('articles', async collection => {
    const response = await axios({
      method: 'get',
      url: 'https://mysite.example/api/articles'
    })
      .then(response => {
        return response.data
      })
      .catch(error => {
        // Handle error
      })
    return response ? response : []
  })
}

This results in an articles collection being successfully created in Eleventy, which is great! However, each collection item does not have the expected collection item data structure. I would expect that the returned data for each item would be added to a data key, and the other default keys would also exist: inputPath, fileSlug, outputPath, url, date, data, templateContent.

Instead, those default keys are not added, and each collection item's data lives at the top level, instead of nested inside a data key.

1. Is there a way to get Eleventy to automatically add this default collection item data structure?
2. Or when I create the collection, do I need to manually add the content to a data key, and programmatically try to create the other default keys?

[danfascia]

This is really interesting and it's one of my own struggles. Data from APIs is something of a second class citizen compared to the data which passes through the collection building API.

It would be great to be able to pipe data from any JS data file through the collections API to get all of the good stuff mentioned and also to hook into the tag system which 11ty relies on quite heavily

[paulshryock]

It would be great to be able to pipe data from any JS data file through the collections API to get all of the good stuff mentioned and also to hook into the tag system which 11ty relies on quite heavily

Yeah, exactly. I feel like that's what eleventyConfig.addCollection() is supposed to do, but for some reason it's not.

[danfascia]

I can understand why some of it doesn't work because it is not necessarily building things to the filesystem (until you paginate) so how would it know what fileSlug to build etc...

But for me hooking into the tags is very important because tags are the (only native) way of carrying out filtering and query building on collections.

[neo7-studio-web]

This is a very good point. I'm facing it as I try to migrate a blog from .md files to headless CMS. The way you implemented it is the natural way that came to my mind. I faced the eact same problem as I have to fix many points that could have been solved. Now I will certainly fix this manually...
This means that REST API data and local/file datas are not treated the same way by eleventy which is an area of improvement IMO.

[polx]

Maybe that helps: https://benmyers.dev/blog/eleventy-data-cascade/
After reading this, I moved my (glob-based) data generation to a file figures.js inside _data which returned the necessary data and I had my data be actually read and apparently sufficiently decorated of default collection item data structure as opposed to when it was in .eleventy.js (I was on 1.0.1). I could use it in a paginated template.

[danburzo]

Or when I create the collection, do I need to manually add the content to a data key, and programmatically try to create the other default keys?

Yes, whatever you return from addCollection() is used as-is, so you’d need to add the properties yourself.

Starting with Eleventy 3.0, you should be able to use Virtual Templates, by taking advantage of the fact that you’ll be able to define an async config file that waits for your API call.