Issue populating document referenced inside embedded discriminator

[adriaanmeuris]

Do you want to request a feature or report a bug?
I like to report what I think is a bug in 5.0.0-rc2

What is the current behavior?
I'm trying to populate a referenced document inside an embedded discriminator.

Populating works when storing the discriminator in a separate collection, but it doesn't when it's embedded.

Situation: Bundle has Item (Books or Ebooks) that refer to an Author

1. Separate collections for Bundle, Item and Author (population of Item and Author works)
2. Collections for Bundle and Author (Item are embedded in Bundle and reference Author, which does not get populated)

Repro script for 1, separate collections, the author is correctly populated

// Modules
const mongoose = require('mongoose');

// DB
let connect = mongoose.connect("mongodb://localhost:27017/test");

// Models
const Author = mongoose.model('Author', new mongoose.Schema({
  firstName: {
    type: String,
    required: true
  },
  lastName: {
    type: String,
    required: true
  }
}));

const ItemSchema = new mongoose.Schema({
  title: {
    type: String,
    required: true
  }
}, {discriminatorKey: 'type'});
const Item = mongoose.model('Item', ItemSchema);

const ItemBookSchema = new mongoose.Schema({
  author: {
    type: mongoose.Schema.ObjectId,
    ref: 'Author'
  }
});
Item.discriminator('Book', ItemBookSchema);
const Book = mongoose.model('Book');

const ItemEBookSchema = new mongoose.Schema({
  author: {
    type: mongoose.Schema.ObjectId,
    ref: 'Author'
  },
  url: {
    type: String
  }
});
Item.discriminator('EBook', ItemEBookSchema);
const EBook = mongoose.model('EBook');

const Bundle = mongoose.model('Bundle', new mongoose.Schema({
  name: {
    type: String,
    required: true
  },
  items: [{
    type: mongoose.Schema.ObjectId,
    ref: 'Item',
    required: false
  }]
}));

// Populate
const population = {
  path: 'items',
  populate: {
    path: 'author'
  }
}

// Create test data
connect.then(() => {
  // Create author
  return new Author({firstName: 'David', lastName: 'Flanagan'}).save();
}).then((author) => {
  // Create items
  let createBook = new Book({title: 'JavaScript: The Definitive Guide', author: author}).save()
  let createEBook = new EBook({
    title: 'JavaScript: The Definitive Guide Ebook',
    url: 'https://google.com',
    author: author
  }).save();
  return Promise.all([createBook, createEBook]);
}).then((items) => {
  // Create bundle with referenced items
  return new Bundle({name: 'Javascript Book Collection', items: items}).save();
}).then(() => {
  return Bundle.findOne({}).populate(population).lean().exec();
}).then((bundle) => {
  console.log(bundle.items[0].author); 
  // { _id: 5a513af060436559217875c2,
  //   firstName: 'David',
  //   lastName: 'Flanagan',
  //   __v: 0 }
});

Repro script for 2, embedded discriminator, the author is not populated

// Modules
const mongoose = require('mongoose');

// DB
let connect = mongoose.connect("mongodb://localhost:27017/test");

// Models
const Author = mongoose.model('Author', new mongoose.Schema({
  firstName: {
    type: String,
    required: true
  },
  lastName: {
    type: String,
    required: true
  }
}));

const ItemSchema = new mongoose.Schema({
  title: {
    type: String,
    required: true
  }
}, {discriminatorKey: 'type'});

const ItemBookSchema = new mongoose.Schema({
  author: {
    type: mongoose.Schema.ObjectId,
    ref: 'Author'
  }
});

const ItemEBookSchema = new mongoose.Schema({
  author: {
    type: mongoose.Schema.ObjectId,
    ref: 'Author'
  },
  url: {
    type: String
  }
});

const BundleSchema = new mongoose.Schema({
  name: {
    type: String,
    required: true
  },
  items: [{
    type: ItemSchema,
    required: false
  }]
});
const Bundle = mongoose.model('Bundle', BundleSchema);

// Add discriminators on items path
BundleSchema.path('items').discriminator('Book', ItemBookSchema);
BundleSchema.path('items').discriminator('EBook', ItemEBookSchema);

// Populate
const population = {
  path: 'items',
  populate: {
    path: 'author'
  }
}

// Create test data
connect.then(() => {
  // Create author
  return new Author({firstName: 'David', lastName: 'Flanagan'}).save();
}).then((author) => {
  // Create bundle with embedded items
  return new Bundle({
    name: 'Javascript Book Collection', items: [
      {type: 'Book', title: 'JavaScript: The Definitive Guide', author: author},
      {
        type: 'EBook',
        title: 'JavaScript: The Definitive Guide Ebook',
        url: 'https://google.com',
        author: author
      }
    ]
  }).save();
}).then(() => {
  // Find bundle, populate author of items
  return Bundle.findOne({}).populate(population).lean().exec();
}).then((bundle) => {
  // Log bundle
  console.log(bundle.items[0].author); // 5a513b01a7a24459239e6c14
});

What is the expected behavior?
Author needs to get populated when Items are embedded

Please mention your node.js, mongoose and MongoDB version.

• node: 9.3.0
• mongoose: 5.0.0-rc2
• mongodb version: 3.6.0

[adriaanmeuris]

I've managed to resolve this by trying out different population methods:

1. populate with string: items.author
result: null

2. populate with object that has a nested object:

{
  path: 'items',
  populate: {
    path: 'author'
  }
};

result: 5a51edae70aa5a59d39b915b

3. populate with object that has a nested object and the referenced model:

{
  path: 'items',
  populate: {
    path: 'author',
    model: 'Author'
  }
};

result: 5a51edae70aa5a59d39b915b

4. populate with object that has a multi-level path:

{
  path: 'items.author'
}

result: null

5. populate with object that has a multi-level path and the model:

{
  path: 'items.author',
  model: 'Author'
}

result:

{
  _id: '5a51edae70aa5a59d39b915b',
  firstName: 'David',
  lastName: 'Flanagan',
  __v: 0
}

summary
I'm happy to have this resolved, but it's kind of confusing whether the right syntax is used for population. An overview of different ways in the populate docs would be of great help.

Also, I was under the impression that referencing the model in the population object was not necessary, since I've already defined ref to Author in the schema - which could be used to determine which model to use for populate() (just like refPath does). Is that correct?

[adriaanmeuris]

Upon further investigating, it's seems there's something up with the logic to determine which model to use for populating when using embedded discriminators.

I've commented to this #4817 which was resolved for regular discriminators, but the issue rises again when using embedded discriminators. Any help or pointers would be greatly appreciated!

[vkarpov15]

Thanks for reporting, will investigate ASAP 👍

[vkarpov15]

This is proving to be a very tricky bug to fix. The right syntax should be:

{
  path: 'items.author'
}

Unfortunately that currently doesn't work because populate doesn't handle embedded discriminators properly yet, so do this as a workaround until we fix this issue.

{
  path: 'items.author',
  model: 'Author'
}

[adriaanmeuris]

No problem, will do. If you'd have a workaround for #4817 as well, that would be awesome. Thanks for your quick feedback!

[vkarpov15]

What's the issue with #4817 ? I thought that one was already fixed...

[adriaanmeuris]

It was, and is, for regular discriminators. But it fails with the same error when using embedded discriminators. See my last comment in #4817 for a repro script. If you'd like me to open up a new issue, let me know!

[vkarpov15]

Please do!

[adriaanmeuris]

here you go #5983

[vkarpov15]

Fixed in master and 4.x branch, fix will be in 5.0.2 and 4.13.10