Spike: Resolver for inclusion of related models

[bajtos]

Continue the works started in #2124, leverage the building blocks implemented as an outcome of #2592 and create a detailed proposal for implementing resolver for navigational properties.

Implementation wise, I think we want to end up with the following user experience:

export class TodoListRepository extends DefaultCrudRepository<
  TodoList,
  typeof TodoList.prototype.id
> {
  public readonly todos: HasManyRepositoryFactory<
    Todo,
    typeof TodoList.prototype.id
  >;

  constructor(
    @inject('datasources.db') dataSource: juggler.DataSource,
    @repository.getter(TodoRepository)
    protected todoRepositoryGetter: Getter<TodoRepository>,
  ) {
    super(TodoList, dataSource);
    this.todos = this._createHasManyRepositoryFactoryFor(
      'todos',
      todoRepositoryGetter,
    );
   ////// THE FOLLOWING LINE IS NEWLY ADDED
   this._registerHasManyInclusion('todos', todoRepositoryGetter);
  }
}

The question is how to implement _register**Inclusion APIs and process the registered inclusion handlers in find/findById implementations provided by DefaultCrudRepository. See #1352 (comment) for more details on that.

It is important to consider the scenario where we want to fetch multiple source models and include their related models in the result, e.g. TodoList.find(1, {include: [{relation: 'todos'}]}). Under the hood, we must avoid SELECT N+1 issue. Inclusion handler must be able to fetch related models for multiple source-model ids in one query (using inq operator under the hood). The design proposed in #1352 (comment) is already taking that into account, so does the existing implementation in juggler.

Acceptance criteria

• A draft pull request demonstrating the proposed solution:
  • Use examples/todo-list to show the proposed user experience
  • Provide a PoC implementation in packages/repository
  • Include a markdown document in the monorepo root (/_SPIKE_.md) describing the proposal at a high level. Keeping the document in git (instead in GH comments) makes it easier to discuss individual parts of the proposal and keep the proposal versioned.
• In Inclusion of related models [MVP] #1352 (comment), I am proposing to leverage the existing code in juggler, e.g. lib/include.js#L609-L634. If we decide to do that, then the spike should outline how we are going to make those bits available to LB4 repositories. Are we going to export it from juggler? Migrate the code over from juggler to LB4 repository module?
• Pay a close attention to test coverage. How are we going to migrate the current test suite in juggler that's extensively covering different aspects of inclusion of related models?
• The follow-up stories created as an outcome of this spike should include tasks to cover the following areas:
  • Update lb4 repository to generate calls of this._registerHasManyInclusion and friends.
  • Documentation updates
  • Announcement blog post

[AnanthGopal]

Hi @bajtos

this._registerHasManyInclusion('todos', todoRepositoryGetter);

When I add this code I got the error "_registerHasManyInclusion" it's not defined. Please provide the full function for this _registerHasManyInclusion

[shadyanwar]

@AnanthGopal this is a spike. The feature is still being worked on and is not implemented or released yet.

[bajtos]

MVP scope

• Run repository tests for PostgreSQL Run repository tests for PostgreSQL #3436
• Run repository tests for Cloudant Run repository tests for Cloudant #3437
• Reject create/update requests when data contains navigational properties Reject create/update requests when data contains navigational properties #3439
• Verify relation type in resolve{Relation}Metadata Verify relation type in resolve{Relation}Metadata #3440
• Add keyFrom to resolved relation metadata Add keyFrom to resolved relation metadata #3441
• Test relations against databases Test relations against databases #3442
• Add findByForeignKeys helper (initial version) Add findByForeignKeys helper (initial version) #3443
• Support inq splitting in findByForeignKeys Support inq splitting in findByForeignKeys #3444
• Introduce InclusionResolver concept Introduce InclusionResolver concept #3445
• Include related models in DefaultCrudRepository Include related models in DefaultCrudRepository #3446
• Implement InclusionResolver for hasMany relation Implement InclusionResolver for hasMany relation #3447
• Implement InclusionResolver for belongsTo relation Implement InclusionResolver for belongsTo relation #3448
• Implement InclusionResolver for hasOne relation Implement InclusionResolver for hasOne relation #3449
• Update todo-list example to use inclusion resolver Update todo-list example to use inclusion resolver #3450
• Add inclusion resolvers to lb4 relation CLI Add inclusion resolvers to lb4 relation CLI #3451
• Blog post: announce Inclusion of related models Blog post: announce Inclusion of related models #3452

Post-MVP

• Include related models with a custom scope Include related models with a custom scope #3453
• Recursive inclusion of related models Recursive inclusion of related models #3454
• Reject queries with incompatible "filter.fields" and "filter.include" Reject queries with incompatible "filter.fields" and "filter.include" #3455
• Spike: robust handling of ObjectID type for MongoDB Spike: robust handling of ObjectID type for MongoDB #3456