Assertion error when foreign key reference to undeclared table

[CBroz1]

Bug Report

Description

When using a foreign key reference of a table that is in the context namespace, but not declared, I hit this assertion error for QueryExpression.support.

Reproducibility

• OS: MACOS
• Python Version: 3.9.13
• MySQL Version: 8.0.28
• MySQL Deployment Strategy: remote
• DataJoint Version: 0.14.0
• Minimum number of steps to reliably reproduce the issue:

import datajoint as dj
from element_lab import lab
from element_lab.lab import Lab

db_prefix = dj.config["custom"].get("database.prefix", "")
schema = dj.schema(db_prefix + "example")

# lab.activate(db_prefix + "lab") # Forgetting this line results in vague assertion error

@schema
class Example(dj.Manual):
    definition = """
    -> Lab
    """

• Error stack

Expected Behavior

Increased error specificity would help new users navigate our design pattern of deferred activation. I'd propose adding to this block in compile_foreign_key.

    if isinstance(ref, type) and issubclass(ref, Table):
        ref = ref()
        try:
            _ = ref.support
        except AssertionError:
            raise DataJointError(
                f"Could not initialize this table as a foreign key reference: {result.ref_table}.\n"
                + "Does this schema need to be activated?"
            )

Screenshots

If applicable, add screenshots to help explain your problem.

Additional Research and Context

This stack overflow user may have encountered similar issues

[dimitri-yatsenko]

@CBroz1, we are intending to get away from these deferred schemas and from the concept of activate. It was intended for building modular code but we have other ways to make code modular. So we will probably not even describe in new tutorials how to create deferred schemas.

[CBroz1]

Makes sense - we can tag with wontfix and close. Are there any docs on the new means of modularization?

[dimitri-yatsenko]

The main problem is not modularization but customization. We created activate so that we can effectively monkey-patch the Elements in custom ways. This allows keeping the Elements themselves unchanged in custom configurations. This saves the hassle of maintaining custom versions of Elements at the expense of extra complexity.

We now thinking that the extra code complexity is a bigger problem. We would rather see people create their custom forks of the pipelines than monkey patch fixed pipelines. The current version control infrastructure makes it more tractable too.

Therefore, no new documentation is necessary. We will simply use GitHub to keep track of multiple forks and branches of pipelines as we customize them, propagating all new improvements across them.