The first step towards pre generated IDs. And how this will improve your Integration tests

Recently on the Magento DevBlog there was an article describing pros and cons for different Entity ID Allocation strategies.

All the authors, as well as most of the readers who left feedbacks for this article, agreed that pre-generated IDs for Magento 2 is the way to go.

But currently, Magento 2 Resource Models are strongly dependent on Database IDs auto-generation as Magento does not provide service for ID allocation and $_isPkAutoIncrement = false is not acceptable for all cases.

Magento\Framework\Model\ResourceModel\Db\AbstractDb::isObjectNotNew(\Magento\Framework\Model\AbstractModel $object)

namespace Magento\Framework\Model\ResourceModel\Db;

abstract class AbstractDb extends AbstractResource
{
//...
    if ($this->isObjectNotNew($object)) {
        $this->updateObject($object);
    } else {
        $this->saveNewObject($object);
    }
//...
    /**
     * Check if object is new
     *
     * @param \Magento\Framework\Model\AbstractModel $object
     * @return bool
     */
    protected function isObjectNotNew(\Magento\Framework\Model\AbstractModel $object)
    {
        return $object->getId() !== null && (!$this->_useIsObjectNew || !$object->isObjectNew());
    }
//...
}

This check $object->getId() !== null does not allow you to specify ID for newly created entities.

The current approach has been working from the ancient times (Magento 1), and most of Magento developers got used to it. So, what's wrong with it?

Let's see how our Integration and Web API Functional tests look like when we rely on Database generated IDs on the example of MSI (Multi Source Inventory) project. We can't just create desired Entity in the test fixture and specify ID for it, to use this pre-specified ID to retrieve the same entity in Test. We can't predict which ID would be specified by MySQL Auto Increment key, it depends whether you re-install database, or re-use it from the previous installation, etc. Thus, what we need to do in the test method to provide a robust implementation to retrieve the entity created in the fixture? - Correct, search for this Entity with getList method specifying search criteria filters given from the fixture.

In our example, we have Sources and Stocks entities. For those entities we have next fixtures:

• Fixture which creates Sources
• Fixture which creates Stocks

Now we want to make Many-to-Many linkage (assignments). To test different use-cases with Source To Stock assignment functionality. Because we don't have another valid method of refering entities which were created in dedicated fixtures as pre-condition. We end up with the following:

Boilerplate code for retrieving Sources by Name

/** @var SourceRepositoryInterface $sourceRepository */
$sourceRepository = Bootstrap::getObjectManager()->get(SourceRepositoryInterface::class);
/** @var SearchCriteriaBuilder $searchCriteriaBuilder */
$searchCriteriaBuilder = Bootstrap::getObjectManager()->get(SearchCriteriaBuilder::class);
/** @var SortOrderBuilder $sortOrderBuilder */
$sortOrderBuilder = Bootstrap::getObjectManager()->create(SortOrderBuilder::class);

$sortOrder = $sortOrderBuilder
    ->setField(SourceInterface::NAME)
    ->setDirection(SortOrder::SORT_ASC)
    ->create();
$searchCriteria = $searchCriteriaBuilder
    ->addFilter(SourceInterface::NAME, ['source-name-1', 'source-name-2', 'source-name-3', 'source-name-4'], 'in')
    ->addSortOrder($sortOrder)
    ->create();
/** @var \Magento\InventoryApi\Api\Data\SourceInterface[] $sources */
$sources = array_values($sourceRepository->getList($searchCriteria)->getItems());

Boilerplate code for retrieving Stocks by Name

$sortOrder = $sortOrderBuilder
    ->setField(StockInterface::NAME)
    ->setDirection(SortOrder::SORT_ASC)
    ->create();
$searchCriteria = $searchCriteriaBuilder
    ->addFilter(StockInterface::NAME, ['stock-name-1', 'stock-name-2'], 'in')
    ->addSortOrder($sortOrder)
    ->create();
/** @var StockInterface[] $stocks */
$stocks = array_values($stockRepository->getList($searchCriteria)->getItems());

The code of Business Logic - creation Source to Stock assignments

/** @var AssignSourcesToStockInterface $assignSourcesToStock */
$assignSourcesToStock = Bootstrap::getObjectManager()->get(AssignSourcesToStockInterface::class);
$assignSourcesToStock->execute([$sources[0]->getSourceId(), $sources[1]->getSourceId()], $stocks[0]->getStockId());
$assignSourcesToStock->execute([$sources[2]->getSourceId(), $sources[3]->getSourceId()], $stocks[1]->getStockId());

Source to Stock assignment - source code.

Especially when we want to cover sophisticated use cases which involve a lot of preconditions to be done - we need to create a lot of Boilerplate code.

How did we solve that issue on MSI project?

We changed the default behavior of Resource Models responsible for entity persistence. Now we make an SQL query checking whether entity already exists in the database or not. And depending on that we build Insert or Update query.

This is done in Trait

namespace Magento\Inventory\Model\ResourceModel;

use Magento\Framework\Exception\LocalizedException;
use Magento\Framework\Model\AbstractModel;

/**
 * Provides possibility of saving entity with predefined/pre-generated id
 *
 * The choice to use trait instead of inheritance was made to prevent the introduction of new layer super type on
 * the module basis as well as better code reusability, as potentially current trait not coupled to Inventory module
 * and other modules could re-use this approach.
 */
trait PredefinedId
{
    /**
     * Overwrite default \Magento\Framework\Model\ResourceModel\Db\AbstractDb implementation of the isObjectNew
     * @see \Magento\Framework\Model\ResourceModel\Db\AbstractDb::isObjectNew()
     *
     * Adding the possibility to check whether record already exists in DB or not
     *
     * @param AbstractModel $object
     * @return bool
     */
    protected function isObjectNotNew(AbstractModel $object)
    {
        $connection = $this->getConnection();
        $select = $connection->select()
            ->from($this->getMainTable(), [$this->getIdFieldName()])
            ->where($this->getIdFieldName() . ' = ?', $object->getId())
            ->limit(1);
        return (bool)$connection->fetchOne($select);
    }

    /**
     * Save New Object
     *
     * Overwrite default \Magento\Framework\Model\ResourceModel\Db\AbstractDb implementation of the saveNewObject
     * @see \Magento\Framework\Model\ResourceModel\Db\AbstractDb::saveNewObject()
     *
     * @param \Magento\Framework\Model\AbstractModel $object
     * @throws LocalizedException
     * @return void
     */
    protected function saveNewObject(\Magento\Framework\Model\AbstractModel $object)
    {
        $bind = $this->_prepareDataForSave($object);
        $this->getConnection()->insert($this->getMainTable(), $bind);

        if ($this->_isPkAutoIncrement) {
            $object->setId($this->getConnection()->lastInsertId($this->getMainTable()));
        }

        if ($this->_useIsObjectNew) {
            $object->isObjectNew(false);
        }
    }
}

We need to overwrite default saveNewObject method to get rid of the unsetting pre-specified ID.

        $bind = $this->_prepareDataForSave($object);
        if ($this->_isPkAutoIncrement) {
            unset($bind[$this->getIdFieldName()]);
        }
        $this->getConnection()->insert($this->getMainTable(), $bind);

This trait is being used by all the ResourceModels in the new Inventory module (for example, Source Resoruce Model).

After applying approach of Pre-Generated IDs, our code of Source to Stock assignments shrunk to

/** @var AssignSourcesToStockInterface $assignSourcesToStock */
$assignSourcesToStock = Bootstrap::getObjectManager()->get(AssignSourcesToStockInterface::class);
$assignSourcesToStock->execute([1, 2], 1);
$assignSourcesToStock->execute([3], 2);

Just for this particular case (Source to Stock assignment) - we got rid of more than 300 lines of boilerplate code in our Integration tests using Pre-Generated IDs approach.