System EAV attributes do not get added to destination attribute set if it was not already associated to the source attribute set

[dfelton]

Environment Information:

Data Migration Tool Version: 2.1.3
Mode: Data
Step: EAV Step
Source Database Version: Magento EE 1.14.2.0
Destination Database Version: Magento EE 2.1.3

Short Description of Issue:

Data migration tool performs an COPY of EAV attribute sets while neglecting to add system attributes (WHERE eav_attribute.is_user_defined = 0) to the destination attribute set, if it was not part of the source attribute set.

Long Description of Issue:

There are a few system EAV attributes in Magento 1, which Magento 1 never adds to the Default attribute set for catalog_product entities. In Magento 2, these system attributes are now part of the Default catalog_product attribute set created during the Magento 2 installation process. Despite this difference in data structure, when the data migration tool migrates the EAV attribute sets over these system attributes are not automatically added to the destination attribute set.

While these attributes can be manually added over post-migration of the EAV data, they should really be added to every migrated catalog_product attribute set automatically as Magento 2 now requires them to be part of the product attribute set for Magento 2 to function properly. For example, one cannot manage a product's categories in the web browser without the category_ids attribute added to the catalog_product entity's attribute set. The entire field will be missing from the category's form since it is no longer part of the attribute set.

In addition to the attributes mentioned above, another issue closely related to this is that system EAV attributes entirely new to Magento 2, also do not get added to the migrated catalog_product attribute sets. At the time of writing this, I am only aware of this affecting the swatch_images catalog_product attribute.

Affected catalog_product attribute_code:

• category_ids (Magento CE & EE)
• giftcard_type (Magento EE)
• price_type (Magento EE)
• shipment_type (Magento EE)
• sku_type (Magento EE)
• weight_type (Magento EE)
• swatch_image (New to Magento 2 CE & EE)

TL;DR;

In Magento 1 category_ids, giftcard_type, price_type, shipment_type, sku_type, weight_type are never added to the eav_entity_attribute table, despite these being system attributes required for basic functionality. Because of this the data migration tool doesn't add it to the destination attribute set since they were not part of the source attribute set.

Additionally, the data migration tool is unaware of brand new Magento 2 system EAV attributes, and therefore neglects to add these attributes to migrated attribute sets. (As of 2.1.3, swatch_images is the only new system attribute I am aware of)

Related GitHub Issues:

If everything I've laid forth here is correct, then I also believe this issue to be the root cause of these issue reports:

• Attribute groups are not taken into consideration while migrating #84
• Product category_ids missing from migrated attribute sets #89
• Categories-option not displayed on Product-page in Catalogue of backend after migration M 1.7.0.3 -> M 2.1.3 #211

[victor-v-rad]

Thank you @dfelton for this detailed issue. It is uncommon for the attributes (category_ids, giftcard_type, price_type ...) not to have its related records in eav_entity_attribute table. But apparently there are such exceptional cases. Currently I am busy with another bug and be back right after that (internal ticket MAGETWO-63104 to fix the current one). Anyway I greatly appreciate your input throughout the issues of this project.

[dfelton]

Your most welcome @victor-v-rad. It is quite an impressive migration tool you guys have built so thanks a ton for providing it to the community in the first place.

[victor-v-rad]

Fixed

[dfelton]

@victor-v-rad Thank you to both you and the rest of the Magento team!