-
Notifications
You must be signed in to change notification settings - Fork 9.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
catalog_product_price index getting stuck #36471
Comments
Hi @vseager. Thank you for your report.
Make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:
For more details, review the Magento Contributor Assistant documentation. Add a comment to assign the issue: To learn more about issue processing workflow, refer to the Code Contributions.
🕙 You can find the schedule on the Magento Community Calendar page. 📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket. ✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel |
Hello, I'm having this problem updating product inventory directly using I believe this problem was introduced by the commit 263c17f, where the db_schema.xml was modified, and an "id" column was added to the catalog_product_index_price_tmp table (and the PK set against just that column), which makes the schema for that table different than catalog_product_index_price. Through the We don't seem to be seeing this error running a full reindex as that code path eventually leads to the function I've been looking to put together a composer patch to get the |
Hi @engcom-November. Thank you for working on this issue.
|
Hi @vseager , Thank you for reporting and collaboration. Verified the issue on Magento 2.4-develop instance but the issue is not reproducible with below steps performed:
|
We have faced the same error, Magento 2.4.5-p1, in the cron_schedule table many error rows having messages: |
Hello @engcom-November, I'd suggest that the conditions to reproduce this issue include the adjustment of some products to change their is_in_stock/manage_stock/etc values. The full If you have your indexes (at least price index) configured to update on save, you can simply take a product that is not managed by inventory (default not managed), go into the product settings and change the configuration to manage inventory, leave the qty at zero so it will be calculated to out of stock, save it, and now it's no longer in categories. Do a full reindex ( The important thing is that you do not see exceptions on the frontend or when running the CLI, because they are caught inside the indexer and written to logs. It does not bubble up from there, and there's no indication in calling code that there was any problem whatsoever, you have to be looking through the logs to see that anything went wrong or see the symptoms from the frontend. |
Hey @vseager and @alek-s-andr, is there any chance that you have custom product types defined, possibly with third party modules? I have been able to determine that the disappearing products and logged query exceptions are limited to my custom product types. I did not declare an "indexerModel" for them, and it looks like the default indexerModel that gets used for any product type which does not have one declared is |
I have two changes that I'm using on a site now to resolve this problem. The first one is a composer patch against magento/module-catalog, and just adjusts that
Next, for my modules defining custom product types, I'm adding a definition for an Hopefully this helps somebody - and if anyone sees anything troubling with these adjustments, by all means please let me know! |
Hi @jaminion, yes we have custom product type from third party module. |
Hi @alek-s-andr , you might peek at the "etc/product_types.xml" file in that third party module, see if there is a declaration for an |
Hi @alek-s-andr / @jaminion , |
Hello @engcom-November , this is a wee bit tricky as it appears that the code which had a recent regression or got "broken" is part of the magento/magento2 repository - however it's only called in the circumstance that a third party module has a product type defined without defining a custom indexerModel, and that is/was optional. The default indexerModel which gets used is not the same indexerModel that is used for a Simple product, instead it's a different class ( I think the only way to confirm it through testing is adding a simple module on develop instance, I'm not sure that's allowed? |
I think #36370 might fix this, could somebody maybe confirm? |
@hostep the changes in the PR look great, simpler form of what I'm using in a composer patch noted above, exciting to see that changes to fix this are in the works with a P1 tag. I have not been having any of the symptoms on one production site since implementing that patch on 2022-11-17, but I'd hesitate to make further changes to that site without an official patch. Getting those target columns defined for the select query fixes it. Thank you for sharing! |
Verified the issue again on Magento 2.4-develop instance having 10k products and by adding custom module: new product type but still could not reproduce the issue with below steps performed:
|
I'm having the same issue on 2.4.5-p1 right now - Price indexer crashes. Which screws up the Catalog Price Rules - the prices get completely screwed up for discounted groups. We have daily import of prices / products / stock for 80,000 products. Total catalog is 250,000. |
@engcom-November can you please try adding Catalog Price Rule discounts? We've noticed the issue increased after we added 10 more groups with different discounts. Since the Price Indexer runs over the Catalog Price rules and creates the discounted prices, it loads up the indexer with new data. |
The full reindex somehow works. I would suggest trying to import lets say 1000 products and then letting them reindex by themselves. |
Thanks @jaminion !! I will try this out over the weekend on my DEV machine and see if this fixes issues. |
Hey @pmonosolo, I'd suggest taking a look at that #36370 mentioned above by hostep - that's generally the same thing at least when I reviewed it earlier, they're trying to solve the same problem and that is code that I'd expect to actually get committed in the future. It would be useful for them to get feedback on that as well - it should be pretty easy to pull those changes as a patch. Either way hopefully you have success, good luck! |
✅ Jira issue https://jira.corp.adobe.com/browse/AC-7468 is successfully created for this GitHub issue. |
✅ Confirmed by @engcom-Hotel. Thank you for verifying the issue. |
Hello, As I can see this issue got fixed in the scope of the internal Jira ticket ACP2E-1401 by the internal team Based on the Jira ticket, the target version is 2.4.6. Thanks |
Any update on the target version for this? Not fixed in 2.4.6-p1. Though I suppose it doesn't matter as the custom indexer model workaround is easy enough. Edit: To be explicit, my workaround for this was adding this in my custom product type tag:
|
Thanks, this saved my day! |
This caused a similar problem for my company (running Magento 2.4.6-p3). The new id primary key on the temporary table slowed down the inventory indexer to the point where it could never complete. We reverted the db_schema.xml so that the 3 previous primary keys were restored then rebuilt the codebase (you need to drop the catalog_product_index_price_tmp table on the first run). Then the inventory indexer ran fine, usually completing in approximatley 5 minutes. |
Preconditions and environment
Steps to reproduce
Let indexes run on cron
Expected result
Indexes run successfully
Actual result
catalog_product_price
.Logs show the following errors:
Changing
catalog_product_price
index to "Update on Save" seems to work as a temporary workaround.Additional information
No response
Release note
No response
Triage and priority
The text was updated successfully, but these errors were encountered: