Unable to push publish plugins #21568

swicken-dotcms · 2022-01-20T14:47:49Z

Describe the bug
Attempting to push publish plugins results in an error attempting to generate a directory

[20/01/22 14:43:49:427 UTC]  INFO publishing.PublisherAPIImpl: Start of Bundler: OSGIBundler
[20/01/22 14:43:49:445 UTC] ERROR publishing.PublisherAPIImpl: com.dotcms.enterprise.publishing.remote.bundler.OSGIBundler : generate()Destination '/osgiBundles/com.dotcms.rest-osgi-debug-0.2.jar' directory cannot be created: Unable to pull content
com.dotcms.publishing.DotBundleException: com.dotcms.enterprise.publishing.remote.bundler.OSGIBundler : generate()Destination '/osgiBundles/com.dotcms.rest-osgi-debug-0.2.jar' directory cannot be created: Unable to pull content
	at com.dotcms.enterprise.publishing.remote.bundler.OSGIBundler.generate(SourceFile:119) ~[ee-22.01.jar:?]
	at com.dotcms.publishing.PublisherAPIImpl.publish_aroundBody0(PublisherAPIImpl.java:118) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publishing.PublisherAPIImpl$AjcClosure1.run(PublisherAPIImpl.java:1) ~[dotcms_22.01_999999.jar:?]
	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjrt-1.9.2.jar:?]
	at com.dotcms.aspects.aspectj.AspectJDelegateMethodInvocation.proceed(AspectJDelegateMethodInvocation.java:42) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.aspects.interceptors.CloseDBIfOpenedMethodInterceptor.invoke(CloseDBIfOpenedMethodInterceptor.java:29) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.aspects.aspectj.CloseDBIfOpenedAspect.invoke(CloseDBIfOpenedAspect.java:41) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publishing.PublisherAPIImpl.publish(PublisherAPIImpl.java:47) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publishing.PublisherAPIImpl.publish(PublisherAPIImpl.java:42) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publisher.business.PublisherQueueJob.execute_aroundBody0(PublisherQueueJob.java:186) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publisher.business.PublisherQueueJob$AjcClosure1.run(PublisherQueueJob.java:1) ~[dotcms_22.01_999999.jar:?]
	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjrt-1.9.2.jar:?]
	at com.dotcms.aspects.aspectj.AspectJDelegateMethodInvocation.proceed(AspectJDelegateMethodInvocation.java:42) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.aspects.interceptors.CloseDBIfOpenedMethodInterceptor.invoke(CloseDBIfOpenedMethodInterceptor.java:29) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.aspects.aspectj.CloseDBIfOpenedAspect.invoke(CloseDBIfOpenedAspect.java:41) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publisher.business.PublisherQueueJob.execute(PublisherQueueJob.java:112) ~[dotcms_22.01_999999.jar:?]
	at org.quartz.core.JobRunShell.run(JobRunShell.java:223) ~[dot.quartz-all-1.8.6_2.jar:?]
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549) ~[dot.quartz-all-1.8.6_2.jar:?]
Caused by: java.io.IOException: Destination '/osgiBundles/com.dotcms.rest-osgi-debug-0.2.jar' directory cannot be created
	at com.dotcms.repackage.org.apache.commons.io.FileUtils.copyFile(FileUtils.java:777) ~[dot.commons-io-2.0.1_2.jar:2.0.1]
	at com.dotcms.repackage.org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:706) ~[dot.commons-io-2.0.1_2.jar:2.0.1]
	at com.dotcms.enterprise.publishing.remote.bundler.OSGIBundler.generate(SourceFile:102) ~[ee-22.01.jar:?]
	... 17 more
[20/01/22 14:43:49:445 UTC] ERROR business.PublisherQueueJob: Unable to publish Bundle '01FSVZ34DPME6FZT8ZEE90KSXC': com.dotcms.enterprise.publishing.remote.bundler.OSGIBundler : generate()Destination '/osgiBundles/com.dotcms.rest-osgi-debug-0.2.jar' directory cannot be created: Unable to pull content
com.dotcms.publishing.DotPublishingException: com.dotcms.enterprise.publishing.remote.bundler.OSGIBundler : generate()Destination '/osgiBundles/com.dotcms.rest-osgi-debug-0.2.jar' directory cannot be created: Unable to pull content
	at com.dotcms.publishing.PublisherAPIImpl.publish_aroundBody0(PublisherAPIImpl.java:154) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publishing.PublisherAPIImpl$AjcClosure1.run(PublisherAPIImpl.java:1) ~[dotcms_22.01_999999.jar:?]
	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjrt-1.9.2.jar:?]
	at com.dotcms.aspects.aspectj.AspectJDelegateMethodInvocation.proceed(AspectJDelegateMethodInvocation.java:42) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.aspects.interceptors.CloseDBIfOpenedMethodInterceptor.invoke(CloseDBIfOpenedMethodInterceptor.java:29) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.aspects.aspectj.CloseDBIfOpenedAspect.invoke(CloseDBIfOpenedAspect.java:41) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publishing.PublisherAPIImpl.publish(PublisherAPIImpl.java:47) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publishing.PublisherAPIImpl.publish(PublisherAPIImpl.java:42) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publisher.business.PublisherQueueJob.execute_aroundBody0(PublisherQueueJob.java:186) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publisher.business.PublisherQueueJob$AjcClosure1.run(PublisherQueueJob.java:1) ~[dotcms_22.01_999999.jar:?]
	at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) ~[aspectjrt-1.9.2.jar:?]
	at com.dotcms.aspects.aspectj.AspectJDelegateMethodInvocation.proceed(AspectJDelegateMethodInvocation.java:42) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.aspects.interceptors.CloseDBIfOpenedMethodInterceptor.invoke(CloseDBIfOpenedMethodInterceptor.java:29) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.aspects.aspectj.CloseDBIfOpenedAspect.invoke(CloseDBIfOpenedAspect.java:41) ~[dotcms_22.01_999999.jar:?]
	at com.dotcms.publisher.business.PublisherQueueJob.execute(PublisherQueueJob.java:112) ~[dotcms_22.01_999999.jar:?]
	at org.quartz.core.JobRunShell.run(JobRunShell.java:223) ~[dot.quartz-all-1.8.6_2.jar:?]
	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549) ~[dot.quartz-all-1.8.6_2.jar:?]
Caused by: com.dotcms.publishing.DotBundleException: com.dotcms.enterprise.publishing.remote.bundler.OSGIBundler : generate()Destination '/osgiBundles/com.dotcms.rest-osgi-debug-0.2.jar' directory cannot be created: Unable to pull content
	at com.dotcms.enterprise.publishing.remote.bundler.OSGIBundler.generate(SourceFile:119) ~[ee-22.01.jar:?]
	at com.dotcms.publishing.PublisherAPIImpl.publish_aroundBody0(PublisherAPIImpl.java:118) ~[dotcms_22.01_999999.jar:?]
	... 16 more
Caused by: java.io.IOException: Destination '/osgiBundles/com.dotcms.rest-osgi-debug-0.2.jar' directory cannot be created
	at com.dotcms.repackage.org.apache.commons.io.FileUtils.copyFile(FileUtils.java:777) ~[dot.commons-io-2.0.1_2.jar:2.0.1]
	at com.dotcms.repackage.org.apache.commons.io.FileUtils.copyFileToDirectory(FileUtils.java:706) ~[dot.commons-io-2.0.1_2.jar:2.0.1]
	at com.dotcms.enterprise.publishing.remote.bundler.OSGIBundler.generate(SourceFile:102) ~[ee-22.01.jar:?]
	at com.dotcms.publishing.PublisherAPIImpl.publish_aroundBody0(PublisherAPIImpl.java:118) ~[dotcms_22.01_999999.jar:?]
	... 16 more
[20/01/22 14:43:49:445 UTC]  INFO util.PushPublishLogger: class com.dotcms.publisher.business.PublisherQueueJob : Status Update: Failed to bundle '01FSVZ34DPME6FZT8ZEE90KSXC'

Reproduced on: 21.06.6, 22.01
Related Ticket: https://dotcms.zendesk.com/agent/tickets/106313

To Reproduce

Create a sender/receiver environment
Add a plugin to the sender
Attempt to push publish that plugin

Expected behavior
The plugin should push successfully.

discussion #21590

The text was updated successfully, but these errors were encountered:

wezell · 2022-01-21T16:08:45Z

This does not work in docker environments. When bundling OSGI plugins, we should use a subfolder of the normal /asset/bundle folder as that folder has the proper permissions, is shared and gets automatically cleaned up etc...

freddyucv · 2022-01-24T16:28:25Z

Hey team! Please add your planning poker estimate with ZenHub @dsilvam @victoralfaro-dotcms

bryanboza · 2022-02-17T20:32:00Z

Hey guys I have a couple of questions here that maybe we need to take in consideration...

Does this work with the OSGI, static ones or both?

Also, what should expect in case you already have the plugin in the receiver side, should we have integrity checker for plugins? or just should replace the existing one without confirmation ?

Taking in consideration this change can broke some features in case of a wrong configuration, maybe at least we should create a new property to enable/disable permissions to override the configuration.

swicken-dotcms · 2022-02-18T14:34:55Z

Additional ticket: https://dotcms.zendesk.com/agent/tickets/106537

…tead. Json serializing instead of xml for OSGi push publish.

victoralfaro-dotcms · 2022-02-18T23:47:20Z

PRs:
https://github.com/dotCMS/enterprise/pull/854
#21752

…ing, centralizing ObjectMapper and XStream instances to one and addind plugin case to Dependency manager.

erickgonzalez · 2022-02-24T20:12:31Z

Need to test in 5.3.8 series, probably not needed since was introduced with the changes in PP (21.05ish)

…ing, centralizing ObjectMapper and XStream instances to one and addind plugin case to Dependency manager.

…ing, centralizing ObjectMapper and XStream instances to one and addind plugin case to Dependency manager. (#21974)

dsilvam · 2022-04-06T21:39:38Z

PR ee: https://github.com/dotCMS/enterprise/pull/866

freddyucv · 2022-04-11T21:20:34Z

Pass Internal QA

it was test with this plugins: https://github.com/dotCMS/plugin-seeds/tree/master/OSGi/com.dotcms.rest

Start the sender in 8080 port.
Start receiver in 8090 port.
Upload the bundle in the sender.
Test the Plugins with: curl -XPUT http://localhost:8080/api/example
Response: {"result":"anonymous user anonymous PUT!"}
Test the Plugins with: curl --head -XPUT http://localhost:8090/api/example
Response: 404 HTTP Code
PP the Plugins to the receiver
Test Again the Plugins with: curl --head -XPUT http://localhost:8090/api/example
Response: {"result":"anonymous user anonymous PUT!"}
Change the code here:

https://github.com/dotCMS/plugin-seeds/blob/master/OSGi/com.dotcms.rest/src/main/java/com/dotcms/plugin/rest/ExampleResource.java#L36

Change " PUT!" by " PUT! With changes"

Upload the bundle again in the sender and PP it to the receiver
Now test with: curl -XPUT http://localhost:8090/api/example
Response: {"result":"anonymous user anonymous PUT! With changes"}

freddyucv · 2022-04-11T21:26:55Z

Hey guys I have a couple of questions here that maybe we need to take in consideration...

Does this work with the OSGI, static ones or both?

Also, what should expect in case you already have the plugin in the receiver side, should we have integrity checker for plugins? or just should replace the existing one without confirmation ?

Taking in consideration this change can broke some features in case of a wrong configuration, maybe at least we should create a new property to enable/disable permissions to override the configuration.

About this one: "Taking in consideration this change can broke some features in case of a wrong configuration, maybe at least we should create a new property to enable/disable permissions to override the configuration."

This is a bug is not a new issue

About this one: "Also, what should expect in case you already have the plugin in the receiver side, should we have integrity checker for plugins? or just should replace the existing one without confirmation ?"

This replace the existing one, this have the same behavior in the sender when you change a Plugin and Upload this again, so I think it is right.

About this one: "Does this work with the OSGI, static ones or both?"

the static plugins are deprecated

bryanboza · 2022-04-13T01:00:57Z

We need some work here, every time you push any plugin we are getting this error in the log:
https://gist.github.com/bryanboza/b40712d8d129c0fc8518288b3493d7c9

bryanboza · 2022-06-06T19:50:32Z

Fixed, tested on release-22.06 and now we don't have the error in the log

swicken-dotcms · 2022-08-30T19:20:18Z

Will not be backporting to LTS releases due to changes to bundle manifest files and an upgrade task requirement

swicken-dotcms added Type : Defect Severity : Support Requested labels Jan 20, 2022

freddyucv added the Team : Falcon label Jan 21, 2022

freddyucv assigned freddyucv and victoralfaro-dotcms and unassigned freddyucv Feb 9, 2022

victoralfaro-dotcms added a commit that referenced this issue Feb 18, 2022

(#21568): Removing xstream class fields and using local variables ins…

b29dfe3

…tead. Json serializing instead of xml for OSGi push publish.

victoralfaro-dotcms added a commit that referenced this issue Feb 18, 2022

(#21568): Removing xstream class fields and using local variables ins…

e9c110a

…tead. Json serializing instead of xml for OSGi push publish.

victoralfaro-dotcms added the Needs Work label Feb 24, 2022

victoralfaro-dotcms added a commit that referenced this issue Feb 24, 2022

(#21568): adding manifest file for plugin PP to avoid XML bundle writ…

409dff2

…ing, centralizing ObjectMapper and XStream instances to one and addind plugin case to Dependency manager.

victoralfaro-dotcms removed the Needs Work label Feb 24, 2022

wezell added the LTS: Needs Backport Ticket that will be added to LTS label Feb 24, 2022

victoralfaro-dotcms added a commit that referenced this issue Feb 28, 2022

(#21568): adding manifest file for plugin PP to avoid XML bundle writ…

5e28ed6

…ing, centralizing ObjectMapper and XStream instances to one and addind plugin case to Dependency manager.

victoralfaro-dotcms added a commit that referenced this issue Feb 28, 2022

(#21568): adding manifest file for plugin PP to avoid XML bundle writ…

1d1f608

…ing, centralizing ObjectMapper and XStream instances to one and addind plugin case to Dependency manager.

victoralfaro-dotcms added a commit that referenced this issue Mar 1, 2022

(#21568): adding manifest file for plugin PP to avoid XML bundle writ…

6f934b9

…ing, centralizing ObjectMapper and XStream instances to one and addind plugin case to Dependency manager.

victoralfaro-dotcms added a commit that referenced this issue Apr 6, 2022

(#21568): adding manifest file for plugin PP to avoid XML bundle writ…

f82a56f

…ing, centralizing ObjectMapper and XStream instances to one and addind plugin case to Dependency manager.

dsilvam pushed a commit that referenced this issue Apr 6, 2022

(#21568): adding manifest file for plugin PP to avoid XML bundle writ…

1ce844c

…ing, centralizing ObjectMapper and XStream instances to one and addind plugin case to Dependency manager. (#21974)

dsilvam linked a pull request Apr 6, 2022 that will close this issue

(#21568): adding manifest file for plugin PP to avoid XML bundle writ… #21974

Merged

dsilvam unassigned victoralfaro-dotcms Apr 6, 2022

dsilvam added QA : Needs Internal Release : 22.05 labels Apr 6, 2022

swicken-dotcms added the LTS: Priority label Apr 7, 2022

freddyucv self-assigned this Apr 11, 2022

freddyucv removed the QA : Needs Internal label Apr 11, 2022

freddyucv added the QA : Passed Internal label Apr 11, 2022

freddyucv removed their assignment Apr 11, 2022

bryanboza added the Needs Work label Apr 13, 2022

wezell added Release : 22.06 and removed Release : 22.05 labels Apr 13, 2022

victoralfaro-dotcms mentioned this issue Apr 13, 2022

#issue-21658: Increasing publishing_pushed_assets.asset_id capacity t… #22028

Closed

dsilvam linked a pull request Apr 27, 2022 that will close this issue

(#21658): Increasing publishing_pushed_assets.asset_id capacity to 2… #22031

Merged

dsilvam added QA : Needs Internal Merged and removed Needs Work QA : Passed Internal labels Apr 27, 2022

freddyucv self-assigned this Apr 27, 2022

freddyucv added QA : Passed Internal and removed QA : Needs Internal labels May 9, 2022

freddyucv removed their assignment May 9, 2022

victoralfaro-dotcms added a commit that referenced this issue May 31, 2022

(#21568): Fix of upgrade task

990aef4

dsilvam pushed a commit that referenced this issue Jun 1, 2022

(#21568): Fix of upgrade task (#22321)

d31edc4

bryanboza added the QA : Approved label Jun 6, 2022

wezell closed this as completed Jun 8, 2022

oidacra pushed a commit that referenced this issue Jun 13, 2022

(#21568): Fix of upgrade task (#22321)

f4bee95

swicken-dotcms added LTS: Excluded Ticket that has been excluded from at least one LTS and removed LTS: Needs Backport Ticket that will be added to LTS LTS: Priority labels Aug 30, 2022

erickgonzalez removed the Severity : Support Requested label Oct 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to push publish plugins #21568

Unable to push publish plugins #21568

swicken-dotcms commented Jan 20, 2022 •

edited by freddyucv

Loading

wezell commented Jan 21, 2022 •

edited

Loading

freddyucv commented Jan 24, 2022

bryanboza commented Feb 17, 2022 •

edited

Loading

swicken-dotcms commented Feb 18, 2022

victoralfaro-dotcms commented Feb 18, 2022 •

edited

Loading

erickgonzalez commented Feb 24, 2022

dsilvam commented Apr 6, 2022

freddyucv commented Apr 11, 2022

freddyucv commented Apr 11, 2022 •

edited

Loading

bryanboza commented Apr 13, 2022

bryanboza commented Jun 6, 2022

swicken-dotcms commented Aug 30, 2022

Unable to push publish plugins #21568

Unable to push publish plugins #21568

Comments

swicken-dotcms commented Jan 20, 2022 • edited by freddyucv Loading

wezell commented Jan 21, 2022 • edited Loading

freddyucv commented Jan 24, 2022

bryanboza commented Feb 17, 2022 • edited Loading

swicken-dotcms commented Feb 18, 2022

victoralfaro-dotcms commented Feb 18, 2022 • edited Loading

erickgonzalez commented Feb 24, 2022

dsilvam commented Apr 6, 2022

freddyucv commented Apr 11, 2022

freddyucv commented Apr 11, 2022 • edited Loading

bryanboza commented Apr 13, 2022

bryanboza commented Jun 6, 2022

swicken-dotcms commented Aug 30, 2022

swicken-dotcms commented Jan 20, 2022 •

edited by freddyucv

Loading

wezell commented Jan 21, 2022 •

edited

Loading

bryanboza commented Feb 17, 2022 •

edited

Loading

victoralfaro-dotcms commented Feb 18, 2022 •

edited

Loading

freddyucv commented Apr 11, 2022 •

edited

Loading