Media encoding is pending

[ludashi2020]

when upload video,it display Media encoding is pending

https://v.qf.al/view?m=IzEkqKQ2q

[ludashi2020]

Please help me,thank you

[mgogoulos]

Hi, how did you install the software, was it through the installation script or through the Docker Compose setup? What operating system does the server have? Did it install well? Checkout logs on logs/ dir, they might contain something useful (celery_long.log or debug.log in this case). Most probably something didn't start well (or wasn't installed correctly)

[gam-ragnar]

Hi @mgogoulos I think logs should be moved to /var/log/mediacms/*

[mgogoulos]

Hi @timfam , valid point, could be part of some small refactoring - on the other side I personally like the fact that you can have everything (media files, logs) on a single directory

[gam-ragnar]

@mgogoulos placing at /var/log/mediacms/* allows for log rotation, better debugging, storage and backup.

[Dan1ell]

How about writing the logs to both places?

[Dan1ell]

I uploaded one mp4 and it went fine. I uploaded another mp4 and it gave 'media encoding is pending'.

Both are there. Both play.
Not sure what made one go well and the other glitch though.
The one that went well was 21 secs long and the one that glitched was 12 min and 30 secs long.

[Dan1ell]

OK, so I dug into this one because I started seeing it on every upload.

When adding a video file to MediaCMS, the file with upload for a few minutes (depends on each individual's internet speed of course), then the message 'Processing' will be displayed for half a minute, and finally a link to 'View your video' will be displayed in the upper right corner.

The 'Processing' message is misleading. It gave me the impression that the transcoding process was taking place and that when 'View your video' was displayed, transcoding had completed and the video would be there ready to be viewed by me.

However, what actually happens is I click the link and the player says:
Media encoding is pending.

Clearly, transcoding has only just begun. In fact, the various resolutions of the video will only gradually become available over the course of several minutes e.g. 240p after 2 min, 360p 3 minutes after that, 480p 5 minutes later, etc, etc, until 1080p becomes available and the transcoding process is done.

Depending on the size of the file, the media encoding message might be displayed briefly or for several minutes. Once the 240p version becomes available, the media encoding message won't be displayed anymore. The user will just see the 240p version in the player when they click the link.

I suggest we change the message from "Media encoding is pending" to:

The media transcoder is working hard! This will take a few minutes. 
In the meantime, you may continue browsing the site. 

Or something along those lines so that our users don't think the site is broken.

Maybe the 'Processing' message can be made better as well, but I don't know what happens during processing, so I can't make a suggestion. What happens during processing?

[gam-ragnar]

There is a line in settings where you can add a message. However +1 to this adjustment

[Dan1ell]

PRE_UPLOAD_MEDIA_MESSAGE, thank you timfam. I added a message now, but since it is displayed before you click the link, it isn't a substitute for improving the messaging in the player itself.

There is also POST_UPLOAD_AUTHOR_MESSAGE_UNLISTED_NO_COMMENTARY, but that is only for the 'unlisted' workflow. We really need to improve that message in the player.

[Dan1ell]

Actually, I found the line in _commons.js!
It can be changed there for now.
At some point, it should probably be moved to the templates/messages/ folder though.

[mgogoulos]

Nice findings, please open an issue so we can work on them

[ttkrpink]

I have the same problem: Media encoding is pending...

The above discussion does not help in my case.
Debug.log is empty even though I turned on setting.py Debug=True. I tried
systemctl restart postgresql nginx celery_beat mediacms

Restarting the services do not trigger the bebug.log.

The attachment is errorlog.txt, which only has some pagination error.

I also checked the admin page encoding. All of them are in the pending state.

Is there other log file I can check to figure out the problem?

Thanks,

errorlog.txt

[ttkrpink]

I have the same problem: Media encoding is pending...

The above discussion does not help in my case.
Debug.log is empty even though I turned on setting.py Debug=True. I tried
systemctl restart postgresql nginx celery_beat mediacms

Restarting the services do not trigger the bebug.log.

The attachment is errorlog.txt, which only has some pagination errors.

I also checked the admin page encoding. All of them are in a pending state.

Is there other log files I can check to figure out the problem?

And there is no "encoded" folder under /home/mediacms.io/mediacms/media_files. The sub dir under that are: chunks,hls, original, uploads, userlogos.

Thanks,

errorlog.txt

[navillusj]

Got something similar, where all my uploaded videos are just sitting on pending... I have enabled the rest of the MP4 encoding setting.
Rebooted, deleted a few videos and reuploaded but still just sits at pending.
I'm a noob, any help is greatly appreciated
Thank you!

[navillusj]

Got it working, only thing I did was since my first post was changing the media type from "Video" to "------" Save going back to the page of the media, then change it back to "Video" Things are now encoding again.

[devilmind81]

Hi, the same issue here. The trick to switch media type like suggested by @navillusj doesn't work for me.
I'm using a docker-compose.yaml configuration.
I have uploaded more then 350 videos and the last 20 are in pending state.
How can troubleshoot the issue or how can I resume/restart the transcoding process?
Please help me I'm stuck!

Thanks in advance

[stoykovstoyk]

Hi,

If you are running MediaCMS inside VPS try increasing the RAM. I had similar issue and I fixed it by increasing the RAM, because I noticed process terminations and out of memory errors while trying to find what the problem might be. So I increased the amount of RAM and everything works fine now.

[devilmind81]

Hi @stoykovstoyk, thank you. It was exattly my problem. During a troubleshooting on the container i saw that celery_worker cpu was at 160%....after i have incresed RAM the issue is fixed. Another option that i have tested was set limit on the worker resources an it was valid also but the encoding time was not acceptable for me because the number of videos to upload is very high.
Thanks a lot again!

[stoykovstoyk]

@devilmind81 I have another problem which I am not able to solve. If you have it too and if you solved it I will highly appreciate if you share how you did it. My problem is the following:
For videos longer then 30 minutes it does not generate video preview in the timeline bar. What I mean is when I move the mouse over the time line it does not show preview of the videos at that specific time. It works for video files that are not longer then 30 minutes, but for longer files it does not works.
My deployment setup is Ubuntu 20.
Only difference that I noticed between working and not working files is:
For video files that does not works I can see in Medias -> Preview file path field which points to a file. For example: /home/mediacms.io/mediacms/media_files/encoded/1/admin/33c786c148704ba5868d771846b84c35.tmpj5p2ftyf.gif
For videos that works same field is empty.
Do you experience the same behavior? If yes, can you share how you solved it? Video preview in timeline bar is a must have for my setup, that is why I ask. Hope you can share some tips.
Thanks a lot.

[mgogoulos]

Hi, this is a different issue, I've seen it and I'm sure it's documented somewhere but cannot find it right now. There is a setting that has to be changed, some default option system wide that prevents sprites from being generated for bigger videos. I'll try to find it and provide the answer here, plus steps to re-run this for all videos already transcoded

[mgogoulos]

edit /etc/ImageMagick-6/policy.xml

and increase width and height to these lines, for example

  <policy domain="resource" name="width" value="16KP"/>
  <policy domain="resource" name="height" value="16KP"/>

[stoykovstoyk]

God bless you @mgogoulos
I did what you told me and everything is working now.
Without your help I wouldn't be able to fix it.
Thank you again for your help, you are really amazing!

[mgogoulos]

Hi, glad this worked, I've updated the documentation to include an example on how to re-run the task that generates the sprites file - https://github.com/mediacms-io/mediacms/blob/main/docs/admins_docs.md#16-frequently-asked-questions

[srvprivate]

Yes, not being able to see encode status is not helpful for me. Been sitting for an hour. Works faster for smaller files, but I don't know what benchmark to work with here since I have no status of encode. Anything planned for that?

[srvprivate]

Did a completely fresh install with docker-compose, wiped out my entire database, and I still have transcode issues. It seems like a lot of media came through, but there's about 20 that are just stuck on "encode pending". Any help on this matter?

[ARaheemPk]

Probably you get "media encoding is pending" issue after server restart.
You should run mediacms by sudo docker compose up after restarting machine.
It's work for me

[mgogoulos]

Using Docker Compose, the celery_worker container is responsible for transcoding, and more specifically celery_long process. I have noticed that if Docker Compose is not shut down well (eg if you hit ctrl+c and don't let it go dow smoothly) some times celery_long does not start, because it is finding the pid of the process on /var/run/mediacms, a solution to this would be to remove the pid files on /var/run/mediacms/ and restart docker-compose. This is a workaround of course, and celery need run on a better way, but could help in some cases explain why videos get into pending state

[piecurus]

Thanks for all this workarounds. I'm experiencing the same issue. Running with docker and I have folder mediacms in /var/run/. So far, still having the problem

[piecurus]

I will test it ASAP.

Indeed I recognise what you say. The issue happens when many videos are uploaded at the same time. I just cancelled everything and restarted the docker as new and there is no issue when uploading one video at the time.

I'll get back to you ASAP with the answer

[mgogoulos]

It shouldn't matter how many videos are uploaded, since they enter a queue and whenever the system has resources it processes them...I don't see what could go wrong except these two things:

1. either system resources are too low, and there are restarts of services (?)
2. that problem where docker-compose is interrupted without stopping correctly, and next time the processes are started there are stale pid files (this is what we're trying to fix here with the rm!)

[piecurus]

I restarted the celery_worker after removing the files in /var/run/mediacms/ and this is what I have now doing ps aux, which looks good

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  0.9  0.0  36916 30028 ?        Ss   13:28   0:00 /usr/bin/python3 /usr/bin/supervisord
www-data      44 34.1  0.2 111136 94396 ?        S    13:28   0:04 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      46 32.2  0.2 111064 94432 ?        S    13:28   0:04 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      49 33.2  0.2 110072 94432 ?        S    13:28   0:04 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      57  1.1  0.2 112304 83588 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      59  0.4  0.2 112352 79032 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      60  1.0  0.2 112308 88200 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      61  0.2  0.2 112356 85048 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      62  0.8  0.2 112312 88076 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      63  0.2  0.2 112360 85052 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      64  0.8  0.2 112316 88076 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      65  0.2  0.2 112364 85060 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      66  1.1  0.2 112320 88332 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      67  0.3  0.2 112368 85060 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      68  0.8  0.2 112324 88208 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      69  0.3  0.2 112372 85060 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      70  0.3  0.2 113036 86544 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      71  0.3  0.2 112376 85064 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      72  0.4  0.2 113048 86544 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      73  0.2  0.2 112380 85064 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      74  1.0  0.2 112336 87960 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      75  0.2  0.2 112384 85068 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      77  1.1  0.2 112340 88220 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      78  0.3  0.2 112388 85076 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      79  0.2  0.2 111312 85028 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      80  0.3  0.2 112392 85080 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      81  0.2  0.2 111316 85032 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      82  0.2  0.2 112396 85080 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      83  0.2  0.2 111320 85032 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      84  0.1  0.2 112400 85084 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      85  0.1  0.2 111324 85036 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      86  0.2  0.2 112404 85084 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      87  0.2  0.2 111328 85036 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      88  0.2  0.2 112408 85084 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      89  0.2  0.2 111332 85036 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      90  0.2  0.2 112412 85088 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/long1.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/c
www-data      91  0.2  0.2 111336 85040 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      92  0.1  0.2 111340 85040 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      93  0.2  0.2 111344 85044 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
www-data      94  0.1  0.2 111348 85052 ?        S    13:29   0:00 /home/mediacms.io/bin/python3 -m celery worker --pidfile=/var/run/mediacms/short2.pid --loglevel=INFO --logfile=/home/mediacms.io/mediacms/logs/
root          95  0.4  0.0   4608  3584 pts/0    Ss   13:29   0:00 bash
root         101 50.0  0.0   8532  4224 pts/0    R+   13:29   0:00 ps aux

[mgogoulos]

this is good! There's a PR that has to be merged that should address this issue

[piecurus]

FYI when I restarted now the docker, the celery worker was not working. I needed to restart it again.

[Abhi3099]

Hii @mgogoulos
I am facing same type of issue . When i try to upload a small video of around 50mb it is taking around 20 min for it to encode and then i am able to play the video. can you tell me like how can i increase the encoding speed of media?
I deployed Mediacms on a k8s cluster having pods celery-worker-long , celery-beat , celery-worker-short , web with local-settings as a configmap.

[okram2000]

Hi guys,
I am facing the same issue here.

Several videos did not encoding after uploading to the server because of some memory issues that are fixed now.
I can upload new videos that get encoded again.

But how can I retrigger the encoding process for these existing uploaded videos. They still show up as pending.

Setting the Media Type to "-----" and back to "Video" does not trigger any encoding progress here.

I also tried playing around with the "Encoding" entries and the Status of the "Media" entries.
No luck so far.

Do I really have to download the files and reupload them again? This can't be the solution, right?

Edit: I found the solution in the code myself...
Very strange that none of the devs is pointing this out here in the board :-/
Solution seems pretty straight forward as there is an unused function in the code already for this:

Steps are:

connect to one of the containers
docker exec -it mediacms-celery_worker-1 /bin/bash

load the code

source  /home/mediacms.io/bin/activate
python manage.py shell

and run

from files.models import Media
from files.tasks import check_pending_states
check_pending_states()
exit()

logout of the container and
check logs of the encoding happening with
tail -f logs/*