ARM Support

[jacobalberty]

I do not presently have any working ARM systems but I am trying cross building this image to run on ARM systems through qemu.

[rundberg]

The image starts up but is inaccessible from the address, https://192.168.1.200:8443/manage/site/default/dashboard

[jacobalberty]

ou sure you mapped at least port 8443? and what does docker exec <containerid> ps give you?

[rundberg]

I have created the container to run in host mode, basically followed https://www.youtube.com/watch?v=35vZWNx5IRw&t=346s.
Cannot map any ports when choosing the host mode!?!?

[jacobalberty]

You are correct host mode should not require mapping any ports. Do you know how to run the equivalent of docker exec <containerid> ps under qnap?

[rundberg]

I Do not, have looked at some docker tutorials today and using a shell won't be a problem BUT I can't find one under QNAP....

[rundberg]

So I found the terminal window and if I run it with /bin/sh I get a primitive shell.... (sry ;) )
However I do not know the syntax for this environment...

[jacobalberty]

ok try running docker exec <containerid> ps aux

For container id use whatever you put in the name field.

[rundberg]

Not sure if I am to use the meta tags or not so here it goes.

With meta tags:
# exec  ps aux                                                                                                                                                                       
/bin/sh: 1: cannot open test: No such file 

Without meta tags:
# exec test ps aux                                                                                                                                                                         
test: missing argument after 'aux'                                                                                                                                                         
                                                                                                                                                                                           
    

[jacobalberty]

Need to include docker before exec

…

On Aug 17, 2017 2:57 PM, "rundberg" ***@***.***> wrote: Not sure if I am to use the meat tags or not so here it goes. *With meta tags:* *# exec ps aux /bin/sh: 1: cannot open test: No such file* *Without meta tags:* # exec test ps aux test: missing argument after 'aux' — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB42gjen4fEQQKWAv3UhXwRs83g-s5u0ks5sZJszgaJpZM4O5T8C> .

[rundberg]

# docker exec test ps aux                                                                                                                                                                  
/bin/sh: 1: docker: not found                                                                                                                                                              
# docker exec  ps aux                                                                                                                                                                
/bin/sh: 2: cannot open test: No such file                                                                                                                                                 
/bin/sh: 2: docker: not found                                                                                                                                                              
#    

[rundberg]

bump

Anything else i can do to help?

[jacobalberty]

I'm not sure the best way to get docker commands going in QNAP, it seems it has package folders and it may be different from system to system. I'm trying running it under qemu instead.

…

On Mon, Aug 21, 2017 at 12:30 PM, rundberg ***@***.***> wrote: *bump* Anything else i can do to help? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#54 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB42gu0po8qCv98IhCF_-vtCBka-4itFks5sab7HgaJpZM4O5T8C> .

[rundberg]

bump*

any progress so far m8?

[jacobalberty]

Sorry been meaning to try and post. I've tried a couple different routes. I'm thinking QEMU may have issues running the jvm try this forum post for running the docker exec <containerid> ps aux.

[kenperkins]

OK So I'm following along testing this on my new Pi3 with arm32v7-beta image.

Container is up and running, currently at 100% utilization in java:

 PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 1856 root      20   0 1155812  32800   9240 S 100.1  3.5   0:12.89 java

After generating the certificate it appears to have started flapping:

[2017-09-10 12:09:30,560] <launcher> INFO  system - Valid keystore is missing. Generating one ...
[2017-09-10 12:09:30,561] <launcher> INFO  system - Generating Certificate[UniFi]... please wait...
[2017-09-10 12:13:24,599] <launcher> INFO  system - Certificate[UniFi] generated!
[2017-09-10 12:14:33,995] <launcher> INFO  system - ======================================================================
[2017-09-10 12:14:34,000] <launcher> INFO  system - UniFi 5.5.20 (build atag_5.5.20_9565 - release) is started
[2017-09-10 12:14:34,000] <launcher> INFO  system - ======================================================================
[2017-09-10 12:14:34,068] <launcher> INFO  system - BASE dir:/usr/lib/unifi
[2017-09-10 12:14:34,240] <launcher> INFO  system - Current System IP: 172.17.0.3
[2017-09-10 12:14:34,241] <launcher> INFO  system - Hostname: 708a27cdb012
[2017-09-10 12:15:45,211] <launcher> INFO  system - ======================================================================
[2017-09-10 12:15:45,215] <launcher> INFO  system - UniFi 5.5.20 (build atag_5.5.20_9565 - release) is started
[2017-09-10 12:15:45,215] <launcher> INFO  system - ======================================================================
[2017-09-10 12:15:45,284] <launcher> INFO  system - BASE dir:/usr/lib/unifi
[2017-09-10 12:15:45,459] <launcher> INFO  system - Current System IP: 172.17.0.3
[2017-09-10 12:15:45,461] <launcher> INFO  system - Hostname: 708a27cdb012
[2017-09-10 12:16:56,460] <launcher> INFO  system - ======================================================================
[2017-09-10 12:16:56,464] <launcher> INFO  system - UniFi 5.5.20 (build atag_5.5.20_9565 - release) is started
[2017-09-10 12:16:56,464] <launcher> INFO  system - ======================================================================
[2017-09-10 12:16:56,532] <launcher> INFO  system - BASE dir:/usr/lib/unifi
[2017-09-10 12:16:56,705] <launcher> INFO  system - Current System IP: 172.17.0.3
[2017-09-10 12:16:56,707] <launcher> INFO  system - Hostname: 708a27cdb012

Further, there's nothing I can see in the error logs.

[jacobalberty]

Hmm, no mongodb instance up? is the docker container itself constantly restarting or just java? ie does docker ps show status constantly being reset or does it stay and keep counting up?

[thowe-switzerland]

I can confirm that the current docker image "jacobalberty/unifi:arm32v7-beta" built by branch "arm32v7-cross-beta" runs just fine on my Asus Tinker Board under Armbian (legacy kernel). Docker version is 17.07.0-ce.

If I find some time I will try it on Raspberry Pi 3 as well. But I think it will be the same there - at least in theory it should...

Thank you very much for your great work. I enjoy your containerized controller a lot. :-)

Please let me know, if I can support your work somehow.

[everflux]

Trying to start on a rpi3 fails, the java process is restarted over and over again and takes much cpu, but no progress.
I don't see a mongodb process.

I used the image from dockerhub, acobalberty/unifi:arm32v7-beta

[jacobalberty]

@everflux you're the second report of those same results, which is weird since it works on the asus tinker board. I suspect its the openjdk jvm. I need to order a rpi3 I suppose and get some hands on testing done, somethings not right there.

[thowe-switzerland]

Interesting that it runs on Tinker Board but not on RPi3. Normally it's the other way around.

As I happen to have a spare RPi3 on my shelf, I'll try to have a look at it this evening (CET).

[thowe-switzerland]

Something came to my:

One of the major differences between RPi3 and Tinker Board is the size of the RAM: 1GB vs 2GB.

I don't know if this is the problem we are seeing here, but having 2GB of RAM on the Tinker Board allows the JVM to use 1GB of heap size as configured by default (JVM_MAX_HEAP_SIZE=1024M). But with only 1GB of RAM at the RPi3 this is at least not an ideal option.

I'll try with my RPi3...

[everflux]

Already tested to reduce the max heap size and define initial heap size. No effect. Resident memory usage of the java process is quite low as far as I can observe from top.
I wonder if there is a problem starting mongodb and the java process fails and restarts as a result. When glancing at the dockerfile and the startup script I didn't see any mongodb startup neither.

[thowe-switzerland]

I spent a quite enjoyable evening with my Raspberry Pi 3. ;-)

Here are my findings:

Both arm32v7 docker images (the cross-prebuilt jacobalberty/unifi:arm32v7-beta one from DockerHub and the custom built one on Raspberry Pi using arm32v7-beta-Dockerfile from GitHub) show the identical problem: On Asus Tinker board both images run fine - on Raspberry Pi 3 both result in a restarting controller.

I had a look into the Docker container:

The default command /usr/local/bin/unifi.sh of the image tries to start the Unifi controller using jsvc. Starting this way leads into an endless loop restarting the controller. Java process is at 100%. Controller is never reachable. As reported by @everflux heap is not the issue here - there is enough free RAM.

When I start the image into the bash, I can start the controller using the alternative command java -jar /usr/lib/unifi/lib/ace.jar start with success! This means, the image basically would work. The problem must be in conjunction with jsvc.

As I do not see the problem on Asus Tinker board under Armbian (with legacy kernel), it could be that jsvc with the (newer) specific kernel used in current Raspbian results in the error.

I hoped that it is the same problem as described here - unfortunately the workaround of setting stack size didn't help (but someone should verify it):
https://community.ubnt.com/t5/UniFi-Wireless/IMPORTANT-Debian-Ubuntu-users-MUST-READ-Updated-07-06-17/td-p/1968252

A side note:
When starting with java -jar /usr/lib/unifi/lib/ace.jar start, the controller runs fine in the container. But after some time the following error is shown on the console:
OpenJDK Zero VM warning: You have loaded library /usr/lib/unifi/lib/native/Linux/armhf/libubnt_webrtc_jni.so which might have disabled stack guard. The VM will try to fix the stack guard now. It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.

This seems not to be problem for the main functionality of the controller. It could be, that the controller can not be remote controlled from the UBNT cloud because of this issue. To get rid of this problem one can:

> apt-get update
> apt-get install prelink
> execstack -c /usr/lib/unifi/lib/native/Linux/armhf/libubnt_webrtc_jni.so

This minor problem seems not related to the main problem. Even with this adjustment, the controller does not start using jsvc.

As for the main problem: I think there is a problem of jsvc in conjunction with Raspbian or its specific kernel.

[everflux]

Nice analysis!
Just fyi I use arch linux (armv7) with the latest kernel.

jsvc seems to be quite old and unmaintained. Perhaps just a systemd unit or similar would suffice?

[jacobalberty]

execstack is a known issue, it had been patched back at the beginning of the year but looks like at some point during another change (possibly when things were moved to using unifi.sh as the entrypoint) the patch got backed out. The issue affects all releases equally, not just arm, so it's not the source of the problem here.

I don't believe systemd runs under docker, but it would be unneeded anyway with docker. To eliminate it the easiest route would be to replace it with the appropriate java call. jsvc is used to match the unifi package (see #17 ) it should be "safe" to go to calling java directly though.

[jacobalberty]

I have recently cleaned up the image quite a bit, it now autodetects architecture in every place I can find that seems relevant. This means the arm build dockerfile and shell scripts are now almost identical to the x86. Cross build still has two differences, First, the qemu cross build binary setup stuff. this is just the two RUN lines at the top and bottom as well as copying the binaries over. Second difference is the gosu binary test crashes qemu. So I am forced to remove this to allow the build to proceed.

The next step I suppose is to kill the jsvc dependency to get this working properly on raspberry pi devices.

[jacobalberty]

And it seems I spoke too soon. Travis failed out on me and upon further digging it seems mongodb-server was dropped for armhf on stretch https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=836435 . It looks like I could bump this to using arm64v8/debian:stretch-slim and put in the right qemu binary on the cross build side and it should work.

[corrideat]

I've submitted two merge requests addressing this issute. Using Ubuntu Xenial instead of Debian Stretch fixes this, as Ubuntu provides 32-bit mongodb. An alternative is using Debian testing, which also has 32-bit mongodb, but the Dockerfile failed in this case.

You may prefer not using Ubuntu, but I believe native 32-bit is better (as in faster) than the emulated 64-bit version, especially if this is running on budget devices.

[jacobalberty]

I think the ubuntu approach is best.

I only have two concerns with ubuntu
#1 is the image larger? I don't have much of a choice, at least when it comes to arm for now. A larger image is preferable to not working
#2 is a little bigger, image compatibility. Ultimately I want to get docker hub going so that it just works on arm or x86. There's a way to define the manifest so this can work. Part of getting it to work is having the docker files close enough that theres minimal changes. I see three changes from the mainline image in this.
First one is pbviously a different source image in FROM.
Secondly It looks like gpg2 is called gnupg2 on ubuntu.
and Finally you have --allow-remove-essential. Did you just find this makes a smaller image and slipped it in or is there some other reason im not missing?

I did go ahead and merge the images into the beta stuff. I'm going to go ahead and delete the 64 bit forks as they will only lead to more confusion as to what to try.

Overall the changes seem like the way to go for getting it working on the raspberry pi.

[corrideat]

I believe the image is slightly larger (around 500MiB total now). However, I tried various methods including trying to compile mongodb myself, and this seems like the least painful way.

I was trying to make the image quickly to resolve an issue. I believe using gnupg, which is the same as the Debian name, will install gpg1, which may be sufficient for the purposes of signature verification.

The --allow-remove-essential was needed because it attempts to remove 'apt', which is marked as 'essential'. Obviously removing apt results in a smaller image overall, but it was added to prevent errors. It should work in Debian without side effects if apt is not marked for removal.

As a suggestion for compatibility, perhaps using Debian testing addresses all of these. I haven't had time to use the working arm build with Debian testing instead of Ubuntu (I tried a different configuration that resulted in build errors).

Now, it would seem like I won't be doing much work on this for the moment, as I did all of this to test an Unifi AP (AP-AC-LR), and was disappointed with the features and performance, so I'll be switching back to my previous AP.

[jacobalberty]

I'm cleaning up the arm stuff now and bringing arm images and x86 images back in sync so the dockerfile is almost the same. I just got a raspberry pi and am working on getting an automated build server set up on it to push the native builds to the hub.

[kenperkins]

Seriously, you didn't have a Pi up until now? I would have bought you one if I had known :(

[wwhchung]

The latest arm image runs perfectly on QNAP TS-431+ via container station. You just have to change the default QNAP web admin ports so it can start as they conflict with the controller ports.

[echoz]

I managed to put this on my RPI3 with Ubuntu 18.04 by cloning the repo and building the image manually from top-of-tree master branch.

Seems like ubuntu:xenial already supports the ARM64v8 instruction set.

Would be awesome if the "latest" tag on the project also supported ARM64v8!

[agherzan]

Are you planning to update the arm tags as well?

[Alpha-Bravo]

I have been using the Dockerfile on an arm32v7 tinkerboard for more than 10 months.

Up to now I always built my own docker image on the tinkerboard using "the old way" feeding the UniFi controller version via build-arg parameter PKGURL to the Dockerfile from Github. That worked perfectly.

Now it is the first time I am trying to create a container using the prebuilt arm32v7-beta image from Docker Hub and feeding the current UniFi controller version (5.10.19) via env variable PKGURL to the container. Somehow I am not able to override the predefined version and always get a running controller of version 5.9.29.

My docker statement:
docker create --restart unless-stopped --init -p 8080:8080 -p 8443:8443 -p 3478:3478/udp -p 10001:10001/udp -e PKGURL='https://dl.ubnt.com/unifi/5.10.19/unifi_sysvinit_all.deb' -e TZ='Europe/Zurich' -v ~/unifi:/unifi --name unifi-5.10.19 jacobalberty/unifi:arm32v7-beta

Does anybody see the problem? Would be so glad to get a hint... ;-)

Thanks!

[Alpha-Bravo]

The problem described in my comment from Mar 10 (seeing an old controller version) seems to be docker image cache related... After purging all cached images, everything is just running fine now.

[imrehg]

I'm using the ARM image (arm32v7-beta tag) for more than half a year now, and seems working well.

Would it be possible to start applying proper tags for these as well, so this caching problems can be the thing of the past? Say 5.10.26-arm32v7 would go a long way, instead of using the same tag repushed all the time. I think it's stable enough to warrant removing the -beta part.

Thanks a lot for all the effort, it's super useful image!

[Alpha-Bravo]

I recommend executing a docker pull jacobalberty/unifi:arm32v7-beta explicitly before docker create or docker run command. This should make sure you have the latest version - and not the one locally cached.

[imrehg]

@Alpha-Bravo this doesn't help much when it's part of a build system, instead of immediately run. The image is pulled at one time as a base. but never quite sure which version was used. This is as opposed to the x86 image which is always clear due to the versioning.
Also, quite often one doesn't want the latest version, but a specific version - here since a tag is overwritten on every new push, there's no turning back, once the local cache is updated.

Is there any specific reason why that arm version is better to be non-version tagged?
Just asking, as discuss different use cases might or might not be convincing, but it would be definitely useful to know what's the reasoning for not version tagging (instead of knowing how to work around that fact in specific circumstances). Cheers!

[naphta]

@jacobalberty I've created a repository using GitHub actions which leverages docker buildx for cross platform builds. The meat of the project is largely a copy/paste of what's currently in this one. (I intend to credit you when I get around to writing the README)

I'm planning on switching the MongoDB installation to be compiled from source so I can support more ARM architectures (and in theory generally many more). I'm happy for you to take from my work as a way to implement support this side. It's currently untested at the time of writing this comment but my personal use case is to run the controller on a Raspberry Pi 4B (with ARM64).

https://github.com/naphta/unifi

[jacobalberty]

@naphta yeah I was just looking at the buildx feature request for the docker hub too hadn't even thought of GitHub actions.

I was thinking of trying to move all the build stuff to my own internal servers at home as that would also allow me to provide signed images.

[naphta]

@naphta yeah I was just looking at the buildx feature request for the docker hub too hadn't even thought of GitHub actions.

I was thinking of trying to move all the build stuff to my own internal servers at home as that would also allow me to provide signed images.

It definitely seems to be working quite well from my point of view. I'm definitely going to do some work around the dependencies though as it's a bit of a mess because of the UniFi deb and more modern OS versions. I saw in your ARMv7 support you dropped it back down to Xenial.

From what I can extrapolate online there's no reason why you shouldn't be able to use a modern version of MongoDB; hence why I'm installing it with dpkg and ignoring those dependencies.

Longer term I'm happy to open up a pull request to send some of these changes upstream if you think it makes sense to. For now I'm quite content with having images which work on my Pi.

[brontide]

OOTB the current git builds and runs just find on arm64. I'm currently testing mongodb 3.6 ( install deps manually and force install of unifi.deb ) and modern jvm replacements as well ( Amazon's corretto, openjdk 15 ). Frankly I think it would be wise to drop many of the elaborate build scripts and go with the official Docker images for mongo and jvm of choice if it were me.

[brontide]

Just as a follow up, the docker hub unifi:5.12.66-arm32v7 throws the following error with the embedded mongo. I can build arm64 from this repo and it runs just fine without complaint.

[2020-04-25T00:34:37,021] <db-server> INFO  db     - 2020-04-25T00:34:37.021+0000 
[2020-04-25T00:34:37,022] <db-server> INFO  db     - 2020-04-25T00:34:37.021+0000 warning: 32-bit servers don't have journaling enabled by default. Please use --journal if you want durability.
[2020-04-25T00:34:37,022] <db-server> INFO  db     - 2020-04-25T00:34:37.021+0000 
[2020-04-25T00:34:37,054] <db-server> INFO  db     - DbServer stopped

Also newer versions of java will not work because they are missing a few classes that were removed in jdk 9. UBNT will have to patch the code to include the missing javax.activiation classes. Shame since I think it feels snappier with the new version, works fine for most functions but don't be deceived.

For those looking to save space it's safe to remove the useless Brotli files from the app directory, this saves 25MB.

find /usr/lib/unifi/webapps/ROOT/app-unifi -type f -name *.br -delete

[jacobalberty]

I believe all ongoing arm issues should be fully corrected with #360 . If there are any issues with the multiarch, multiarch-5 or beta tags then #360 would be the correct place to document them so I can get them fixed before the next 6.0.x release.