Update OZW to 1.6 and use the same version in all context #675

sescandell · 2020-03-04T21:46:26Z

Pull Request check-list

To ensure your Pull Request can be accepted as fast as possible, make sure to review and check all of these items:

If your changes affects code, did your write the tests?
Are tests passing? (npm test on both front/server)
Is the linter passing? (npm run eslint on both front/server)
Did you run prettier? (npm run prettier on both front/server)
If your changes modify the API (REST or Node.js), did you modify the API documentation? (Documentation is based on comments in code)
If you are adding a new features/services which needs explanation, did you modify the user documentation? See the GitHub repo and the website.
Did you add fake requests data for the demo mode (front/src/config/demo.json) so that the demo website is working without a backend? (if needed) See https://demo.gladysassistant.com.

NOTE: these things are not required to open a PR and can be done afterwards / while the PR is open.

Description of change

As explained in #614 (comment) image used in Docker reference OZW 1.6.

Let's try to make CircleCI in sync with the "production" environment and use node-openzwave-share version 1.6 (to stay sync regarding the OZW version).

codecov · 2020-03-04T22:38:53Z

Codecov Report

❗ No coverage uploaded for pull request base (master@2e9c66d). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #675   +/-   ##
=========================================
  Coverage          ?   92.05%           
=========================================
  Files             ?      411           
  Lines             ?     5145           
  Branches          ?        0           
=========================================
  Hits              ?     4736           
  Misses            ?      409           
  Partials          ?        0

Flag	Coverage Δ
#server	`92.05% <ø> (?)`

Impacted Files	Coverage Δ
server/lib/scene/scene.getBySelector.js	`100.00% <0.00%> (ø)`
server/lib/house/house.userSeen.js	`100.00% <0.00%> (ø)`
server/lib/calendar/calendar.getEvents.js	`100.00% <0.00%> (ø)`
server/lib/device/index.js	`100.00% <0.00%> (ø)`
server/services/xiaomi/api/xiaomi.controller.js	`100.00% <0.00%> (ø)`
server/lib/device/device.add.js	`100.00% <0.00%> (ø)`
server/services/example/index.js	`100.00% <0.00%> (ø)`
server/services/philips-hue/lib/models/color.js	`100.00% <0.00%> (ø)`
server/models/room.js	`100.00% <0.00%> (ø)`
server/utils/accessToken.js	`100.00% <0.00%> (ø)`
... and 401 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e9c66d...30d4f13. Read the comment docs.

sescandell · 2020-03-04T23:10:00Z

Hi @Pierre-Gilles

With these limited changes:

the docker container used for tests is now using OZW 1.6 with the same revision as the alpine3.11 (revision 974... as far as I think...)
node library has been updated to the 1.6.2
alpine version has been forced so Gladys stay in control about the OZW version running in prod (with Docker at least... don't know how your RPI image is generated and what should I change to "force" a specific version for OZW)

All tests are green. BUT it doesn't necessary means everything is working. It simply means that all interfaces with the lib' are okay. If anything has changed inside OZW, it will not appear in the current tests.

That being said, the OZW v. 1.6.974 is 3 months old. As you can see, I have to manually checkout and compile the lib'. 3 months is a lot. Many fixes have been made on the lib' since then. I would like to use a newer version by using, let's say one of the latest commit ID. And doing this in the Docker container used for tests AND in the Dockerfile used for generated images (so removing the apk add openzwave ... and replacing it with a custom install like in the test container). Before doing this, I would like your feedbacks? Since updating to the 1.6 version, we might as well use the latest version available. What do you think? Should I checkout a newer version or stay with this one? (my preference is for a new one...)

As soon as possible, I will try to connect my ZWave key and see if things are working with a real device and real events (but I have a very limited set of devices to test "everything"). If you want to check on your side with your devices if everything is working, that would be great!

Thanks,

Pierre-Gilles · 2020-03-07T17:16:09Z

Hi @sescandell ! Thanks for this great PR ! :)

Before doing this, I would like your feedbacks? Since updating to the 1.6 version, we might as well use the latest version available.

I'm ok with using a newer version, but we really need to lock the version used in the Dockerfile, because we don't want to have unwanted side effects when building at a random time if one OZW guy pushed some buggy code on master at the same time (it happens a lot with OZW).

Do you know how they work? They don't seem to use tags frequently on their repo...

I don't understand how they version/release their lib

sescandell · 2020-03-08T19:36:17Z

we really need to lock the version used in the Dockerfile

Totally agree with this!

Do you know how they work? They don't seem to use tags frequently on their repo...

Unfortunately, I don't. I'm not even sure they have any rule. They have a milestone for the 1.7 version, but nothing very clear about patches or fixes for a given version. Looking at Alpine distribution, I think they just compile "one version" at the time of updating...

Let's try with the latest one. Then we would have to follow the OZW repo to update it "when we think it deserve it".

Update of the PR coming.

sescandell · 2020-03-08T21:11:04Z

Hi @Pierre-Gilles

PR updated. Forced on a given commit. Dockerfile and CircleCI updated.

I just don't have any idea how you generate the RPI image and if I should change anything for it.
Do not hesitate to tell me if something else must be updated.

I didn't yet tested this on a real zWave Controller. I should be able to do it in the coming days. If you have any chance to test it, let me know.

sescandell · 2020-03-21T17:35:30Z

I've made some very simple tests: the zWave doesn't initialize as expected.

I'll dive into the changes of the NPM lib in the coming days,

Still not ready to merge for now.

Pierre-Gilles · 2020-04-06T10:01:03Z

Hey ! I saw you pushed more things, is it ready for review? :)

Thanks for your great work !

sescandell · 2020-04-06T17:03:50Z

Hi @Pierre-Gilles ,

I tested yesterday to run Gladys with OZW on a MacOSX and it worked (well... actually the communication with the OZW library worked... I had other issues but I think they are not tied to the update).

I'm now going to test the same thing in Docker and let you know.

By the way, @Pierre-Gilles could you please precise your environment? What's your OS? How do you develop Gladys? Which tools? How do you test ZWave integration? How do you debug? What's your IDE?

Thanks,

sescandell · 2020-04-06T21:40:34Z

Well, actually I have issues to test the Docker environment as you cannot mount an USB device into the Docker container.
It's working on the Mac. If you want more tests, I'll try to reconfigure another computer on a Linux environment and test from there.

Do not hesitate if you have any question,

Albenss · 2020-04-07T07:14:12Z

I don't understand why you cannot mount an USB device into Docker, have you tried to use the volume section to mount the device directly?
Like it is used in Zigbee2Mqtt (https://www.zigbee2mqtt.io/information/docker.html#running).

VonOx · 2020-04-07T09:43:56Z

Without your Docker run command, it's hard to help.
You need to launch with privileged and mount dev, like this.

docker run -d \
--restart=always \
--privileged \
--network=host \
--name gladys \
-e NODE_ENV=production \
-e SERVER_PORT=80 \
-e TZ=Europe/Paris \
-e SQLITE_FILE_PATH=/var/lib/gladysassistant/gladys-production.db \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /var/lib/gladysassistant:/var/lib/gladysassistant \
-v /dev:/dev \
gladysassistant/gladys:4.0.0-beta-arm

sescandell · 2020-04-07T11:12:52Z

I already tried the following, but none of them is working:

docker run -d \
--restart=always \
--privileged=true \
--name gladys \
-e NODE_ENV=production \
-e SERVER_PORT=80 \
-e TZ=Europe/Paris \
-e SQLITE_FILE_PATH=/var/lib/gladysassistant/gladys-production.db \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /Users/Shared/Gladys:/var/lib/gladysassistant \
--device=/dev/tty.usbmodem14201 \
-p 8080:80 \
gladystest

docker run -d \
--restart=always \
--privileged=true \
--name gladys \
-e NODE_ENV=production \
-e SERVER_PORT=80 \
-e TZ=Europe/Paris \
-e SQLITE_FILE_PATH=/var/lib/gladysassistant/gladys-production.db \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /Users/Shared/Gladys:/var/lib/gladysassistant \
-v /dev:/dev \
-p 8080:80 \
gladystest

Are you working with OSX (10.15.4) on your side too?

VonOx · 2020-04-07T11:19:32Z

Windows and Linux most of the time

Access USB stick on Mac is a bit tricky, I saw this article https://christopherjmcclellan.wordpress.com/2019/04/21/using-usb-with-docker-for-mac/

sescandell · 2020-04-07T11:21:18Z

Actually, it seems this is not supported by Docker on Mac OSX at all... See docker/for-mac#900

Otherwise, I need to falback on the boot2docker and VM customization... Going back to Windows / Linux

sescandell · 2020-04-07T11:23:13Z

@VonOx

Windows and Linux most of the time

Should I understand USB share in Docker is supported on Windows host?

VonOx · 2020-04-07T11:51:31Z

Sorry I don't know, I'm not using USB share on Windows.

sescandell · 2020-04-07T22:00:28Z

I've made some additional tests on my Mac with a zWave controller playing with a binary device: and it works pretty well.

Appart from this, I have a Walli Dimmer Fibaro device, and Dimmer are not working at all. But I will propose a fix into another PR, this is not the aim of that one.

sescandell · 2020-04-22T16:59:46Z

@Pierre-Gilles not sure if you followed everything here, but it is ready to merge

Pierre-Gilles

Thanks for this PR ! :)

This is really nice and it'll help many people. Congrats 👏

I noted some minor changes to your PR, can you have a look?

docker/Dockerfile

front/src/components/boxs/device-in-room/device-features/BinaryDeviceFeature.jsx

sescandell · 2020-04-27T16:04:44Z

@Pierre-Gilles all fixed

docker/Dockerfile

Pierre-Gilles · 2020-05-04T09:16:18Z

Many thanks for the PR!! :) Except my last question, I'm good to go!

Pierre-Gilles · 2020-05-18T10:10:44Z

@VonOx How can we merge both your changes on Docker + this PR ? Is your PR compatible with that ?

VonOx · 2020-05-18T11:44:41Z

@Pierre-Gilles I think so

PR #643 , the only modification on Dockerfile is base image ( node-lts from node-12 )
But merge #675 before, i will rebase my PR

PR #793 use independent Dockerfile ( buildx ) , i will implement OZW part as @sescandell did before you merge.

Based on GladysAssistant#675

* Create Dockerfile.buildx * Create pr-docker-image.yml * Move yaml in the correct folder * OZW 1.6 use the same version in all context Based on #675 Co-authored-by: Pierre-Gilles Leymarie <pierregilles.leymarie@gmail.com>

* Force alpine version to force OZW version * Install OZW 1.6 in CircleCI & use 1.6.2 OZW Node * Force library update * Install latest OZW version (but fixed on a given commit) * Fix already pushed in a previous PR... Error in commit 5242a1a ? * Reorder Docker commands for local dev' * Revert change not for this PR * Fix Dockerfile * Revert forcing alpine version as now OZW is manually compiled Co-authored-by: Pierre-Gilles Leymarie <pierregilles.leymarie@gmail.com>

* Create Dockerfile.buildx * Create pr-docker-image.yml * Move yaml in the correct folder * OZW 1.6 use the same version in all context Based on GladysAssistant#675 Co-authored-by: Pierre-Gilles Leymarie <pierregilles.leymarie@gmail.com>

* Force alpine version to force OZW version * Install OZW 1.6 in CircleCI & use 1.6.2 OZW Node * Force library update * Install latest OZW version (but fixed on a given commit) * Fix already pushed in a previous PR... Error in commit 5242a1a ? * Reorder Docker commands for local dev' * Revert change not for this PR * Fix Dockerfile * Revert forcing alpine version as now OZW is manually compiled Co-authored-by: Pierre-Gilles Leymarie <pierregilles.leymarie@gmail.com>

* Create Dockerfile.buildx * Create pr-docker-image.yml * Move yaml in the correct folder * OZW 1.6 use the same version in all context Based on GladysAssistant#675 Co-authored-by: Pierre-Gilles Leymarie <pierregilles.leymarie@gmail.com>

sescandell changed the title ~~[WIP] Sync & use OZW 1.6~~ Update OZW to 1.6 and use the same version in all context Mar 8, 2020

VonOx mentioned this pull request Mar 12, 2020

CI and 🐳 improvement #643

Merged

sescandell added 4 commits April 4, 2020 12:42

Force alpine version to force OZW version

abe75ec

Install OZW 1.6 in CircleCI & use 1.6.2 OZW Node

f22fb11

Force library update

36008f0

Install latest OZW version (but fixed on a given commit)

b6776b4

sescandell force-pushed the pr-ozwUpdate branch from 896cc4d to b6776b4 Compare April 4, 2020 10:42

Fix already pushed in a previous PR... Error in commit 5242a1a ?

f8a75f5

Reorder Docker commands for local dev'

87fc2e3

sescandell force-pushed the pr-ozwUpdate branch from befffcd to 87fc2e3 Compare April 7, 2020 07:57

Pierre-Gilles requested changes Apr 27, 2020

View reviewed changes

docker/Dockerfile Outdated Show resolved Hide resolved

front/src/components/boxs/device-in-room/device-features/BinaryDeviceFeature.jsx Outdated Show resolved Hide resolved

Revert change not for this PR

e118b3c

sescandell added 2 commits April 27, 2020 18:00

Fix Dockerfile

d15365a

Useless change

992fd11

Pierre-Gilles reviewed May 4, 2020

View reviewed changes

docker/Dockerfile Outdated Show resolved Hide resolved

Revert forcing alpine version as now OZW is manually compiled

379209b

sescandell mentioned this pull request May 6, 2020

zWave - Dimmer integration #757

Closed

7 tasks

VonOx added a commit to VonOx/Gladys that referenced this pull request May 18, 2020

OZW 1.6 use the same version in all context

5e9bb65

Based on GladysAssistant#675

Merge branch 'master' into pr-ozwUpdate

30d4f13

Pierre-Gilles merged commit 1cdaca8 into GladysAssistant:master May 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update OZW to 1.6 and use the same version in all context #675

Update OZW to 1.6 and use the same version in all context #675

sescandell commented Mar 4, 2020 •

edited

Loading

codecov bot commented Mar 4, 2020 •

edited

Loading

sescandell commented Mar 4, 2020 •

edited

Loading

Pierre-Gilles commented Mar 7, 2020

sescandell commented Mar 8, 2020

sescandell commented Mar 8, 2020

sescandell commented Mar 21, 2020

Pierre-Gilles commented Apr 6, 2020

sescandell commented Apr 6, 2020 •

edited

Loading

sescandell commented Apr 6, 2020

Albenss commented Apr 7, 2020

VonOx commented Apr 7, 2020

sescandell commented Apr 7, 2020

VonOx commented Apr 7, 2020

sescandell commented Apr 7, 2020

sescandell commented Apr 7, 2020

VonOx commented Apr 7, 2020

sescandell commented Apr 7, 2020

sescandell commented Apr 22, 2020

Pierre-Gilles left a comment

sescandell commented Apr 27, 2020

Pierre-Gilles commented May 4, 2020

Pierre-Gilles commented May 18, 2020

VonOx commented May 18, 2020 •

edited

Loading

Update OZW to 1.6 and use the same version in all context #675

Update OZW to 1.6 and use the same version in all context #675

Conversation

sescandell commented Mar 4, 2020 • edited Loading

Pull Request check-list

Description of change

codecov bot commented Mar 4, 2020 • edited Loading

Codecov Report

sescandell commented Mar 4, 2020 • edited Loading

Pierre-Gilles commented Mar 7, 2020

sescandell commented Mar 8, 2020

sescandell commented Mar 8, 2020

sescandell commented Mar 21, 2020

Pierre-Gilles commented Apr 6, 2020

sescandell commented Apr 6, 2020 • edited Loading

sescandell commented Apr 6, 2020

Albenss commented Apr 7, 2020

VonOx commented Apr 7, 2020

sescandell commented Apr 7, 2020

VonOx commented Apr 7, 2020

sescandell commented Apr 7, 2020

sescandell commented Apr 7, 2020

VonOx commented Apr 7, 2020

sescandell commented Apr 7, 2020

sescandell commented Apr 22, 2020

Pierre-Gilles left a comment

Choose a reason for hiding this comment

sescandell commented Apr 27, 2020

Pierre-Gilles commented May 4, 2020

Pierre-Gilles commented May 18, 2020

VonOx commented May 18, 2020 • edited Loading

sescandell commented Mar 4, 2020 •

edited

Loading

codecov bot commented Mar 4, 2020 •

edited

Loading

sescandell commented Mar 4, 2020 •

edited

Loading

sescandell commented Apr 6, 2020 •

edited

Loading

VonOx commented May 18, 2020 •

edited

Loading