Host set in predeploy not available in deploy

[yvespp]

Sometime the host set via Liima rest service in the predeploy phase is not available in the deployment run of the same deployment. In a subsequent deployment the host property is available and the deployment succeeds.

Relevant log entries:

# Starting deployment script on AMW server liima-6569ffff67-xhjsz
# Mode: PREDEPLOY
2019-09-02 16:12:45,779 INFO refact.amwVMBuilder Hostname of VM is: ld9t007a
2019-09-02 16:12:45,779 INFO refact.amwVMBuilder Setting hostname on relation from defa_wrapper KR-19.07 to node node_01 RL-14.10
2019-09-02 16:12:46,709 INFO refact.amwVMBuilder Update request 36676 finished, success = True
2019-09-02 16:12:45,779 INFO refact.amwVMBuilder Setting hostname on relation from defa_wrapper KR-19.07 to node node_01 RL-14.10
2019-09-02 16:13:35,072 INFO AmwDeployRunner Marking predeployment as success
# Starting deployment script on AMW server liima-6569ffff67-p7fr8
# Mode: DEPLOY
2019-09-02 16:13:43,776 INFO refact.amwConfiguration Config: 
2019-09-02 16:13:43,777 INFO refact.amwConfiguration     host: 

[yvespp]

This is very strange and can't really happen:

• PREDEPLOY runs asynchronous and the java transaction ends when the deploy scripts starts.
• setting the hostname happens via rest and run's in its own transaction.
• marking the deployment as success also happens via rest and runs in its own transaction. This also triggers the DEPLOY phase via event.
• The DEPLOY phase runs on a different Liima instance.

Unless we have some cache somewhere this should be impossible?!

[yvespp]

We should try to update to a newer Wildfly version and see if it fixes it

[sdedijer]

Hi @yvespp ,

Sorry that I don't know it, but could you please help me to reproduce this issue. Where is predeploy? Could you please describe step by step how to reproduce this issue? Sorry really once again for asking it.

And also once I know how to do it, do you think that we should also take the latest version of Wildfly and try to reproduce it with latest version as well?

Thanks a lot for your help in advance and BR.

[yvespp]

I can only reproduce this if there is more then one Liima server.
The reproduce you have to:

• Setup two Liima servers connecting to the same database
• Set a hostname via REST PUT /hostNames/{node}/{release}/{env}/{appServer} to one of the servers
• Immediately after the above call read the hostname from the other server via GET /hostNames

[yvespp]

Test calls:

curl -X PUT "$LIIMA_HOST/AMW_rest/resources/resources/test_app1/test_rl/properties/Version?env=Global" -H "accept: application/json" -H "Content-Type: text/plain" -d "MyVersion3"
curl -X GET "$LIIMA_HOST/AMW_rest/resources/resources/test_app1/test_rl/properties/Version?env=Global" -H "accept: application/json"

[mburri]

I'm confused - first you wrote to set a host name via /hostNames resource, but your test calls use a different endpoint (/resources/resources/...) instead of /resources/hostNames

What is the correct endpoint to reproduce this issue?

with the following calls:

curl -X PUT "http://localhost:8081/AMW_rest/resources/hostNames/node_01/RL-14.10/B/adam_sandbox" -H "accept: application/json" -H "Content-Type: text/plain" -d "my-host-name-that-i-just-set"

and immidiatly afterwards:

curl -X GET "http://localhost:8080/AMW_rest/resources/hostNames?host=my-host-name-that-i-just-set&disableMerge=false" -H "accept: application/json"

(note the different ports)

I got the following response:

[
  {
    "host": "my-host-name-that-i-just-set",
    "appServer": "adam_sandbox",
    "appServerRelease": "multiple",
    "runtime": "EAP6",
    "node": "node_01",
    "nodeRelease": "RL-14.10",
    "environment": "B",
    "domain": "Dev",
    "definedOnNode": false
  }
]

Is this wrong? Looks good to me...

Currently running on EAP 7.2 locally (both servers)

[mburri]

Hmm - just tried once again - and the new host name is not set. It looks like It's only possible to set the hostname once - the first time a new entry is created with "appServerRelease": "multiple",

I think that I'm on the wrong path - @yvespp could you point me in the right direction please?

edit: did some more experiments - there are multiple put endpoints for hostnames, I tried the others - but the hostname is always set correctly and available on the other server...

[yvespp]

I tested again and I can't reproduce this issue anymore... Also found no errors in our prod related to this issue.
I attached my tests scripts that I used, in case it shows up again
test_via_rest.txt
test_via_generate.txt