Feature/proxy-fmu #633

markaren · 2021-04-06T12:23:51Z

Adds proxy-fmu support to libcosim. proxy-fmu is implemented in C++ and supports automatic spawning of new processes. Thus it replaces the current usage of the JVM based fmu-proxy solution.

Closes #628

markaren · 2021-04-06T16:28:33Z

Integration works locally at this point.

# Conflicts: # conanfile.py

markaren · 2021-04-09T13:02:33Z

Not finding FMILibrary on linux when building proxyfmu is such a weird error.
conan create works when triggered from proxyfmu repository. Why does it fail here?
I tried running plain conan install without going through conan create for libcosim first. Still the same error.

Note that the same error is reproduceable on a plain ubuntu installation.

@kyllingstad any ideas?

markaren · 2021-04-22T15:11:28Z

Doubt it. I ran with that update susessfully yesterday and here non proxy builds fails also.

markaren · 2021-04-22T16:00:06Z

cheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 22, 14:00 UTC - 18:00 UTC
Scheduled - Detail: The DNS record http://conan.bintray.com will be routed to the new ConanCenter repository.
Impact: Temporary instability, none lasting.

If you receive this error, it is related to this maintenance:

ljamt · 2021-05-06T14:55:33Z

Just build cosim-cli locally with the conan package from this branch:
libcosim/0.8.0@osp/testing-feature_proxy-server

Then reconfigured the House demo configuration to the following:

	<Simulators>
        <Simulator name="Clock" source="proxyfmu://localhost?file=Clock.fmu"/>
        <Simulator name="InnerWall" source="proxyfmu://localhost?file=InnerWall.fmu"/>
        <Simulator name="OuterWall1" source="proxyfmu://localhost?file=OuterWall1.fmu">
            <InitialValues>
                <InitialValue variable="T_outside">
                    <Real value="5.3"/>
                </InitialValue>
            </InitialValues>
        </Simulator>
        <Simulator name="OuterWall2" source="proxyfmu://localhost?file=OuterWall2.fmu">
            <InitialValues>
                <InitialValue variable="T_outside">
                    <Real value="4.9"/>
                </InitialValue>
            </InitialValues>
        </Simulator>
        <Simulator name="Room1" source="proxyfmu://localhost?file=Room1.fmu"/>
        <Simulator name="Room2" source="proxyfmu://localhost?file=Room2.fmu"/>
        <Simulator name="TempController" source="proxyfmu://localhost?file=TempController.fmu"/>
    </Simulators>

Testing with
cosim run c:\dev\cse-demos\house\OspSystemStructure_.xml -d 10

At first successful result:

@progress 1 1.000000 1.000000
@progress 2 2.000000 2.000000
@progress 3 3.000000 3.000000
@progress 4 4.000000 4.000000
@progress 5 5.000000 5.000000
@progress 6 6.000000 6.000000
@progress 7 7.000000 7.000000
@progress 8 8.000000 8.000000
@progress 9 9.000000 9.000000
@progress 10 10.000000 10.000000
[proxyfmu] Shutting down proxy for Clock::Clock done..
[proxyfmu] External proxy process for instance 'Clock' returned with status 0
[proxyfmu] Shutting down proxy for TempController::TempController done..
[proxyfmu] External proxy process for instance 'TempController' returned with status 0
[proxyfmu] Shutting down proxy for OuterWall1::OuterWall1 done..
[proxyfmu] External proxy process for instance 'OuterWall1' returned with status 0
[proxyfmu] Shutting down proxy for InnerWall::InnerWall done..
[proxyfmu] External proxy process for instance 'InnerWall' returned with status 0
[proxyfmu] Shutting down proxy for OuterWall2::OuterWall2 done..
[proxyfmu] External proxy process for instance 'OuterWall2' returned with status 0
[proxyfmu] Shutting down proxy for Room1::Room1 done..
[proxyfmu] External proxy process for instance 'Room1' returned with status 0
[proxyfmu] Shutting down proxy for Room2::Room2 done..
[proxyfmu] External proxy process for instance 'Room2' returned with status 0

But subsequent simulations gave:

Attempt 2:
Thrift: Thu May  6 16:42:20 2021 TServerSocket::listen() BIND 64466
Unhandled Exception reached the top of main: Could not bind: errno = 10013, application will now exit
[proxyfmu] External proxy process for instance 'Room1' returned with status 2
Thrift: Thu May  6 16:42:23 2021 TSocket::open() connect() <Host: localhost Port: 64466>: errno = 10061
Thrift: Thu May  6 16:42:25 2021 TSocket::open() connect() <Host: localhost Port: 64466>: errno = 10061

Attempt 3:
Thrift: Thu May  6 16:42:29 2021 TServerSocket::listen() BIND 53205
Unhandled Exception reached the top of main: Could not bind: errno = 10013, application will now exit
[proxyfmu] External proxy process for instance 'OuterWall1' returned with status 2
Thrift: Thu May  6 16:42:32 2021 TSocket::open() connect() <Host: localhost Port: 53205>: errno = 10061
Thrift: Thu May  6 16:42:34 2021 TSocket::open() connect() <Host: localhost Port: 53205>: errno = 10061

Attempt 4:
Thrift: Thu May  6 16:42:49 2021 TServerSocket::listen() BIND 53308
Unhandled Exception reached the top of main: Could not bind: errno = 10013, application will now exit
[proxyfmu] External proxy process for instance 'InnerWall' returned with status 2
Thrift: Thu May  6 16:42:52 2021 TSocket::open() connect() <Host: localhost Port: 53308>: errno = 10061
Thrift: Thu May  6 16:42:54 2021 TSocket::open() connect() <Host: localhost Port: 53308>: errno = 10061

Curious to hear if you see the same.

markaren · 2021-05-06T15:52:17Z

Hmm, runs fine here (same case). You might have been unlucky with the port selection?

ljamt · 2021-05-07T06:33:25Z

Hmm, runs fine here (same case). You might have been unlucky with the port selection?

You are probably right. For me, this specific case works ~3 out of 10 attempts.

kyllingstad

This looks really good, great work! I have some comments about error handling and formatting that I'd like you to address, but otherwise, I think this is good to go.

(Sorry for taking so long before doing the review, btw.)

src/cosim/proxy/proxy_uri_sub_resolver.cpp

src/cosim/proxy/remote_fmu.hpp

src/cosim/proxy/remote_slave.cpp

kyllingstad · 2021-05-10T07:03:52Z

src/cosim/proxy/remote_slave.cpp

+
+cosim::proxy::remote_slave::~remote_slave()
+{
+    remote_slave::end_simulation();


I haven't looked into the functions you're calling here, but I just want to note that one cannot throw exceptions from destructors(*). So if there is a non-neglectible chance that these functions may throw, you may want to wrap the calls in a try-catch-statement and log a message on error.

(*) Or, you can, but it causes the program to immediately crash via std::terminate().

tools/housekeeping

tools/tmp_cleanup

kyllingstad

Great job!

markaren · 2021-06-01T06:54:57Z

Can merge this once a new release has been made from master and @ljamt has vetted it. I'd also like to have #637 merged first, so I can make use of it in this PR.

# Conflicts: # src/cosim/fmuproxy/fmuproxy_client.cpp # tests/ssp_loader_fmuproxy_test.cpp

ljamt · 2021-08-06T10:22:18Z

Can merge this once a new release has been made from master and @ljamt has vetted it. I'd also like to have #637 merged first, so I can make use of it in this PR.

Done testing and approving the PR now. Great work!!
Apologies for the delay.

markaren · 2021-08-07T09:08:47Z

There is still the issue that this feature is not part of the unit testing though...

ljamt · 2021-08-09T08:25:19Z

There is still the issue that this feature is not part of the unit testing though...

This PR includes two unit test files. What feature is not part of the unit testing?

markaren · 2021-08-09T08:31:00Z

Conan builds are not tested through the CI.

markaren · 2021-08-14T08:04:48Z

Enabled testing for this feature. Tests passes, but some other issues appeared for gcc8.

# Conflicts: # .github/workflows/ci-conan.yml # conanfile.py

Lars Ivar Hatledal and others added 4 commits April 6, 2021 14:09

wip proxy integration

b756af5

link against proxy

39a9e12

wip

b33356b

sucess

93dcba3

update

a551450

markaren changed the title ~~Feature/proxy server~~ Feature/proxy-fmu Apr 7, 2021

Lars Ivar Hatledal added 8 commits April 7, 2021 08:58

Merge branch 'master' into feature/proxy-server

5602210

# Conflicts: # conanfile.py

adding cmake option for proxy and misc fixes

b6c882b

fix option name

29421a3

add newline

9b9b13c

fix option name

d8a9406

typo

9a49a27

fix option name

77ba5c7

update target

c3e6e57

markaren force-pushed the feature/proxy-server branch 2 times, most recently from 1297921 to c3e6e57 Compare April 9, 2021 12:45

Lars Ivar Hatledal and others added 12 commits April 10, 2021 01:17

actually use remote proxy feature..

9af60ff

copy proxy_server.exe

eb88472

update

1710002

updated namespace

9dd2a9c

check status

ef930b5

change uri

a99531e

update

44f0d52

Update readme

2835d6a

Update conanfile.py

8f47dd6

Update conanfile.py

f80a4fe

gcc7 filesystem fix

f0d6fd2

exclude debug for proxyfmu

784f1a0

Update proxyfmu version

73f8d87

kyllingstad requested changes May 10, 2021

View reviewed changes

markaren and others added 5 commits May 11, 2021 09:34

cleanup

ec99217

throw on bad file and remove empty url code path

bf3e079

add more tests

77dd4a4

copy booter and unix executables for proxyfmu to bin

0bf6954

replace use of panic with exception

2d4ef45

kyllingstad approved these changes Jun 1, 2021

View reviewed changes

markaren added 2 commits June 3, 2021 15:31

Merge branch 'master' into feature/proxy-server

f51e8d5

# Conflicts: # src/cosim/fmuproxy/fmuproxy_client.cpp # tests/ssp_loader_fmuproxy_test.cpp

use cosim::filesystem

9484ec5

ljamt approved these changes Aug 6, 2021

View reviewed changes

Lars Ivar Hatledal added 3 commits August 14, 2021 09:42

testing

7b53041

testing

a1d1611

testing

038c3ca

kyllingstad mentioned this pull request Aug 17, 2021

Clean up Conan CI build environment using Docker and run tests in Conan CI builds #644

Merged

markaren mentioned this pull request Aug 17, 2021

Unable to change the value of boolean inputs when running FMUs under FMU-Proxy. NTNU-IHB/FMU-proxy#131

Closed

Merge branch 'master' into feature/proxy-server

5b23aef

# Conflicts: # .github/workflows/ci-conan.yml # conanfile.py

markaren merged commit d23a7b4 into master Aug 18, 2021

markaren deleted the feature/proxy-server branch August 18, 2021 08:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/proxy-fmu #633

Feature/proxy-fmu #633

markaren commented Apr 6, 2021 •

edited

Loading

markaren commented Apr 6, 2021

markaren commented Apr 9, 2021 •

edited

Loading

markaren commented Apr 22, 2021

markaren commented Apr 22, 2021

ljamt commented May 6, 2021 •

edited

Loading

markaren commented May 6, 2021 •

edited

Loading

ljamt commented May 7, 2021

kyllingstad left a comment •

edited

Loading

kyllingstad May 10, 2021 •

edited

Loading

kyllingstad left a comment

markaren commented Jun 1, 2021

ljamt commented Aug 6, 2021

markaren commented Aug 7, 2021

ljamt commented Aug 9, 2021

markaren commented Aug 9, 2021

markaren commented Aug 14, 2021 •

edited

Loading

Feature/proxy-fmu #633

Feature/proxy-fmu #633

Conversation

markaren commented Apr 6, 2021 • edited Loading

markaren commented Apr 6, 2021

markaren commented Apr 9, 2021 • edited Loading

markaren commented Apr 22, 2021

markaren commented Apr 22, 2021

ljamt commented May 6, 2021 • edited Loading

markaren commented May 6, 2021 • edited Loading

ljamt commented May 7, 2021

kyllingstad left a comment • edited Loading

Choose a reason for hiding this comment

kyllingstad May 10, 2021 • edited Loading

Choose a reason for hiding this comment

kyllingstad left a comment

Choose a reason for hiding this comment

markaren commented Jun 1, 2021

ljamt commented Aug 6, 2021

markaren commented Aug 7, 2021

ljamt commented Aug 9, 2021

markaren commented Aug 9, 2021

markaren commented Aug 14, 2021 • edited Loading

markaren commented Apr 6, 2021 •

edited

Loading

markaren commented Apr 9, 2021 •

edited

Loading

ljamt commented May 6, 2021 •

edited

Loading

markaren commented May 6, 2021 •

edited

Loading

kyllingstad left a comment •

edited

Loading

kyllingstad May 10, 2021 •

edited

Loading

markaren commented Aug 14, 2021 •

edited

Loading