Discovery of Comsol 6.0 fails on Linux

[louisreg]

Hi,
I've upgraded my comsol version to 6.0 and now mph is failing to locate comsol. I did a quick research and it seems that the following code is failing with comsol 6.0:
backends.append({ 'name': name, 'major': major, 'minor': minor, 'patch': patch, 'build': build, 'root': root, 'jvm': jvm, 'java': java, 'server': [comsol, 'mphserver'],
I've downgraded and it works as usual.
Louis

[john-hen]

Thanks for reporting this.

Which part of the code fails isn't all that relevant. The root cause must be that something in Comsol's folder structure has changed between 5.x and 6.0. So far, there is nothing version-specific in the discovery mechanism. I don't have access to a Comsol 6.0 installation at this point, so cannot look into this or test possible solutions.

Pull request welcome.

[john-hen]

I see nothing in the Comsol 6.0 release notes that would indicate any changes to the folder layout. So, no clue what's happening there.

[doub1emint]

Hi,
Is MPh suitable for COMSOL 6.0? It will be more efficient and convenient to use the COMSOL 6.0.

[john-hen]

I agree, you should be using Comsol 6.0 if you have it. But according to the original post it does not work with MPh. I can't verify if that's true, but it probably is.

Let me know if you want to help fix this. The information provided so far is obviously insufficient. You would need Comsol 6.0 and run some tests, then report back. I can walk you through the steps, it's not difficult. I may be able to fix this, but I need feedback I can work with.

[doub1emint]

Thanks for your quick reply.
Now I still use COMSOL 5.6 version, later I'll try to use COMSOL 6.0 because it's a significant update and I'm willing to report some info with you to improve MPh.

[john-hen]

Okay, great. Here's what you can do…

Basically, if the discovery mechanism does indeed not work with Comsol 6.0, we want to see what's different compared to Comsol 5.6. You could look at the folder structure of the two installations and see if you notice anything. But there's also a test script you can run, which should report everything that's important.

• Download the current main branch of MPh.
  You can download the ZIP archive from the front page of the repo (the green "Code" button) and unpack it. Or use Git if you prefer: git clone --depth 1 https://github.com/MPh-py/MPh.git.
• Open a console in the root folder of that local copy.
  This folder would be called MPh-main or MPh depending on which method you used.
• Run python tools/test.py discovery --log.

Here's what the output looks like with Comsol 5.6 on Windows.

C:\temp\MPh-main
$ python tools/test.py discovery --log
Running test group "discovery".
[00:00.717] Checking registry node "SOFTWARE\Comsol\COMSOL56".
[00:00.717] Checking installation folder "C:\Program Files\COMSOL\COMSOL56\Multiphysics".
[00:01.504] Reported version info is "COMSOL Multiphysics 5.6.0.401".
[00:01.504] Assigned name "5.6" to this installation.
Passed in 2 s.

Those messages are logged by the search_Windows() function, by the way. They're essentially debug messages to avoid having to step through that function with a debugger. You best run the test once before you install 6.0 and once after. If it doesn't work with 6.0, there should be a message explaining at which point it failed. This will hopefully clue me in and we can take it from there.

[max3-2]

Hi @john-hen
I was away for quite a while, new job and some more changes. But let me tag in here again: I tested against 5.5, 5.6 and 6.0 and had no issues running the full tests and some models. Some new feature look great!

PS: This holds for macOS - I will have access to windows soon but honestly do not see any issues. I think we need more input from @louisreg on this specific issue. Maybe some other path problems?

[john-hen]

Hi Max, great to hear from you! Job changes, I know all about that. I don't actually work with Comsol right now, but obviously we want to get 6.0 working a.s.a.p. Appreciate your feedback.

So it's not a problem across all operating systems then. Makes it more likely to be a Linux issue, as most people are on Windows, I would assume, and there hasn't exactly been an uproar over 6.0 not working, even though it's been out for over a month now.

When you get a chance to test on Windows, make sure you run the full test suite there too (python tools/test.py --log). I half-expect a new release to break the way MPh assigns tags to newly created nodes (because there would be new features). Then again, if it doesn't happen on macOS, I don't see why it would on Windows. And I could also relax the test assertions there. Should maybe do that anyway, as it is bound to happen eventually.

PS: I didn't add that many features. But the one, I think, that stands out is that we can now do if (model/'').problems(): … to check if there are any warnings or errors anywhere in the model. That would be like 50 lines of code in Java. (Though now that I look at that example, maybe I should expose the problems() method in the Model class as well.)

[max3-2]

Will do. We have COMSOL windows licenses available but need a project to justify the cost per department. I do plan on running simulations where COMSOL is a viable option but this will take some time (weeks...).
I might see if I can install COMSOL and run the discovery since most of it should run without acquiring a license - I should be able do override the server call

[john-hen]

Ah, okay. Then just run the discovery test, that doesn't need a license (if you manage to install without one). I'm not really worried about the other tests. For one, it's out of scope as far as this issue is concerned. And for another, it would just mean the tests need fixing, not the actual library code. If the discovery works, then everything else should be fine, from a user perspective.

[john-hen]

I just future-proofed that one iffy test I alluded to. Also, I had forgotten that we actually skip that test on macOS / Linux, because it's unbearably slow in client–server mode. So it could not have failed when you ran the test suite on macOS.

[max3-2]

The below holds for Windows 10, COMSOL 6.0:
I can confirm that I can run and test everything that does not need a license, and if this is the issue the exception is quite clear.

So if the installation is performed correctly, MPh works with COMSOL 6.0 on either Windows and macOS, I would assume it should work on Linux too.

[john-hen]

Thanks for looking into this, @max3-2. This is good news. Means most users won't be affected by this.

[john-hen]

I'll leave this open until we hear from a Linux user. Note that there is no need to "downgrade" even if 6.0 doesn't work. We can have multiple Comsol versions installed side by side. After all, that's the whole point of the "discovery mechanism". Just check client.version after installing the new Comsol. If it returns "6.0" then everything's fine. Otherwise please run the discovery test as described in the earlier comment and be available for follow-up questions. If there is a problem, it should be an easy fix.

[mchekr]

Hi
I confirm that on a linux server mph fails to find the comsol installation.
the proposed discovery test returns :

$ python3 tools/test.py discovery --log
Running test group "discovery".
[00:00.138] Checking installation folder "/usr/local/comsol60/multiphysics".
[00:01.060] Reported version info is "COMSOL Multiphysics 6.0.0.318".
[00:01.061] Assigned name "6.0" to this installation.`
[00:01.061] Did not find Java virtual machine.
[00:01.061] Could not locate any Comsol installation.
Traceback (most recent call last):
  File "XXXXX/tests/MPh-main/tests/test_discovery.py", line 40, in <module>
    test_backend()
  File "XXXXX/tests/MPh-main/tests/test_discovery.py", line 25, in test_backend
    backend = mph.discovery.backend()
  File "XXXXX/tests/MPh-main/mph/discovery.py", line 416, in backend
    raise RuntimeError(error)
RuntimeError: Could not locate any Comsol installation.
Failed after 1 s.

tell me if you want additional information

[john-hen]

Hi @mchekr, thanks for running the test. Yes, I'll need a bit more information.

So it looks like the dynamic library libjvm.so is not where it used to be. With Comsol 5.6, it was here:

/usr/local/comsol56/multiphysics/java/glnxa64/jre/lib/amd64/server/libjvm.so

Can you search the comsol60 folder for libjvm.so? Or, alternatively, look it up in comsol.ini:

$ cat /usr/local/comsol60/multiphysics/bin/glnxa64/comsol.ini

In Comsol 5.6, that file contains:

# Settings for COMSOL desktop on glnxa64
-startup
../../plugins/org.eclipse.equinox.launcher_1.5.700.v20200527-1300.jar
--launcher.library
../glnxa64
-install ../..
-vm
../../java/glnxa64/jre/lib/amd64/server/libjvm.so
-vmargs
#-Djava.net.useSystemProxies=true
-Dosgi.splashPath=platform:/base/plugins/com.comsol.util/
-Dosgi.clean=true
-Dorg.osgi.service.log.admin.loglevel=DEBUG
-Dosgi.configuration.cascaded=true
-Dosgi.checkConfiguration=true
-Dosgi.sharedConfiguration.area=file:configuration/comsol
-Dosgi.sharedConfiguration.area.readOnly=true
-Dosgi.configuration.area=@user.home/.comsol/v56/configuration/comsol
-Dosgi.instance.area=@user.home/.comsol/v56/workspace/comsol
#-Dorg.eclipse.swt.browser.DefaultType=mozilla
-Declipse.security=
-Djava.security.policy=${osgi.install.area}/bin/glnxa64/comsol.policy
-Dcs.logoutput=file
-Dcs.logfileprefix=comsol
-Dcs.client=true
-Xss4m
-Xms40m
-Xmx2g
-XX:MaxMetaspaceSize=512m

The 8th line is the relative path from there to libjvm.so. MPh does not parse that file (though it could, actually…), the (relative) location is hard-coded. I did not expect that to change, but apparently it has, that's what that log entry means.

Maybe they vendored a new Java version? You could check that too. Like with Comsol 5.6, the output is this:

$ /usr/local/comsol56/multiphysics/java/glnxa64/jre/bin/java -version
openjdk version "1.8.0_265"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_265-b01)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.265-b01, mixed mode)

Should be the same call, except comsol60 in the path (unless that somehow changed too).

[mchekr]

The libjvm.so has indeed changed location : it appears to be in

../../java/glnxa64/jre/lib/server/libjvm.so

(the amd64 folder has been removed)

as for the Java version, the output is:

openjdk version "11.0.13" 2021-10-19
OpenJDK Runtime Environment Temurin-11.0.13+8 (build 11.0.13+8)
OpenJDK 64-Bit Server VM Temurin-11.0.13+8 (build 11.0.13+8, mixed mode)

[john-hen]

Ah, okay, they upgraded the JDK from 8 to 11. That's probably a good thing, might alleviate (or even solve) the issues discussed in #38.

I fixed that, we're now simply checking both locations. Was the easiest thing to do, but maybe I'll have it parse comsol.ini at some point, to future-proof this even more. That's actually what the start-up script comsol does too on Linux and macOS.

Can you run the discovery test again with the patch? I'll do a new release a.s.a.p. if the test now passes, which it should.

[mchekr]

Great!
the test now passes with the patch.

Many Thanks for the work, I'll wait for the release when you'll have the time.

[john-hen]

Perfect. Thanks for your help! Release will be out shortly…

[john-hen]

Fixed in MPh 1.1.4, released today.

[normanisrael]

Hi guys,

I am using comsol 6 on linux and mph 1.1.6 and I am running into this issue. My comsol is in a custom folder. I tried creating a symbolic link in ~/.local as stated here. Still no luck. Would love some help.

[john-hen]

Hi. Then it must be something else, as this issue was fixed in MPh 1.1.4. Probably just the symbolic link that has an incorrect target or name?