Signal v1.15.0 and newer crash on startup on Fedora Linux 28/29 (and others), libcrypto related during DB operations (induced by sqlcipher binaries) #2634
Comments
|
In our Thanks for jumping in on this! |
|
With the default Signal v1.15.3 from Git (without any node-sqlcipher upgrades), I get this - the ctx=0x0 looks suspicious to me, since the manual page states: I still can't make head or tails of this since I don't know how/where/why these are called. |
|
As can be seen from the stacktrace, the system's OpenSSL 1.1.0 library is used instead of the statically linked one. This can happen if the library is already loaded into the Electron process and its symbols take precedence. The crash is likely a result of SQLCipher expecting the older version of OpenSSL it was built against. @scottnonnenberg-signal You can prevent this by compiling your version of |
|
@scottnonnenberg-signal I think the approach of packing a static OpenSSL on Linux is not working for me although I understand it is trying to be helpful. Basically we should not be mixing SSL stuff in the same program, and up to the lastest v1.14.x things went smoothly for me. I am not sure if @richardschuetz 's proposal is right, the only way I have found through experimenting for hours on end is this script (snippet): I can not package this build through yarn build-release --linux rpm though, which will rebuild sqlcipher in a different manner and then fail at run-time, not with SIGSEGV, but through not finding libcrypto.so.1.0.0. It appears it picks up a binary from journeyapps in this case, since that tries to bind against libcrypto.so.1.0.0. - I don't have that decrepit version on my system (Fedora 28). I cannot yet your sqlcipher fork to work at all, the original one requires the rebuild-from-source as shown above. |
|
@mandree That's not the right version of |
|
@richardschuetz where do you run this Configure line you mention? I use a spec file to build an RPM of signal-desktop for myself on Fedora 27 and I can't see where I'd insert this Configure line. The %build part looks like this in my spec file. I also get segfault but even in the latest 1.15.3 version. I tried going back to 1.15.1 and still get segfault. Also tried removing my .config/Signal dir but that didn't help. The last version I can successfully build and run with my RPM spec file is 1.14.4. They all build but only 1.14.4 runs without segfault. |
|
@scottnonnenberg-signal what I am trying to do is build Fedora 28 packages of Signal for several computers I oversee and deploy it. A statically linked obsolete OpenSSL version from Ubuntu 14.04 appears not to work on Fedora 28 (or Arch for that matter) for various reasons: they do not blend in with a different distribution that uses different policies, versions, paths, and possibly other reasons. So IMO what I/we need to do is rebuild sqlcipher all from source, link dynamically against the system's OpenSSL library (as the original journeyapps/sqlcipher claims to do) and forbid it to use any precompiled binaries. The electron-rebuild run on the original sqlcipher exacts just that for the "yarn run start", but if I run yarn build-release, it clobbers the results of the electron-rebuild, rebuilds sqlcipher again (pulling in the binary) and then boom. Bottom line, I want to recompile sqlcipher and waive any binaries |
|
Success. Following up my own post, after researching, reading, experimenting, learning, I have achieved my goal. I have ended up using the original @journeyapps/sqlcipher, hack its "install" configuration in package.json to --build-from-source, install the hacked module as requisite. My build script for Fedora 28 currently used as shown below, works with v1.15.4 and generates a functioning app that is dynamically linked against libcrypto.so.1.1. I understand that may not be good for distribution, but the same approach should satisfy anything that builds from source and goes shipwrecked by the statically linked Ubuntu 14.04 OpenSSL crap. WARNING this is not for developers who hack on Signal due to the brutal (side RANT: Note that Ubuntu "long-term-support" is a red herring, it only applies to a few dozen base packages that are core to ubuntu server, on the Desktop it's useless as most packages go out of support after a few months, and we should not encourage users to use privacy/security-sensitive stuff on top of a rocking and sliding pile of unmaintained packages. Don't build on Ubuntu 14.04. It enhances compatibility but only permits users to stay on their outdated/insecure installs longer.) |
mandree
changed the title
|
Reproduced with Signal v1.15.5 on Fedora 28, building from the zip file linked to on https://github.com/signalapp/Signal-Desktop/releases/tag/v1.15.5 . |
|
Reproduced on a LibreSSL-based Void Linux system. |
|
It should be noted that "So the Signal-Desktop build uses a fork of this project," instead of the upstream for node-sqlcipher. journeyapps/node-sqlcipher#8 |
|
FWIW: the above steps of patching @journeyapps/sqlcipher (and performed in principal) worked for 'git checkout tags/v1.17.2'. |
|
Unfortunately @mandree's shell script no longer works because the node-sqlcipher fork Signal is using added support for FTS5, which isn't present in the original. I created journeyapps/node-sqlcipher#17 to get it added to the original package. |
|
I found https://copr.fedorainfracloud.org/coprs/luminoso/Signal-Desktop/ (and rebuilt the RPMs on F29 from commit |
|
Once journeyapps/node-sqlcipher#17 is merged the Signal maintainers could switch back to from the fork they're currently using. The fork links statically against libcrypto which is causing issues with a number of distributions. Edit: typo |
|
Yeah, i noticed. We'll need to wait for a new version to be published to npm... Edit: Edit2:
👍 (FYI i had to clear my ~/.cache/Signal directory to make it work) |
|
@fholzer which version did you try that with? I did it with v1.23.1 but it crashed at startup. |
|
Arch Linux are using a patch to link libcrypto dynamically rather than statically - this appears to fix crashes for me on Fedora 29, too. Link: |
|
@stemid I'm on 1.22.0 at the moment. Had some issues with 1.23.x myself, but no time to look into it in more detail yet. @mandree not sure which sqlcipher package Arch uses, but the file that seems to get patched with the file you linked is actually linking dynamically already: https://github.com/journeyapps/node-sqlcipher/blob/master/deps/sqlite3.gyp#L75 |
@fholzer as of v1.23.1, sqlcipher isn't pulled from in its master branch, but from a frozen version of Scott Nonnenberg's clone: |
|
@fholzer thanks for that but it failed during yarn install. Tried it with a new git repo, your mods. Output: https://bpaste.net/show/9fc922f28d05 Fedora 28 with node and node-gyp installed from packages. Maybe I need later versions of those? |
|
@stemid not entirely sure what to make of this... have you had a look at nodejs/node-gyp#809? there's one comment in particular that seemed to have help a lot of ppl, maybe give that a try? nodejs/node-gyp#809 (comment) In case that doesn't help, can you run
That's a possibility, though I'm no expert in regards to compiling native node modules. Some ppl suggested outdated vars in ~/.npmrc in the thread I linked. Not sure... sorry Have you ever successfully built Signal with your setup? (sqlcipher in particular) |
|
The patch from comment 478328776 worked for me, but applying could only be done after the first wget "https://aur.archlinux.org/cgit/aur.git/plain/openssl-linking.patch?h=signal" -O ~/openssl-linking.patch sed -i.bak "s/node\": \"10.13.0/node\": \"$(node --version | sed 's/v//')/" package.json yarn install --frozen-lockfile patch -p1 < ~/openssl-linking.patch sudo chattr +i node_modules/@journeyapps/sqlcipher/deps/sqlite3.gyp # Err... yarn generate SIGNAL_ENV=production yarn build-release NODE_ENV=production NODE_APP_INSTANCE=production yarn run start --disable-gpu Still, this bug is open for way too long and I don't understand why Signal desktop should only be a thing for MacOS or Windows users, especially considering most tinfoil hats (hello, myself!) would run Linux anyway. This report even includes fixes, can't they just be applied to upstream to finally resolve this one? |
|
@ckujau I only skipped the chattr line. |
I agree this is why I just gave up on Signal... I can't see why they cannot take this info (or info in multiple threads in regards to RPMs/Fedora functionality) and create a build. |
I haven't tested these builds, but it appears there's a COPR: |
Yeah I've seen these. Thing is if I am using a security application I want it built and provided by the company producing it. |
|
The snap-packaged application is also broken because of this, along with openSUSE Tumbleweed. Maybe when Signal sigfaults in absolutely every Linux distro something will be done about it? |
Signal is currently using a forked version of node-sqlcipher. I'm not entirely sure as to all the motivations for forking, but it seems that one reason may have been because upstream didn't support FTS5. This reason is no longer relevant because of this PR [^1], which adds FTS5 support upstream. Another reason for the forked version of node-sqlcipher may be to bundle libcrypto. This is causing problems with many Linux users, especially those who are not using Ubuntu [^2][^3]. It seems to me that statically linking is not generally considered to be a best practice on Linux. The upstream project's README [^4] seems to have a sensible policy: bundling OpenSSL on Windows and OS X platforms and dynamically linking to the system's library on Linux platforms. This PR moves away from the forked node-sqlcipher and returns back to upstream. This seems to fix OpenSSL problems on non-Ubuntu Linux distributions. [^1]: journeyapps/node-sqlcipher#17 [^2]: signalapp#2634 [^3]: signalapp#2662 [^4]: https://github.com/journeyapps/node-sqlcipher#openssl
|
Hello! I have a crashing Signal too, my backtrace looks similar to the initial posted.
This is the scenario on my laptop. Gentoo btw. Conclusion: Signal can not be used Linux installations with (encrypted fs and) cryptsetup with OpenSSL backend? OR Hypothesis: Signal can be used on Linux installations with cryptsetup with OpenSSL backend IF they match the OpenSSL version used in the Signal version? or is there any other workaround which I missed? :) |
|
@buscher workaround is patching one of the dependencies (see above #2634 (comment)) |
|
@buscher as hsanjuan said, you have to either patch a dependency, or replace the dependency After that you can start the binary from the signal/release/linux-unpacked directory |
|
@scottnonnenberg-signal can this issue please be addressed in the next release? It's been open for one and a half years and the cause is known. To all other users: please refrain from posting "me too" with just changed minor details of solutions, the cause and its workaround are sufficiently documented already. If in doubt whether your information is already contained here, the slightest doubt => do not post comments here. |
|
@mandree We don't officially support any Linux platform other than Debian-based Linux distributions, via our official apt packages. We welcome assistance in making Signal Desktop more accessible to other distributions, but right now we can't spend much time on it. And we won't be changing our SQLCipher/OpenSSL build approach, statically linking everything. So the best way to help us is to research then provide low-impact changes we could apply to our existing setup. |
|
@scottnonnenberg-signal Sorry to jump in like this, but I think there is misconumication Currently you can't run signal on any linux distribution except ubuntu. This is because your forked version of sqlcipher links against a static openssl library. The solution would be to either change your forked sqlcipher repository to not link against a static openssl, or use the upstream one. To be clear: |
This is not correct. I've tested Signal Desktop successfully not just on Ubuntu, but also Debian. And Linux Mint, and Kubuntu, a step or two away from Ubuntu itself. Perhaps you haven't tried with the most recent builds? |
|
My apologizes, did you change anything regarding sqlcipher? Could somebody else comment if it stills fails for them? |
|
Arch patches the |
My guess would be that it builds (for now) due to this commit (updating the forked version of node-sqlcipher to 4.3.0 and updates the "packaged" openssl library to 1.1.1d: scottnonnenberg-signal/node-sqlcipher@b10f232 |
This was not a build problem. It always built. The problem is the application died on start. I checked and it seems to run fine now that openssl was updated to 1.1.1d (openSUSE Tumbleweed).
It seems the upstream does not include a statically-linked openssl for linux and is more outdated. Now, are those compiled by hand on a dev machine? If so, it seems there is a small dilemma here between using manually compiled binaries by a project dev vs. using the distribution's provided libraries. |
|
Quoting @d-hansen:
For me, 68ee557 apparently fixes the issue on Fedora 31 (i. e. v1.32.0-beta.2 and younger), yet I'd like to stress @d-hansen's "for now". My concern is that the error will come back as OpenSSL is updated again in the distros over time on Linux. It would seem that working now is just a coincidence because Given that electron at least on Linux links to libcrypto.so dynamically (see below), it seems important that we prevent ever mixing a different version in, so I ask @scottnonnenberg-signal to Downstream packagers have been patching This is on Fedora 31 amd64: |
|
It is very nonsensical to me to have a product that touts security (of your messages) but then does not follow or track the latest updates to highly-vulnerable cryptography libraries. One would think that a product like signal would want to stay as current as possible and track the most recent versions and updates to OpenSSL and libcrypto specifically! Using a version of node/sqlcipher from 2 years ago (with fairly minimal updates) and then statically linking against libcrypto from a particular version of OpenSSL just seems so wrong in the security world. We want systems to quickly respond to CVE's. So, it likely won't be very long until the next rev of OpenSSL is released and of course, systems like Fedora, Ubuntu, etc are going to quickly upgrade to it. :( |
|
Not only that, but the encryption key is stored next to the database, while the attachments are unencrypted. I don't understand what scenario the encryption mitigates. |
Scott, I understand time constraints, but I do not understand what the technical merit of "linking statically" is if it breaks portability. But then we need to do two things:
The proper course of action would seem to declare Ubuntu 16 unsupported and require a dynamic libcrypto.so.1.1 for now and build against 1.1 headers. What currently happens is that some part of the build on Fedora (included module) pulls in libcrypto.so (as seen by my The given low-profile change is this: |
Bug description
Signal crashes on startup with a SIGSEGV.
Steps to reproduce
Actual result:
Gets killed with SIGSEGV. This is the console output:
And then it crashes. This is the backtrace from GDB. If more details are needed, please let me know the magic of Signal's build rig to make it a build with debug symbols. I am a C programmer but unfamiliar with node, yarn, and thereabouts.
Expected result:
Proper start up.
Screenshots
Platform info
Signal version:
v1.15.2 as Git tag.
Operating System:
Fedora Linux 28
Linked device version:
8.0.0
Link to debug log
can't obtain one. See above for what I've got.
The text was updated successfully, but these errors were encountered: