Skip to content
This repository has been archived by the owner on Mar 30, 2023. It is now read-only.

checksum fails with "no properly formatted SHA1 checksum lines found" for esVersion < 6.0.0 #237

Closed
ttsu opened this issue Oct 19, 2018 · 2 comments
Closed

checksum fails with "no properly formatted SHA1 checksum lines found" for esVersion < 6.0.0 #237

ttsu opened this issue Oct 19, 2018 · 2 comments

Comments

@ttsu
Copy link

ttsu commented Oct 19, 2018 •
edited
Loading

The ARM deployment fails when running the VM scripts to install elasticsearch/kibana/logstash if the "esVersion" parameter set to 5.6.12. I haven't tested other versions but I suspect that it will fail for any versions < 6.0.0 since the -c compatible checksum formats are only available for 6.0.0+. https://www.elastic.co/blog/sha512-checksums-for-elastic-stack-artifacts

This affects template version 6.4.2.

The failures occur where the shasum -c command is used:

azure-marketplace/src/scripts/elasticsearch-install.sh

Line 379 in bde918a

shasum -a 512 -c $SHASUM

azure-marketplace/src/scripts/kibana-install.sh

Line 173 in bde918a

shasum -a 512 -c $SHASUM

azure-marketplace/src/scripts/logstash-install.sh

Line 167 in bde918a

shasum -a 512 -c $SHASUM

Here is the deployment error message:

{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-debug for usage details.","details":[{"code":"Conflict","message":"{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"VMExtensionProvisioningError\",\r\n \"message\": \"VM has reported a failure when processing extension 'script'. Error message: \\\"Enable failed: failed to execute command: command terminated with exit status=1\\n[stdout]\\nefault\\\" package.\\r\\nE.g.: sudo apt install oracle-java8-set-default\\r\\nOn Ubuntu systems, oracle-java8-set-default is most probably installed\\r\\nautomatically with this package.\\r\\n######################\\r\\n\\r\\nSelecting previously unselected package oracle-java8-set-default.\\r\\n(Reading database ... \\r(Reading database ... 5%\\r(Reading database ... 10%\\r(Reading database ... 15%\\r(Reading database ... 20%\\r(Reading database ... 25%\\r(Reading database ... 30%\\r(Reading database ... 35%\\r(Reading database ... 40%\\r(Reading database ... 45%\\r(Reading database ... 50%\\r(Reading database ... 55%\\r(Reading database ... 60%\\r(Reading database ... 65%\\r(Reading database ... 70%\\r(Reading database ... 75%\\r(Reading database ... 80%\\r(Reading database ... 85%\\r(Reading database ... 90%\\r(Reading database ... 95%\\r(Reading database ... 100%\\r(Reading database ... 55379 files and directories currently installed.)\\r\\nPreparing to unpack .../oracle-java8-set-default_8u191-1~webupd8~1_all.deb ...\\r\\nUnpacking oracle-java8-set-default (8u191-1~webupd8~1) ...\\r\\nSelecting previously unselected package gsfonts.\\r\\nPreparing to unpack .../gsfonts_1%3a8.11+urwcyr1.0.7~pre44-4.2ubuntu1_all.deb ...\\r\\nUnpacking gsfonts (1:8.11+urwcyr1.0.7~pre44-4.2ubuntu1) ...\\r\\nSelecting previously unselected package libfontenc1:amd64.\\r\\nPreparing to unpack .../libfontenc1_1%3a1.1.3-1_amd64.deb ...\\r\\nUnpacking libfontenc1:amd64 (1:1.1.3-1) ...\\r\\nSelecting previously unselected package libxfont1:amd64.\\r\\nPreparing to unpack .../libxfont1_1%3a1.5.1-1ubuntu0.16.04.4_amd64.deb ...\\r\\nUnpacking libxfont1:amd64 (1:1.5.1-1ubuntu0.16.04.4) ...\\r\\nSelecting previously unselected package x11-common.\\r\\nPreparing to unpack .../x11-common_1%3a7.7+13ubuntu3.1_all.deb ...\\r\\ndpkg-query: no packages found matching nux-tools\\r\\nUnpacking x11-common (1:7.7+13ubuntu3.1) ...\\r\\nSelecting previously unselected package xfonts-encodings.\\r\\nPreparing to unpack .../xfonts-encodings_1%3a1.0.4-2_all.deb ...\\r\\nUnpacking xfonts-encodings (1:1.0.4-2) ...\\r\\nSelecting previously unselected package xfonts-utils.\\r\\nPreparing to unpack .../xfonts-utils_1%3a7.7+3ubuntu0.16.04.2_amd64.deb ...\\r\\nUnpacking xfonts-utils (1:7.7+3ubuntu0.16.04.2) ...\\r\\nSelecting previously unselected package gsfonts-x11.\\r\\nPreparing to unpack .../gsfonts-x11_0.24_all.deb ...\\r\\nUnpacking gsfonts-x11 (0.24) ...\\r\\nProcessing triggers for libc-bin (2.23-0ubuntu10) ...\\r\\nProcessing triggers for man-db (2.7.5-1) ...\\r\\nProcessing triggers for systemd (229-4ubuntu21.4) ...\\r\\nProcessing triggers for ureadahead (0.100.0-19) ...\\r\\nSetting up oracle-java8-set-default (8u191-1~webupd8~1) ...\\r\\nSetting up gsfonts (1:8.11+urwcyr1.0.7~pre44-4.2ubuntu1) ...\\r\\nSetting up libfontenc1:amd64 (1:1.1.3-1) ...\\r\\nSetting up libxfont1:amd64 (1:1.5.1-1ubuntu0.16.04.4) ...\\r\\nSetting up x11-common (1:7.7+13ubuntu3.1) ...\\r\\nupdate-rc.d: warning: start and stop actions are no longer supported; falling back to defaults\\r\\nSetting up xfonts-encodings (1:1.0.4-2) ...\\r\\nSetting up xfonts-utils (1:7.7+3ubuntu0.16.04.2) ...\\r\\nSetting up gsfonts-x11 (0.24) ...\\r\\nProcessing triggers for libc-bin (2.23-0ubuntu10) ...\\r\\nProcessing triggers for systemd (229-4ubuntu21.4) ...\\r\\nProcessing triggers for ureadahead (0.100.0-19) ...\\r\\n/var/lib/dpkg/info /var/lib/waagent/custom-script/download/0\\n[18102018-22:07:14] [install_java] [install_java_package] update oracle-java8-installer to 8u181\\n/var/lib/waagent/custom-script/download/0\\n[18102018-22:07:14] [install_java] [install_java_package] updated oracle-java8-installer\\nReading package lists...\\nBuilding dependency tree...\\nReading state information...\\noracle-java8-installer is already the newest version (8u191-1~webupd8~1).\\n0 upgraded, 0 newly installed, 0 to remove and 49 not upgraded.\\n[18102018-22:07:14] [install_java] installed java!\\n[18102018-22:07:14] [install_es] installing Elasticsearch 5.6.12\\n[18102018-22:07:14] [install_es] download location - https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.12.deb?ultron=msft&gambit=azure\\n[18102018-22:07:16] [install_es] downloaded Elasticsearch 5.6.12\\n[18102018-22:07:18] [install_es] error validating checksum for Elasticsearch 5.6.12\\n\\n[stderr]\\nmke2fs 1.42.13 (17-May-2015)\\ngpg: keyring `/tmp/tmp5qgzhp19/secring.gpg' created\\ngpg: keyring `/tmp/tmp5qgzhp19/pubring.gpg' created\\ngpg: requesting key EEA14886 from hkp server keyserver.ubuntu.com\\ngpg: /tmp/tmp5qgzhp19/trustdb.gpg: trustdb created\\ngpg: key EEA14886: public key \\\"Launchpad VLC\\\" imported\\ngpg: no ultimately trusted keys found\\ngpg: Total number processed: 1\\ngpg: imported: 1 (RSA: 1)\\nshasum: elasticsearch-5.6.12.deb.sha512: no properly formatted SHA1 checksum lines found\\n\\\".\"\r\n }\r\n ]\r\n }\r\n}"}]}
@russcam
Copy link
Contributor

russcam commented Oct 23, 2018

Thanks for opening @ttsu. Will look into

@russcam
Copy link
Contributor

russcam commented Oct 30, 2018

From 5.6.2 onwards, the Elastic Stack build changed from generating sha1 checksums to sha512 checksums - this was missed when implementing the checking of SHA Checksums in c473220.

Working on a fix now

russcam added a commit that referenced this issue Oct 30, 2018
@russcam
Check sha1 checksum for versions less than 5.6.2
e8c6d80 
This commit updates the install scripts to download and use a
sha1 checksum for Elastic Stack versions less than 5.6.2.

Fixes #237
@russcam russcam mentioned this issue Oct 30, 2018
Merged
russcam added a commit that referenced this issue Oct 30, 2018
@russcam
Check sha1 checksum for versions less than 5.6.2 (#241)
a6ce061 
This commit updates the install scripts to download and use a
sha1 checksum for Elastic Stack versions less than 5.6.2.

Fixes #237

* Include package name in sha file

This commit adds the package name to the sha checksum
file, to validate it with shasum utility. Earlier versions
of checksum file only include the checksum and not the package name.

* Use esVersion for parameter instead of version

With the update to gulp, the version parameter is now used by
gulp to return details of the gulp version. Change our version
command line flag to esVersion, in line with the template parameters

* don't delete resource groups on bailout when nodestroy passed

This commit changes the behaviour of bailout to not delete resource groups
when nodestroy flag has been passed. When there's a failure that causes a
bailout, user may want to investigate by passing --nodestroy.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@russcam @ttsu