Error messages when running scriplets make debugging failures difficult

[dilyn-corner]

Bug Description

When a part in some snap's snapcraft.yaml includes a scriplet, error messages surrounding failures of that scriplet are "incorrect" in that they do not correspond to problems within the YAML file itself, but instead in some ancillary file which (I presume) snapcraft is creating and then executing during the parts lifecycle event. This means that debugging any given scriplet failure is never deterministically difficult; instead, the difficulty of determining what failed, where it failed, and why it failed can range from 'trivial' (the context surrounding the error easily disambiguates the failure) to 'impossible' (the error message is a nonsequitor, the surrounding context isn't why it failed but instead some earlier part which failed, etc.).

In short: instead of debugging the scriplet in my snapcraft.yaml which is, for all intents and purposes, a shell script, I have to debug the transformation of my scriplet into some nonobvious, unavailable-by-default, created-by-snapcraft script.

To Reproduce

Reproducing the issue is straightforward. The YAML posted below will produce an error of the relevant kind.

To replicate my environment:
snap refresh --channel=7.x/stable snapcraft
snapcraft --use-lxd

Environment

Some variant of Ubuntu 23.04 with snapcraft from 7.x/stable, revision 9542. Build the example snap with snapcraft --use-lxd.

snapcraft.yaml

name: my-snap-name # you probably want to 'snapcraft register <name>'
base: core22 # the base snap is the execution environment for this snap
version: '0.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Single-line elevator pitch for your amazing snap # 79 char long summary
description: |
  This is my-snap's description. You have a paragraph or two to tell the
  most important story about your snap. Keep it under 100 words though,
  we live in tweetspace and your description wants to look good in the snap
  store.

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: devmode # use 'strict' once you have the right plugs and slots

parts:
  my-part:
    # See 'snapcraft plugins'
    plugin: nil
    override-build: |
      cat >> test.sh << EOF
      #!/bin/sh -eux

      foo=bar
      bar=$baz
      EOF

Relevant log output

Starting Snapcraft 7.5.2                                                                                                                               
Logging execution to '/home/dilyn/.local/state/snapcraft/log/snapcraft-20230921-105432.570321.log'                                                     
Running on amd64 for amd64                                                                                                                             
Cleaning build provider                                                                                                                                
Cleaned build provider                                                                                                                                 
Starting Snapcraft 7.5.2                                                                                                                               
Logging execution to '/home/dilyn/.local/state/snapcraft/log/snapcraft-20230921-105434.907064.log'                                                     
Running on amd64 for amd64                                                                                                                             
Launching instance...                                                                                                                                  
Checking for instance 'snapcraft-my-snap-name-on-amd64-for-amd64-2193722' in project 'snapcraft' in remote 'local'                                     
Instance 'snapcraft-my-snap-name-on-amd64-for-amd64-2193722' does not exist.                                                                           
Checking for base instance 'base-instance-snapcraft-buildd-base-v10--2d55678e0339df02cd48' in project 'snapcraft' in remote 'local'                    
Creating instance from base instance 'base-instance-snapcraft-buildd-base-v10--2d55678e0339df02cd48'                                                   
Starting instance.                                                                                                                                     
Starting Snapcraft 7.5.2                                                                                                                               
Logging execution to '/tmp/snapcraft.log'                                                                                                              
Running on amd64 for amd64                                                                                                                             
Initializing parts lifecycle                                                                                                                           
Executing parts lifecycle...                                                                                                                           
Executing parts lifecycle: pull my-part                                                                                                                
Executing action                                                                                                                                       
Executed: pull my-part                                                                                                                                 
Executing parts lifecycle: build my-part                                                                                                               
Executing action                                                                                                                                       
:: + cat                                                                                                                                               
:: /bin/bash: line 41: baz: unbound variable                                                                                                           
'override-build' in part 'my-part' failed with code 1.
Review the scriptlet and make sure it's correct.                                                
Failed to execute pack in instance.                                                                                                                    
Full execution log: '/home/dilyn/.local/state/snapcraft/log/snapcraft-20230921-105434.907064.log'

Additional context

Ideally, the error message will at least indicate which line of the YAML is causing the failure. I have no idea how tricky this reverse-engineering would be in practice.

An alternative would be to increase the visibility on build failures to the host machine. Perhaps by including the transformed scriplet which is being executed in the build environment for that part in the generated log file?

[dilyn-corner]

I believe this can be closed due to work done by @bepri

[bepri]

I agree, it sounds like this was closed with canonical/craft-parts#966 and canonical/craft-parts#897, which both made it into Snapcraft 8.6. Thanks Dilyn!