[colic] KeyError on execution of ELK with ScanCode-CLI

[inishchith]

• ScanCode-CLI, when executed with ELK, showed the following error. Executes well with other repositories(eg. Graal) when tested.

• Error Log

Traceback (most recent call last):
  File "/Users/Nishchith/GitHub/CHAOSS/exec/scancode-toolkit/etc/scripts/scancli.py", line 73, in <module>
    for s in scan(args):
  File "/Users/Nishchith/GitHub/CHAOSS/exec/scancode-toolkit/etc/scripts/scancli.py", line 64, in scan
    results = channel.receive()
  File "/usr/local/lib/python3.6/site-packages/execnet/gateway_base.py", line 728, in receive
    raise self._getremoteerror() or EOFError()
execnet.gateway_base.RemoteError: Traceback (most recent call last):
  File "<string>", line 1063, in executetask
  File "<string>", line 1, in do_exec
  File "<remote exec>", line 53, in <module>
  File "<remote exec>", line 44, in run_scan
  File "/Users/Nishchith/GitHub/CHAOSS/exec/scancode-toolkit/src/scancode/cli.py", line 813, in run_scan
    raise ScancodeError(msg + '\n' + traceback.format_exc())
ScancodeError: ERROR: failed to collect codebase at: u'/tmp/worktrees/grimoirelab-elk-git/utils/perceval'
Traceback (most recent call last):
  File "/Users/Nishchith/GitHub/CHAOSS/exec/scancode-toolkit/src/scancode/cli.py", line 809, in run_scan
    max_in_memory=max_in_memory
  File "/Users/Nishchith/GitHub/CHAOSS/exec/scancode-toolkit/src/scancode/resource.py", line 281, in __init__
    self._populate()
  File "/Users/Nishchith/GitHub/CHAOSS/exec/scancode-toolkit/src/scancode/resource.py", line 440, in _populate
    parent = parent_by_loc.pop(top)
KeyError: u'/tmp/worktrees/grimoirelab-elk-git/utils/perceval/backends'


[2019-08-16 20:16:10,589] - Analysis failed at 41af2b5984ec2ce9b6781bfb1b3e133aedba19a6
Traceback (most recent call last):
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-graal/graal/backends/core/analyzers/scancode.py", line 90, in __analyze_scancode_cli
    msg = subprocess.check_output(cmd_scancli).decode("utf-8")
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['python3', '/Users/Nishchith/GitHub/CHAOSS/exec/scancode-toolkit/etc/scripts/scancli.py', '/tmp/worktrees/grimoirelab-elk-git/perceval/__init__.py', '/tmp/worktrees/grimoirelab-elk-git/perceval/backends/__init__.py', '/tmp/worktrees/grimoirelab-elk-git/perceval/backends/backend.py', '/tmp/worktrees/grimoirelab-elk-git/perceval/utils.py', '/tmp/worktrees/grimoirelab-elk-git/utils/bugzilla2el.py', '/tmp/worktrees/grimoirelab-elk-git/utils/perceval']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-perceval/perceval/backend.py", line 472, in run
    for item in items:
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-perceval/perceval/backend.py", line 589, in fetch
    raise e
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-perceval/perceval/backend.py", line 583, in fetch
    for item in items:
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-perceval/perceval/backend.py", line 162, in fetch
    for item in self.fetch_items(category, **kwargs):
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-graal/graal/graal.py", line 182, in fetch_items
    raise e
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-graal/graal/graal.py", line 175, in fetch_items
    commit['analysis'] = self._analyze(commit)
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-graal/graal/backends/core/colic.py", line 170, in _analyze
    analysis = self.analyzer.analyze(local_paths)
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-graal/graal/backends/core/colic.py", line 221, in analyze
    analysis = self.analyzer.analyze(**kwargs)
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-graal/graal/backends/core/analyzers/scancode.py", line 127, in analyze
    result = self.__analyze_scancode_cli(kwargs['file_paths'])
  File "/Users/Nishchith/GitHub/CHAOSS/grimoirelab-graal/graal/backends/core/analyzers/scancode.py", line 93, in __analyze_scancode_cli
    e.output.decode("utf-8")))
graal.graal.GraalError: Scancode failed at ['/tmp/worktrees/grimoirelab-elk-git/perceval/__init__.py', '/tmp/worktrees/grimoirelab-elk-git/perceval/backends/__init__.py', '/tmp/worktrees/grimoirelab-elk-git/perceval/backends/backend.py', '/tmp/worktrees/grimoirelab-elk-git/perceval/utils.py', '/tmp/worktrees/grimoirelab-elk-git/utils/bugzilla2el.py', '/tmp/worktrees/grimoirelab-elk-git/utils/perceval'], [

[valeriocos]

The error can be reproduced with the following conf:

colic
https://github.com/chaoss/grimoirelab-elk
--git-path
/tmp/graal_issue_55
--exec-path
/home/slimbook/Escritorio/sources/graal-libs/scancode-toolkit/etc/scripts/scancli.py
--category
code_license_scancode_cli
--from-date
2015-11-12

or

colic
https://github.com/chaoss/grimoirelab-elk
--git-path
/tmp/graal_issue_55
--exec-path
/home/slimbook/Escritorio/sources/graal-libs/scancode-toolkit/etc/scripts/scancli.py
--category
code_license_scancode_cli
--from-date
2017-07-15

git show 2fca45d