Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Changing subtype of filp returned by Lsof #922

Closed
vobst opened this issue Mar 10, 2023 · 4 comments · Fixed by #1048
Closed

Changing subtype of filp returned by Lsof #922

vobst opened this issue Mar 10, 2023 · 4 comments · Fixed by #1048

Comments

@vobst
Copy link

vobst commented Mar 10, 2023 •
edited
Loading

Describe the bug
When calling the list_fds method of the lsof plugin the subtype of the returned filp behaves in (at least to me) unexpected ways.
According to this snippet

file_type = symbol_table + constants.BANG + "file"

fds = objects.utility.array_of_pointers(
     fd_table, count=max_fds, subtype=file_type, context=context
)

for fd_num, filp in enumerate(fds):
     if filp != 0:
                # added by me ^^
                print(filp.dereference().vol.get("type_name"))
                filp = filp.dereference().cast(file_type)
                print(context.modules["kernel"].get_symbols_by_absolute_location(filp.f_op))
                # end
                full_path = LinuxUtilities.path_for_file(context, task, filp)

                yield fd_num, filp, full_path

It should always be symbol_table + constants.BANG + "file", i.e., symbol_table_name1!file. However, the actual output when running my plugin is

...
symbol_table_name1!file
['symbol_table_name1!bpf_map_fops']
symbol_table_name1!btf_type
...

i.e., there is a distinct point where it switches - the first time it hits an fd that represents a BPF map. In that case I use the returned file pointer's private_data to construct a BpfMap object, which internally does quite a lot of stuff (also involving BTF). However, I would not expect this to change anything in the fd iterator.

Context
Volatility Version: a35afd4
Operating System: Kali container on Arch host
Python Version: 3.11.2
Suspected Operating System: self-compiled 6.0.12
Command:

To Reproduce
Steps to reproduce the behavior:

  1. You can find the plugin that I was writing here
  2. An example memory image (+symbols) that triggers the behavior can be downloaded here
    (Note that the project contains a patch to hack around the issue and it gets applied if you use the container)

Expected behavior
To me, the fact that something I do with the filp changes the subtype of the iterator for future items is unexpected. I am not familiar enough with your framework or Python to tell if this a mistake on my end, but if it is, I'd like to understand how I can change my code to avoid it.

Example output

vol -vvvvv -f /io/dumps/krie-3410c66d-26be0e1ef560.elf linux.bpf_proc
Volatility 3 Framework 2.4.2
INFO     volatility3.cli: Volatility plugins path: ['/opt/vol/volatility3/volatility3/plugins', '/opt/vol/volatility3/volatility3/framework/plugins']
INFO     volatility3.cli: Volatility symbols path: ['/opt/vol/volatility3/volatility3/symbols', '/opt/vol/volatility3/volatility3/framework/symbols']
Level 7  volatility3.cli: Cache directory used: /root/.cache/volatility3
INFO     volatility3.framework.automagic: Detected a linux category plugin
INFO     volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist
INFO     volatility3.framework.automagic: Running automagic: SymbolCacheMagic
INFO     volatility3.framework.automagic: Running automagic: LayerStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 7  volatility3.framework.layers.resources: Available URL handlers: HTTPErrorProcessor, HTTPDefaultErrorHandler, HTTPRedirectHandler, ProxyHandler, HTTPBasicAuthHandler, ProxyBasicAuthHandler, HTTPDigestAuthHandler, ProxyDigestAuthHandler, AbstractHTTPHandler, HTTPHandler, HTTPSHandler, HTTPCookieProcessor, UnknownHandler, FileHandler, FTPHandler, CacheFTPHandler, DataHandler, VolatilityHandler, JarHandler, OfflineHandler, LeechCoreHandler
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using Elf64Stacker
Level 8  volatility3.framework.automagic.stacker: Stacked Elf64Layer using Elf64Stacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LinuxIntelStacker
DEBUG    volatility3.framework.automagic.symbol_cache: Duplicate entry for identifier b'Linux version 6.0.12 (user@26be0e1ef560) (Ubuntu clang version 14.0.0-1ubuntu1, Ubuntu LLD 14.0.0) #1 SMP PREEMPT_DYNAMIC Fri Mar 3 21:31:29 UTC 2023\n\x00': file:///opt/vol/volatility3/volatility3/symbols/linux/6.0.12_x86_64_dfrws_vol_ebpf-26be0e1ef560.json and file:///opt/vol/volatility3/volatility3/symbols/linux/c3404494065f55e73e9459dd23f8808f.isf.json
DEBUG    volatility3.framework.automagic.linux: Identified banner: b'Linux version 6.0.12 (user@26be0e1ef560) (Ubuntu clang version 14.0.0-1ubuntu1, Ubuntu LLD 14.0.0) #1 SMP PREEMPT_DYNAMIC Fri Mar 3 21:31:29 UTC 2023\n\x00'
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!assoc_array_ptr
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!io_notif_slot
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mtd_info
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!iopf_device_param
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!nf_ct_event_notifier
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!mem_cgroup
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!sfp_bus
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!udp_tunnel_nic
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!pcpu_dstats
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!phylink
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!smc_hashinfo
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!reset_control
DEBUG    volatility3.framework.symbols: Unresolved reference: LintelStacker1!gpio_desc
DEBUG    volatility3.framework.automagic.linux: Linux ASLR shift values determined: physical 0 virtual 0
DEBUG    volatility3.framework.automagic.linux: DTB was found at: 0x300c000
Level 8  volatility3.framework.automagic.stacker: Stacked IntelLayer using LinuxIntelStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker
Level 8  volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist.kernel
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name.memory_layer
Level 9  volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.BpfPslist.kernel.layer_name.memory_layer.base_layer
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist.kernel
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
Level 9  volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.BpfPslist
DEBUG    volatility3.framework.automagic.stacker: Stacked layers: ['IntelLayer', 'Elf64Layer', 'FileLayer']
INFO     volatility3.framework.automagic: Running automagic: SymbolFinder
INFO     volatility3.framework.automagic: Running automagic: LinuxSymbolFinder
Level 9  volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.BpfPslist.kernel.symbol_table_name
DEBUG    volatility3.framework.automagic.symbol_cache: Duplicate entry for identifier b'Linux version 6.0.12 (user@26be0e1ef560) (Ubuntu clang version 14.0.0-1ubuntu1, Ubuntu LLD 14.0.0) #1 SMP PREEMPT_DYNAMIC Fri Mar 3 21:31:29 UTC 2023\n\x00': file:///opt/vol/volatility3/volatility3/symbols/linux/6.0.12_x86_64_dfrws_vol_ebpf-26be0e1ef560.json and file:///opt/vol/volatility3/volatility3/symbols/linux/c3404494065f55e73e9459dd23f8808f.isf.json
DEBUG    volatility3.framework.automagic.symbol_finder: Identified banner: b'Linux version 6.0.12 (user@26be0e1ef560) (Ubuntu clang version 14.0.0-1ubuntu1, Ubuntu LLD 14.0.0) #1 SMP PREEMPT_DYNAMIC Fri Mar 3 21:31:29 UTC 2023\n\x00'
DEBUG    volatility3.framework.automagic.symbol_finder: Using symbol library: file:///opt/vol/volatility3/volatility3/symbols/linux/6.0.12_x86_64_dfrws_vol_ebpf-26be0e1ef560.json
INFO     volatility3.framework.automagic: Running automagic: KernelModule

PID     COMM    PROGS   MAPS    LINKS
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!assoc_array_ptr
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!io_notif_slot
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mtd_info
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!iopf_device_param
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!nf_ct_event_notifier
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!mem_cgroup
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!sfp_bus
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!udp_tunnel_nic
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!pcpu_dstats
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!phylink
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!smc_hashinfo
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!reset_control
DEBUG    volatility3.framework.symbols: Unresolved reference: symbol_table_name1!gpio_desc
Level 9  volatility3.utility.common: Program does not have BTF info attached
Level 9  volatility3.utility.common: Program does not have BTF info attached
Level 9  volatility3.utility.common: Program does not have BTF info attached
Level 9  volatility3.utility.common: Program does not have BTF info attached
Level 9  volatility3.utility.common: Program does not have BTF info attached

1       systemd 6,7,8,9,10              Traceback (most recent call last):
  File "/opt/vol/volatility3/vol.py", line 10, in <module>
    volatility3.cli.main()
  File "/opt/vol/volatility3/volatility3/cli/__init__.py", line 797, in main
    CommandLine().run()
  File "/opt/vol/volatility3/volatility3/cli/__init__.py", line 445, in run
    renderers[args.renderer]().render(constructed.run())
  File "/opt/vol/volatility3/volatility3/cli/text_renderer.py", line 193, in render
    grid.populate(visitor, outfd)
  File "/opt/vol/volatility3/volatility3/framework/renderers/__init__.py", line 241, in populate
    for level, item in self._generator:
  File "/opt/vol/volatility3/volatility3/plugins/linux/bpf_proc.py", line 127, in _generator
    for task, progs, maps, links in self.list_bpf_procs(
  File "/opt/vol/volatility3/volatility3/plugins/linux/bpf_proc.py", line 96, in list_bpf_procs
    for pid, comm, _task, fd_fields in fds_generator:
  File "/opt/vol/volatility3/volatility3/framework/plugins/linux/lsof.py", line 69, in list_fds
    for fd_fields in fd_generator:
  File "/opt/vol/volatility3/volatility3/framework/symbols/linux/__init__.py", line 229, in files_descriptors_for_process
    full_path = LinuxUtilities.path_for_file(context, task, filp)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/vol/volatility3/volatility3/framework/symbols/linux/__init__.py", line 172, in path_for_file
    dentry = filp.get_dentry()
             ^^^^^^^^^^^^^^^
  File "/opt/vol/volatility3/volatility3/framework/objects/__init__.py", line 453, in __getattr__
    return getattr(self.dereference(), attr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/vol/volatility3/volatility3/framework/objects/__init__.py", line 968, in __getattr__
    raise AttributeError(
AttributeError: StructType has no attribute: symbol_table_name1!btf_type.get_dentry
@github-actions
Copy link

This issue is stale because it has been open for 200 days with no activity.

@github-actions github-actions bot added the stale label Sep 27, 2023
@github-actions GitHub Actions
Copy link

This issue was closed because it has been inactive for 60 days since being marked as stale.

@ikelos
Copy link
Member

ikelos commented Nov 26, 2023

I'd like to keep this open because I do still want to get to the bottom of it...

@ikelos ikelos reopened this Nov 26, 2023
@ikelos ikelos removed the stale label Nov 26, 2023
ikelos added a commit that referenced this issue Nov 26, 2023
@ikelos
Core: Fix array_of_pointers to act only on those pointers
e8d70b5 
Previously in array_of_pointers we were updating the vol dict for the
base pointer type.  Now we clone it before modifying it, so we're only
interfering with that one type, not any future types.

Fixes #922
@ikelos ikelos mentioned this issue Nov 26, 2023
Merged
@ikelos
Copy link
Member

ikelos commented Nov 26, 2023

Hi there, thanks very much for the detailed report and I can only apologize that it's taken us this long to dig into it. It was on our radar, but we didn't communicate that and it didn't have the priority it should have given it was a core issue.

You were correct, there was an issue with the way that array_of_pointers was constructing its arrays. It was taking the default pointer type class and updating its subtype (which then affected all subsequent copies of that type, ie, all pointers) whereas it should have cloned it and used only that. I've merged in the appropriate fix as of commit 9f9afbf. Thank you again for bringing it to our attention and the detailed information you provided.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Issues/issue922 volatilityfoundation/volatility3
2 participants
@ikelos @vobst