Merge branch 'volatilityfoundation:develop' into vmayarascan

.github/workflows/install.yml

@@ -0,0 +1,31 @@
+name: Install Volatility3 test
+on: [push, pull_request]
+jobs:
+
+  install_test:
+    runs-on: ${{ matrix.host }}
+    strategy:
+      fail-fast: false
+      matrix:
+        host: [ ubuntu-latest, windows-latest ]
+        python-version: [ "3.7", "3.8", "3.9", "3.10", "3.11" ]
+    steps:
+    - uses: actions/checkout@v3
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v4
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Setup python-pip
+      run: python -m pip install --upgrade pip
+
+    - name: Install dependencies
+      run: |
+        pip install -r requirements.txt
+
+    - name: Install volatility3
+      run: pip install .
+
+    - name: Run volatility3
+      run: vol --help

.github/workflows/stale.yml

@@ -0,0 +1,23 @@
+name: Close inactive issues
+on:
+  schedule:
+    - cron: "30 1 * * *"
+
+jobs:
+  close-issues:
+    runs-on: ubuntu-latest
+    permissions:
+      issues: write
+      pull-requests: write
+    steps:
+      - uses: actions/stale@v5
+        with:
+          days-before-issue-stale: 200
+          days-before-issue-close: 60
+          stale-issue-label: "stale"
+          stale-issue-message: "This issue is stale because it has been open for 200 days with no activity."
+          close-issue-message: "This issue was closed because it has been inactive for 60 days since being marked as stale."
+          days-before-pr-stale: -1
+          days-before-pr-close: -1
+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          exempt-issue-labels: "enhancement,plugin-request,question"

API_CHANGES.md

@@ -4,6 +4,10 @@ API Changes
 When an addition to the existing API is made, the minor version is bumped.
 When an API feature or function is removed or changed, the major version is bumped.
 
+2.5.0
+=====
+Add in support for specifying a type override for object_from_symbol
+
 2.4.0
 =====
 Add a `get_size()` method to Windows VAD structures and fix several off-by-one issues when calculating VAD sizes.

doc/source/basics.rst

@@ -1,7 +1,7 @@
 Volatility 3 Basics
 ===================
 
-Volatility splits memory analysis down to several components:
+Volatility splits memory analysis down to several components.  The main ones are:
 
 * Memory layers
 * Templates and Objects
@@ -13,22 +13,65 @@ which acts as a container for all the various layers and tables necessary to con
 Memory layers
 -------------
 
-A memory layer is a body of data that can be accessed by requesting data at a specific address.  Memory is seen as
-sequential when accessed through sequential addresses, however, there is no obligation for the data to be stored
-sequentially, and modern processors tend to store the memory in a paged format.  Moreover, there is no need for the data
-to be stored in an easily accessible format, it could be encoded or encrypted or more, it could be the combination of
-two other sources.  These are typically handled by programs that process file formats, or the memory manager of the
-processor, but these are all translations (either in the geometric or linguistic sense) of the original data.
-
-In Volatility 3 this is represented by a directed graph, whose end nodes are
-:py:class:`DataLayers <volatility3.framework.interfaces.layers.DataLayerInterface>` and whose internal nodes are
-specifically called a :py:class:`TranslationLayer <volatility3.framework.interfaces.layers.TranslationLayerInterface>`.
-In this way, a raw memory image in the LiME file format and a page file can be
-combined to form a single Intel virtual memory layer.  When requesting addresses from the Intel layer, it will use the
-Intel memory mapping algorithm, along with the address of the directory table base or page table map, to translate that
+A memory layer is a body of data that can be accessed by requesting data at a specific address.  At its lowest level
+this data is stored on a phyiscal medium (RAM) and very early computers addresses locations in memory directly.  However,
+as the size of memory increased and it became more difficult to manage memory most architectures moved to a "paged" model 
+of memory, where the available memory is cut into specific fixed-sized pages.  To help further, programs can ask for any address 
+and the processor will look up their (virtual) address in a map, to find out where the (physical) address that it lives at is,
+in the actual memory of the system.
+
+Volatility can work with these layers as long as it knows the map (so, for example that virtual address `1` looks up at physical
+address `9`).  The automagic that runs at the start of every volatility session often locates the kernel's memory map, and creates
+a kernel virtual layer, which allows for kernel addresses to be looked up and the correct data returned.  There can, however, be
+several maps, and in general there is a different map for each process (although a portion of the operating system's memory is
+usually mapped to the same location across all processes).  The maps may take the same address but point to a different part of 
+physical memory.  It also means that two processes could theoretically share memory, but having an virtual address mapped to the 
+same physical address as another process.  See the worked example below for more information.
+
+To translate an address on a layer, call :py:meth:`layer.mapping(offset, length, ignore_errors) <volatility3.framework.interfaces.layers.TranslationLayerInterface.mapping>` and it will return a list of chunks without overlap, in order,
+for the requested range.  If a portion cannot be mapped, an exception will be thrown unless `ignore_errors` is true.  Each 
+chunk will contain the original offset of the chunk, the translated offset, the original size and the translated size of 
+the chunk, as well as the lower layer the chunk lives within.
+
+Worked example
+^^^^^^^^^^^^^^
+
+The operating system and two programs may all appear to have access to  all of physical memory, but actually the maps they each have
+mean they each see something different:
+
+.. code-block::
+    :caption: Memory mapping example
+
+    Operating system map                        Physical Memory
+    1 -> 9                                      1  - Free
+    2 -> 3                                      2  - OS.4, Process 1.4, Process 2.4
+    3 -> 7                                      3  - OS.2
+    4 -> 2                                      4  - Free
+                                                5  - Free
+    Process 1 map                               6  - Process 1.2, Process 2.3
+    1 -> 12                                     7  - OS.3
+    2 -> 6                                      8  - Process1.3
+    3 -> 8                                      9  - OS.1
+    4 -> 2                                      10 - Process2.1
+                                                11 - Free
+    Process 2 map                               12 - Process1.1
+    1 -> 10                                     13 - Free
+    2 -> 15                                     14 - Free
+    3 -> 6                                      15 - Process2.2
+    4 -> 2                                      16 - Free
+
+In this example, part of the operating system is visible across all processes (although not all processes can write to the memory, there
+is a permissions model for intel addressing which is not discussed further here).)
+
+In Volatility 3 mappings are represented by a directed graph of layers, whose end nodes are
+:py:class:`DataLayers <volatility3.framework.interfaces.layers.DataLayerInterface>` and whose internal nodes are :py:class:`TranslationLayers <volatility3.framework.interfaces.layers.TranslationLayerInterface>`.
+In this way, a raw memory image in the LiME file format and a page file can be combined to form a single Intel virtual 
+memory layer.  When requesting addresses from the Intel layer, it will use the Intel memory mapping algorithm, along 
+with the address of the directory table base or page table map, to translate that
 address into a physical address, which will then either be directed towards the swap layer or the LiME layer.  Should it
-be directed towards the LiME layer, the LiME file format algorithm will be translated to determine where within the file
-the data is stored and that will be returned.
+be directed towards the LiME layer, the LiME file format algorithm will be translate the new address to determine where 
+within the file the data is stored.  When the :py:meth:`layer.read() <volatility3.framework.interfaces.layers.TranslationLayerInterface.read>` 
+method is called, the translation is done automatically and the correct data gathered and combined.
 
 .. note:: Volatility 2 had a similar concept, called address spaces, but these could only stack linearly one on top of another.