-
Notifications
You must be signed in to change notification settings - Fork 35
Home
Welcome to the iExtractor wiki!
iExtractor is a collection of tools and scripts to automate data extraction from iOS firmware files (i.e. IPSW - iPhone Software - files). It runs on macOS and partially on Linux (certain tools and features only work on macOS).
iExtractor may be used by people interested in iOS internals to automate the processing of data for multiple iOS versions or simply to learn more about IPSW files, kernelcaches and sandbox profiles. By using a single command you will go through all steps from downloading the public Apple-provided IPSW file, to unpacking the kernel, to reversing the sandbox profiles. You can customize the scripts or build your own to process multiple IPSW files or to get a particular piece of information.
A tool similar to iExtractor is ipsw_decrypt.py. ipsw_decrypt.py
is a Python script that decrypts all encrypted information inside an IPSW file. iExtractor also extracts kernel extensions, reverses sandbox profiles and extracts system dynamic libraries from the dyld shared cache.
Check the README file for information about installing and using iExtractor. Check the sections below for detailed information.
iExtractor requires the IPSW download URL and the decryption keys (for iOS <= 9) to do the downloading, unpacking, extraction and processing of data. For a given firmware, it follows the steps below:
- Download the IPSW file from a publicly available link provided by Apple.
- Unpack the contents of the IPSW file.
- Decrypt the disk image file for iOS <= 9.
- If running on Linux, convert the disk image file (
.dmg
) to a filesystem image file (.img
). - With the image file mounted, copy the
sandboxd
file storing the sandbox profiles for iOS <= 8. - With the image file mounted, extract the system dynamic library files from the dyld shared cache.
- With the image file mounted, pack the entire filesystem contents for "offline" use.
- Decrypt the raw kernelcache file for iOS <= 9 into a kernelcache dump file.
- Unpack the kernelcache dump file into the actual kernelcache.
- Extract the sandbox operations from the kernelcache.
- Extract the sandbox kernel extension from the kernelcache.
- Extract the binary sandbox profiles from the kernel sandbox extension (iOS >= 9) or from
sandboxd
(iOS <= 8). - Reverse the binary sandbox profiles into their original human readable form.
The firmware-metadata/
folder in the repository stores metadata for each firmware; each metadata file inside the folder is named after the firmware ID (e.g. iPhone5,1_9.3_13E237
). Inside each metadata file we store the download URL and the decryption keys; for example:
$ cat firmware-metadata/iPhone5,1_9.3_13E237
firmware_keys_url: https://www.theiphonewiki.com/wiki/Eagle_13E237_(iPhone5,1)
firmware_download_url: http://appldnld.apple.com/iOS9.3/031-55518-20160328-92D092CC-F29A-11E5-A1E1-DC0EF816D560/iPhone5,1_9.3_13E237_Restore.ipsw
rootfs_key: 2a66fd6377af8f60d5e300ac3aa8d9c44a1c0dee94579ad3f8a26515debbf381bb971ae8
kernelcache_iv: 17026ef62f8da01dc0f4e7d2b8b54d4c
kernelcache_key: 4ca5ec9e8dc9c5f77412c699fe724a5bfe101a79e1f2134e53bbb990e62a8c96
Keys are listed on The iPhone Wiki. The download URL is available on ipsw.me and, for most versions, on The iPhone Wiki.
The config
file defines the IPSW_STORE
and OUT_STORE
variables. This where the downloaded IPSW files and the extracted files are stored, respectively. The downloaded IPSW files are stored directly in the IPSW_STORE
folder. Extracted files are stored in a subfolder of the OUT_STORE
named after the firmware ID.
The listing below shows the hierarchy of the IPSW_STORE
folder (in/
) and the OUT_STORE
folder (out/
):
.
|-- in/
| |-- iPhone5,1_9.3_13E237_Restore.ipsw
| `-- iPhone_4.0_64bit_11.1.2_15B202_Restore.ipsw
`-- out/
|-- iPhone5,1_9.3_13E237/
| |-- 058-25481-332.dmg
| |-- 058-25512-331.dmg
| |-- 058-25517-331.dmg
| |-- BuildManifest.plist
| |-- Firmware/
| |-- Restore.plist
| |-- com.apple.security.sandbox.kext
| |-- decrypted.dmg
| |-- dyld_shared_cache/
| |-- fs.tar.gz
| |-- kernelcache.decrypted
| |-- kernelcache.mach.arm
| |-- kernelcache.release.n41
| |-- reversed_profiles/
| |-- sandbox_bundle
| `-- sb_ops
`-- iPhone_4.0_64bit_11.1.2_15B202/
|-- 058-84556-102.dmg
|-- 058-84705-099.dmg
|-- 058-85048-102.dmg
|-- BuildManifest.plist
|-- Firmware/
|-- Restore.plist
|-- com.apple.security.sandbox.kext
|-- com.apple.security.sandbox.kext.ARM64.48E3D31D-C7FB-300C-9BDC-1C578EE2AA5B
|-- decrypted.dmg
|-- dyld_shared_cache/
|-- fs.tar.gz
|-- kernelcache.decrypted
|-- kernelcache.mach.arm
|-- kernelcache.release.iphone6
|-- kernelcache.release.iphone8b
|-- reversed_profiles/
|-- sandbox_bundle
`-- sb_ops
Let's do a walkthrough of the files and folders in the IPSW_STORE
and OUT_STORE
listed above, in the order they are downloaded, unpacked, extracted and processed by iExtractor, mirroring the overview of steps above:
- The
iPhone5,1_9.3_13E237_Restore.ipsw
andiPhone_4.0_64bit_11.1.2_15B202_Restore.ipsw
files in thein/
subfolder are the firmware files. They are downloaded from a publicly available link provided by Apple. - The contents of the IPSW files are unpacked in the
out/iPhone5,1_9.3_13E237/
andout/iPhone_4.0_64bit_11.1.2_15B202/
folders. Contents include the058....dmg
files, theBuildManifest.plist
andRestore.plist
files, theFirmware/
folder the thekernelcache.release...
files. - The largest
.dmg
file is the root disk image file. For iOS <= 9 it is encrypted. The decrypted disk image file (or, if not encrypted, a link to the original file) isdecrypted.dmg
. - If iExtractor were to be run on Linux, there would have been a
rootfs.img
file converted fromdecrypted.dmg
. macOS is able to directly attach and mount.dmg
files, while Linux requires.img
files. - For iOS <= 8, a
sandboxd
file would have been present. The output folders are for iOS 9.3 and iOS 11.1.2 and thesandboxd
file is absent. - The
dyld_shared_cache/
folder stores the system library files extracted from the dyld shared cache. - The
fs.tar.gz
file is an archive of the entire filesystem contents. - The
kernelcache.decrypted
file stores the decrypted kernelcache dump of one of thekernelcache.release....
files. For iOS >= 10 the kernelcache release file is not encrypted so thekernelcache.decrypted
file is a link to it. - The
kernelcache.mach.arm
file is the actual kernelcache file unpacked from thekernelcache.decrypted
file. - The
sb_ops
file stores the sandbox operations, a list of strings required for reversing the sandbox profiles. - The
com.apple.security.sandbox.kext
file stores the kernel sandbox extension. - The
sandbox_bundle
file (orsandbox_profiles/
folder for iOS <= 8) stores the binary sandbox profiles. - The
reversed_profiles/
folder stores the reversed sandbox profiles in human readable form.
iExtractor uses existing tools for all the steps of processing IPSW files. These are non-interactive command line tools that enable iExtractor to use a single script to process one or multiple IPSW files. Some of them are part of the installed environment while others are part of the repository (directly or as submodules) in the tools/
subfolder. Mirroring the steps above, the tools employed are:
-
wget
downloads the IPSW file from the publicly available link provided by Apple. -
unzip
unpacks the IPSW file (a ZIP file) into the output folder. -
vfdecrypt decrypts the largest
.dmg
file intodecrypted.dmg
for iOS <= 9 using the community provided keys from The iPhone Wiki. -
dmg2img
convertsdecrypted.dmg
intorootfs.img
. -
hdutil
on macOS andmount
andumount
on Linux mount and unmount thedecrypted.dmg
androotfs.img
files respectively.cp
copiessandboxd
from the mount point to the output folder. - (only on macOS) After mounting the image file, dsc_extractor extracts the system dynamic library files from the
System/Library/Caches/com.apple.dyld/
path from the mount point todyld_shared_cache/
. - After mounting the image file,
tar
packs the filesystem contents infs.tar.gz
. -
xpwntool decrypts
kernelcache.release....
tokernelcache.decrypted
for iOS <= 9. -
lzssdec unpacks
kernelcache.decrypted
tokernelcache.mach.arm
. - (only on macOS) extract_sbops extracts the sandbox operations in the
sb_ops
file fromkernelcache.mach.arm
. -
joker extracts
com.apple.security.sandbox.kext
fromkernelcache.mach.arm
. -
extract_sbprofiles extracts binary sandbox profiles from
sandboxd
insandbox_profiles/
for iOS <= 8 (only on macOS);dd
inside a custom script extractssandbox_bundle
fromcom.apple.security.sandbox.kext
for iOS >= 9. -
SandBlaster reverses binary sandbox profiles from
sandbox_profiles/
orsandbox_bundle
toreversed_profiles/
.
Let's get a glimpse of how the above tools are actually used. Below is a set of commands that have been run manually to go through all steps. iExtractor automates the commands below and hides the particularities of iOS versions. Commands have been run on a macOS 10.13.2 system; paths to commands and files are particular to the environment. The input file is an iOS 8 IPSW (firmare ID iPad2,1_8.4.1_12H321
), with the the download URL and keys listed in the corresponding firmware metadata file. Commands mirror the steps above.
# Download IPSW file.
cd in/
wget http://appldnld.apple.com/ios8.4.1/031-31231-20150812-751BEF2C-3C8F-11E5-94D1-BB1A3A53DB92/iPad2,1_8.4.1_12H321_Restore.ipsw
# Unpack IPSW file.
cd ../out/
mkdir iPad2,1_8.4.1_12H321
cd iPad2,1_8.4.1_12H321
unzip -qq ../../in/iPad2,1_8.4.1_12H321_Restore.ipsw
# Decrypt .dmg file.
../../../iExtractor.git/tools/vfdecrypt/vfdecrypt -i 058-24482-024.dmg -o decrypted.dmg -k d33bfc10cc397ae008d2e56685acb172b74f0eccbe652ecd52e234ab12705980b6483b60
# Do not convert .dmg to .img (not required on macOS).
# Attach and mount disk image.
sudo mkdir -p /mnt/ios/iPad2,1_8.4.1_12H321
sudo hdiutil attach -noverify -mountpoint /mnt/ios/iPad2,1_8.4.1_12H321 decrypted.dmg
# Copy sandboxd.
cp /mnt/ios/iPad2,1_8.4.1_12H321/usr/libexec/sandboxd .
# Extract dyld shared cache.
mkdir dyld_shared_cache
../../../iExtractor.git/tools/dyld/dsc_extractor /mnt/ios/iPad2,1_8.4.1_12H321/System/Library/Caches/com.apple.dyld/dyld_shared_cache_armv7 dyld_shared_cache/
# Pack filesystem.
sudo tar -C /mnt/ios/iPad2,1_8.4.1_12H321 -czf fs.tar.gz .
# Unmount and dettach disk image.
sudo hdiutil detach /dev/disk2
# Decrypt kernelcache.
../../../iExtractor.git/tools/xpwn/builddir/ipsw-patch/xpwntool kernelcache.release.k93 kernelcache.decrypted -k 6dea51edc4c6f6c205e25ca1d9af5b9ee7167ed5685a937ad892e7fcbfb3e615 -iv 38d12734c845d1dcdb84846caf619da7 -decrypt
# Unpack kernelcache.
../../../iExtractor.git/bin/get_lzss_section_offset.py kernelcache.decrypted # command output is 448 (offset)
../../../iExtractor.git/tools/lzssdec/lzssdec -o 448 < kernelcache.decrypted > kernelcache.mach.arm
# Extract sandbox operations.
../../../iExtractor.git/tools/sandblaster/tools/sandbox_toolkit/extract_sbops/extract_sbops kernelcache.mach.arm > sb_ops
# Extract kernel sandbox extension.
JOKER_DIR=. ../../../iExtractor.git/tools/joker/joker.universal -K com.apple.security.sandbox kernelcache.mach.arm
# Extract sandbox profiles.
mkdir sandbox_profiles
cd sandbox_profiles/
../../../../iExtractor.git/tools/sandblaster/tools/sandbox_toolkit/extract_sbprofiles/extract_sbprofiles ../sandboxd
cd ..
# Reverse sandbox profiles.
mkdir reversed_profiles
cd ../../../iExtractor.git/tools/sandblaster/reverse-sandbox/
for i in ../../../../store/out/iPad2,1_8.4.1_12H321/sandbox_profiles/*; do python reverse_sandbox.py -r 8.4.1 -o ../../../../store/out/iPad2,1_8.4.1_12H321/sb_ops -d ../../../../store/out/iPad2,1_8.4.1_12H321/reversed_profiles/ "$i"; done
To make it easier to automate and to hide specifics of iOS versions, iExtractor uses two layers of scripts on top of which wrapper scripts are built.
The lower layer scripts are located in the bin/
subfolder. They take care of iOS version specifics and call the required tools in the tools/
subfolder. For example, bin/decrypt_fs
decrypts the disk image file if a key is available (i.e. if it is encrypted), otherwise it just creates a link; if it runs on Linux it also converts the decrypted disk image file (decrypted.dmg
) to a filesystem image file (rootfs.img
). There is a bin/common
configuration script that's being sourced by all higher layer scripts.
The higher layer scripts are located in the scripts/
subfolder. All these scripts are passed a single argument: the firmware ID. They will parse firmware ID information from the corresponding file in the firmware-metadata/
subfolder, construct parameters and call lower layer scripts in the bin/
subfolder or existing tools. There is a scripts/common
configuration script that's being sourced by all higher layer scripts.
Wrapper scripts are custom scripts created by the user to call higher layer scripts according to need. Sample scripts are part of the repository: scripts named with run_...
. A wrapper script is passed the firmware ID as a parameter and calls corresponding higher layer scripts. For example the run_all
wrapper script calls all higher layer scripts, while the run_sandblaster
wrapper script only calls SandBlaster-specific higher layer scripts.
In short, wrapper scripts in the root of the repository call higher layer scripts in the scripts/
subfolder which in turn call lower layer scripts in the bin/
subfolder.
You will usually run a wrapper script, but you can run a higher layer or lower layer script as well. All will work with files in the folders defined by the IPSW_STORE
and OUT_STORE
variables in the config
file.
Running a wrapper script is as simple as passing it the firmware ID as a parameter:
./run_all iPhone5,1_9.3_13E237
./run_sandblaster iPhone5,1_9.3_13E237
Output files will be stored in the OUT_STORE/iPhone5,1_9.3_13E237/
folder.
Either for testing, debugging or to do a specific action, you can directly call a higher layer script or a lower layer script. Running a higher layer script requires scripts/
to be the working directory; similarly, running a lower layer script requires bin/
to be the working directory. For example, if you wanted to decrypt the disk image for a given firmware you could run one of the two commands below (a higher layer and a lower layer script):
cd scripts/
./decrypt_fs iPhone5,1_9.3_13E237
cd bin/
./decrypt_fs ../../store/out/iPhone5,1_9.3_13E237/058-25512-331.dmg 2a66fd6377af8f60d5e300ac3aa8d9c44a1c0dee94579ad3f8a26515debbf381bb971ae8
Similarly to running a wrapper script, the decrypted.dmg
output file will be stored in the OUT_STORE/iPhone5,1_9.3_13E237/
folder.
Higher layer scripts do sanity checks for input and output files. If output files already exists, the higher layer script will prompt if you want to overwrite them. This is why wrapper scripts use a construct such as the one below, to prevent the likely useless action of overwriting existing output files with the same content. Wrapper scripts can, of course, be updated to force the overwrite.
echo "* Extract dynamic library cache"
yes N | ./extract_dyld_shared_cache "$firmware_id"
In case of issues you can turn on debugging by altering the DEBUG
variable in the config
file and setting it to 1
. That causes lower and higher layer scripts to print out detailed information about what they are doing including printing out commands they run. Debug messages use the [debug]
prefix. For example, running the decrypt_fs
lower layer script in the bin/
subfolder with debugging enabled renders the output below:
$ ./decrypt_fs ../../store/out/iPhone5,1_9.3_13E237/058-25512-331.dmg 2a66fd6377af8f60d5e300ac3aa8d9c44a1c0dee94579ad3f8a26515debbf381bb971ae8
[debug] Decrypting dmg file ../../store/out/iPhone5,1_9.3_13E237/058-25512-331.dmg to ../../store/out/iPhone5,1_9.3_13E237/decrypted.dmg ...
[debug] ../tools/vfdecrypt/vfdecrypt -k2a66fd6377af8f60d5e300ac3aa8d9c44a1c0dee94579ad3f8a26515debbf381bb971ae8 -i../../store/out/iPhone5,1_9.3_13E237/058-25512-331.dmg -o../../store/out/iPhone5,1_9.3_13E237/decrypted.dmg
If you only want certain actions to automate you can create a custom wrapper script using existing run_...
scripts as starting point. Then you can use the for_each_firmware
script to run the custom wrapper script on each firmware.
To add support for a new firmware, create a new file in the firmware-metadata/
using existing files as starting point. Extract the download firmare ID and download URL from ipsw.me and decryption keys for iOS <= 9 from The iPhone Wiki.
If you want a new action as part of iExtractor (e.g. extracting only Mach-O files from the filesystem, extracting all kernel sandbox extensions, extracting the launchd
executable etc.) you need to create a higher layer script in the scripts/
subfolder and, if required, a lower layer script in the bin/
subfolder using existing scripts as starting points.
iExtractor runs on firmware files from iOS 7 onward. The sandbox profile reverser doesn't work for iOS <= 6. All other components should likely work for earlier iOS versions as well.
iExtractor run on macOS. It partially runs on Linux except for:
- extracting the dyld shared cache: dsc_extractor only works on macOS
- extracting the sandbox operations: extract_sbops only works on macOS
- extracting sandbox profiles for iOS <= 8: extract_sbprofiles only works on macOS
- mounting root filesystem image files for iOS >= 10.3: since iOS 10.3 the filesystem is Apple File System (APFS) for which there is currently (December 2017) no support on Linux