Reduce memory footprint of the Swift compiler #14
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Use malloc/free for allocating/freeing SIL instructions instead of using the BumpPtrAllocator. This allows for memory reuse and significantly reduces the memory footprint of the compiler. For example, a peak memory usage during a compilation of the standard library and StdlibUnitTest is reduced by 25%-30%. The performance of the compiler seems to be not affected by this change, i.e. no slowdown is measured. The use-after-free issues reported by build bots are fixed now. rdar://23303031
|
DON'T MERGE IT INTO TRUNK YET! We are going to test on build bots for a couple of days or for a week and if everything is fine, we'll merge it afterwards. |
|
Set your branch title into |
|
👍 |
|
But there is no branch named 'trunk' in this repo ¯_(ツ)_/¯. |
|
@swiftix 👍 You are hero. @yurikoles May be they use SVN internally. Am I right @swiftix |
swiftix
force-pushed
the
recycle-sil-instructions-memory
branch
from
bafb0e9
to
0768c76
Compare
|
@codestergit We used to use SVN internally. These days we switched to git, but we sometimes still use trunk and master interchangeably. |
swiftix
force-pushed
the
recycle-sil-instructions-memory
branch
from
0768c76
to
5c64804
Compare
|
I think it would be great if anyone would be willing to help us test this pull request. For example, you can build it on Linux and use valgrind to catch bugs, etc. |
|
Yes, I agree with @nadavrot. But please use the most up-to-date version from the swiftix:recycle-il-instructions-memory branch. GitHub does not update it properly here. |
swiftix
force-pushed
the
recycle-sil-instructions-memory
branch
from
5c64804
to
46b58ac
Compare
|
Since this PR does not get automatically updated with changes from my branch, I'll submit a new one. |
airspeedswift
added a commit
to airspeedswift/swift
that referenced
this pull request
Oct 26, 2016
airspeedswift
added a commit
that referenced
this pull request
Oct 27, 2016
[stdlib] Make various free functions mutating an _ArrayBufferProtocol into extensions. Resolves ABI FIXME #14
slavapestov
pushed a commit
to slavapestov/swift
that referenced
this pull request
Nov 27, 2018
…ake3 Propagate linker flags for kqueue and pthread_workqueue
slavapestov
pushed a commit
to slavapestov/swift
that referenced
this pull request
Nov 27, 2018
…ake3 Propagate linker flags for kqueue and pthread_workqueue Signed-off-by: Daniel A. Steffen <dsteffen@apple.com>
kateinoigakukun
referenced
this pull request
in kateinoigakukun/swift
Nov 15, 2019
New WASI SDK package should contain a new version of the linker. Also, the script should now create an installable package on macOS so that one could create a full SwiftWasm package later after the build. * Update WASI SDK, create installable package on macOS * Fix sysroot path, update wasi-sdk on Linux * Exclude module net for wasm SDK in glibc.modulemap * Remove module termios from glic.modulemap for wasm * Disable _stdlib_mkstemps for wasm
lorentey
pushed a commit
that referenced
this pull request
Dec 11, 2019
kateinoigakukun
referenced
this pull request
in kateinoigakukun/swift
Dec 14, 2019
New WASI SDK package should contain a new version of the linker. Also, the script should now create an installable package on macOS so that one could create a full SwiftWasm package later after the build. * Update WASI SDK, create installable package on macOS * Fix sysroot path, update wasi-sdk on Linux * Exclude module net for wasm SDK in glibc.modulemap * Remove module termios from glic.modulemap for wasm * Disable _stdlib_mkstemps for wasm
kateinoigakukun
referenced
this pull request
in kateinoigakukun/swift
Jan 11, 2020
New WASI SDK package should contain a new version of the linker. Also, the script should now create an installable package on macOS so that one could create a full SwiftWasm package later after the build. * Update WASI SDK, create installable package on macOS * Fix sysroot path, update wasi-sdk on Linux * Exclude module net for wasm SDK in glibc.modulemap * Remove module termios from glic.modulemap for wasm * Disable _stdlib_mkstemps for wasm
kateinoigakukun
referenced
this pull request
in kateinoigakukun/swift
Jan 24, 2020
New WASI SDK package should contain a new version of the linker. Also, the script should now create an installable package on macOS so that one could create a full SwiftWasm package later after the build. * Update WASI SDK, create installable package on macOS * Fix sysroot path, update wasi-sdk on Linux * Exclude module net for wasm SDK in glibc.modulemap * Remove module termios from glic.modulemap for wasm * Disable _stdlib_mkstemps for wasm
MaxDesiatov
referenced
this pull request
in MaxDesiatov/swift
May 1, 2020
New WASI SDK package should contain a new version of the linker. Also, the script should now create an installable package on macOS so that one could create a full SwiftWasm package later after the build. * Update WASI SDK, create installable package on macOS * Fix sysroot path, update wasi-sdk on Linux * Exclude module net for wasm SDK in glibc.modulemap * Remove module termios from glic.modulemap for wasm * Disable _stdlib_mkstemps for wasm
This was referenced Jul 13, 2016
This was referenced Oct 4, 2022
DougGregor
pushed a commit
to DougGregor/swift
that referenced
this pull request
Apr 28, 2024
Windows: add ICU packaging rules
nate-chandler
added a commit
to nate-chandler/swift
that referenced
this pull request
Jun 26, 2024
Apply the same control-flow changes to the method that were applied on main to avoid duplicated implementations one of which crashes with ``` Metadata for archetype not bound in function. The metadata could be missing entirely because it needs to be passed to the function. Or the metadata is present and not bound in which case setScopedLocalTypeMetadata or similar must be called. Archetype without metadata: Value (primary_archetype_type address=0x14207ebe0 conforms_to="Swift.(file).Escapable" name="Value" (interface_type=generic_type_param_type depth=0 index=0 decl="Synchronization.(file)._Cell.Value@/Volumes/ams1_ext4/Submit/SWE/DevTools/swift/6_0_0_6/swift/stdlib/public/Synchronization/Cell.swift:19:23")) Function: ; Function Attrs: noinline nounwind define linkonce_odr hidden ptr @"$s15Synchronization5_CellVyxGRi_zlWOb"(ptr %0, ptr %1, ptr %2) swiftlang#14 !dbg !8 { %4 = alloca i1, align 1 } LocalTypeData: (0x14207f5f8 (_Cell<Value>), FormalTypeMetadata) => [ (universal) concrete: 0x600003d1add0 ptr %2 ] ``` when building Synchronization.
nate-chandler
added a commit
to nate-chandler/swift
that referenced
this pull request
Jun 26, 2024
Apply the same control-flow changes to the method that were applied on main to avoid duplicated implementations one of which crashes with ``` Metadata for archetype not bound in function. The metadata could be missing entirely because it needs to be passed to the function. Or the metadata is present and not bound in which case setScopedLocalTypeMetadata or similar must be called. Archetype without metadata: Value (primary_archetype_type address=0x14207ebe0 conforms_to="Swift.(file).Escapable" name="Value" (interface_type=generic_type_param_type depth=0 index=0 decl="Synchronization.(file)._Cell.Value@/Volumes/ams1_ext4/Submit/SWE/DevTools/swift/6_0_0_6/swift/stdlib/public/Synchronization/Cell.swift:19:23")) Function: ; Function Attrs: noinline nounwind define linkonce_odr hidden ptr @"$s15Synchronization5_CellVyxGRi_zlWOb"(ptr %0, ptr %1, ptr %2) swiftlang#14 !dbg !8 { %4 = alloca i1, align 1 } LocalTypeData: (0x14207f5f8 (_Cell<Value>), FormalTypeMetadata) => [ (universal) concrete: 0x600003d1add0 ptr %2 ] ``` when building Synchronization.
nate-chandler
added a commit
to nate-chandler/swift
that referenced
this pull request
Jun 26, 2024
Apply the same control-flow changes to the method that were applied on main to avoid duplicated implementations one of which crashes with ``` Metadata for archetype not bound in function. The metadata could be missing entirely because it needs to be passed to the function. Or the metadata is present and not bound in which case setScopedLocalTypeMetadata or similar must be called. Archetype without metadata: Value (primary_archetype_type address=0x14207ebe0 conforms_to="Swift.(file).Escapable" name="Value" (interface_type=generic_type_param_type depth=0 index=0 decl="Synchronization.(file)._Cell.Value@/Volumes/ams1_ext4/Submit/SWE/DevTools/swift/6_0_0_6/swift/stdlib/public/Synchronization/Cell.swift:19:23")) Function: ; Function Attrs: noinline nounwind define linkonce_odr hidden ptr @"$s15Synchronization5_CellVyxGRi_zlWOb"(ptr %0, ptr %1, ptr %2) swiftlang#14 !dbg !8 { %4 = alloca i1, align 1 } LocalTypeData: (0x14207f5f8 (_Cell<Value>), FormalTypeMetadata) => [ (universal) concrete: 0x600003d1add0 ptr %2 ] ``` when building Synchronization.
nate-chandler
added a commit
to nate-chandler/swift
that referenced
this pull request
Jun 26, 2024
Apply the same control-flow changes to the method that were applied on main to avoid duplicated implementations one of which crashes with ``` Metadata for archetype not bound in function. The metadata could be missing entirely because it needs to be passed to the function. Or the metadata is present and not bound in which case setScopedLocalTypeMetadata or similar must be called. Archetype without metadata: Value (primary_archetype_type address=0x14207ebe0 conforms_to="Swift.(file).Escapable" name="Value" (interface_type=generic_type_param_type depth=0 index=0 decl="Synchronization.(file)._Cell.Value@/path/to/swift/6_0_0_6/swift/stdlib/public/Synchronization/Cell.swift:19:23")) Function: ; Function Attrs: noinline nounwind define linkonce_odr hidden ptr @"$s15Synchronization5_CellVyxGRi_zlWOb"(ptr %0, ptr %1, ptr %2) swiftlang#14 !dbg !8 { %4 = alloca i1, align 1 } LocalTypeData: (0x14207f5f8 (_Cell<Value>), FormalTypeMetadata) => [ (universal) concrete: 0x600003d1add0 ptr %2 ] ``` when building Synchronization.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a cleaned-up version of my previous patch to use malloc/free instead of BumpPtrAllocator for SILInstructions. It fixed a few remaining errors reported by build bots.
Since this feature can eventually destabilise the compiler if there are some not yet detected errors, let's test is more thoroughly on our build bots for a while before we merge it into trunk.