-
Notifications
You must be signed in to change notification settings - Fork 12.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AssumeBundles] Dereferenceable used in bundle only applies at assume. #126117
Conversation
b2f29f6
to
efda984
Compare
efda984
to
0608821
Compare
@llvm/pr-subscribers-llvm-transforms @llvm/pr-subscribers-llvm-analysis Author: Florian Hahn (fhahn) ChangesUpdate LangRef and code using
Update code using This follows up on #123196. With that change, it should be safe to expose dereferenceable assumptions more widely as in Full diff: https://github.com/llvm/llvm-project/pull/126117.diff 8 Files Affected:
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 8891aedcb58e552..f4d6affab8f9c3e 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -1474,7 +1474,11 @@ Currently, only the following parameter attributes are defined:
``null_pointer_is_valid`` function attribute is present.
``n`` should be a positive number. The pointer should be well defined,
otherwise it is undefined behavior. This means ``dereferenceable(<n>)``
- implies ``noundef``.
+ implies ``noundef``. When ``dereferenceable(<n>)`` is used in an
+ :ref:`assume operand bundls <assume_opbundles>`, the pointer is only
+ guaranteed to be dereferenceable at the point of the assumption and
+ may not be dereferenceable at later pointers, e.g. because it could have
+ been freed.
``dereferenceable_or_null(<n>)``
This indicates that the parameter or return value isn't both
diff --git a/llvm/lib/Analysis/Loads.cpp b/llvm/lib/Analysis/Loads.cpp
index 733a7988e5a730a..91ae944d2e68366 100644
--- a/llvm/lib/Analysis/Loads.cpp
+++ b/llvm/lib/Analysis/Loads.cpp
@@ -26,8 +26,6 @@
using namespace llvm;
-extern cl::opt<bool> UseDerefAtPointSemantics;
-
static bool isAligned(const Value *Base, Align Alignment,
const DataLayout &DL) {
return Base->getPointerAlignment(DL) >= Alignment;
@@ -171,7 +169,7 @@ static bool isDereferenceableAndAlignedPointer(
Size, DL, CtxI, AC, DT, TLI,
Visited, MaxDepth);
- if (CtxI && (!UseDerefAtPointSemantics || !V->canBeFreed())) {
+ if (CtxI && !V->canBeFreed()) {
/// Look through assumes to see if both dereferencability and alignment can
/// be proven by an assume if needed.
RetainedKnowledge AlignRK;
diff --git a/llvm/lib/Analysis/ValueTracking.cpp b/llvm/lib/Analysis/ValueTracking.cpp
index 6eba6c0f08c3f40..eac11ca379932ba 100644
--- a/llvm/lib/Analysis/ValueTracking.cpp
+++ b/llvm/lib/Analysis/ValueTracking.cpp
@@ -626,7 +626,7 @@ static bool isKnownNonZeroFromAssume(const Value *V, const SimplifyQuery &Q) {
*I, I->bundle_op_info_begin()[Elem.Index])) {
if (RK.WasOn == V &&
(RK.AttrKind == Attribute::NonNull ||
- (RK.AttrKind == Attribute::Dereferenceable &&
+ (RK.AttrKind == Attribute::Dereferenceable && !V->canBeFreed() &&
!NullPointerIsDefined(Q.CxtI->getFunction(),
V->getType()->getPointerAddressSpace()))) &&
isValidAssumeForContext(I, Q.CxtI, Q.DT))
diff --git a/llvm/test/Analysis/ValueTracking/assume-queries-counter.ll b/llvm/test/Analysis/ValueTracking/assume-queries-counter.ll
index a53b90b7b448ae8..0223ab8c4b677ab 100644
--- a/llvm/test/Analysis/ValueTracking/assume-queries-counter.ll
+++ b/llvm/test/Analysis/ValueTracking/assume-queries-counter.ll
@@ -47,7 +47,7 @@ define dso_local i1 @test2(ptr readonly %0) {
ret i1 %2
}
-define dso_local i32 @test4(ptr readonly %0, i1 %cond) {
+define dso_local i32 @test4(ptr readonly %0, i1 %cond) nofree nosync {
; COUNTER1-LABEL: @test4(
; COUNTER1-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[TMP0:%.*]], i32 4) ]
; COUNTER1-NEXT: br i1 [[COND:%.*]], label [[A:%.*]], label [[B:%.*]]
diff --git a/llvm/test/Analysis/ValueTracking/assume.ll b/llvm/test/Analysis/ValueTracking/assume.ll
index 4c4b46c41996868..298facfa3aa9d05 100644
--- a/llvm/test/Analysis/ValueTracking/assume.ll
+++ b/llvm/test/Analysis/ValueTracking/assume.ll
@@ -57,7 +57,7 @@ define dso_local i1 @test2(ptr readonly %0) {
ret i1 %2
}
-define dso_local i32 @test4(ptr readonly %0, i1 %cond) {
+define dso_local i32 @test4(ptr readonly %0, i1 %cond) nofree nosync {
; CHECK-LABEL: @test4(
; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[TMP0:%.*]], i32 4) ]
; CHECK-NEXT: br i1 [[COND:%.*]], label [[A:%.*]], label [[B:%.*]]
@@ -91,7 +91,7 @@ A:
ret i32 %6
}
-define dso_local i32 @test4a(ptr readonly %0, i1 %cond) {
+define dso_local i32 @test4a(ptr readonly %0, i1 %cond) nofree nosync {
; CHECK-LABEL: @test4a(
; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[TMP0:%.*]], i32 4), "align"(ptr [[TMP0]], i32 8) ]
; CHECK-NEXT: br i1 [[COND:%.*]], label [[A:%.*]], label [[B:%.*]]
diff --git a/llvm/test/Transforms/LICM/hoist-speculatable-load.ll b/llvm/test/Transforms/LICM/hoist-speculatable-load.ll
index 752cf3e99cbf5a6..cf43ff9913813e3 100644
--- a/llvm/test/Transforms/LICM/hoist-speculatable-load.ll
+++ b/llvm/test/Transforms/LICM/hoist-speculatable-load.ll
@@ -11,12 +11,12 @@ define void @f(i32 %ptr_i, ptr %ptr2, i1 %cond) {
; CHECK-NEXT: store i32 0, ptr [[PTR2:%.*]], align 4
; CHECK-NEXT: br label [[FOR_BODY_LR_PH]]
; CHECK: for.body.lr.ph:
-; CHECK-NEXT: [[TMP0:%.*]] = load i32, ptr [[PTR]], align 4
; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body:
; CHECK-NEXT: [[I_08:%.*]] = phi i32 [ 0, [[FOR_BODY_LR_PH]] ], [ [[INC:%.*]], [[IF_END:%.*]] ]
; CHECK-NEXT: br i1 [[COND]], label [[IF_END]], label [[IF:%.*]]
; CHECK: if:
+; CHECK-NEXT: [[TMP0:%.*]] = load i32, ptr [[PTR]], align 4, !invariant.load [[META0:![0-9]+]]
; CHECK-NEXT: store i32 [[TMP0]], ptr [[PTR2]], align 4
; CHECK-NEXT: br label [[IF_END]]
; CHECK: if.end:
@@ -56,4 +56,58 @@ exit: ; preds = %if.end, %entry
ret void
}
+define void @f_nofree_nosync(i32 %ptr_i, ptr %ptr2, i1 %cond) nofree nosync {
+; CHECK-LABEL: @f_nofree_nosync(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[PTR:%.*]] = inttoptr i32 [[PTR_I:%.*]] to ptr
+; CHECK-NEXT: call void @llvm.assume(i1 true) [ "align"(ptr [[PTR]], i32 16), "dereferenceable"(ptr [[PTR]], i32 16) ]
+; CHECK-NEXT: br i1 [[COND:%.*]], label [[FOR_BODY_LR_PH:%.*]], label [[IF0:%.*]]
+; CHECK: if0:
+; CHECK-NEXT: store i32 0, ptr [[PTR2:%.*]], align 4
+; CHECK-NEXT: br label [[FOR_BODY_LR_PH]]
+; CHECK: for.body.lr.ph:
+; CHECK-NEXT: br label [[FOR_BODY:%.*]]
+; CHECK: for.body:
+; CHECK-NEXT: [[I_08:%.*]] = phi i32 [ 0, [[FOR_BODY_LR_PH]] ], [ [[INC:%.*]], [[IF_END:%.*]] ]
+; CHECK-NEXT: br i1 [[COND]], label [[IF_END]], label [[IF:%.*]]
+; CHECK: if:
+; CHECK-NEXT: [[TMP0:%.*]] = load i32, ptr [[PTR]], align 4, !invariant.load [[META0]]
+; CHECK-NEXT: store i32 [[TMP0]], ptr [[PTR2]], align 4
+; CHECK-NEXT: br label [[IF_END]]
+; CHECK: if.end:
+; CHECK-NEXT: [[INC]] = add nuw nsw i32 [[I_08]], 1
+; CHECK-NEXT: [[CMP:%.*]] = icmp slt i32 [[INC]], 2
+; CHECK-NEXT: br i1 [[CMP]], label [[FOR_BODY]], label [[EXIT:%.*]]
+; CHECK: exit:
+; CHECK-NEXT: ret void
+;
+entry:
+ %ptr = inttoptr i32 %ptr_i to ptr
+ call void @llvm.assume(i1 true) [ "align"(ptr %ptr, i32 16), "dereferenceable"(ptr %ptr, i32 16) ]
+ br i1 %cond, label %for.body.lr.ph, label %if0
+
+if0:
+ store i32 0, ptr %ptr2, align 4
+ br label %for.body.lr.ph
+
+for.body.lr.ph: ; preds = %entry
+ br label %for.body
+
+for.body: ; preds = %for.body.lr.ph, %if.end
+ %i.08 = phi i32 [ 0, %for.body.lr.ph ], [ %inc, %if.end ]
+ br i1 %cond, label %if.end, label %if
+
+if:
+ %0 = load i32, ptr %ptr, align 4, !invariant.load !{}
+ store i32 %0, ptr %ptr2, align 4
+ br label %if.end
+
+if.end: ; preds = %for.body
+ %inc = add nuw nsw i32 %i.08, 1
+ %cmp = icmp slt i32 %inc, 2
+ br i1 %cmp, label %for.body, label %exit
+
+exit: ; preds = %if.end, %entry
+ ret void
+}
declare void @llvm.assume(i1 noundef)
diff --git a/llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-constant-size.ll b/llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-constant-size.ll
index 90671689f1dce05..13780a684f9ce79 100644
--- a/llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-constant-size.ll
+++ b/llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-constant-size.ll
@@ -1,5 +1,5 @@
; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 5
-; RUN: opt -p loop-vectorize -force-vector-width=2 -use-dereferenceable-at-point-semantics -S %s | FileCheck %s
+; RUN: opt -p loop-vectorize -force-vector-width=2 -S %s | FileCheck %s
declare void @llvm.assume(i1)
@@ -1411,7 +1411,6 @@ exit:
}
; %a may be freed between the dereferenceable assumption and accesses.
-; It is not safe to use with -use-dereferenceable-at-point-semantics.
define void @may_free_align_deref_assumption_in_header_constant_trip_count_loop_invariant_ptr(ptr noalias %a, ptr noalias %b, ptr noalias %c) {
; CHECK-LABEL: define void @may_free_align_deref_assumption_in_header_constant_trip_count_loop_invariant_ptr(
; CHECK-SAME: ptr noalias [[A:%.*]], ptr noalias [[B:%.*]], ptr noalias [[C:%.*]]) {
@@ -1505,7 +1504,6 @@ exit:
}
; %a may be freed between the dereferenceable assumption and accesses.
-; It is not safe to use with -use-dereferenceable-at-point-semantics.
define void @may_free_local_ptr_align_deref_assumption_in_header_constant_trip_count_loop_invariant_ptr(ptr noalias %b, ptr noalias %c) nofree nosync {
; CHECK-LABEL: define void @may_free_local_ptr_align_deref_assumption_in_header_constant_trip_count_loop_invariant_ptr(
; CHECK-SAME: ptr noalias [[B:%.*]], ptr noalias [[C:%.*]]) #[[ATTR1]] {
diff --git a/llvm/test/Transforms/SimplifyCFG/speculate-derefable-load.ll b/llvm/test/Transforms/SimplifyCFG/speculate-derefable-load.ll
index 0138433312ed84a..b8c999d700aa7da 100644
--- a/llvm/test/Transforms/SimplifyCFG/speculate-derefable-load.ll
+++ b/llvm/test/Transforms/SimplifyCFG/speculate-derefable-load.ll
@@ -4,10 +4,14 @@
define i64 @align_deref_align(i1 %c, ptr %p) {
; CHECK-LABEL: define i64 @align_deref_align(
; CHECK-SAME: i1 [[C:%.*]], ptr [[P:%.*]]) {
-; CHECK-NEXT: [[ENTRY:.*:]]
+; CHECK-NEXT: [[ENTRY:.*]]:
; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[P]], i64 8), "align"(ptr [[P]], i64 8) ]
+; CHECK-NEXT: br i1 [[C]], label %[[IF:.*]], label %[[EXIT:.*]]
+; CHECK: [[IF]]:
; CHECK-NEXT: [[V:%.*]] = load i64, ptr [[P]], align 8
-; CHECK-NEXT: [[RES:%.*]] = select i1 [[C]], i64 [[V]], i64 0
+; CHECK-NEXT: br label %[[EXIT]]
+; CHECK: [[EXIT]]:
+; CHECK-NEXT: [[RES:%.*]] = phi i64 [ [[V]], %[[IF]] ], [ 0, %[[ENTRY]] ]
; CHECK-NEXT: ret i64 [[RES]]
;
entry:
@@ -23,9 +27,64 @@ exit:
ret i64 %res
}
+define i64 @align_deref_align_nofree_nosync(i1 %c, ptr %p) nofree nosync {
+; CHECK-LABEL: define i64 @align_deref_align_nofree_nosync(
+; CHECK-SAME: i1 [[C:%.*]], ptr [[P:%.*]]) #[[ATTR0:[0-9]+]] {
+; CHECK-NEXT: [[ENTRY:.*:]]
+; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[P]], i64 8), "align"(ptr [[P]], i64 8) ]
+; CHECK-NEXT: [[V:%.*]] = load i64, ptr [[P]], align 8
+; CHECK-NEXT: [[SPEC_SELECT:%.*]] = select i1 [[C]], i64 [[V]], i64 0
+; CHECK-NEXT: ret i64 [[SPEC_SELECT]]
+;
+entry:
+ call void @llvm.assume(i1 true) [ "dereferenceable"(ptr %p, i64 8), "align"(ptr %p, i64 8) ]
+ br i1 %c, label %if, label %exit
+
+if:
+ %v = load i64, ptr %p, align 8
+ br label %exit
+
+exit:
+ %res = phi i64 [ %v, %if ], [ 0, %entry ]
+ ret i64 %res
+}
+
define i64 @assume_deref_align2(i1 %c1, i32 %x, ptr %p) {
; CHECK-LABEL: define i64 @assume_deref_align2(
; CHECK-SAME: i1 [[C1:%.*]], i32 [[X:%.*]], ptr [[P:%.*]]) {
+; CHECK-NEXT: [[ENTRY:.*]]:
+; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[P]], i64 8), "align"(ptr [[P]], i64 8) ]
+; CHECK-NEXT: br i1 [[C1]], label %[[IF1:.*]], label %[[EXIT:.*]]
+; CHECK: [[IF1]]:
+; CHECK-NEXT: [[C2:%.*]] = icmp ugt i32 [[X]], 10
+; CHECK-NEXT: br i1 [[C2]], label %[[IF2:.*]], label %[[EXIT]]
+; CHECK: [[IF2]]:
+; CHECK-NEXT: [[V:%.*]] = load i64, ptr [[P]], align 8
+; CHECK-NEXT: br label %[[EXIT]]
+; CHECK: [[EXIT]]:
+; CHECK-NEXT: [[RES:%.*]] = phi i64 [ [[V]], %[[IF2]] ], [ 1, %[[IF1]] ], [ 0, %[[ENTRY]] ]
+; CHECK-NEXT: ret i64 [[RES]]
+;
+entry:
+ call void @llvm.assume(i1 true) [ "dereferenceable"(ptr %p, i64 8), "align"(ptr %p, i64 8) ]
+ br i1 %c1, label %if1, label %exit
+
+if1:
+ %c2 = icmp ugt i32 %x, 10
+ br i1 %c2, label %if2, label %exit
+
+if2:
+ %v = load i64, ptr %p, align 8
+ br label %exit
+
+exit:
+ %res = phi i64 [ %v, %if2 ], [ 1, %if1 ], [ 0, %entry ]
+ ret i64 %res
+}
+
+define i64 @assume_deref_align2_nofree_nosync(i1 %c1, i32 %x, ptr %p) nofree nosync {
+; CHECK-LABEL: define i64 @assume_deref_align2_nofree_nosync(
+; CHECK-SAME: i1 [[C1:%.*]], i32 [[X:%.*]], ptr [[P:%.*]]) #[[ATTR0]] {
; CHECK-NEXT: [[ENTRY:.*:]]
; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[P]], i64 8), "align"(ptr [[P]], i64 8) ]
; CHECK-NEXT: [[C2:%.*]] = icmp ugt i32 [[X]], 10
@@ -51,9 +110,10 @@ exit:
ret i64 %res
}
-define i64 @assume_deref_align_not_dominating(i1 %c, ptr %p) {
+
+define i64 @assume_deref_align_not_dominating(i1 %c, ptr %p) nofree nosync {
; CHECK-LABEL: define i64 @assume_deref_align_not_dominating(
-; CHECK-SAME: i1 [[C:%.*]], ptr [[P:%.*]]) {
+; CHECK-SAME: i1 [[C:%.*]], ptr [[P:%.*]]) #[[ATTR0]] {
; CHECK-NEXT: [[ENTRY:.*]]:
; CHECK-NEXT: br i1 [[C]], label %[[IF:.*]], label %[[EXIT:.*]]
; CHECK: [[IF]]:
|
@llvm/pr-subscribers-llvm-ir Author: Florian Hahn (fhahn) ChangesUpdate LangRef and code using
Update code using This follows up on #123196. With that change, it should be safe to expose dereferenceable assumptions more widely as in Full diff: https://github.com/llvm/llvm-project/pull/126117.diff 8 Files Affected:
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 8891aedcb58e552..f4d6affab8f9c3e 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -1474,7 +1474,11 @@ Currently, only the following parameter attributes are defined:
``null_pointer_is_valid`` function attribute is present.
``n`` should be a positive number. The pointer should be well defined,
otherwise it is undefined behavior. This means ``dereferenceable(<n>)``
- implies ``noundef``.
+ implies ``noundef``. When ``dereferenceable(<n>)`` is used in an
+ :ref:`assume operand bundls <assume_opbundles>`, the pointer is only
+ guaranteed to be dereferenceable at the point of the assumption and
+ may not be dereferenceable at later pointers, e.g. because it could have
+ been freed.
``dereferenceable_or_null(<n>)``
This indicates that the parameter or return value isn't both
diff --git a/llvm/lib/Analysis/Loads.cpp b/llvm/lib/Analysis/Loads.cpp
index 733a7988e5a730a..91ae944d2e68366 100644
--- a/llvm/lib/Analysis/Loads.cpp
+++ b/llvm/lib/Analysis/Loads.cpp
@@ -26,8 +26,6 @@
using namespace llvm;
-extern cl::opt<bool> UseDerefAtPointSemantics;
-
static bool isAligned(const Value *Base, Align Alignment,
const DataLayout &DL) {
return Base->getPointerAlignment(DL) >= Alignment;
@@ -171,7 +169,7 @@ static bool isDereferenceableAndAlignedPointer(
Size, DL, CtxI, AC, DT, TLI,
Visited, MaxDepth);
- if (CtxI && (!UseDerefAtPointSemantics || !V->canBeFreed())) {
+ if (CtxI && !V->canBeFreed()) {
/// Look through assumes to see if both dereferencability and alignment can
/// be proven by an assume if needed.
RetainedKnowledge AlignRK;
diff --git a/llvm/lib/Analysis/ValueTracking.cpp b/llvm/lib/Analysis/ValueTracking.cpp
index 6eba6c0f08c3f40..eac11ca379932ba 100644
--- a/llvm/lib/Analysis/ValueTracking.cpp
+++ b/llvm/lib/Analysis/ValueTracking.cpp
@@ -626,7 +626,7 @@ static bool isKnownNonZeroFromAssume(const Value *V, const SimplifyQuery &Q) {
*I, I->bundle_op_info_begin()[Elem.Index])) {
if (RK.WasOn == V &&
(RK.AttrKind == Attribute::NonNull ||
- (RK.AttrKind == Attribute::Dereferenceable &&
+ (RK.AttrKind == Attribute::Dereferenceable && !V->canBeFreed() &&
!NullPointerIsDefined(Q.CxtI->getFunction(),
V->getType()->getPointerAddressSpace()))) &&
isValidAssumeForContext(I, Q.CxtI, Q.DT))
diff --git a/llvm/test/Analysis/ValueTracking/assume-queries-counter.ll b/llvm/test/Analysis/ValueTracking/assume-queries-counter.ll
index a53b90b7b448ae8..0223ab8c4b677ab 100644
--- a/llvm/test/Analysis/ValueTracking/assume-queries-counter.ll
+++ b/llvm/test/Analysis/ValueTracking/assume-queries-counter.ll
@@ -47,7 +47,7 @@ define dso_local i1 @test2(ptr readonly %0) {
ret i1 %2
}
-define dso_local i32 @test4(ptr readonly %0, i1 %cond) {
+define dso_local i32 @test4(ptr readonly %0, i1 %cond) nofree nosync {
; COUNTER1-LABEL: @test4(
; COUNTER1-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[TMP0:%.*]], i32 4) ]
; COUNTER1-NEXT: br i1 [[COND:%.*]], label [[A:%.*]], label [[B:%.*]]
diff --git a/llvm/test/Analysis/ValueTracking/assume.ll b/llvm/test/Analysis/ValueTracking/assume.ll
index 4c4b46c41996868..298facfa3aa9d05 100644
--- a/llvm/test/Analysis/ValueTracking/assume.ll
+++ b/llvm/test/Analysis/ValueTracking/assume.ll
@@ -57,7 +57,7 @@ define dso_local i1 @test2(ptr readonly %0) {
ret i1 %2
}
-define dso_local i32 @test4(ptr readonly %0, i1 %cond) {
+define dso_local i32 @test4(ptr readonly %0, i1 %cond) nofree nosync {
; CHECK-LABEL: @test4(
; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[TMP0:%.*]], i32 4) ]
; CHECK-NEXT: br i1 [[COND:%.*]], label [[A:%.*]], label [[B:%.*]]
@@ -91,7 +91,7 @@ A:
ret i32 %6
}
-define dso_local i32 @test4a(ptr readonly %0, i1 %cond) {
+define dso_local i32 @test4a(ptr readonly %0, i1 %cond) nofree nosync {
; CHECK-LABEL: @test4a(
; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[TMP0:%.*]], i32 4), "align"(ptr [[TMP0]], i32 8) ]
; CHECK-NEXT: br i1 [[COND:%.*]], label [[A:%.*]], label [[B:%.*]]
diff --git a/llvm/test/Transforms/LICM/hoist-speculatable-load.ll b/llvm/test/Transforms/LICM/hoist-speculatable-load.ll
index 752cf3e99cbf5a6..cf43ff9913813e3 100644
--- a/llvm/test/Transforms/LICM/hoist-speculatable-load.ll
+++ b/llvm/test/Transforms/LICM/hoist-speculatable-load.ll
@@ -11,12 +11,12 @@ define void @f(i32 %ptr_i, ptr %ptr2, i1 %cond) {
; CHECK-NEXT: store i32 0, ptr [[PTR2:%.*]], align 4
; CHECK-NEXT: br label [[FOR_BODY_LR_PH]]
; CHECK: for.body.lr.ph:
-; CHECK-NEXT: [[TMP0:%.*]] = load i32, ptr [[PTR]], align 4
; CHECK-NEXT: br label [[FOR_BODY:%.*]]
; CHECK: for.body:
; CHECK-NEXT: [[I_08:%.*]] = phi i32 [ 0, [[FOR_BODY_LR_PH]] ], [ [[INC:%.*]], [[IF_END:%.*]] ]
; CHECK-NEXT: br i1 [[COND]], label [[IF_END]], label [[IF:%.*]]
; CHECK: if:
+; CHECK-NEXT: [[TMP0:%.*]] = load i32, ptr [[PTR]], align 4, !invariant.load [[META0:![0-9]+]]
; CHECK-NEXT: store i32 [[TMP0]], ptr [[PTR2]], align 4
; CHECK-NEXT: br label [[IF_END]]
; CHECK: if.end:
@@ -56,4 +56,58 @@ exit: ; preds = %if.end, %entry
ret void
}
+define void @f_nofree_nosync(i32 %ptr_i, ptr %ptr2, i1 %cond) nofree nosync {
+; CHECK-LABEL: @f_nofree_nosync(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: [[PTR:%.*]] = inttoptr i32 [[PTR_I:%.*]] to ptr
+; CHECK-NEXT: call void @llvm.assume(i1 true) [ "align"(ptr [[PTR]], i32 16), "dereferenceable"(ptr [[PTR]], i32 16) ]
+; CHECK-NEXT: br i1 [[COND:%.*]], label [[FOR_BODY_LR_PH:%.*]], label [[IF0:%.*]]
+; CHECK: if0:
+; CHECK-NEXT: store i32 0, ptr [[PTR2:%.*]], align 4
+; CHECK-NEXT: br label [[FOR_BODY_LR_PH]]
+; CHECK: for.body.lr.ph:
+; CHECK-NEXT: br label [[FOR_BODY:%.*]]
+; CHECK: for.body:
+; CHECK-NEXT: [[I_08:%.*]] = phi i32 [ 0, [[FOR_BODY_LR_PH]] ], [ [[INC:%.*]], [[IF_END:%.*]] ]
+; CHECK-NEXT: br i1 [[COND]], label [[IF_END]], label [[IF:%.*]]
+; CHECK: if:
+; CHECK-NEXT: [[TMP0:%.*]] = load i32, ptr [[PTR]], align 4, !invariant.load [[META0]]
+; CHECK-NEXT: store i32 [[TMP0]], ptr [[PTR2]], align 4
+; CHECK-NEXT: br label [[IF_END]]
+; CHECK: if.end:
+; CHECK-NEXT: [[INC]] = add nuw nsw i32 [[I_08]], 1
+; CHECK-NEXT: [[CMP:%.*]] = icmp slt i32 [[INC]], 2
+; CHECK-NEXT: br i1 [[CMP]], label [[FOR_BODY]], label [[EXIT:%.*]]
+; CHECK: exit:
+; CHECK-NEXT: ret void
+;
+entry:
+ %ptr = inttoptr i32 %ptr_i to ptr
+ call void @llvm.assume(i1 true) [ "align"(ptr %ptr, i32 16), "dereferenceable"(ptr %ptr, i32 16) ]
+ br i1 %cond, label %for.body.lr.ph, label %if0
+
+if0:
+ store i32 0, ptr %ptr2, align 4
+ br label %for.body.lr.ph
+
+for.body.lr.ph: ; preds = %entry
+ br label %for.body
+
+for.body: ; preds = %for.body.lr.ph, %if.end
+ %i.08 = phi i32 [ 0, %for.body.lr.ph ], [ %inc, %if.end ]
+ br i1 %cond, label %if.end, label %if
+
+if:
+ %0 = load i32, ptr %ptr, align 4, !invariant.load !{}
+ store i32 %0, ptr %ptr2, align 4
+ br label %if.end
+
+if.end: ; preds = %for.body
+ %inc = add nuw nsw i32 %i.08, 1
+ %cmp = icmp slt i32 %inc, 2
+ br i1 %cmp, label %for.body, label %exit
+
+exit: ; preds = %if.end, %entry
+ ret void
+}
declare void @llvm.assume(i1 noundef)
diff --git a/llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-constant-size.ll b/llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-constant-size.ll
index 90671689f1dce05..13780a684f9ce79 100644
--- a/llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-constant-size.ll
+++ b/llvm/test/Transforms/LoopVectorize/dereferenceable-info-from-assumption-constant-size.ll
@@ -1,5 +1,5 @@
; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 5
-; RUN: opt -p loop-vectorize -force-vector-width=2 -use-dereferenceable-at-point-semantics -S %s | FileCheck %s
+; RUN: opt -p loop-vectorize -force-vector-width=2 -S %s | FileCheck %s
declare void @llvm.assume(i1)
@@ -1411,7 +1411,6 @@ exit:
}
; %a may be freed between the dereferenceable assumption and accesses.
-; It is not safe to use with -use-dereferenceable-at-point-semantics.
define void @may_free_align_deref_assumption_in_header_constant_trip_count_loop_invariant_ptr(ptr noalias %a, ptr noalias %b, ptr noalias %c) {
; CHECK-LABEL: define void @may_free_align_deref_assumption_in_header_constant_trip_count_loop_invariant_ptr(
; CHECK-SAME: ptr noalias [[A:%.*]], ptr noalias [[B:%.*]], ptr noalias [[C:%.*]]) {
@@ -1505,7 +1504,6 @@ exit:
}
; %a may be freed between the dereferenceable assumption and accesses.
-; It is not safe to use with -use-dereferenceable-at-point-semantics.
define void @may_free_local_ptr_align_deref_assumption_in_header_constant_trip_count_loop_invariant_ptr(ptr noalias %b, ptr noalias %c) nofree nosync {
; CHECK-LABEL: define void @may_free_local_ptr_align_deref_assumption_in_header_constant_trip_count_loop_invariant_ptr(
; CHECK-SAME: ptr noalias [[B:%.*]], ptr noalias [[C:%.*]]) #[[ATTR1]] {
diff --git a/llvm/test/Transforms/SimplifyCFG/speculate-derefable-load.ll b/llvm/test/Transforms/SimplifyCFG/speculate-derefable-load.ll
index 0138433312ed84a..b8c999d700aa7da 100644
--- a/llvm/test/Transforms/SimplifyCFG/speculate-derefable-load.ll
+++ b/llvm/test/Transforms/SimplifyCFG/speculate-derefable-load.ll
@@ -4,10 +4,14 @@
define i64 @align_deref_align(i1 %c, ptr %p) {
; CHECK-LABEL: define i64 @align_deref_align(
; CHECK-SAME: i1 [[C:%.*]], ptr [[P:%.*]]) {
-; CHECK-NEXT: [[ENTRY:.*:]]
+; CHECK-NEXT: [[ENTRY:.*]]:
; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[P]], i64 8), "align"(ptr [[P]], i64 8) ]
+; CHECK-NEXT: br i1 [[C]], label %[[IF:.*]], label %[[EXIT:.*]]
+; CHECK: [[IF]]:
; CHECK-NEXT: [[V:%.*]] = load i64, ptr [[P]], align 8
-; CHECK-NEXT: [[RES:%.*]] = select i1 [[C]], i64 [[V]], i64 0
+; CHECK-NEXT: br label %[[EXIT]]
+; CHECK: [[EXIT]]:
+; CHECK-NEXT: [[RES:%.*]] = phi i64 [ [[V]], %[[IF]] ], [ 0, %[[ENTRY]] ]
; CHECK-NEXT: ret i64 [[RES]]
;
entry:
@@ -23,9 +27,64 @@ exit:
ret i64 %res
}
+define i64 @align_deref_align_nofree_nosync(i1 %c, ptr %p) nofree nosync {
+; CHECK-LABEL: define i64 @align_deref_align_nofree_nosync(
+; CHECK-SAME: i1 [[C:%.*]], ptr [[P:%.*]]) #[[ATTR0:[0-9]+]] {
+; CHECK-NEXT: [[ENTRY:.*:]]
+; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[P]], i64 8), "align"(ptr [[P]], i64 8) ]
+; CHECK-NEXT: [[V:%.*]] = load i64, ptr [[P]], align 8
+; CHECK-NEXT: [[SPEC_SELECT:%.*]] = select i1 [[C]], i64 [[V]], i64 0
+; CHECK-NEXT: ret i64 [[SPEC_SELECT]]
+;
+entry:
+ call void @llvm.assume(i1 true) [ "dereferenceable"(ptr %p, i64 8), "align"(ptr %p, i64 8) ]
+ br i1 %c, label %if, label %exit
+
+if:
+ %v = load i64, ptr %p, align 8
+ br label %exit
+
+exit:
+ %res = phi i64 [ %v, %if ], [ 0, %entry ]
+ ret i64 %res
+}
+
define i64 @assume_deref_align2(i1 %c1, i32 %x, ptr %p) {
; CHECK-LABEL: define i64 @assume_deref_align2(
; CHECK-SAME: i1 [[C1:%.*]], i32 [[X:%.*]], ptr [[P:%.*]]) {
+; CHECK-NEXT: [[ENTRY:.*]]:
+; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[P]], i64 8), "align"(ptr [[P]], i64 8) ]
+; CHECK-NEXT: br i1 [[C1]], label %[[IF1:.*]], label %[[EXIT:.*]]
+; CHECK: [[IF1]]:
+; CHECK-NEXT: [[C2:%.*]] = icmp ugt i32 [[X]], 10
+; CHECK-NEXT: br i1 [[C2]], label %[[IF2:.*]], label %[[EXIT]]
+; CHECK: [[IF2]]:
+; CHECK-NEXT: [[V:%.*]] = load i64, ptr [[P]], align 8
+; CHECK-NEXT: br label %[[EXIT]]
+; CHECK: [[EXIT]]:
+; CHECK-NEXT: [[RES:%.*]] = phi i64 [ [[V]], %[[IF2]] ], [ 1, %[[IF1]] ], [ 0, %[[ENTRY]] ]
+; CHECK-NEXT: ret i64 [[RES]]
+;
+entry:
+ call void @llvm.assume(i1 true) [ "dereferenceable"(ptr %p, i64 8), "align"(ptr %p, i64 8) ]
+ br i1 %c1, label %if1, label %exit
+
+if1:
+ %c2 = icmp ugt i32 %x, 10
+ br i1 %c2, label %if2, label %exit
+
+if2:
+ %v = load i64, ptr %p, align 8
+ br label %exit
+
+exit:
+ %res = phi i64 [ %v, %if2 ], [ 1, %if1 ], [ 0, %entry ]
+ ret i64 %res
+}
+
+define i64 @assume_deref_align2_nofree_nosync(i1 %c1, i32 %x, ptr %p) nofree nosync {
+; CHECK-LABEL: define i64 @assume_deref_align2_nofree_nosync(
+; CHECK-SAME: i1 [[C1:%.*]], i32 [[X:%.*]], ptr [[P:%.*]]) #[[ATTR0]] {
; CHECK-NEXT: [[ENTRY:.*:]]
; CHECK-NEXT: call void @llvm.assume(i1 true) [ "dereferenceable"(ptr [[P]], i64 8), "align"(ptr [[P]], i64 8) ]
; CHECK-NEXT: [[C2:%.*]] = icmp ugt i32 [[X]], 10
@@ -51,9 +110,10 @@ exit:
ret i64 %res
}
-define i64 @assume_deref_align_not_dominating(i1 %c, ptr %p) {
+
+define i64 @assume_deref_align_not_dominating(i1 %c, ptr %p) nofree nosync {
; CHECK-LABEL: define i64 @assume_deref_align_not_dominating(
-; CHECK-SAME: i1 [[C:%.*]], ptr [[P:%.*]]) {
+; CHECK-SAME: i1 [[C:%.*]], ptr [[P:%.*]]) #[[ATTR0]] {
; CHECK-NEXT: [[ENTRY:.*]]:
; CHECK-NEXT: br i1 [[C]], label %[[IF:.*]], label %[[EXIT:.*]]
; CHECK: [[IF]]:
|
llvm/lib/Analysis/ValueTracking.cpp
Outdated
@@ -626,7 +626,7 @@ static bool isKnownNonZeroFromAssume(const Value *V, const SimplifyQuery &Q) { | |||
*I, I->bundle_op_info_begin()[Elem.Index])) { | |||
if (RK.WasOn == V && | |||
(RK.AttrKind == Attribute::NonNull || | |||
(RK.AttrKind == Attribute::Dereferenceable && | |||
(RK.AttrKind == Attribute::Dereferenceable && !V->canBeFreed() && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think you need this change. Even if the pointer is freed later, the fact that it was dereferenceable at some point implies that it is not null.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, removed again, thanks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How effective are we at proving nofree and nosync these days? Do we need to consider some heuristic for a local "canBeFreedBetween(Assume,CtxI)"?
@@ -11,12 +11,12 @@ define void @f(i32 %ptr_i, ptr %ptr2, i1 %cond) { | |||
; CHECK-NEXT: store i32 0, ptr [[PTR2:%.*]], align 4 | |||
; CHECK-NEXT: br label [[FOR_BODY_LR_PH]] | |||
; CHECK: for.body.lr.ph: | |||
; CHECK-NEXT: [[TMP0:%.*]] = load i32, ptr [[PTR]], align 4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you'd preserve test intent by adding nofree nosyc to this one (maybe with a name change), and then adding a separate one without it.
It isn't clear to my why with the additional attributes we still can't hoist the load. That's concerning?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added f_nofree_nosync
below; the problem seems to be that %ptr
is formed using inttoptr
and Value::canBeFreed
only applies to function arguments I think (it would be valid for the function to allocate and free locally).
So eventually we will probably need more powerful analysis.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add a variant which doesn't involve the inttoptr then?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a variant at the bottom, thanks
@@ -2922,6 +2924,11 @@ the behavior is undefined, unless one of the following exceptions applies: | |||
(including a zero alignment). If this is the case, then the pointer value | |||
must be a null pointer, otherwise the behavior is undefined. | |||
|
|||
* ``dereferenceable(<n>)`` operand bundles only guarantee the pointer is | |||
dereferenceable at the point of the assumption and may not be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"and may not be" seems like you're referring to the assumption at first. Maybe: ". The pointer may not be " (the rest of the sentence is fine)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated, thanks!
@@ -11,12 +11,12 @@ define void @f(i32 %ptr_i, ptr %ptr2, i1 %cond) { | |||
; CHECK-NEXT: store i32 0, ptr [[PTR2:%.*]], align 4 | |||
; CHECK-NEXT: br label [[FOR_BODY_LR_PH]] | |||
; CHECK: for.body.lr.ph: | |||
; CHECK-NEXT: [[TMP0:%.*]] = load i32, ptr [[PTR]], align 4 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add a variant which doesn't involve the inttoptr then?
I think nofree/nosync inference is reasonably effective, but the attributes can only be used with arguments; a function still could allocate/free memory used internally. I am planning to add such a heuristic as follow-up; as long as the assumption is in the preheader, we would need to there and the blocks in the loop. |
398e9a4
to
8d76c39
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…s at assume. (#126117) Update LangRef and code using `Dereferenceable` in assume bundles to only use the information if it is safe at the point of use. `Dereferenceable` in an assume bundle is only guaranteed at the point of the assumption, but may not be guaranteed at later points, because the pointer may have been freed. Update code using `Dereferenceable` to only use it if the pointer cannot be freed. This can further be refined to check if the pointer could be freed between assume and use. This follows up on llvm/llvm-project#123196. With that change, it should be safe to expose dereferenceable assumptions more widely as in llvm/llvm-project#121789 PR: llvm/llvm-project#126117
llvm#126117) Update LangRef and code using `Dereferenceable` in assume bundles to only use the information if it is safe at the point of use. `Dereferenceable` in an assume bundle is only guaranteed at the point of the assumption, but may not be guaranteed at later points, because the pointer may have been freed. Update code using `Dereferenceable` to only use it if the pointer cannot be freed. This can further be refined to check if the pointer could be freed between assume and use. This follows up on llvm#123196. With that change, it should be safe to expose dereferenceable assumptions more widely as in llvm#121789 PR: llvm#126117
@@ -2926,6 +2928,11 @@ the behavior is undefined, unless one of the following exceptions applies: | |||
(including a zero alignment). If this is the case, then the pointer value | |||
must be a null pointer, otherwise the behavior is undefined. | |||
|
|||
* ``dereferenceable(<n>)`` operand bundles only guarantee the pointer is | |||
dereferenceable at the point of the assumption. The pointer may not be |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is indented too far and ends up rendering weirdly.
Update LangRef and code using
Dereferenceable
in assume bundles to only use the information if it is safe at the point of use.Dereferenceable
in an assume bundle is only guaranteed at the point of the assumption, but may not be guaranteed at later points, because the pointer may have been freed.Update code using
Dereferenceable
to only use it if the pointer cannot be freed. This can further be refined to check if the pointer could be freed between assume and use.This follows up on #123196.
With that change, it should be safe to expose dereferenceable assumptions more widely as in
#121789