Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ospf6d: fix heap use after free #1217

Merged
merged 1 commit into from
Sep 24, 2017

Conversation

qlyoung
Copy link
Member

@qlyoung qlyoung commented Sep 21, 2017

Refcount issue
fixes #1135

During the loop we save a pointer to the next route in the table in case
brouter is deleted during the course of the loop iteration. However when
we call ospf6_route_remove this can trigger ospf6_route_remove on other
routes in the table, one of which could be pointed at by said pointer.
Since ospf6_route_next locks the route that it returns, it won't
actually be deleted, instead the refcount will go to 1. In the next loop
iteration, nbrouter becomes brouter, and calling ospf6_route_next on
this one will finally decrement the refcount to 0, resulting in a free,
which causes subsequent reads on brouter to be UAF. Since the route will
have OSPF6_ROUTE_WAS_REMOVED set, provided the memory was not
overwritten before we got there, we'll continue on to the next one so it
is unlikely this will cause a crash in production.

Solution implemented is to check if we've deleted the route and continue
if so.

Signed-off-by: Quentin Young qlyoung@cumulusnetworks.com

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: FAILED

See below for issues.
CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1731/

This is a comment from an EXPERIMENTAL automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.

Get source and apply patch from patchwork: Successful

Building Stage: Successful

Basic Tests: Failed

Topology tests on Ubuntu 16.04 amd64: Successful
IPv6 protocols on Ubuntu 14.04: Successful
Ubuntu 16.04 deb pkg check: Successful
IPv4 ldp protocol on Ubuntu 16.04: Successful
Ubuntu 14.04 deb pkg check: Successful
CentOS 7 rpm pkg check: Successful
Fedora 24 rpm pkg check: Successful
Ubuntu 12.04 deb pkg check: Successful
Static analyzer (clang): Successful
Debian 8 deb pkg check: Successful

IPv4 protocols on Ubuntu 14.04: Failed

RFC Compliance Test ANVL-BGP4-15.28 failing:
Test Summary
If an optional attribute is recognized, then the value of this
attribute MUST be checked. If an error is detected, the attribute MUST
be discarded, and the Error Subcode MUST be set to Optional Attribute
Error. The Data field MUST contain the attribute
(type, length and value).
(This test checks for AGGREGATOR attribute)
Test Reference
NEGATIVE
RFC4271, Sect. 6.3, p 34,
UPDATE message error handling
Test Classification
MUST
Test ANVL-BGP4-15.28: !FAILED!
External peer did not receive expected
BGP4 Notification Message from DUT


CLANG Static Analyzer Summary

  • Github Pull Request 1217, comparing to Git base SHA f2a3a14

No Changes in Static Analysis warnings compared to base

138 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1731/artifact/shared/static_analysis/index.html

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1731/

This is a comment from an EXPERIMENTAL automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.


CLANG Static Analyzer Summary

  • Github Pull Request 1217, comparing to Git base SHA f2a3a14

No Changes in Static Analysis warnings compared to base

138 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1731/artifact/shared/static_analysis/index.html

@qlyoung
Copy link
Member Author

qlyoung commented Sep 21, 2017

false positives

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1732/

This is a comment from an EXPERIMENTAL automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.


CLANG Static Analyzer Summary

  • Github Pull Request 1217, comparing to Git base SHA f2a3a14

No Changes in Static Analysis warnings compared to base

138 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1732/artifact/shared/static_analysis/index.html

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1734/

This is a comment from an EXPERIMENTAL automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.


CLANG Static Analyzer Summary

  • Github Pull Request 1217, comparing to Git base SHA f2a3a14

No Changes in Static Analysis warnings compared to base

138 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1734/artifact/shared/static_analysis/index.html

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1733/

This is a comment from an EXPERIMENTAL automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.


CLANG Static Analyzer Summary

  • Github Pull Request 1217, comparing to Git base SHA f2a3a14

No Changes in Static Analysis warnings compared to base

138 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1733/artifact/shared/static_analysis/index.html

@donaldsharp
Copy link
Member

@qlyoung can you make sure you run ospf-smoke with this change?

@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1735/

This is a comment from an EXPERIMENTAL automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.


CLANG Static Analyzer Summary

  • Github Pull Request 1217, comparing to Git base SHA f2a3a14

No Changes in Static Analysis warnings compared to base

138 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1735/artifact/shared/static_analysis/index.html

@mwinter-osr
Copy link
Member

Rerunning tests - Address Sanitizer Tests were disabled for some reason

@FRRouting FRRouting deleted a comment from NetDEF-CI Sep 22, 2017
@FRRouting FRRouting deleted a comment from NetDEF-CI Sep 22, 2017
@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1736/

This is a comment from an EXPERIMENTAL automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.


CLANG Static Analyzer Summary

  • Github Pull Request 1217, comparing to Git base SHA f2a3a14

No Changes in Static Analysis warnings compared to base

138 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1736/artifact/shared/static_analysis/index.html

@mwinter-osr
Copy link
Member

I still see many heap-after-free Address Sanitizer errors:
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1736/artifact/ASANTOPO/AddressSanitizerError/AddressSanitzer.txt

@qlyoung
Copy link
Member Author

qlyoung commented Sep 22, 2017

@mwinter-osr check the line numbers, you are testing master and not stable/3.0.

@mwinter-osr
Copy link
Member

Yes, something went wrong... troubleshooting the CI - ignore the next CI reruns

@FRRouting FRRouting deleted a comment from NetDEF-CI Sep 22, 2017
@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1738/

This is a comment from an EXPERIMENTAL automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.


CLANG Static Analyzer Summary

  • Github Pull Request 1217, comparing to Git base SHA f2a3a14

No Changes in Static Analysis warnings compared to base

138 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1738/artifact/shared/static_analysis/index.html

@FRRouting FRRouting deleted a comment from NetDEF-CI Sep 22, 2017
@mwinter-osr
Copy link
Member

Ok, address sanitizer fixed. There was a bad bug where leftover errors from previous results got reported. Issue is fixed.

Your PR correctly fixes the issue.

During the loop we save a pointer to the next route in the table in case
brouter is deleted during the course of the loop iteration. However when
we call ospf6_route_remove this can trigger ospf6_route_remove on other
routes in the table, one of which could be pointed at by said pointer.
Since ospf6_route_next locks the route that it returns, it won't
actually be deleted, instead the refcount will go to 1. In the next loop
iteration, nbrouter becomes brouter, and calling ospf6_route_next on
this one will finally decrement the refcount to 0, resulting in a free,
which causes subsequent reads on brouter to be UAF. Since the route will
have OSPF6_ROUTE_WAS_REMOVED set, provided the memory was not
overwritten before we got there, we'll continue on to the next one so it
is unlikely this will cause a crash in production.

Solution implemented is to check if we've deleted the route and continue
if so.

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
@NetDEF-CI
Copy link
Collaborator

Continuous Integration Result: SUCCESSFUL

Congratulations, this patch passed basic tests

Tested-by: NetDEF / OpenSourceRouting.org CI System

CI System Testrun URL: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1739/

This is a comment from an EXPERIMENTAL automated CI system.
For questions and feedback in regards to this CI system, please feel free to email
Martin Winter - mwinter (at) opensourcerouting.org.


CLANG Static Analyzer Summary

  • Github Pull Request 1217, comparing to Git base SHA f2a3a14

No Changes in Static Analysis warnings compared to base

138 Static Analyzer issues remaining.

See details at
https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1739/artifact/shared/static_analysis/index.html

@qlyoung
Copy link
Member Author

qlyoung commented Sep 22, 2017

just some style nits

} else {
nbrouter = ospf6_route_next(brouter);
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be simpler:

nbrouter = ospf6_route_next(brouter);
if (brouter->lock == 1)
    continue;

And I'm not sure but maybe we could check the OSPF6_ROUTE_WAS_REMOVED flag instead of the refcount lock.

Copy link
Member Author

@qlyoung qlyoung Sep 24, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Negative, your code is a use after free. brouter is deleted when the refcount drops to zero, which happens inside ospf6_route_next as a call to ospf6_route_unlock (ref). I should have been more specific in the PR description.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Your commit message couldn't be better, it's me who did a poor review here.

Thanks for fixing this long-standing issue, will merge now.

@rwestphal rwestphal merged commit d31b6db into FRRouting:stable/3.0 Sep 24, 2017
@qlyoung qlyoung deleted the fix-heap-uaf-ospf6 branch December 6, 2017 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants