Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SPARK-2582. Make Block Manager Master pluggable. #1506

Closed
wants to merge 4 commits into from

Commits on Jul 21, 2014

  1. SPARK-2582. Make Block Manager Master pluggable.

    This patch makes the BlockManagerMaster a trait and makes the current BlockManagerMaster one of
    the possible implementations and renames it to StandaloneBlockManagerMaster. An additional (as yet undocumented)
    configuration parameter is added which can be used to set the BlockManagerMaster type to use. At some
    point, when we add BlockManagerMasters which write metadata to HDFS or replicate, we can add other possible
    values which will use other implementations.
    
    There is no change in current behavior. We must also enforce other implementations to use the current Akka actor
    itself, so the code in the BlockManager does not need to care what implementation is used on the BMM side. I am not sure
    how to enforce this. This is not too much of a concern as we don't have to make it pluggable - so the only options would
    be part of Spark - so this should be fairly easy to enforce.
    harishreedharan committed Jul 21, 2014
    Configuration menu
    Copy the full SHA
    840b3ce View commit details
    Browse the repository at this point in the history
  2. SPARK-2582. Remove unnecessary braces and also remove Logging inherit…

    …ance in BlockManagerMaster
    harishreedharan committed Jul 21, 2014
    Configuration menu
    Copy the full SHA
    bfc327e View commit details
    Browse the repository at this point in the history
  3. Fix test failures.

    harishreedharan committed Jul 21, 2014
    Configuration menu
    Copy the full SHA
    eaaf500 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2014

  1. Merge remote-tracking branch 'asf/master' into pluggable-BMM

    Conflicts:
    	core/src/test/scala/org/apache/spark/storage/BlockManagerSuite.scala
    harishreedharan committed Jul 28, 2014
    Configuration menu
    Copy the full SHA
    ac2b7e9 View commit details
    Browse the repository at this point in the history