Code Development Notes

Overview of How We Developed the Codebase

• Found some research to support our plan for parallelizing SGD
• Sketched out the algorithm on paper
• Implemented algorithm in unit test "simulation"
• Transposed test algorithm into Iterative Reduce primitives
  • ComputableMaster
  • Worker
  • Updateable
• Which Gave Us
  • POLRMasterNode
  • POLRWorkerNode
  • ParameterVectorGradientUpdatable

Core Stochastic Gradient Descent Notes

• com.cloudera.knittingboar.sgd
  • Contains most of the core logic for running the SGD process
  • Based on core code from the Mahout Multinomial Logistic Regression / SGD implementation
  • Interesting Classes
    • ParallelOnlineLogisticRegression
      • Modified Mahout SGD / LR
    • POLRMasterDriver
      • Simulated Master node - used in unit test simulations to craft the initial parallel SGD logic
    • POLRWorkerDriver
      • Simulated Worker node - used in unit test simulations to craft the initial parallel SGD logic

Iterative Reduce Based Parallel SGD Implementation

• com.cloudera.knittingboar.sgd.iterativereduce
  • Contains the Iterative Reduce based nodes to run on YARN
  • Interesting Classes
    • POLRMasterNode
      • Master Node Iterative Reduce implementation for SGD
      • Processes the "super step"
    • POLRWorkerNode
      • Processes a shard/split of input data per epoch/iteration
      • Sends updates to master after each iteration

Unit Tests

• We ended up writing quite a few unit tests
• Not all unit tests are for regression testing, some were used to hash out chunks of parallel SGD functionality
  • Specifically this namespace:
    • https://github.com/jpatanooga/KnittingBoar/tree/master/src/test/java/com/cloudera/knittingboar/sgd