Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new generic optimization API #1028

Merged
merged 6 commits into from
Jul 14, 2023
Merged

Added new generic optimization API #1028

merged 6 commits into from
Jul 14, 2023

Conversation

m4rs-mt
Copy link
Owner

@m4rs-mt m4rs-mt commented Jul 7, 2023

This PR adds a newly created optimization API to build massively-parallel (particle based) optimizers. The current PR includes a set of classes to build custom optimizers while sharing a unified problem-focused API surface.

The optimization classes support vectorized IO and computation operations using the type parameters TNumericType and TElementType. TElementType refers to the underlying scalar type, like float. TNumericType has to implement the IVectorizedType interface to support combined vectorized types like Float32x4.

Users can implement their own objective function by implementing to IOptimizationFunction interface as shown below:

  public readonly struct Objective : IOptimizationFunction<Float32x2, float, float>
  {
      public float Evaluate(LongIndex1D index, Index1D dimension, SingleVectorView<Float32x2> positionView)
      {
          float result = 0;
          for (Index1D i = 0; i < dimension; ++i)
          {
              var vec = positionView[i];
              var dist = vec - new Float32x2(5.0f, 5.0f); // The closer we are to (5.0f, 5.0f, ...), the better!
              result += dist.X * dist.X + dist.Y * dist.Y;
          }
          return result / dimension;
      }
 
      public bool CurrentIsBetter(float current, float proposed) => current <= proposed;
 }

Note that this code uses a specialized implementation operating on Float32x2 vectors.

The high level idea is to derive your custom optimizer from OptimizationEngine<...>. Such an engine is meant to manage all buffers and most of the reusable kernels. An engine is then used to create a specialized Optimizer instance to take a given objective function into account. Using this design allows to reuse the same memory allocations with different objective functions.

A sample use case of an imaginary optimization engine is shown here:

// Create the current optimization engine and a specialized optimizer using our previously shown objective function
using var myEngine = new CustomOptimizationEngine<Float32x2, float, float, ...>(...);
using var optimizer = myEngine.CreateOptimizer(stream, random, new Objective());

// Load all variable bounds and custom use-case and domain-specific parameters
optimizer.LoadBounds(stream, lowerBounds, upperBounds);
optimizer.LoadParameters(stream, /* Custom optimizer parameters */);

// Initialize best-known positions and our best-known distance for a 4-dimensional problem
var bestPosition = new float[] { 0.0f, 0.0f, 0.0f, 0.0f };
var run = optimizer.BeginOptimization(stream, bestPosition, float.MaxValue);

// Realize a user-defined optimization loop
for (int i = 0; i < 100; ++i)
    run.Step();

// a) Finish the optimization run by accumulating all results into GPU accessible buffers
var gpuResult = run.Finish();

// b) Finish the optimization run by accumulating all results and fetching them into CPU memory
var result = run.FinishToCPUAsync();

Note that this PR requires PR #1022, PR #1023, and PR #1027.

@m4rs-mt m4rs-mt added the feature A new feature (or feature request) label Jul 7, 2023
@m4rs-mt m4rs-mt added this to the v1.5 milestone Jul 7, 2023
@m4rs-mt m4rs-mt force-pushed the optimization branch 2 times, most recently from 4fc1c44 to ea1b7e8 Compare July 7, 2023 13:00
m4rs-mt added a commit that referenced this pull request Jul 7, 2023
m4rs-mt added a commit that referenced this pull request Jul 13, 2023
@m4rs-mt m4rs-mt merged commit 39924e6 into master Jul 14, 2023
@m4rs-mt m4rs-mt deleted the optimization branch July 14, 2023 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new feature (or feature request)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants