Support for custom Java-backed models (eg. factorization machine)

[lcx517]

Hi,
Spark ML support FM Model in 3.x.
Is there any plan to support factorization machine in Spark 3.x?

[vruusmann]

The PMML specification does not define a specialized model element for representing factorization machines.

AFAIK, the business logic of factorization machines cannot be effectively represented using the available PMML building blocks (eg. a chain of regression tables).

Possible workarounds:

1. Independently design and implement a PMML-like representation for factorization models into the JPMML software project (both converter and evaluator sides).

2. Implement as a custom Java-backed model. Simply create a subclass of org.jpmml.evaluator.java.JavaModel, and implement a factorization machine business logic in its evaluate<MiningFunction>(ValueFactory, EvaluationContext) method. For example, could simply invoke Apache Spark API methods in there.