Allowing to use Memory in Pipeline

[c-w-feldmann]

No description provided.

[JochenSiegWork]

I have some problems with the design of unit test. These should be fixed before merging.

[JochenSiegWork]

Testing timings in unit tests is considered a bad thing. Timings depend on many external circumstances. Our code can be correct even when these external circumstances lead to bad runtimes. Instead, I would just test to ensure that the results are correct.

You could also add a test that tests that the cached element is used and not recreated every time the pipeline is called. Maybe wrap the featurization-element in a Wrapper class that counts executions of the fit_transform method, like:

class MyWrapper:

def __init__(element):
  self.counter = 0
  self.element = element

def fit_transform(X, y):
  self.counter += 1
  return self.element.fit_transform(X, y)

featurization_element_wrapped = MyWrapper(MorganFP())
pipeline = Pipeline([
  ..,
  ("featurizater", featurization_element_wrapped),
 ..]
pipeline.fit(X,y)

self.assertEqual(featurization_element_wrapped.counter, 1)

Note that this should only work when you don't parallelize the execution.

[c-w-feldmann]

Implemented. I had to make the counter var a global var, as otherwise the hash of the object would be changed. It is kind of ugly, do you have a better idea?

[c-w-feldmann]

Currently fails docsig. Assumingly, this is a docsig error, as this only occurs for 0.59.0. See this issue.