Lower the memory footprint of shogun library

[vigsterkr]

travis is failing more and more with:

ld terminated with signal 9

which means that the library linking was killed because the instance was running out of memory. we need to start doing something about shogun's memory footprint ASAP

[karlnapf]

could #2005 help?

[vigsterkr]

@karlnapf noup, unfortunately we are having that flag since march apparently and we are running into troubles still on travis

[lambday]

@lisitsyn how about after this gsoc we go ahead with the plugin project? I think that will also help reducing in the memory footprint. I can join with you in this. if we can make this ready before next year's gsoc it would be a great boost!

[iglesias]

I second that, @lambday.

[lambday]

@iglesias either we can go with existing libraries like pugg, or we can create from scratch our own as required like @lisitsyn started. It will be really cool!

[vigsterkr]

@lambday as always i'll point my finger on gstreamer

[lambday]

@vigsterkr didn't know about this one. can we define our core components, plugin managers, plugins etc using this?

[vigsterkr]

@lambday noup, as it's actually a media library, but they are thoroughly tested as it's being shiped with a lot of set-top boxes, browsers etc. and their plugin framework is just awesome.

[lambday]

@vigsterkr so can we use it in shogun? or you're suggesting that we follow what they did.

[vigsterkr]

@lambday no not at all, it's just a good reference point how we should do things.
as that one has a very good pipelining method that basically makes sure that the pipeline you've defined, i.e. load_feature -> preprocess -> train -> test -> evaluate -> save
a) makes sense at all
b) it automagically loads all the requried dlls

[lambday]

@vigsterkr sounds really exciting. will check it out. as a starter I was just having a look at Dr. Dobb's tutorial on this.

[vigsterkr]

btw the problem with travis is actually with the swig interface code compilation itself, e.g. modshogun_PYTHON.cxx. we should try again to modularize the swig interface, as that file takes more than 4gigs of ram to compile.

[iglesias]

@vigsterkr, could you quickly point out for me the previous attempt to modularize the swig interface, or somewhere I could start looking at?

[vigsterkr]

@iglesias i suppose it should be among the commits, but i've just heard from @sonney2k that once it was done but that it was very unstable that's why it was dropped on the end. :S