Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random init #60

Open
MLnick opened this issue Oct 1, 2016 · 1 comment
Open

Random init #60

MLnick opened this issue Oct 1, 2016 · 1 comment

Comments

@MLnick
Copy link
Contributor

MLnick commented Oct 1, 2016

Would there be interest in some random initialisation mechanism for vectors / matrices? Often it is ok to init with 0s (the implicit case with new Glint data structures). But sometimes you want to have random normal or random uniform init.

It's possible to do it yourself via a forEachPartition and using a push, but it is a bit of a hack really. It seems much simpler & cleaner to provide an interface for this, like client.random.vector.randn[Double](dim, mean = 0, std = 1.0).

What do you think?

@rjagerman
Copy link
Owner

This seems quite useful and relatively easy to do. I feel like Glint is starting to look more like a distributed linear algebra library and less of a parameter server, which is quite an interesting direction.

I would need to refactor some code that deals with initialization though. The easiest approach to me seems to design an abstract Initializer class, which can be implemented as a ZeroInitializer or a RandomNormalInitializer. The class should be serializable and shipped to the parameter servers when we construct the actors (e.g. by providing it in the Akka Props, such as at Client.scala#L175).

This would allow us to easily write some high-level wrapper functions in Client such as client.randn(...). I'm still quite busy in the next few weeks, but after Spark Summit EU I should have some more time to look into this. Nevertheless, I like the suggestion!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants