You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Would there be interest in some random initialisation mechanism for vectors / matrices? Often it is ok to init with 0s (the implicit case with new Glint data structures). But sometimes you want to have random normal or random uniform init.
It's possible to do it yourself via a forEachPartition and using a push, but it is a bit of a hack really. It seems much simpler & cleaner to provide an interface for this, like client.random.vector.randn[Double](dim, mean = 0, std = 1.0).
What do you think?
The text was updated successfully, but these errors were encountered:
This seems quite useful and relatively easy to do. I feel like Glint is starting to look more like a distributed linear algebra library and less of a parameter server, which is quite an interesting direction.
I would need to refactor some code that deals with initialization though. The easiest approach to me seems to design an abstract Initializer class, which can be implemented as a ZeroInitializer or a RandomNormalInitializer. The class should be serializable and shipped to the parameter servers when we construct the actors (e.g. by providing it in the Akka Props, such as at Client.scala#L175).
This would allow us to easily write some high-level wrapper functions in Client such as client.randn(...). I'm still quite busy in the next few weeks, but after Spark Summit EU I should have some more time to look into this. Nevertheless, I like the suggestion!
Would there be interest in some random initialisation mechanism for vectors / matrices? Often it is ok to init with 0s (the implicit case with new Glint data structures). But sometimes you want to have random normal or random uniform init.
It's possible to do it yourself via a
forEachPartition
and using apush
, but it is a bit of a hack really. It seems much simpler & cleaner to provide an interface for this, likeclient.random.vector.randn[Double](dim, mean = 0, std = 1.0)
.What do you think?
The text was updated successfully, but these errors were encountered: