Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How should I run the streaming tensor correctly? #8

Closed
lwqq00 opened this issue Dec 4, 2024 · 4 comments
Closed

How should I run the streaming tensor correctly? #8

lwqq00 opened this issue Dec 4, 2024 · 4 comments

Comments

@lwqq00
Copy link

lwqq00 commented Dec 4, 2024

Hello, I am learning about streaming tensor decompositions and very interested in "Streaming Generalised Canonical Polyadic Tensor Decompositions" published in PASC23.
However, when I try to reproduce it, I use the following command.

./bin/genten --input ../data/chicago-crime-comm-filtered3.txt --streaming-solver online-cp --history-method ktensor-fro --window-method reservoir --window-size 10 --window-weight 1.0 --output-file stream_output.ktns --history-file chicago_crime_history.txt

But I can't observe if the streaming-solver is called successfully or not. Since this is my first encounter with a codebase as complex as genten, I have some overwhelmed.
I would like to ask if I am executing the command correctly? Also, how should I verify that streaming-solver is being used?

THANKS!

@etphipp
Copy link
Collaborator

etphipp commented Dec 4, 2024

Thanks for reaching out. Unfortunately the streaming solver currently can't be run from the command-line as you discovered (I should at least add an error message stating that). The results in that paper were created by running GenTen through the Matlab interface here. The issue is the command-line driver doesn't have an interface for reading the streaming slices, mainly because it wasn't clear to me what the utility would be in actually doing that. If you are mainly just interested in playing around with this, would the Matlab interface work for you? I could post some code as an example to get you up and running with it. Alternatively, I could also probably get it up and running with the Python interface. I could also get the command-line interface setup for it, but it might take a little time and I would have to decide on an interface for specifying how the tensor slices are brought in.

@lwqq00
Copy link
Author

lwqq00 commented Dec 5, 2024

Thank you so much for your patient response! I greatly admire your excellent work, which has been incredibly inspiring for my research. I am planning to cite your paper and conduct some comparative experiments to further understand and study the streaming solver.
Therefore, if possible, I would be very excited to see the command-line interface support the streaming solver functionality in the future. This would significantly help us use GenTen more efficiently. If it’s convenient for you, I’d also like to see some related usage examples.

@etphipp
Copy link
Collaborator

etphipp commented Dec 15, 2024

I created a python interface for the streaming GCP algorithm, and created two python notebooks that demonstrate how to use it: one for synthetic data and Gaussian loss here and one using the Chicago Crime data from FROSTT with Bernoulli loss here. As you can see from the Chicago Crime example, there is a lot of preprocessing that is required for the streaming algorithm to reorder the modes, split the tensor into sub-tensors along the temporal mode, and compute a warm-start, not to mention the multiple parameter lists that are needed for the streaming solver. It really isn't practical to include all of those preprocessing steps into the command-line driver, which is why I'm reluctant to do that. I am hoping the python interface will be sufficient for your needs.

@lwqq00
Copy link
Author

lwqq00 commented Dec 17, 2024

Thanks for your generous help and excellent work! They are extremely helpful for me to understand and work with the streaming solver.

@lwqq00 lwqq00 closed this as completed Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants