-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Request for model training #43
Comments
Hi @laura300, Thanks for the feedback! Training a model for excitatory and inhibitory cells combined would in theory be possible but I would not recommend it (but you can still convince me otherwise!). The model trained on both datasets would try to find a compromise and therefore result in sub-optimal predictions for excitatory neurons. I have done this upon a question of a reviewer of our paper, but we did not include this result in the paper. Instead, I would rather recommend to train a model for excitatory and another model for inhibitory cells. If you think that this is a good idea, I can do this for you (the model for excitatory neurons is already available, I would only train the model for inhibitory neurons). The limitation of this approach is that there are only few training data for inhibitory neurons. Alternatively, one could use the model trained for excitatory neurons for both excitatory and inhibitory neurons, and simply accept that the predictions will be still correlated with inhibitory spike rates (Figure 3a) but off by a large factor (Figure 3c,e). I hope this helps you to decide what model to choose for your recordings! If not, let me know. About GCaMP7f: Yes, it is not yet in included in the ground truth. I will include one available dataset in the near future (approx. end of 2022), but I want to do some analyses first. The expected performance is that a model trained on all available datasets (e.g., one of the global_EXC models) will perform equally well on GCaMP7f data. However, due to the shorter rise time of GCaMP7f, the inferred spike rate will be slightly earlier than the true spike rate (this systematic shift will occur for any spike inference algorithm optimized for GCaMP6). This effect is small for GCaMP7f (I think it is clearly less than 100 ms) but a bit stronger for GCaMP8. Together, this effect is not detrimental, but one needs to keep it in mind. Let me know if you have any other questions! |
Hi Peter, Thanks a lot for your quick answer and your input on the GCaMP7. All the best, |
Ok, great! P.S. @laura300: Since your data are sampled a relatively low temporal resolution (5 Hz), maybe check out whether the typical noise levels of your data is covered by the models (default coverage is noise levels between 2 and 9). There was recently an issue (issue #39), where we found that noise levels of the 4-Hz dataset were much higher than 9, and training a model with higher noise levels improved performance a lot. So, just in case you also work with low-signal recordings :-) |
Hi Peter, |
Hi Jana @nguyemi5, That's a good question. I think your intuition is correct. The overall shape of the inferred spiking should be correct but the absolute spike rate might be overestimated. Luckily, the GCaMP8 paper includes also a ground truth dataset with jGCaMP7f. I have extracted this dataset and used it to check your question. Below, I plot the average spike rate inferred using the "Global_EXC_30Hz_smoothing25ms_causalkernel" model, and the ground truth spike rate. The spike rate is actually overestimated by a factor of 3-4 (median overestimate of 3.6x): Of course, take this with a grain of salt, since it is based on only one single jGCaMP7f ground truth dataset with 21 neurons. I'm still in the process of analyzing how well the existing models generalize to GCaMP8f/m/s and GCaMP7f. This requires some attention to the detail and therefore took me longer than expected (and I was also busy with transitioning from postdoc to PI ...). The brief analyses like this one can be done quickly, but to bring things together and understand the effects will still take me a few more months. But if you have specific questions that you would find interesting to have analyzed, just let me know. Best, |
Hi Peter, |
Hi @nguyemi5, There is an update about using spike inference with Cascade on data acquired with GCaMP7f. We have now tested models that were trained on either GCaMP6 or GCaMP8 and applied them to GCaMP7f, with the result that GCaMP8-trained models (which are already available for Cascade) performed better. However, GCaMP6-based models ("Global_EXC" models) also performed quite well. It is described in this new preprint on spike inference with GCaMP8 in Figure 1 Supplement 5. If somebody is using MLSpike or OASIS (Suite2p) for spike inference with GCaMP7f, we also provide the best-fit values for GCaMP7f in the Methods section :-) Best, |
Hi Peter,
Thanks for developing this tool, it's really useful!
Would it be possible to have to train a model for Excitatory and inhibitory cells combined at 5Hz?
I am also wondering since we have been using GCaMP7f and this is not included in the ground truth, do you have any idea of the expected performance?
Thanks!
Laura
The text was updated successfully, but these errors were encountered: