Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

investigate weighting of edge scores by primary source (semmeddb) or primary source type (text-mining) #550

Closed
andrewsu opened this issue Jan 20, 2023 · 2 comments

Comments

@andrewsu
Copy link
Member

In NCATSTranslator/Feedback#21, "water" is returned as a treatment for cerebal palsy. The result comes from BTE (result 89 from https://arax.ncats.io/?r=83fa0ad2-e666-43db-a932-b02fceb335d6). All of the key edges are from text-mined sources (semmeddb and text-mining provider).

Right now, I believe result scores are the sum of the NGD edge scores for all edges in a result. We should investigate a weighted sum, where weights could be determined by the primary source or primary source type. Initially, we could try something very naive like "text-mined edges will have half the weight of non-text-mined edges". Evaluation of this naive scheme initially would have to be done by eye/smell test, but ideally we test via benchmarks in the future.

@tokebe
Copy link
Member

tokebe commented Jan 20, 2023

It may make sense to make this a final pass in the score.js.

I'd be tempted to say the easiest way to handle this may be to add a scoreWeight property to the API List file that is then used in score.js. That said, by primary source type makes more sense than purely by API -- perhaps a map of different types in a new config?

I'll also say that we probably want to change it from sum to average or something for combining scores in creative mode to avoid scores going too high from just a lot of results being merged, though maybe we do want to keep the idea that more duplicate results being merged means a result is probably "better" because it's coming from multiple sources...

@tokebe
Copy link
Member

tokebe commented Aug 1, 2023

Superseded by #634

@tokebe tokebe closed this as completed Aug 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants