Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Features/cached graphs #1656

Merged
merged 59 commits into from
Aug 8, 2024
Merged

Features/cached graphs #1656

merged 59 commits into from
Aug 8, 2024

Conversation

shivamka1
Copy link
Collaborator

@shivamka1 shivamka1 commented Jun 12, 2024

What changes were proposed in this pull request?

#1649

Why are the changes needed?

Same as above

Does this PR introduce any user-facing change? If yes is this documented?

No

How was this patch tested?

Uni, integration and manual tests

Issues

If this resolves any issues, please link to them here, the format is a KEYWORD followed by @_
KEYWORDS available are close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved.
Please delete this text before creating your PR

Are there any further changes required?

No

@shivamka1 shivamka1 marked this pull request as ready for review June 14, 2024 18:19
@shivamka1 shivamka1 requested a review from miratepuffin June 14, 2024 18:19
Copy link
Collaborator

@miratepuffin miratepuffin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many random notes - as we discussed I think we need to rethink the different pathways to upload a graph tomorrow and cut some of the confusion out of these functions.

@miratepuffin
Copy link
Collaborator

After discussion, we should tidy the APIs to provide the following:

server = RaphtoryServer(working_dir="/graphs",port=1736,config = ...)

server.start()

server.stop()

server.wait()

server.run()



c=server.get_client()

c=RaphtoryClient("localhost:1736")

c.send_graphs(hashmap_of_graphs,overwrite=True,namespace="ben")

c.load_from_path(path_to_folder_of_graphs,overwrite=True,namespace="shivam")

c.query(...)

@shivamka1
Copy link
Collaborator Author

After discussion, we should tidy the APIs to provide the following:

server = RaphtoryServer(working_dir="/graphs",port=1736,config = ...)

server.start()

server.stop()

server.wait()

server.run()



c=server.get_client()

c=RaphtoryClient("localhost:1736")

c.send_graphs(hashmap_of_graphs,overwrite=True,namespace="ben")

c.load_from_path(path_to_folder_of_graphs,overwrite=True,namespace="shivam")

c.query(...)

Implemented this leaving the namespace argument which we will address once load is sorted

This was referenced Jun 27, 2024
* impl gqlgraphs

* properties, unique layers to gqlgraphs

* impl default configs, impl config load precedence, add tests

* impl auth configs

* wait for server to come online when started, also expose an api on client to see if server is already online, add tests

* add client tests

* fix tests

* add load graphs tests

* rid duplicate tests

* wait internalised

* rid silly auth config defaults
@fabianmurariu
Copy link
Collaborator

if we absolutely must println, we should use tracing https://docs.rs/tracing/latest/tracing/attr.instrument.html otherwise we should remove them

@miratepuffin miratepuffin mentioned this pull request Aug 6, 2024
22 tasks
Copy link
Collaborator

@miratepuffin miratepuffin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Disable load graph from path for both GraphQL and python apis (keep internal function as we may want to add back).
  • Change the return type of python mv,copy,delete to just Result<()>
  • Decode the graph within python receive (you have the code already in a test)
  • Nuke the tracing as we are going to do our own one
  • Disable auth for user APIs until we have audited

pub fn load_graph_from_path(
work_dir: &Path,
path_on_server: &Path,
namespace: &Option<String>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to an Option which would also allow renaming of the graph on copy

Copy link
Collaborator

@miratepuffin miratepuffin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final bits to check through before merge

Copy link
Collaborator

@miratepuffin miratepuffin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The final review!

@miratepuffin miratepuffin merged commit bcc2e87 into master Aug 8, 2024
19 checks passed
@miratepuffin miratepuffin deleted the features/cached-graphs branch August 8, 2024 09:34
@miratepuffin miratepuffin mentioned this pull request Aug 19, 2024
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants