Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add store/forward cache capabilities #4

Merged
merged 13 commits into from
Jan 15, 2021
Merged

Add store/forward cache capabilities #4

merged 13 commits into from
Jan 15, 2021

Conversation

adobeDan
Copy link
Contributor

@adobeDan adobeDan commented Jan 6, 2021

Summary

  • Add an sqlite cache for both activation and deactivation requests
  • Respond from the cache if possible when COPS is unavailable
  • Provide for store (cache-only) and forward (COPS-only) proxy operating modes

Testing Steps

  • Test in passthrough mode (for SSL, you will need to update your cert to pkcs12 format with a password)
  • Test in cache mode
  • Test in store mode
  • Test in forward mode

Fixes #3

brotskydotcom and others added 10 commits November 27, 2020 09:16
Includes merge of all upstream changes through release as Adobe Open Source.
Includes merge of all upstream changes through release as Adobe Open Source.
All the bones are there, but apparently you can't use an enum as a configuration value.
So this required a bunch of changes in secure.rs:

- move to native-tls rather than rustssl, which was broken on Mac at the version we were using
- move to latest versions of tokio (now at 1.0) and hyper and related packages
- use pkcs12 certificate form (required by native-tls and better because it's a single package)
- rework the socket stream stuff to be compatible with latest tokio and hyper

Unfortunately, it looks like sqlx is not compatible with the latest tokio.  So that's the next thing to work on.
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	src/proxy/secure.rs
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	src/cache.rs
#	src/cli.rs
#	src/main.rs
#	src/proxy.rs
#	src/res/defaults.toml
#	src/res/template.toml
#	src/settings.rs
@adobeDan adobeDan self-assigned this Jan 6, 2021
@adobeDan
Copy link
Contributor Author

adobeDan commented Jan 6, 2021

@adorton-adobe this is FYI about a lot of work on caching I did over the break. I still need to get sqlx working the latest version of tokio before I can fully test the cache. I should be done with this in about a week. Then I would be happy to turn it over to one of your folks.

I found a wonderful open source developer who had a fork of sqlx that he had ported to the latest tokio and native-tls.

I got all three cache modes working.

There's still one problem when there is a cache miss in store mode: the 502 response I am using to indicate I can't reach the server seems to crash the client.  I have to find out from the server people what I should be returning in that case.
The code wasn't formatting correctly on Linux due to the width_heuristics option.
@adobeDan
Copy link
Contributor Author

Hey, @adorton-adobe, it's definitely time to review this PR, because I believe it works great!

@adobeDan
Copy link
Contributor Author

adobeDan commented Jan 14, 2021

I'm attaching a working config file and certificate to make it easier to test. Just add these lines to your local host file:

## section: SSL
# certificate to use with localhost
127.0.0.1	frl-proxy.brotsky.net
::1			frl-proxy.brotsky.net
## end of section: SSL

and it should work fine. You can control the mode with the --mode command line argument, as in

frl-online-proxy start -c config.toml --mode store

@adorton-adobe
Copy link
Contributor

Thanks. I started looking at it yesterday.

Copy link
Contributor

@adorton-adobe adorton-adobe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. I just have a few comments including one bug I encountered. Once we get those cleared up I think we're ready to merge. I will continue to test it in the meantime.

Cargo.toml Outdated Show resolved Hide resolved
src/cli.rs Outdated Show resolved Hide resolved
src/cops.rs Outdated Show resolved Hide resolved
@adorton-adobe
Copy link
Contributor

One other thing I forgot to ask - have you made any updates to the User Guide yet? If not, I will plan to update it to cover this new functionality.

@adobeDan
Copy link
Contributor Author

One other thing I forgot to ask - have you made any updates to the User Guide yet? If not, I will plan to update it to cover this new functionality.

No, I hate doing docs, I'm slow at it and overthink them terribly. If you would be willing that would be great.

1. Updated to use captive repo version of updated sqlx while waiting for official release.
2. Added explicit tokio-stream dependency to fix dependency conflict between hyper and updated sqlx.
3. Integrated deactivation code fix from code review.
4. Tested deactivation and fixed schema error in deactivations table

NOTE: You must delete any prior databases, because the db schema has changed.
@adobeDan
Copy link
Contributor Author

@adorton-adobe I've integrated your requested fixes, and I've tested deactivation in cache, store, and forward modes. I think we're ready to merge unless your testing reveals other problems.

Copy link
Contributor

@adorton-adobe adorton-adobe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're good for now. I will be doing additional testing as I update the docs and will open issues for anything I encounter.

@adorton-adobe adorton-adobe merged commit 7e63af4 into adobe:dev Jan 15, 2021
@adorton-adobe adorton-adobe added this to the 0.9.0 milestone Jan 15, 2021
@adobeDan adobeDan deleted the feature-cache branch January 29, 2021 05:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Provide store/forward cache capabilities
4 participants