Add initial scaffolding #1
Conversation
Signed-off-by: José Ulises Niño Rivera <junr03@users.noreply.github.com>
There was a problem hiding this comment.
This is good @junr03. Few minor comments and questions.
For the envoy stats access this would be an issue. I also didn't find a way to interact with envoy stats from proxy-wasm interface. To add stats in ABI it would require change on proxy-wasm, envoy (as host) and getting buy-off from open-source community.
envoyfilter/Cargo.toml
Outdated
| @@ -4,4 +4,9 @@ version = "0.1.0" | |||
| authors = ["Katanemo Inc <info@katanemo.com>"] | |||
| edition = "2018" | |||
There was a problem hiding this comment.
2021 is the last stable edition
|
|
||
| ## Building | ||
|
|
||
| ```sh |
There was a problem hiding this comment.
Also need to add support for runtime wasm32-wasi before cross-compiling
$ rustup target add wasm32-wasi
| info!("Access granted."); | ||
| // This call allows the filter to continue operating on the HTTP request sent by the user. | ||
| // In Katanemo's use case the call would continue after the LLM host has responded with routing | ||
| // decisions. |
There was a problem hiding this comment.
At this point we need to look at response from llm and log additional metrics
| // This call allows the filter to continue operating on the HTTP request sent by the user. | ||
| // In Katanemo's use case the call would continue after the LLM host has responded with routing | ||
| // decisions. | ||
| self.resume_http_request(); |
There was a problem hiding this comment.
There was a problem hiding this comment.
Figuratively: yes
Literally: no, Envoy is event driven, not blocking. So that means that the filter has paused processing the http request until there is an event that initiates processing again. i.e it's not the case that execution stopped on line 73. The next line of execution would be on_http_request_body or similar. Does that make sense?
| // RootContext allows the Rust code to reach into the Envoy Config | ||
| impl RootContext for HttpHeaderRoot { | ||
| fn on_configure(&mut self, _: usize) -> bool { | ||
| if let Some(config_bytes) = self.get_plugin_configuration() { |
There was a problem hiding this comment.
In upcoming PRs this would be converted to yaml file and verified according to schema spec.
There was a problem hiding this comment.
It already comes to us converted and verified. However, it looks like the WASM filter schema is dynamic. I need to see if there is a way to define a structured schema via our own protobuf, so we can rely on Envoy's protobuf parser to verify for us.
There was a problem hiding this comment.
that would be nice if we could use protbuf for validation etc.
|
@junr03 I did see that c++ wasm filter does support interaction with stats sink https://github.com/proxy-wasm/proxy-wasm-cpp-sdk/blob/3c7f786bf1e8bdd9537a640eeff1558e18bc9855/docs/wasm_filter.md#stats-api |
Nice -- I don't mean to say that it is supported or not. Just a callout that of all the functionality we discussed using, the only immediate one I did not find (not that I looked too hard yet) was stats manipulation. Lmk if my responses make sense and you feel comfortable landing this so we can work off of it. |
Sure let's keep exploring and see if we can find a way to create metrics inside envoy using proxy-wasm |
This PR adds the initial scaffolding for an Envoy filter using the WASM runtime written in rust. The filter shows quite a few of Envoy's capabilities relevant for the katanemo gateway:
Important callout: the only piece of functionality that is critical for the gateway that I have not explored yet is interacting with Envoy's stats objects. i.e, are we able to create and manipulate counters/gauges/histograms from the rust code? I am not sure yet. Figuring this out will be my next priority
In the event that that functionality is lacking we will need to either:
Tests will come in a separate PR -- usually I would write tests in the same PR, but I wanted to get the scaffolding working and out so that @adilhafeez could start taking a look.