A light mock http proxy tool to support your End to End automation test. The best way to mock thrid parties API dependencies (inspired by https://httpdump.io)
Test automation is the key component of quality software, we all love it. However nowaday we relay more and more on Saas services to fasten our product delivery (ex: google map, firebase, twilio, etc...) From a development perspective if feels good to use reliable services, but from a test automation perspective... This is where the headach starts.
External dependencies are the main Pain in the a** of E2E automation test !!!
This is how HTTP MOXY starts growing in our mind... We had to solve this remaining pain because it will just increase with the time.
Basically Http Moxy is just a simple http proxy on Steroid, It provides the ability to :
- listen to passing http request
- Mock any http response
And the best is that as a http proxy, Moxy doesn't bring any changes into the way software engineers are coding.
Http Moxy is born from the insight that as my test automation project, i believe that external dependencies are not a part of my scope... Why should i test an external service (i believe the third party provider should have their own End to End test) However it's a part of my scope to test the mapping from my business object to this external interface.
Luckely in 2020 Rest API is a key standard to connect 2 systems.
After a few days of brainstorming http moxy was borned, based on anyproxy from alibaba. Http Moxy* will serve you as middle man between between 2 systems to listen and mock the response from the external party.
The concept relies on the Distributed tracing, each request has a tracable id that we can follow across the system... By having the ability to trace each request through a unique identifier, we can also refer to that same Id to mock the response of the external dependency for happy and unhappy scenario.
Installing the http moxy requires 2 steps:
- Deploy the http moxy in your infrastructure or use the Saas version (https://moxy.restqa.io)
- Inside the microservice that needs to connect an external service, set the proxies environment variable to point to the moxy server
export http_proxy=http://url-of-your-just-deployed-moxy-server:8080in case you deploy your own moxyexport https_proxy=https://moxy.restqa.ioin case you want to use the saas version
Don't forget to set both of them : http_proxy and https_proxy
| Variable | Description | Default |
|---|---|---|
PROXY_PORT |
The port to expose the proxy | 8080 |
ADMIN_PORT |
The port to expose the administration | 8000 |
HEADER_REQUEST_ID_PROPERTY |
The request header property you want to use to identify the request (default: x-request-id) | x-request-id |
MQTT_HOST |
The Host of mqtt broker | localhost |
MQTT_TOPIC |
The topic used to send broker messages | restqa/api/moxy |
Small Demo (click to run the Video)
The service is exposing 2 ports:
- 8080 the proxy port, you can configure it by setting up the
PROXY_PORTenvironment variable - 8000 the admin port, you can configure it by setting up the
ADMIN_PORTenvironment variable
Moxy shares on 2 endpoints into the admin endpoint:
http://url-of-your-just-deployed-moxy-server:8080/requests/{request id}(POST) : To create a mock related to a request idhttp://url-of-your-just-deployed-moxy-server:8080/requests/{request id}(GET) : To inspect a request related to a request id
in the following use case we want to mock the response from our sms provider for unhappy scenarios
We will call:
curl --request POST \
--url http://url-of-your-just-deployed-moxy-server:8000/requests/test-e2e-xxx-yyy-zzz \
--header 'content-type: application/json' \
--data '{
"mock": {
"statusCode": 404,
"header": {
"Content-Type": "application/json"
},
"body": {
"message": "The phone number needs to be defined"
}
}
}'
In the previous request the key informations are:
- We are creating a mock for a future request. This request will need to include the
x-request-id: test-e2e-xxx-yyy-zzzin the request header to be catched by the mock - The mock will repond with a 404 status code and a specific json response body
Once you will run your test, you will need to be sure the request Id : test-e2e-xxx-yyy-zzz, has been set in the request header of every transaction in your system to be sure that the Moxy can catch it.
Example if i call directly the sms provider:
curl 'https://my-sms-provider.com/send/message' \
-X POST \
-H 'Host: my-sms-provider.com' \
-H 'Accept: */*' \
-H 'content-type: application/x-www-form-urlencoded' \
-H 'x-request-id: test-e2e-xxx-yyy-zzz' \
-H 'Content-Length: 32' \
-d 'ACCOUNT=ww&PASSWORD=ww&OPTION=ww'
-k \
-x http://url-of-your-just-deployed-moxy-server:8080
On we catched a request after running our tests, we will want to inspect the request by using the same request id (in our case test-e2e-xxx-yyy-zzz)
curl --request GET \
--url http://url-of-your-just-deployed-moxy-server:8000/requests/test-e2e-xxx-yyy-zzz
And the response will look like:
{
"id": "test-e2e-xxx-yyy-zzz",
"method": "POST",
"url": "my-sms-provider.com",
"path": "/send/message",
"headers": {
"Host": "my-sms-provider.com",
"User-Agent": "curl/7.64.1",
"Accept": "*/*",
"Cookie": "JSESSIONID=EB87299D9439D54FE71A7B5EBBF9FE8B; __cfduid=d72c2f067e0b9024049b8bbe48a7f28be1581260566",
"content-type": "application/x-www-form-urlencoded",
"x-request-id": "test-e2e-xxx-yyy-zzz",
"Content-Length": "32"
},
"body": "ACCOUNT=ww&PASSWORD=ww&OPTION=ww",
"mock": {
"statusCode": 404,
"header": {
"Content-Type": "application/json"
},
"body": {
"message": "The phone number needs to be defined"
}
}
}
Here as you can see you will have all the request information available.
All you need to do is to connect a mock to a request id...
Don't forget you need to ensure that your microservice add the x-request-id into the request headers otherwise moxy won't be able to mock the request
- If you don't mock some request Moxy will just forward the request to the expected target
- Moxy is Acting as a MAN IN THE MIDDLE so do not deploy it in PRODUCTION...
- A self certification is created by Moxy so you will need to ignore the ignore invalid self-signed ssl certificate
TODO
TODO
You can access to a monitoring dashboard through the admin URL on the port 8000 (default)
To run the container you just need to run :
docker run --name http-moxy -p 8080:8080 -p 8000:8000 --rm restqa/http-moxy
Then you can access to the admin panel : http://localhost:8000
And config your http_proxy environement variable : export http_proxy=http://localhost:8080
Let say you want to deploy moxy in the tests namespace you will just need to run :
kubectl apply -f https://raw.githubusercontent.com/restqa/http-moxy/master/kubernetes-definition.yml -n tests
The definition file includes :
- config map
- deployment ( 1 replicas)
- service
You can still get insipire from our example definition file : kubernetes-definition.yml