Merge pull request #40 from laluka/jtof-fap-jsonl-output

Add JSON-Lines option

README.md

@@ -39,8 +39,8 @@ A tool that tests MANY url bypasses to reach a 40X protected page.
 
 Usage:
     bypass-url-parser (-u <URL> | -R <file>) [-m <mode>] [-o <outdir>] [-S <level>] [ (-H <header>)...] [-r <num>]
-                           [-s <ip>] [--spoofip-replace] [-p <port>] [--spoofport-replace] [--request-tls]
-                           [--dump-payloads] [-t <threads>] [-T <timeout>] [-x <proxy_url>] [-v | -d | -dd]
+                      [-s <ip>] [--spoofip-replace] [-p <port>] [--spoofport-replace] [-t <threads>] [-T <timeout>]
+                      [--request-tls] [--jsonl] [--dump-payloads] [-x <proxy_url>] [-v | -d | -dd]
 
 Program options:
     -u, --url <URL>           URL (path is optional) to run bypasses against
@@ -49,10 +49,10 @@ Program options:
     -m, --mode <mode>         Bypass modes. See 'Bypasser.BYPASS_MODES' in code [Default: all]
     -o, --outdir <outdir>     Output directory for results
     -x, --proxy <proxy_url>   Set a proxy in the format http://proxy_ip:port.
-    -S, --save-level <level>  Save results level. From 0 (DISABLE) to 3 (FULL) [Default: 1]
+    -S, --save-level <level>  Save results level. From 0 (DISABLE) to 3 (FULL) [Default: 2]
     -s, --spoofip <ip>        IP(s) to inject in ip-specific headers
     -p, --spoofport <port>    Port(s) to inject in port-specific headers
-    -r, --retry <num>         Retry attempts of failed requests. Set 0 to disable all retry tentatives [Default: 3]
+    -r, --retry <num>         Retry attempts of failed requests. Set 0 to disable all retry tentatives [Default: 1]
     -t, --threads <threads>   Scan with N parallel threads [Default: 1]
     -T, --timeout <timeout>   Request times out after N seconds [Default: 5]
 
@@ -68,6 +68,7 @@ Misc options:
     --spoofport-replace       Disable list of default internal ports in 'http_headers_port' bypass mode
     --request-tls             Force usage of TLS/HTTPS for the request load with the '-R, --request' option
     --dump-payloads           Print all payloads (curls) generated by this tool.
+    --jsonl                   Print results in JSON lines format (pipe command output)
 
 Examples:
     bypass-url-parser -u "http://127.0.0.1/juicy_403_endpoint/" -s 8.8.8.8 -d
@@ -184,6 +185,16 @@ bypass-url-parser -u /path/urls -s /path/custom_ip --spoofip-replace
 bypass-url-parser -u /path/urls -p "3000, 9443, 10443"
 ```
 
+### JSON-Lines output and command piping
+
+With the `--jsonl` option, it's possible to print the results on `stdout` in `JSON-Lines` format. The standard tool's output and results are displayed with a logger on `stderr`, so it is possible to pipe the `JSON-Line` output format with other tools:
+
+```bash
+bypass-url-parser -u "https://thinkloveshare.com/juicy_403_endpoint/" -t 20 -S 0 -m case_substitution,char_encode --jsonl | jq
+```
+
+***Notes:** With `-S 2` ou `-S 3`, the JSON-Lines output also includes the path and the name of saved html files.*
+
 ### Results saving
 
 By default, if target url is unique, the tool saves a copy of the results in `/tmp/tmpXXX-bypass-url-parser/` directory.