Skip to content

Commit

Permalink
Add new checksum feature for more detection case
Browse files Browse the repository at this point in the history
  • Loading branch information
j3ssie committed Feb 1, 2021
1 parent f9ca5a0 commit 04f2d28
Showing 19 changed files with 424 additions and 109 deletions.
41 changes: 26 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

<p align="center">
<img alt="Jaeles" src="https://github.com/jaeles-project/jaeles-plugins/blob/master/assets/jaeles.png?raw=true" height="140" />
<p align="center">
@@ -8,12 +7,11 @@
</p>
</p>

**Jaeles** is a powerful, flexible and easily extensible framework written in Go for building your own Web Application Scanner.
**Jaeles** is a powerful, flexible and easily extensible framework written in Go for building your own Web Application
Scanner.

![Architecture](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/jaeles-architecture.png?raw=true)



## Painless integrate Jaeles into your recon workflow?

<p align="center">
@@ -72,9 +70,18 @@ docker run j3ssie/jaeles scan -s '<selector>' -u http://example.com

## Showcases

| ![apache-status.png](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/apache-status.png?raw=true) [**Apache Server Status**](https://youtu.be/nkBcIvzi3H4) | ![tableau-dom-xss.png](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/tableau-dom-xss.png?raw=true) [**Tableau DOM XSS CVE-2019-19719**](https://youtu.be/EG7Qmt8kt58) |
|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| ![rabbitmq-cred.png](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/rabbitmq-cred.png?raw=true) [**RabbitMQ Default Credentials**](https://youtu.be/ed4n1sCNu3s) | ![jenkins-xss.png](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/jenkins-xss.png?raw=true) [**Jenkins XSS CVE-2020-2096**](https://youtu.be/JfihhEOEWSE) |
| ![apache-status.png](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/apache-status.png?raw=true) [**
Apache Server Status**](https://youtu.be/nkBcIvzi3H4)
| ![tableau-dom-xss.png](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/tableau-dom-xss.png?raw=true) [**
Tableau DOM XSS CVE-2019-19719**](https://youtu.be/EG7Qmt8kt58) | |:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:
|:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:
|
| ![rabbitmq-cred.png](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/rabbitmq-cred.png?raw=true) [**
RabbitMQ Default Credentials**](https://youtu.be/ed4n1sCNu3s)
| ![jenkins-xss.png](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/jenkins-xss.png?raw=true) [**
Jenkins XSS CVE-2020-2096**](https://youtu.be/JfihhEOEWSE) |

<h4 align='center'> More showcase can be found <a href="https://jaeles-project.github.io/showcases/">here</a></h4>

@@ -86,13 +93,13 @@ docker run j3ssie/jaeles scan -s '<selector>' -u http://example.com

![Burp Integration](https://github.com/jaeles-project/jaeles-plugins/blob/master/imgs/Burp-Integration.gif?raw=true)

Plugin can be found [here](https://github.com/jaeles-project/jaeles-plugins/blob/master/jaeles-burp.py) and Video Guide [here](https://youtu.be/1lxsYhfTq3M)
Plugin can be found [here](https://github.com/jaeles-project/jaeles-plugins/blob/master/jaeles-burp.py) and Video
Guide [here](https://youtu.be/1lxsYhfTq3M)

## Mentions

[My introduction slide about Jaeles](https://speakerdeck.com/j3ssie/jaeles-the-swiss-army-knife-for-automated-web-application-testing)


### Planned Features

* Adding more signatures.
@@ -105,14 +112,16 @@ Plugin can be found [here](https://github.com/jaeles-project/jaeles-plugins/blob

## Contribute

If you have some new idea about this project, issue, feedback or found some valuable tool feel free to open an issue for just DM me via @j3ssiejjj.
Feel free to submit new signature to this [repo](https://github.com/jaeles-project/jaeles-signatures).
If you have some new idea about this project, issue, feedback or found some valuable tool feel free to open an issue for
just DM me via @j3ssiejjj. Feel free to submit new signature to
this [repo](https://github.com/jaeles-project/jaeles-signatures).

### Credits

* Special thanks to [chaitin](https://github.com/chaitin/xray) team for sharing ideas to me for build the architecture.

* React components is powered by [Carbon](https://www.carbondesignsystem.com/) and [carbon-tutorial](https://github.com/carbon-design-system/carbon-tutorial).
* React components is powered by [Carbon](https://www.carbondesignsystem.com/)
and [carbon-tutorial](https://github.com/carbon-design-system/carbon-tutorial).

* Awesomes artworks are powered by [Freepik](http://freepik.com) at [flaticon.com](http://flaticon.com).

@@ -129,15 +138,17 @@ This project exists thanks to all the people who contribute. [[Contribute](CONTR

### Financial Contributors

Become a financial contributor and help us sustain our community. [[Contribute](https://opencollective.com/jaeles-project/contribute)]
Become a financial contributor and help us sustain our
community. [[Contribute](https://opencollective.com/jaeles-project/contribute)]

#### Individuals

<a href="https://opencollective.com/jaeles-project"><img src="https://opencollective.com/jaeles-project/individuals.svg?width=890"></a>

#### Organizations

Support this project with your organization. Your logo will show up here with a link to your website. [[Contribute](https://opencollective.com/jaeles-project/contribute)]
Support this project with your organization. Your logo will show up here with a link to your
website. [[Contribute](https://opencollective.com/jaeles-project/contribute)]

<a href="https://opencollective.com/jaeles-project/organization/0/website"><img src="https://opencollective.com/jaeles-project/organization/0/avatar.svg"></a>
<a href="https://opencollective.com/jaeles-project/organization/1/website"><img src="https://opencollective.com/jaeles-project/organization/1/avatar.svg"></a>
@@ -152,7 +163,7 @@ Support this project with your organization. Your logo will show up here with a

## License

`Jaeles` is made with ♥ by [@j3ssiejjj](https://twitter.com/j3ssiejjj) and it is released under the MIT license.
`Jaeles` is made with ♥ by [@j3ssiejjj](https://twitter.com/j3ssiejjj) and it is released under the MIT license.

## Donation

11 changes: 6 additions & 5 deletions cmd/config.go
Original file line number Diff line number Diff line change
@@ -250,13 +250,13 @@ Core Flags:
-s, --signs strings Signature selector (Multiple -s flags are accepted)
-x, --exclude strings Exclude Signature selector (Multiple -x flags are accepted)
-L, --level int Filter signatures by level (default 1)
-G, --passive Turn on passive detections
-G, --passive Turn on passive detections (default: false)
-p, --params strings Custom params -p='foo=bar' (Multiple -p flags are accepted)
-H, --headers strings Custom headers (e.g: -H 'Referer: {{.BaseURL}}') (Multiple -H flags are accepted)
Mics Flags:
--proxy string proxy
--timeout int HTTP timeout (default 20)
--proxy string Proxy for sending request
--timeout int HTTP timeout (default 20s)
--debug Debug
-v, --verbose Verbose
--no-db Disable Database
@@ -280,8 +280,9 @@ Mics Flags:
--html string Enable generate HTML reports after the scan done
--hh string Full help message
--dr Shortcut for disable replicate request (avoid sending many timeout requests)
--at Enable Always True Detection for observe response
--fi Enable filtering mode (to use Diff() detection)
--lc Shortcut for '--proxy http://127.0.0.1:8080'
--at Enable Always True Detection for observe response
--ba Shortcut for take raw input as '{{.BaseURL}}'
`
h += "\n\nExamples Commands:\n"
@@ -297,7 +298,7 @@ Mics Flags:
h += " cat list_target.txt | jaeles scan -c 100 -s <signature>\n"

h += "\nOthers Commands:\n"
h += " jaeles server -s '/tmp/custom-signature/sensitive/.*' -L 2\n"
h += " jaeles server -s '/tmp/custom-signature/sensitive/.*' -L 2 --fi\n"
h += " jaeles server --host 0.0.0.0 --port 5000 -s '/tmp/custom-signature/sensitive/.*' -L 2\n"
h += " jaeles config reload --signDir /tmp/standard-signatures/\n"
h += " jaeles config add -B /tmp/custom-active-signatures/\n"
1 change: 1 addition & 0 deletions cmd/root.go
Original file line number Diff line number Diff line change
@@ -94,6 +94,7 @@ func init() {
RootCmd.PersistentFlags().IntVar(&options.ChunkLimit, "chunk-limit", 200000, "Limit size to trigger chunk run")
// some shortcuts
RootCmd.PersistentFlags().StringVarP(&options.InlineDetection, "inline", "I", "", "Inline Detections")
RootCmd.PersistentFlags().BoolVar(&options.EnableFiltering, "fi", false, "Enable filtering mode (to use Diff() detection)")
RootCmd.PersistentFlags().BoolVar(&options.Mics.DisableReplicate, "dr", false, "Shortcut for disable replicate request (avoid sending many request to timeout)")
RootCmd.PersistentFlags().BoolVar(&options.Mics.BaseRoot, "ba", false, "Shortcut for take raw input as {{.BaseURL}}'")
RootCmd.PersistentFlags().BoolVar(&options.Mics.BurpProxy, "lc", false, "Shortcut for '--proxy http://127.0.0.1:8080'")
39 changes: 27 additions & 12 deletions cmd/scan.go
Original file line number Diff line number Diff line change
@@ -116,20 +116,28 @@ func runScan(cmd *cobra.Command, _ []string) error {
}, ants.WithPreAlloc(true))
defer p.Release()

for _, signFile := range options.SelectedSigns {
sign, err := core.ParseSign(signFile)
if err != nil {
utils.ErrorF("Error parsing YAML sign: %v", signFile)
continue
}
// filter signature by level
if sign.Level > options.Level {
continue
for _, url := range urls {
wg.Add(1)
// calculate filtering result first if enabled from cli
baseJob := libs.Job{URL: url}
if options.EnableFiltering {
core.BaseCalculateFiltering(&baseJob, options)
}

// Submit tasks one by one.
for _, url := range urls {
wg.Add(1)
for _, signFile := range options.SelectedSigns {
sign, err := core.ParseSign(signFile)
if err != nil {
utils.ErrorF("Error parsing YAML sign: %v", signFile)
continue
}

// filter signature by level
if sign.Level > options.Level {
continue
}
sign.Checksums = baseJob.Checksums

// Submit tasks one by one.
job := libs.Job{URL: url, Sign: sign}
_ = p.Invoke(job)
}
@@ -172,6 +180,13 @@ func CreateRunner(j interface{}) {
}

for _, job := range jobs {
// custom calculate filtering if enabled inside signature
if job.Sign.Filter || len(job.Sign.FilteringPaths) > 0 {
core.CalculateFiltering(&job, options)
}

utils.DebugF("Raw Checksum: %v", job.Sign.Checksums)

if job.Sign.Type == "routine" {
routine, err := core.InitRoutine(job.URL, job.Sign, options)
if err != nil {
18 changes: 18 additions & 0 deletions core/detecter.go
Original file line number Diff line number Diff line change
@@ -5,6 +5,7 @@ import (
"fmt"
"github.com/jaeles-project/jaeles/sender"
"github.com/jaeles-project/jaeles/utils"
"github.com/thoas/go-funk"
"regexp"
"strconv"
"strings"
@@ -243,6 +244,22 @@ func (r *Record) RequestScripts(scriptType string, scripts []string) bool {
return result
})

// if checksum is different with all previous checksum
vm.Set("Diff", func(call otto.FunctionCall) otto.Value {
rchecksum := record.Response.Checksum
isDiff := true
if funk.ContainsString(record.Sign.Checksums, rchecksum) {
isDiff = false
}

utils.DebugF("Checksums: %v", record.Sign.Checksums)
utils.DebugF("Current Checksum: %v", rchecksum)
utils.DebugF("Diff() -- %v", isDiff)

result, _ := vm.ToValue(isDiff)
return result
})

// Origin field
vm.Set("OriginStatusCode", func(call otto.FunctionCall) otto.Value {
statusCode := record.OriginRes.StatusCode
@@ -272,6 +289,7 @@ func (r *Record) RequestScripts(scriptType string, scripts []string) bool {
result, _ := vm.ToValue(componentLength)
return result
})

// Origins('1', 'status')
// Origins('response')
vm.Set("Origins", func(call otto.FunctionCall) otto.Value {
83 changes: 83 additions & 0 deletions core/filter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
package core

import (
"fmt"
"github.com/jaeles-project/jaeles/libs"
"github.com/jaeles-project/jaeles/sender"
"github.com/jaeles-project/jaeles/utils"
"github.com/thoas/go-funk"
)

var baseFiltering = []string{
"hopetoget404" + RandomString(6),
fmt.Sprintf("%s", RandomString(16)+"/"+RandomString(5)),
fmt.Sprintf("%s.html", RandomString(16)),
fmt.Sprintf("%s.php~", RandomString(16)),
fmt.Sprintf("%s.%00", RandomString(16)),
fmt.Sprintf("%s.json", RandomString(16)),
}

func BaseCalculateFiltering(job *libs.Job, options libs.Options) {
utils.DebugF("Start Calculate Basic Filtering: %s", job.URL)
for _, filterPath := range baseFiltering {
var req libs.Request
req.Method = "GET"
req.EnableChecksum = true
req.URL = utils.JoinURL(job.URL, filterPath)

res, err := sender.JustSend(options, req)
// in case of timeout or anything
if err != nil {
return
}

if res.Checksum != "" {
utils.DebugF("[Checksum] %s - %s", req.URL, res.Checksum)
job.Checksums = append(job.Checksums, res.Checksum)
}
}
job.Checksums = funk.UniqString(job.Checksums)
}

func CalculateFiltering(job *libs.Job, options libs.Options) {
var filteringPaths []string

// ignore old result
if job.Sign.OverrideFilerPaths {
job.Sign.Checksums = []string{}
} else {
// mean doesn't have --fi in cli
if len(job.Sign.Checksums) == 0 {
filteringPaths = append(filteringPaths, baseFiltering...)
}
}

if len(job.Sign.FilteringPaths) > 0 {
filteringPaths = append(filteringPaths, job.Sign.FilteringPaths...)
}

if len(filteringPaths) == 0 {
return
}

for _, filterPath := range filteringPaths {
var req libs.Request
req.Method = "GET"
req.EnableChecksum = true
//req.URL = job.URL + "/" + filterPath
req.URL = utils.JoinURL(job.URL, filterPath)

res, err := sender.JustSend(options, req)
// in case of timeout or anything
if err != nil {
return
}

if res.Checksum != "" {
utils.DebugF("[Checksum] %s - %s", req.URL, res.Checksum)
job.Sign.Checksums = append(job.Sign.Checksums, res.Checksum)
}
}

job.Sign.Checksums = funk.UniqString(job.Sign.Checksums)
}
6 changes: 6 additions & 0 deletions core/runner.go
Original file line number Diff line number Diff line change
@@ -202,6 +202,7 @@ func (r *Runner) SendOrigin(originReq libs.Request) (libs.Origin, map[string]str
var origin libs.Origin
var err error
var originRes libs.Response
originReq.EnableChecksum = true

originSign := r.Sign
if r.Opt.Scan.RawRequest != "" {
@@ -239,6 +240,11 @@ func (r *Runner) SendOrigin(originReq libs.Request) (libs.Origin, map[string]str
origin.ORequest = originReq
origin.OResponse = originRes
r.Origin = originRec

if originRes.Checksum != "" {
utils.DebugF("[Checksum Origin] %s - %s", originReq.URL, originRes.Checksum)
r.Sign.Checksums = append(r.Sign.Checksums, originRes.Checksum)
}
return origin, r.Target
}

8 changes: 8 additions & 0 deletions core/sending.go
Original file line number Diff line number Diff line change
@@ -98,6 +98,10 @@ func (r *Record) DoSending() {
}

req := r.Request
if r.Opt.EnableFiltering || r.Sign.Filter {
req.EnableChecksum = true
}

// if middleware return the response skip sending it
var res libs.Response
if r.Response.StatusCode == 0 && r.Request.Method != "" && r.Request.MiddlewareOutput == "" && req.Res == "" {
@@ -114,6 +118,10 @@ func (r *Record) DoSending() {
}
r.Request = req
r.Response = res

if r.Response.Checksum != "" {
utils.DebugF("[Checksum] %s - %s", req.URL, res.Checksum)
}
r.Analyze()
}

Loading

0 comments on commit 04f2d28

Please sign in to comment.