-
Notifications
You must be signed in to change notification settings - Fork 44
Mallet Tutorial Getting started
git clone https://github.com/SensePost/Mallet
cd Mallet
mvn package
After a ton of downloads, you should end up with a target directory containing all of the libraries that Mallet requires, as well as a mallet-$VERSION.jar file, where $version indicates the tagged version of Mallet.
Running Mallet is a simple java -jar target/mallet-1.0-SNAPSHOT.jar, if the version you have is 1.0-SNAPSHOT.
Mallet starts up with a graph implementing a very simple SOCKS proxy, listening on localhost:1080.
Mallet includes a very simple demo Client and Server, that go along with the examples/json*.mxe files in the repository. You can run the Server app as follows:
java -cp target/mallet-1.0-SNAPSHOT.jar demo.Server localhost 9999
You can change the port if you want to, just make it correspond with the port you tell the client to connect to! You can run the Client app as follows:
java -cp target/mallet-1.0-SNAPSHOT.jar demo.Client localhost 9999
In this state, the Client can communicate with the Server, but Mallet is not in the loop at all. Normally, I'd try using something like tsocks or proxychains to intercept the socket calls, but for some reason I have not figured out, neither of those work with Java. Go figure! As a work around, I implemented a proxy setup using the commonly referred to socksProxyHost and socksProxyPort variables. When you run the client as follows, the Client will route its connections through Mallet:
java -DsocksProxyHost=localhost -DsocksProxyPort=1080 -cp target/mallet-1.0-SNAPSHOT.jar demo.Client localhost 9999
You can use load examples/json1.mxe into the graph via the File menu, or the "Open" icon in the toolbar. This is really the same default SOCKS proxy graph that is loaded on startup.
Use the Client to send a message to the Server, and then check the Connections tab in Mallet to be sure that you are intercepting the traffic. You should see the connection on the left, and if you select it, you should see the events within that connection on the right. Mallet essentially shows all of the Netty events that have occurred on any Channel (inbound or outbound) that has an InterceptHandler node in the graph. The inbound and outbound channels are joined by a RelayHandler node, which is responsible for copying inbound read events on one channel to outbound write events on the other channel, and vice versa. So, if you are wondering why you are seeing events twice, you are probably seeing the READ event on one channel, and the corresponding WRITE event on the other channel! You can remove one of the InterceptHandler nodes if you want to see fewer events. I like to see both sides, to make sure that what I think is happening really is!
Looking at the events, you should see several READ events, paired with their corresponding WRITE. The first READ event shows a ByteBuf (Netty's version of an array of bytes) representing "{\n", followed by several more events with more of the message, until the Client's message is complete, and the Server responds to it. Eventually, the connection is closed.
You can also select the Intercept checkbox in the menu bar. This will hold any events that pass through an InterceptHandler, so that they can be reviewed and/or modified. Sending a new message from the Client should result in a few events being shown in the UI, but no READ events. Selecting the last entry in the event list, and clicking "Send" will forward all preceding events, triggering more events, until you get several READ events showing up. At this point, you can modify them in the Byte editor (only by changing the hex values), and eventually drop or forward the events when you are satisfied.
Lesson objective: Understand the UI, a bit about how the graph is structured, and what the events look like.
Most protocols are composed of messages - a stream of bytes that collectively make up a message. The question then becomes, how do I know where the message starts and ends? Netty provides a number of FrameDecoder implementations which allow you to easily extract complete messages from a stream. Commonly used classes include LengthFieldFrameDecoder, FixedLengthFrameDecoder, DelimeterBasedFrameDecoder, LineBasedFrameDecoder, and several more. Naturally, if you have a LengthBasedFrameDecoder that strips the preceding length from the message, you will need a corresponding LengthFieldPrepender on the outbound pipeline to add it again, before sending the message. And so on for the other different FrameDecoder strategies.
In this instance, we have seen that the protocol in use between the Client and Server is based on JSON objects. One simple way of telling where the JSON object messages start and end is to look for the first curly brace, then increment a counter for each subsequent opening brace, and decrement it for each closing brace. When you get back down to 0, the message is complete. This is exactly what the JsonObjectDecoder class does!
If you open examples/json2.mxe, you will see a graph that includes a JsonObjectDecoder node. Now we know that any messages passed along past this point in the graph will be a ByteBuf containing a complete JSON message. If you trigger a new connection in the Client, you should see only a single READ event from the Client on the inbound channel, followed by the matching WRITE to the Server on the outbound channel, then a READ on the outbound channel, followed by the matching WRITE to the Client. The initial READ event from the Client should contain the entire JSON message.
Now, should you want to tamper with the message, it is a lot easier to work with a complete message. It's still just bytes, though, and Mallet's byte editor is not so great!
Lesson objective: Understand how to collate or separate a stream of bytes into individual messages, and emit them again.
As a digression from the prepackaged graphs, starting from examples/json2.mxe, find the SimpleBinaryModificationHandler node in the palette, and drag it into the graph on the line between the first InterceptHandler and the Relay node. Double click on the node, and change the two parameters to some text that the Client will send, and a replacement value. Both strings must be the same length!
Now, trigger a new connection from the Client, and review the events. The first READ event should show the original text sent by the Client, but the subsequent WRITE event should show the replacement text. You should also see a ``UserEvent``` indicating that the substitution was made.
This is a useful technique for tampering with protocols that have a short timeout (or just for repetitive changes that you would like to automate). By default, the SimpleBinaryModificationHandler only operates on READ events, although this can be changed by referencing the 4 parameter constructor (String find, String replace, boolean modifyReads, boolean modifyWrites).
There is also a ComplexBinaryModificationHandler that can find and replace across packet boundaries. This can be useful if you are not sure that you have a complete message yet (you have not included a suitable FrameDecoder in the graph), BUT it can also cause problems, if it manages a partial match on the last packet before the server response. If it has a partial match, it simply keeps the packet back, expecting to receive the next packet where it can either complete the match, or fail. If there are no more packets arriving, however, that packet would be kept back indefinitely, breaking the communications! So, prefer the simple over the complex until there is no other option!
Mallet doesn't have to just work with ByteBuf instances, though. Netty uses the concept of FrameDecoders and FrameEncoders to translate from one representation to another. You may start with a ByteFuf, and convert it to a Java Object that provides protocol specific methods to interact with. In this case, let's add a StringDecoder and StringEncoder node into our graph. You can load examples/json3.mxe to get the pre-defined graph. Trigger an event in the Client and see that the messages passing through Mallet are now String's, and consequently much easier to understand and edit!
Important concept: Decoder classes work by intercepting the READ events on the Channel, and Encoder classes work by intercepting the WRITE events on the Channel. As a result, it doesn't matter in which order the StringEncoder and StringDecoder nodes are in the graph, so long as both of them are in a suitable position relative to the rest of the nodes. For instance, make sure that both the decoder and encoder are between the JsonObjectDecoder and the InterceptHandler nodes.
Important concept: Netty (and by extension Mallet) can only read and write bytes from and to the network. So, whatever protocol decoders you add to convert bytes to a Java Object need to have a matching protocol encoder on the opposite Channel to convert the Object back to bytes.
Important concept: You don't have to have the same protocol decoder/encoder on both channels. For instance, starting with examples/json3.mxe, delete the StringEncoder on the inbound Channel, and the StringDecoder on the outbound Channel. Trigger an event in the Client, and observe that you have inbound Strings from the client, and to the Server, but ByteBufs from the Server, and to the Client.
Important concept: Sometimes a decoder and encoder are linked together in a single class, because there may be a dependency between them. To illustrate, consider an HTTP client: An outbound HEAD request triggers a response header with a Content-Length header indicating the number of bytes that a GET request would receive. A naive HttpResponseDecoder would hang, waiting for the content that would never arrive. Knowing that the response is for a HEAD request allows the decoder to stop at the appropriate point, and move on.