Add annotation support to drcachesim to delimit trace boundaries and app phases

[fhahn]

For some of our uses cases we found it useful to limit instrumentation (e.g. mem tracing) to a certain code region, e.g.

void foo(void){
    uninteresting();
    __START_TRACE();
    for (....) {
      interesting accesses
    }
    __STOP_TRACE();
    uninteresting2();
}

We have been using invalid encodings on AArch64, which we replace with labels in app2app and then in later stages we keep track of of whether tracing is enabled or disabled.

#define __START_TRACE() { asm volatile (".inst 0x2520e020"); }
#define __STOP_TRACE() { asm volatile (".inst 0x2520e040"); }

Would something like this be useful as a generic API or does something similar already exist and I missed it?

[derekbruening]

Yes, something exactly like this exists: annotations. However, it is not implemented on aarch*. That is covered by #1672.

The other method of doing this is using the start/stop API to create a trace burst: see clients/drcachesim/tests/*burst*.cpp. We have been using start/stop rather than annotations for even lower overhead when not tracing, but also adding annotation support to drcachesim's tracer for an alternative start/stop seems good to me. We can consider this issue to cover the drcachesim annotation side.

[fhahn]

Great thanks! I'll see if I can get that working next week!

[derekbruening]

I'll see if I can get that working next week!

Do you mean the start/stop approach? I see you closed the issue which I had repurposed to be about using annotations in drcachesim: was that deliberate? I'll reopen here. There are some advantages to annotations over start/stop: they can be compiled into the binary with minimal changes as opposed to linking with drcachesim and DR, and they are finer-grained and could be used to do things while tracing like turn on and off online cache filtering or add labels into the trace demarking regions of interest, so I think it's worth considering this as a feature request.

[fhahn]

Ah sorry, I did not realize the re-purposing! I meant annotation support.

[derekbruening]

Right, so there are 2 things for the annotations: implement the underlying annotation instruction sequences for aarch*, which is #1672, and then adding a series of named annotations and handlers in drcachesim's tracer, which is this issue.

[derekbruening]

It looks like this is the same feature request as #2478. Though this was filed later it has more participants so let's fold #2478 into here.

[derekbruening]

Pasting some xref links from #2478:

Xref #3995 which is asking for using instruction counts like -trace_after_instrs to delineate multiple app regions

Note that one existing method of doing this is to use the start/stop interface. We have specific tests of this for drcachesim offline traces with static linking: the simplest is here: https://github.com/DynamoRIO/dynamorio/blob/master/clients/drcachesim/tests/burst_static.cpp

[derekbruening]

Pasing from #2478 to further clarify that this covers annotations not just for starting and stopping a trace but for marking pieces of the trace:

For correlating which part of a trace corresponds to which phase of an
application, one approach would be to add annotations to the app using DR's
existing annotation mechanism and having the trace recognize them (and either
mark up the trace or start/stop the trace). If the app cannot be changed, we'd
want to look for certain functions or addresses. This issue covers building
some general support for this type of feature.