-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RUM-4371 Extract OpenTelemetry support sdk into a dedicated module #2021
RUM-4371 Extract OpenTelemetry support sdk into a dedicated module #2021
Conversation
@@ -0,0 +1,3 @@ | |||
# Datadog Implementation SDK for OpenTelemetry |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# Datadog Implementation SDK for OpenTelemetry | |
# Datadog Trace SDK for Android with OpenTelemetry support |
or something similar
) | ||
consumerProguardFiles("consumer-rules.pro") | ||
} | ||
namespace = "com.datadog.android.opentelemetry" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
both namespace and root package should be com.datadog.android.trace.opentelemetry
, because the common definitions are in com.datadog.android.trace
. Thus, since it is a subset with a specific implementation belonging to the Trace feature, it should be com.datadog.android.trace.opentelemetry
imo. This keeps the consistency.
"The Tracing feature to use with the Datadog monitoring " + | ||
"library for Android applications." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need to be updated
@@ -60,7 +52,6 @@ android { | |||
} | |||
|
|||
dependencies { | |||
coreLibraryDesugaring(libs.desugarJdk) | |||
api(project(":dd-sdk-android-core")) | |||
api(libs.openTelemetryApi) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if we are extracting OpenTelemetry
into a dedicated module, why do we keep this one here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just a bad copy / paste sorry
public static CoreTracerBuilder builder(InternalLogger internalLogger) { | ||
try { | ||
Class.forName("java.util.function.Function"); | ||
return new DefaultCoreTracerBuilder(internalLogger); | ||
} catch (ClassNotFoundException e) { | ||
internalLogger.log( | ||
InternalLogger.Level.ERROR, | ||
Arrays.asList(InternalLogger.Target.USER, InternalLogger.Target.MAINTAINER), | ||
() -> "Trying to CoreTracer implementation on Android 23 and below. " + | ||
"In order for this to properly work you will need to enable coreDesugaring " + | ||
"in your compileOptions", | ||
e, | ||
false, | ||
null); | ||
return new NoOpCoreTracerBuilder(); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why we are doing this check? as I understand, Otel is moved to the dedicated module and it was Otel which was using API 24+ classes. So, to my understanding, after this move dd-sdk-android-trace
should be completely compatible with API 21?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so I like you asked. Not quite, as I explained the copied code from APM also uses java.util.function
packages so this is my extra safety in case someone internally in our team uses this code wrongly. There was another approach I had in mind and that was about adding the java9.util.function
package inside to replicate the java.util.function
and replace all its usage in the CoreTracer
code. It worked when I tested this morning with the code from this repo: https://github.com/retrostreams/android-retrostreams/tree/master/src/main/java/java9/util/function
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think once we remove the code we don't need, majority of such usages will go away, and for the rest we can do something java9.util.function
indeed.
The problem with this check is that most probably during the development devs use emulators with the most recent API. And even if there are instrumented tests on pre-API devices, these are just tests, nobody is checking logcat there. So I don't think application developers will ever see this error before shipping app to prod.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But at least if they ship it to prod by mistake it is not going to crash...that's the reason why I wanted at least to have this safety net. Without it will crash. I agree that once we have the artifact from APM team the CoreTracer
here will be replaced by that and problem will no longer be there. We will do code cleanup for sure but I already did some tests and some parts cannot be removed so in the end we will have 3 options:
- keep it like before, no safety net, no NoOpAgentTracer and make sure we do not use it wrongly
- keep it like it is proposed in this POC
- use
java9.util.function
substitution. This I tested and works but might have some limitations in some conditions (who knows what we might miss in those substitutions as that package was not maintained in the last 7 years). I this this is a bit hacky in a way, @xgouchet already had an opinion on this when I discussed this morning. Can you add your thoughts here @xgouchet please, tks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My point is that we can come to the complete API 21 support without need for any check/desugaring if we remove redundant code and replace the sites using API 23+ symbols with API 21 compatible equivalents.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Duly noted, and this can be done in 2 steps, keep it like this for now and then later cleaning and replacing those java.util.function
with compatible equivalents. This will allow us to go ahead with dogfooding and even release, I don't see this a blocker here. I am waiting for other comments here and then we take a final decision.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also this reflection check is not needed for API 24+.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, aware of that will fix it in the real PR now, tks !!
CoreTracerBuilder config(Config config); | ||
|
||
AgentTracer.TracerAPI build(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
CoreTracerBuilder config(Config config); | |
AgentTracer.TracerAPI build(); | |
CoreTracerBuilder config(Config config); | |
AgentTracer.TracerAPI build(); |
public interface CoreTracerBuilder { | ||
CoreTracerBuilder serviceName(String serviceName); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
public interface CoreTracerBuilder { | |
CoreTracerBuilder serviceName(String serviceName); | |
public interface CoreTracerBuilder { | |
CoreTracerBuilder serviceName(String serviceName); |
interface _InternalCoreWriterProvider : StorageBackedFeature { | ||
fun getCoreTracerWriter(): com.datadog.trace.common.writer.Writer | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please describe this one? How it is used, why does it start with _
and extends StorageBackedFeature
? We are mixing here feature and provider concept. This one, probably, can be a standalone interface, without extending StorageBackedFeature
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is being used here: https://github.com/DataDog/dd-sdk-android/pull/2021/files#diff-1db0ecd609920bfa147ebcdc3403f1534b1d66e6506d8a055297399552783c5fR140
As you can see it is outside of the module and unwrap
function only accepts a <T:Feature>
, therefor the need for that interface to extend StorageBackedFeature
.
The reason why I added _
as a prefix is to follow the _InternalProxy
approach for RN and Flutter to expose that class but mark it as Internal.
Please let me know if you have a better idea to export that coreTracerWriter
from there in the dd-sdk-android-trace-otel
module.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still think it is better to keep them separate and don't use _
prefix. Feature
interface is indented for communication with SDK core, and here we are adding something completely not related.
IMO Instead, it is better to have just CoreWriterProvider
interface, not extending any interface, annotated with @InternalApi
, which has only one method getCoreTracerWriter
.
And then in the code:
val tracingFeature = sdkCore.getFeature(Feature.TRACING_FEATURE_NAME).unwrap()
...
val coreTracerWriter = (tracingFeature as? `CoreWriterProvider`).getCoreTracerWriter()
3b954ad
to
9ae513b
Compare
b7ffd61
to
9a11404
Compare
9a11404
to
0500eac
Compare
@@ -0,0 +1,16 @@ | |||
class com.datadog.android.trace.opentelemetry.OtelTracerProvider : io.opentelemetry.api.trace.TracerProvider | |||
constructor(com.datadog.android.api.feature.FeatureSdkCore, com.datadog.trace.bootstrap.instrumentation.api.AgentTracer.TracerAPI, com.datadog.android.api.InternalLogger, Boolean) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we shouldn't have a public constructor when Builder
is given
// Build | ||
id("com.android.library") | ||
kotlin("android") | ||
id("com.google.devtools.ksp") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is there a need for KSP in this module?
"OPENTELEMETRY_API_VERSION_NAME", | ||
"\"${libs.versions.openTelemetry.get()}\"" | ||
) | ||
consumerProguardFiles("consumer-rules.pro") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is no such file under dd-sdk-android-trace-otel
, at least in this PR
* @return the function result | ||
*/ | ||
R apply(T t); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
} | |
} | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is a typo in the folder name and in the package name respectively: intenal
-> internal
. This applies to other similar test files as well.
sample/kotlin/build.gradle.kts
Outdated
repositories { | ||
mavenLocal() | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't be needed, we consume from the project, not from the published artifact
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes was only for local testing
return try { | ||
// we are forcing here the checkup as it could be triggered only at runtime and will be too late | ||
Class.forName("java.util.function.Function") | ||
action() | ||
} catch (@Suppress("TooGenericExceptionCaught") e: Exception) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if this code block will be executed in the hot path, we need an optimization to use reflection only once
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, I thought about it myself, this code will only be called in the OtelTracerProvider
builder method which normally should be happening few times in the clients code when an OtelTracerProvider
is created. I don't think we should not anything else here.
internal fun <T : Any?> executeSafelyOnAndroid23AndBelow( | ||
action: () -> T, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you may want to place action
as last argument to be able to use syntax like foo(args) { lambda }
for the invocation
val activeTraceId = activeSpanContext.traceId.toString() | ||
sdkCore.addActiveTraceToContext(activeTraceId, activeSpanId) | ||
fun build(): TracerProvider { | ||
return executeSafelyOnAndroid23AndBelow( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: current method name gives an impression that code will be executed only on API 23 and below, which is not true. maybe it is worth to rename it to executeIfAboveAndroid24
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, but that name suggestion will mean that it will only be executed on Android24
and above for me :). What if I just call it executeSafely
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Normally the method name should reflect when lambda will be executed, not when it won't.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes definitely and actually the name your proposed it's fine, my mind is a bit blurry after all this nordic weather :)
// We need to add the DatadogContextStorageWrapper and this should be executed before | ||
// before io.opentelemetry.context.LazyStorage is loaded in the class loader to take effect. | ||
// For now we should assume that our users are using `OtelTracerProvider` first in their code. | ||
// Later on maybe we should consider a method DatdogOpenTelemetry.initialize() to be called |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// Later on maybe we should consider a method DatdogOpenTelemetry.initialize() to be called | |
// Later on maybe we should consider a method DatadogOpenTelemetry.initialize() to be called |
0500eac
to
1e98819
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## feature/otel-support #2021 +/- ##
========================================================
- Coverage 63.92% 63.62% -0.31%
========================================================
Files 752 751 -1
Lines 28382 28359 -23
Branches 4686 4682 -4
========================================================
- Hits 18143 18041 -102
- Misses 9036 9119 +83
+ Partials 1203 1199 -4
|
fun setSampleRate(Double): Builder | ||
fun setBundleWithRumEnabled(Boolean): Builder | ||
override fun toString(): String | ||
companion object |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor: need to make companion
object private
or internal
, because it doesn't expose any public members in the current state
} | ||
|
||
dependencies { | ||
api(project(":dd-sdk-android-core")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this import is redundant, :features:dd-sdk-android-trace
already brings it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes but I mentioned this usage on other modules also, I think for visibility
return try { | ||
// we are forcing here the checkup as it could be triggered only at runtime and will be too late | ||
@Suppress("UnsafeThirdPartyFunctionCall") | ||
Class.forName("java.util.function.Function") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are we sure that in case if desugaring applied there will be actually such class in the classpath (btw, if we use reflection, we need to add a rule to consumer Proguard rules) and desugaring won't instead inline function calls?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked a desugared application and there is a java.util.function
package added in the .dex
package at compile time. I am not really sure we need proguard rule here. Everything we are doing here is quite experimental and we rely on the fact that the desugaring will always work like that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is for obfuscation. Production app will have java.util.function.Function
renamed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in fact, once we get rid of all NewApi
errors, this check/method won't be needed
mockInternalLogger.verifyLog( | ||
InternalLogger.Level.ERROR, | ||
InternalLogger.Target.USER, | ||
OtelTracerProvider.TRACING_NOT_ENABLED_ERROR_MESSAGE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how then we are going distinguish between the cases when Tracing is not enabled vs there is a mistake from our side by not implementing the necessary interface? Shouldn't we introduce different errors for these cases?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
good point, it makes sense in this case I guess to have different messages. Maybe we should also change the target
in this case to MAINTAINER
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for the case when checking internal interface - yes, for the case when checking Feature - it should stay with User target.
@@ -890,6 +909,23 @@ internal class OtelTracerBuilderProviderTest { | |||
|
|||
// endregion | |||
|
|||
class TracingFeature : Feature, InternalCoreWriterProvider { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not using mock? change on the L126 looks redundant.
upd: Ok, I see that TracingFeature
is internal and in another module. Then at least let's rename it to StubTracingFeature
and remove TODO
calls in methods with Unit
return type.
) { | ||
// Given | ||
val exception = Exception() | ||
val action: () -> String = { throw exception } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if some class other than Function
is missing, then it will be Error
, not exception. This might be the case if something changes in the code running action
and requirement is not API 24 anymore, but higher (so that Function
exists, but something else doesn't exist on the classpath).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's have a chat about this as I am not sure I understand exactly what you mean. One note from my part is that we will not be able to check the whole package existence through reflection unless we will go one by one. The more we discuss about this the more I think that maybe this safety mechanism is not the best idea and we should just go with desugaring required.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's say Otel code is updated with some class which exists on say API 25. Class.forName("function")
will be fine, but when action()
is executed it will be NoClassDefFound**Error**
for this new class, not exception. But imo with current state of the code it is fine.
} | ||
|
||
@Test | ||
fun `M return action result W executeSafelyOnAndroid23AndBelow{action does not throw exception, below 24}`( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fun `M return action result W executeSafelyOnAndroid23AndBelow{action does not throw exception, below 24}`( | |
fun `M return action result W executeIfJavaFunctionPackageExists {action does not throw exception, below 24}`( |
|
||
@TestTargetApi(24) | ||
@Test | ||
fun `M return action result W executeSafelyOnAndroid23AndBelow{action does not throw exception, above 23}`( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fun `M return action result W executeSafelyOnAndroid23AndBelow{action does not throw exception, above 23}`( | |
fun `M return action result W executeIfJavaFunctionPackageExists {action does not throw exception, above 23}`( |
<?xml version="1.0" encoding="UTF-8"?> | ||
<lint> | ||
<issue id="NewApi"> | ||
<ignore path="src/main/java/com/datadog/trace"/> /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But it means it is still temporary, only for dogfooding? Since it is dd-sdk-android-trace
module which is already used by the customers, we cannot ship any API here that doesn't exist on API 21, we cannot release like that.
// region InternalCoreWriterProvider | ||
override fun getCoreTracerWriter(): com.datadog.trace.common.writer.Writer { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// region InternalCoreWriterProvider | |
override fun getCoreTracerWriter(): com.datadog.trace.common.writer.Writer { | |
// region InternalCoreWriterProvider | |
override fun getCoreTracerWriter(): com.datadog.trace.common.writer.Writer { |
return try { | ||
// we are forcing here the checkup as it could be triggered only at runtime and will be too late | ||
@Suppress("UnsafeThirdPartyFunctionCall") | ||
Class.forName("java.util.function.Function") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw, Class.forName
may throw not only exceptions, but also Errors https://developer.android.com/reference/java/lang/Class#forName(java.lang.String), they won't be called by catch block in the current state.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yup maybe I will change it for Throwable
instead to make sure.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Docs were flagged to review due to features/dd-sdk-android-trace-otel/README.md
. Do you intend to add more content here? Let me know, and we can take a look!
@brett0000FF yes we have a special task for docs in the README.md + docs in the public page. |
1e98819
to
b49d5b0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approving this PR, but there are things which should be addressed in the later PRs:
- we shouldn't have
NewApi
errors suppressed - all
NewApi
should be addressed
return try { | ||
// we are forcing here the checkup as it could be triggered only at runtime and will be too late | ||
@Suppress("UnsafeThirdPartyFunctionCall") | ||
Class.forName("java.util.function.Function") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in fact, once we get rid of all NewApi
errors, this check/method won't be needed
What does this PR do?
Following previous discussions and plans in this PR we are performing the following:
dd-sdk-android-trace-otel
dd-java-agent
project containing theCoreTracer
code will remain as part of thedd-sdk-android-trace
module as this is considered the core tracing module. The idea behind this decision is to have later a better split of tracing API implementations support into dedicated module and all to delegate to theCoreTracer
into thedd-sdk-android-trace
module.dd-sdk-android-trace-otel
module will not require desugaring in its .aar metadata anymore and we added a safety mechanism in place in case our clients will omit the docs and use this module without desugaringMotivation
What inspired you to submit this pull request?
Additional Notes
dd-java-agent
code containing theCoreTracer
logic is still using thejava.util.function
package and therefor will require desugaring for Android 23 and below. By discussing with the team we decided that it is fine for this code to stay in thedd-sdk-android-trace
module as for now it will only be used by thedd-sdk-android-trace-otel
module which will require desugaring in the release docs. In any case this code will be stripped out by R8 in the clients apps if it is not used.CoreTracer
code and see if we can actually make it compatible for Android 23 and below by removing all thejava.util.function
logic. This is highly doubtable as I already performed some research and this turns out to be quite complicated.ContextStorage.addWrapper
functionality as this is weirdly initialized in the OpenTelemetry code (when the class is loaded) and these tests are too flaky as there can be already the class loaded before the test started due to some weird Junit behaviour.Review checklist (to be filled by reviewers)