Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removes double-conversion when collecting into Elasticsearch #1700

Merged
merged 1 commit into from
Aug 23, 2017

Conversation

codefromthecrypt
Copy link
Member

@codefromthecrypt codefromthecrypt commented Aug 22, 2017

Before, when accepting span2 format, we needlessly converted data to and
from the legacy Span format. This removes the double-conversion in favor
of a version 2 native span consumer.

Before:

             ┌──────────────────────────────────────────────────┐
             │           ┌──────────────┐        KafkaCollector │
             │           │ Kafka Message│                       │
             │           └──────┬───────┘                       │
             │┌─────────────────┼───────────────────┐           │
             ││           ┌─────┴───────┐  Collector│           │
             ││           │ byte array  │           │           │
             ││           └─────┬───────┘           │           │
             ││              .──▼──.                │           │
             ││             ╱       ╲               │           │
             ││       ┌────( decode  )────┐         │           │
             ││    ┌──┴───┐ `.     ,' ┌───┴──┐      │           │
             ││    │json2 │   `───'   │thrift│      │           │
             ││    └──┬───┘       │   └───┬──┘      │           │
             ││       │           │       │         │           │
             ││       │         ┌─┴────┐  │         │           │
             ││       ▼         │ json │  │         │           │
             ││┌────────────┐   └─┬────┘  │         │           │
             │││List<Span2> │     │       │         │           │
             ││└────┬───────┘     │       │         │           │
┌────────────┴┴─────┼─────────────┼───────┼───┐     │           │
│                   ▼             ▼       ▼   │     │           │
│               .───────.       ┌────────────┐│     │           │
│              ( convert )─────▶│ List<Span> ││     │           │
│               `───────'       └──────┬─────┘│     │           │
│ This double converts when input      │      ├─────┘           │
│ is already in span2 format           │      ├─────────────────┘
│                                      │      ├─────────────────┐
│              ┌────────────┐      .───▼───.  │Elasticsearch    │
│              │List<Span2> │◀────( convert ) │AsyncSpanConsumer│
│              └─────┬──────┘      `───────'  │                 │
│                    │                        │                 │
└────────────┬───────▼────────────────────────┘                 │
             │  .───────.       ┌────────────┐                  │
             │ ( convert )─────▶│BulkRequest │                  │
             │  `───────'       └──────┬─────┘                  │
             └─────────────────────────┼────────────────────────┘
                                       ▼                         
                               ┌──────────────┐                  
                               │ Elasticsearch│                  
                               └──────────────┘                  

Now, the storage layer is simpler and only converts when required

               ││              .─────.                │      │   
               ││             ╱       ╲               │      │   
               ││       ┌────( decode  )────┐         │      │   
               ││    ┌──┴───┐ `.     ,' ┌───┴──┐      │      │   
               ││    │json2 │   `───'   │thrift│      │      │   
               ││    └──┬───┘       │   └───┬──┘      │      │   
               ││       │           │       │         │      │   
               ││       │         ┌─┴────┐  │         │      │   
               ││       │         │ json │  │         │      │   
               ││       │         └─┬────┘  │         │      │   
               ││       │           │       │         │      │   
               ││       ▼           ▼       ▼         │      │   
               ││ ┌────────────┐  ┌────────────┐      │      │   
               ││ │List<Span2> │  │ List<Span> │──────┼─┐    │   
               ││ └─────┬──────┘  └────────────┘      │ │    │   
               │└───────┼─────────────────────────────┘ │    │   
               └────────┼───────────────────────────────┼────┘   
┌───────────────────────┼───────┐ ┌─────────────────────┼───────┐
│                   .───▼───.   │ │ ┌────────────┐  .───▼───.   │
│                  ( convert )◀─┼─┼─│List<Span2> ◀── convert )  │
│ Elasticsearch     `───┬───'   │ │ └────────────┘  `───────'   │
│ AsyncSpanConsumer     │       │ │  Conditional conversion     │
│                 ┌─────▼──────┐│ └─────────────────────────────┘
│                 │BulkRequest ││                                
│                 └─────┬──────┘│                                
└───────────────────────┼───────┘                                
                        ▼                                        
                ┌──────────────┐                                 
                │ Elasticsearch│                                 
                └──────────────┘                                 

Before, when accepting span2 format, we needlessly converted data to and
from the legacy Span format. This removes the double-conversion in favor
of a version 2 native span consumer.
@codefromthecrypt codefromthecrypt changed the base branch from collector-deprecate to master August 23, 2017 01:36
@codefromthecrypt codefromthecrypt merged commit 8892aa3 into master Aug 23, 2017
@codefromthecrypt codefromthecrypt deleted the es-span2native branch August 23, 2017 02:06
abesto pushed a commit to abesto/zipkin that referenced this pull request Sep 10, 2019
…kin#1700)

Before, when accepting span2 format, we needlessly converted data to and
from the legacy Span format. This removes the double-conversion in favor
of a version 2 native span consumer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant