Skip to content
This repository has been archived by the owner on Apr 22, 2022. It is now read-only.

Simulate users/clicks using large batch dataset #10

Open
bjgbeelen opened this issue Jul 13, 2018 · 1 comment
Open

Simulate users/clicks using large batch dataset #10

bjgbeelen opened this issue Jul 13, 2018 · 1 comment
Assignees
Labels
de-assessment DataEngineer assessment

Comments

@bjgbeelen
Copy link
Contributor

Use one of the large datasets to produce events on Kafka.
Realistic replay or every second an event, up to you... :-)

@bjgbeelen bjgbeelen added the de-assessment DataEngineer assessment label Jul 13, 2018
@krisgeus krisgeus added this to the New DE assessment milestone Jul 13, 2018
@krisgeus krisgeus self-assigned this Aug 23, 2018
@krisgeus
Copy link
Contributor

Use the json endpoint of divolte and a hdfs sink to generate a large enough dataset.
Apache bench could be used maybe to generate the calls.

@krisgeus krisgeus assigned krisgeus and bbrumi and unassigned krisgeus Sep 27, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
de-assessment DataEngineer assessment
Projects
None yet
Development

No branches or pull requests

3 participants