Add dp3 paper to version control #19
Conversation
wkalt
force-pushed
the
task/add-paper
branch
14 times, most recently
from
12a0401 to
0fb682f
Compare
doc/paper/paper.tex
Outdated
| allowing the user to quickly pivot between searching for interesting events, | ||
| and replaying those events in full context. | ||
|
|
||
| For the ML developer, dp3 supports both client/server and embedded modes of |
There was a problem hiding this comment.
These aren't defined clearly IMO - is there a way to put this that does not introduce new terms?
wkalt
force-pushed
the
task/add-paper
branch
3 times, most recently
from
3acc1e4 to
fbe8322
Compare
doc/paper/paper.tex
Outdated
| all apply to server logs as well. Many organizations employ more infrastructure | ||
| engineers than all the employees in the largest robotics company, and some of | ||
| them undoubtedly produce petabytes of logs and analyze them with Kafka | ||
| \cite{kafka} or Spark \cite{spark}. |
There was a problem hiding this comment.
Not sure the above paragraph is needed. Can't we move on to the next paragraph directly?
doc/paper/paper.tex
Outdated
| traditional log databases are generally driven by organizational scale – i.e | ||
| how many thousands of internal microservices there are. Organizational scale is | ||
| slower-growing because it usually requires some form of sustained business | ||
| success over years. |
There was a problem hiding this comment.
I feel that there are some untestable hypotheses in here. Could we shorten this to simply state the challenge(s) brought on by large scales of multimodal log data
doc/paper/paper.tex
Outdated
| serve for ML purposes, will be image and point-cloud data. In these kinds of | ||
| messages, there is effectively one column: a giant byte array. This means that | ||
| row-oriented and columnar files for these kinds of data are physically pretty | ||
| similar. So intuitively, it seems like the performance benefits we would hope |
There was a problem hiding this comment.
"Intuitively," "it seems," "we would hope," "mostly," "perhaps less," etc. are subjective claims that are especially weak when paired together (as in this sentence). Can we omit or more clearly describe/quantify these?
| extremely complex. Although one workable option is to hire SQL experts to | ||
| interface between the database and the robotics developer, one of dp3's | ||
| experimental goals is to avoid this solution and attempt instead to empower | ||
| the robotics developer directly. Second, analytics databases are not |
There was a problem hiding this comment.
If this is one of the motivations of dp3, can we indicate this somewhere previously? This project has lots of potential, but I feel that the motivations are a bit scattered in the paper
| user to decide whether to associate producers with \q{devices} or | ||
| \q{simulation runs} or another internal concept. dp3 supports multiple | ||
| logical databases within a single instance, and operations must be scoped | ||
| with a database. |
There was a problem hiding this comment.
"with" --> was "within" intended here?
| only with producers and topics. Trees are constructed based on a start and end | ||
| time, branching factor, and target leaf width in seconds. The constructor then | ||
| picks a height that results in a tree satisfying the dimensional constraints. | ||
| The height obtained by a default tree is 5. |
There was a problem hiding this comment.
"obtained by" --> "of"?
| incomparable. However, support for this may become important as we develop | ||
| more focus on subqueries. | ||
|
|
||
| Continuous queries are also not supported in dp3 today, however support |
There was a problem hiding this comment.
--> "; however, support would not require much work."
Should we mention this in our feature development plans if not mentioned already?
This adds the dp3 paper to version control, along with a job to ensure the pdf matches the tex file.
No description provided.