MDC4 Meetings 2014 W36

Wednesday September 3rd, 2014

Present: Kresten, Jon, Rune

Kresten/Rune completed first round of implementation of first two lumps, based on Joe's prototype of pipelining. Minimal change implementation, single fold instead of n and switching strategy where start off with gets/random reads.
Also have function on hash tree that estimates size of tree. By sampling first 1000 segments/objects. Gives us withing a few percentage points.
Rune has run AAE tests against develop branch and against Kresten's branch and pass tests.
Branch started today jdb-pipeline-aae-fullsync-wip on core/kv and krab-fullsync-wip
Next step is more testing.

Jon reset Kresten's password. Jon added Rune and Kresten to jira-users and jira-developers.
Machines r2s10.bos1 and r2s11.bos1 allocated to project. They need to be re-imaged and will be released to us soon.
Email out to Steve/Paul on testing information
Requested help for Paul H from MVM, says it would be his third priority and wouldn't be able to start for a few days.

Turns out Joe was already working on L2 and has an implementation, email pending from him on that later today.
Joe was planning to extend rtcloud to support provisioning in multiple AZs to support repl testing.

Have the first 'sprint' set up to run for 2.0.1. Jon went through the JIRA board with Rune, and he's going to start focussing there.
Joe sent fullsync status email - has pushed his branch as well as Krestens. He ran microbenchmarks, but the key sizes were way too small so only showed how awful old AAE fullsync was rather than how good the AAE changes are. He will be running larger tests today on EC2.
Rune lost the day to production issues with user on 1.4.2 having entropy drift.
Kresten has been off hacking quietly somewhere.
Micah is wrapping up coverage work before starting on realtime balance.
Jon still writing up fullsync/riak_net between meetings.
Test hardware r2s10-11.bos1 being reprovisioned today, PaulH sending VPN instructions to Kresten/Rune.

For the L2.5 testing, we discussed

making sure volumes were large
that we ran fullsync with realtime operating
adding 0.1%/0.01% differences to more accurately represent realtime being run
making a distinction between keys that are different on both sides, vs present on only one side.

None new.

None new.

Mikael Lixenstrand from ESL will be joining the project as additional resource. He's got a good Erlang background and has been working on presales and proserve work out of EMEA, so this gives him a chance to learn more of Riak and accelerate things.
Rune tied up on support issue. Hoping to get some fullsync runs in over the weekend.
Kresten out.
JoeB ran into issues with AWS when converting rtcloud to run across two Amazon regions. Has a workaround that should let him benchmark soon, but sends disterl traffic over the open internet, so undesirable long term.
Micah expected to start on the realtime balancing work next week.
Jon tied up with other projects/meetings. Still working to get riak_net and independent fullsync write-ups completed.