Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic AAE Status #654

Merged
merged 7 commits into from
Sep 7, 2013
Merged

Generic AAE Status #654

merged 7 commits into from
Sep 7, 2013

Conversation

rzezeski
Copy link
Contributor

@rzezeski rzezeski commented Sep 6, 2013

This is replacing #596. It's been rebased against develop.

This pull request makes AAE entropy info parameterized on type. This allows both KV and Yokozuna to share almost all the same status code without copy pasta.

  • Add a Type to entropy info. It represents the type of AAE being performed. Currently riak_kv or yokozuna but could be other things in the future.
  • Export the get_build_time function as it can be reused by Yokozuna.
  • Parameterize the module and function name used to compute all exchanges. KV has inter-partition exchange whereas Yokozuna is purely intra. I.e. Yokozuna exchange is always between the same partition, different services. By parameterizing the "all exchanges" calculation can let different services have different exchange patterns.

Adding a type namespace to entropy info allows me to reuse it for
Yokozuna AAE status as well as future uses of AAE.
This function can be reused by Yokozuna.
The exchanges that need to take place in Yokozuna are not the same as
those in KV.
@rzezeski rzezeski mentioned this pull request Sep 6, 2013
Fix the filtering logic which removes AAE info for indices no longer
owned by the local node.  The key is now `{Type, Index}`.
Others = ordsets:subtract(ordsets:from_list(Indices),
ordsets:from_list(Primaries)),
[ets:delete(?ETS, {index, Idx}) || Idx <- Others],
[ets:match_delete(?ETS, {{index, {'_', Idx}}, '_'}) || Idx <- Others],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moving comments from #596 (comment) here:

I'm fine w/ the quick fix for now. I think we should open an issue as well since the table size does seem related to ring size (number of partitions owned by a single node). May not be that much of a hit vs. calling multiple deletes but since we did the large ring work we should be careful.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add functions which provide a default `Type` so that the riak_kv calls
can stay as they currently are.
@jrwest
Copy link
Contributor

jrwest commented Sep 6, 2013

Changes look good. As mentioned in chat I'm +1 on this with the caveat that I think issues should be opened to export tree_built/2 and to change to using ets:delete over match_delete (as pre2 issues). I didn't get a chance to run it with the changes but I can do that later today if we want to hold off a little longer on the merge.

@@ -284,6 +284,7 @@ next_state_with_timeout(StateName, State, Timeout) ->
{next_state, StateName, State, Timeout}.

exchange_complete({LocalIdx, _}, {RemoteIdx, RemoteNode}, IndexN, Repaired) ->
riak_kv_entropy_info:exchange_complete(LocalIdx, RemoteIdx, IndexN, Repaired),
riak_kv_entropy_info:exchange_complete(LocalIdx, RemoteIdx,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nitpick but do we even need this file in the patch now?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🐟

coderoshi and others added 2 commits September 6, 2013 11:39
Use default type of `riak_kv` when calling `tree_built/2`.
rzezeski added a commit that referenced this pull request Sep 7, 2013
@rzezeski rzezeski merged commit 22f0a70 into develop Sep 7, 2013
@seancribbs seancribbs deleted the feature/rz/generic-aae-status branch April 1, 2015 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants