Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backwards compat for users running map reduce jobs that use riak_kv_counter #841

Merged
merged 3 commits into from
Feb 14, 2014

Conversation

russelldb
Copy link
Member

Also add similar convenience functions for 2.0+ data types to riak_kv_crdt.

targetted at Map reduce users, pass an object, get value. Avoids
us exposing the types.hrl to users. Also re-instate riak_kv_counter
but just for the `value` function, so as not to break old counter
mr jobs when we ship.
%%
%% -------------------------------------------------------------------

%% @doc Backwads compatibility with 1.4 counters `value' function as
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo "Backwards"

-spec value(riak_object:riak_object()) ->
integer().
value(RObj) ->
{{_Ctx, Count}, _Stats} = riak_kv_crdt:value(RObj, ?V1_COUNTER_TYPE),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this raise an error if the type is a 2.0 counter instead of a 1.4 counter? What happens after the upgrade completes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, you'll get a zero, like if you asked for a counter and it was a set, map, or regular riak_object

@seancribbs
Copy link
Contributor

So these are helpful, but I'd really love to see riak_kv_mapreduce wrappers around these so you can actually use them without writing your own functions. That said, it's a "nice-to-have" and can wait.

I made this module to test with:

-module(crdtmr).
-export([map_value/3]).
-include_lib("riak_kv/include/riak_kv_types.hrl").

map_value(RObj, <<"legacy_counter">>, _) ->
    [ riak_kv_counter:value(RObj) ];
map_value(RObj, _, <<"legacy_counter">>) ->
    [ riak_kv_counter:value(RObj) ];
map_value(RObj, _, _) ->
    {Type, Value} = riak_kv_crdt:value(RObj),
    [ riak_kv_crdt_json:fetch_response_to_json(Type, Value, undefined, ?MOD_MAP) ].

Here's the JSON of a sample job I sent from the Ruby client:

{"inputs":["maps","test"],"query":[{"map":{"language":"erlang","keep":true,"module":"crdtmr","function":"map_value"}}]}

...and here was the output:

[{"type"=>"map", "value"=>{"likes_counter"=>10, "friends_set"=>["Joe", "Justin"]}}]

Running over counters with the 'legacy_counter' arg turned on verified that for a non-legacy counter you'll just get 0:

irb(main):068:0> mr.query.first.arg = 'legacy_counter'
=> "legacy_counter"
irb(main):069:0> mr.run
=> [0]
irb(main):070:0> mr.query.first.arg = nil
=> nil
irb(main):071:0> mr.run
=> [{"type"=>"counter", "value"=>13}]

Other than the missing functionality, this works as advertised. 👍

@russelldb
Copy link
Member Author

I see what you mean (re MR wrappers.) The main aim here was to not break existing code, and provide similar support for migrating existing code to 2.0 data types. I'll try and add some funs to riak_kv_mapreduce as time permits though.

Thanks!

russelldb added a commit that referenced this pull request Feb 14, 2014
Backwards compat for users running map reduce jobs that use riak_kv_counter
@russelldb russelldb merged commit dee15e7 into develop Feb 14, 2014
@russelldb russelldb deleted the rdb/bug/counter-mr-backcompat branch February 14, 2014 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants