Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validate consistent bucket type properties #747

Closed
wants to merge 1 commit into from

Conversation

jrwest
Copy link
Contributor

@jrwest jrwest commented Dec 5, 2013

cc/ @jtuple @russelldb

A bucket type that is consistent must remain so (the property is
immutable) and the n_val of the type may not be changed (for the type
or its buckets). A type that has consistent=true is consisdered to be
strongly-cosnsitent regardless of the values of other properties
including the datatype property -- strong-consistency may use this
property for its own purposes in the future. Untyped buckets (default
bucket type) cannot be strongly consistent.

The eqc tests have been updated to reflect the changes.

Other Notes:

  • The consistent property is validated in two paths depending on if the
    type is consistent or not. This is a bit meh.
  • It is assumed that the value of allow_mult does not matter to strong
    consistency and thus it is not validated (besides being a valid boolean)
    for this path
  • The datatype property is not validated if the type is strongly-cosnsitent.
    This is because, what the property will mean is still undefined.
  • I may have butchered prop_merges() a bit (the most at least). As validation
    grows we may want to re-think how we build the expected set in the property

A bucket type that is consistent must remain so (the property is
immutable) and the n_val of the type may not be changed (for the type
or its buckets). A type that has consistent=true is consisdered to be
strongly-cosnsitent regardless of the values of other properties
including the datatype property -- strong-consistency may use this
property for its own purposes in the future. Untyped buckets (default
bucket type) cannot be strongly consistent.

The eqc tests have been updated to reflect the changes.

Other Notes:
* The consistent property is validated in two paths depending on if the
  type is consistent or not. This is a bit meh.
* It is assumed that the value of allow_mult does not matter to strong
  consistency and thus it is not validated (besides being a valid boolean)
  for this path
* The datatype property is not validated if the type is strongly-cosnsitent.
  This is because, what the property will mean is still undefined.
* I may have butchered prop_merges() a bit (the most at least). As validation
  grows we may want to re-think how we build the expected set in the property
@jrwest
Copy link
Contributor Author

jrwest commented Dec 5, 2013

Also, should we just merge #667 into this PR?

NValChanged = n_val_changed(Existing, New),
IsConsistent = is_consistent(Existing),
NewConsistent = proplists:get_value(consistent, New),
Expected = case {DefaultBucket, HasAllowMult, AllowMult, HasDatatype,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, this is quite the truth table now. I know it is my fault for structuring it that way. Oh boy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea I know. I made it pretty bad. As we generate more properties it will continue to get worse too. I had a few thoughts on how to fix but none are really concrete. We need a better way of determining what should be in the Expected set, but most soln's I can come up w/ mirror most of the validation or other properties. I wonder if we could instead know ahead of time which properties we generated are invalid (or we expect to be invalid rather) and then building the Expected set is just a matter of removing the invalid entries the generator told us about...not sure how clear that is...thoughts?

@russelldb
Copy link
Member

Tests pass, and it seems to work with simple tests by hand. We need to add an issue to refactor that EQC test. It can probably even happen after the freeze since it is a test. Imma 👍

@russelldb
Copy link
Member

And I agree, we should merge in #667, too.

@jrwest
Copy link
Contributor Author

jrwest commented Dec 5, 2013

@jtuple would you mind taking a look still?

@russelldb thanks! issue opened: #750. I'll merge #667 into this PR and test before merging.

@jtuple
Copy link
Contributor

jtuple commented Dec 18, 2013

Since @russelldb already did a more detailed review, I just built this branch and played around with things as a user would. As far as I can tell, things work as they should.

I'm still not sure if data types should or shouldn't be mutable on a consistent bucket. I need to revisit my consistent data types branch soon and determine if it gracefully handles mixed types in a bucket / to the same key. But, that's outside the scope of this pull-request.

I did notice an issue when passing in a mispelled datatype that is unrelated to this PR. Get a pretty lame/nasty error message when trying to create unknown atom. But, not sure we can easily fix this; and imagine it's a general problem with how we erlify JSON. Eg.

./dev/dev1/bin/riak-admin bucket-type update strong '{"props": {"datatype": "setz"}}' 
RPC to 'dev1@127.0.0.1' failed: {'EXIT',
                                 {badarg,
                                  [{erlang,binary_to_existing_atom,
                                    [<<"setz">>,utf8],
                                    []},
                                   {riak_kv_wm_utils,erlify_bucket_prop,1,
                                    [{file,"src/riak_kv_wm_utils.erl"},
                                     {line,360}]},
                                   {riak_kv_console,
                                    '-bucket_type_update/2-lc$^0/1-0-',1,
                                    [{file,"src/riak_kv_console.erl"},
                                     {line,506}]},
                                   {riak_kv_console,bucket_type_update,2,
                                    [{file,"src/riak_kv_console.erl"},
                                     {line,506}]},
                                   {rpc,'-handle_call_call/6-fun-0-',5,
                                    [{file,"rpc.erl"},{line,205}]}]}}

In any case, +1 from me.

@jrwest
Copy link
Contributor Author

jrwest commented Dec 18, 2013

@jtuple yea I didn't try to do anything around datatype when consistent=true really. as for the error you found, that was another issue i hoped to address by the command line changes that I unfortunately did not get to. Will merge this with #667 as part of it as promised shortly.

@jrwest
Copy link
Contributor Author

jrwest commented Dec 19, 2013

closing for rebased/merged version in #770

@jrwest jrwest closed this Dec 19, 2013
@seancribbs seancribbs deleted the jrw-sc-btype-validation branch April 1, 2015 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants