Create a property service

[allenxwang]

Goals

The purpose of having a service for properties are the following

Enable the dynamic nature of the properties

Once we have a standard service for properties, we can create a standard PolledConfigurationSource on the client side to get all the properties, which can be changed dynamically on the server.

Hide persistence details and rely on API

So that you can switch persistence store without affecting clients

Provide property management functionalities

We are aiming to support these functionalities initially

• Pluggable persistence, with implementation on Cassandra using Astyanax
• Life cycle management
• Validation
• Query

A set of REST APIs as well as some UI will be provided.

Support scopes and property filtering based on scopes

With the service, you can create properties with the same name but different scopes. A client can query the service to get the properties for certain scopes. For example, there can be properties with the same name, but different values for different applications. One application can query the service to get the property for that specific application.

Architecture

The service will be a web application that manages the properties as resources. It is a Jersey based RESTful service. Configuration and wiring of implementation specific objects will be done via dependency injection with Guice.

Property Model

The following is the proposed Java model for Property

public class Property {
     // an ID associated with persistence store
     private String propertyId;
     // property name observed from client side
     private String key; 
     private String value;
     // validation rule, can be null if none or not supported
     private String constraints;
     private String description;
     private long lastUpdated;

     // attribute name/value pair like region=us-east-1
     // affect selection or filtering of properties.
     private Map<String, String> scopes;

     // attributes that are related to the server side 
     // operation of the property which do not have any effect
     // usage on the client side
     private Map<String, String> operationalAttributes;

     // setters and getters ...    
}

The following interfaces defines some meta data and operational attributes which are implementation specific:

public interface PropertyMetaData {
    /**
     * Return list of supported scopes for the property,
     * for example, "region", "env", "serverId"
     */ 
    public List<String> getSupportedScopes();

    /**
     * List of supported operational attributes,
     * for example, "lastUpdatedBy"
     */
    public List<String> getSupportedOperationalAttributes();
}

Data Access

The following is the interface for data access for the persistence store.

public interface PropertiesDAO {
    public List<Property> getAllProperties() throws Exception;

    public Property getProperty(String id) throws Exception;

    /**
     * @param skipPropWithNonexistCriteria whether we should skip a property if a specified scope 
     *          in the query does not apply to the property
     */
    public List<Property> queryProperties(Map<String, Collection<String>> scopes, boolean skipPropWithNonexistCriteria);

    public void createOrUpdateProperty(Property property, Map<String, Object> operationalAttributes) throws Exception;

    public void deleteProperty(String id, Map<String, Object> operationalAttributes) throws Exception;

    public void init() throws Exception;

    public void shutDown() throws Exception;

    public void updateJournal(Property property, Constants.Action action, 
            Map<String, Object> operationalAttributes) throws Exception;

    public boolean propertiesChangedSince(long timeStamp);

    // get the time stamp any property is last updated in the persistence store
    public long getLastModified();
}

Some implementation loads all properties from persistence into memory. Additional functionalities are required for such implementation like refresh and keeping a certain level of data consistency. These functionalities will be defined in additional class/interface.

REST API

The REST API should support creation, retrieval, query/search, update and deletion of property.

The response code follows this convention:

200 - request is successful
400 - malformed request
404 - requested property does not exist (in retrieval, update and deletion)
405 - trying to update a field of property that is supposed to immutable
409 - In creation, property with the same key and scopes already exist
500 - unexpected server side exception occurred

Creation

POST <path>/property

Body of the POST request will be the serialized version of Property.

Response payload is the serialized version of the final property created, including property ID.

Example of request payload:

{"key": "foo", "value": "bar", 
 "description": "this property is used to determine behavior of foo",
 "scopes": {"region":"us-east-1", "env":"prod"}, 
 "operationalAttributes":{"lastUpdatedBy": "allen", "source":"asgard", "ticket": "JIRA12345"},
}

Example of response payload:

{"propertyId": "1234567",
 "key": "foo", "value": "bar", 
 "description": "this property is used to determine behavior of foo",
 "scopes": {"region":"us-east-1", "env":"prod"}, 
 "operationalAttributes":{"lastUpdatedBy": "allen", "source":"asgard", "ticket": "JIRA12345"},
 "lastUpdated":"2013-02-04T23:34:02.968Z"
}

Update

Once a property is created, only the following fields allow update:

• value
• constraint
• description
• operational attributes

Scopes should not be updated. Instead, new property should be created with the desired new scope and the old property can be deleted if the scopes are no longer applicable.

The HTTP Verb for update should ideally be "PATCH". However, since it just gets standardized and there is limited support, we are going to use "POST" instead.

POST <path>/property?propertyId=<id>

An example of UPDATE payload:

{ "value": "fooAndBar", 
 "description": "this property is used to determine behavior of foo and bar",
 "operationalAttributes":{"lastUpdatedBy": "allen", "source":"asgard", "ticket": "JIRA1234567"}
}

An example of UPDATE response:

{"propertyId": "1234567",
 "key": "foo", "value": "fooAndBar", 
 "description": "this property is used to determine behavior of foo and bar",
 "scopes": {"region":"us-east-1", "env":"prod"}, 
 "operationalAttributes":{"lastUpdatedBy": "allen", "source":"asgard", "ticket": "JIRA234567"},
 "lastUpdated":"2013-03-05T20:12:02.000Z"
}

Retrieval of a single property

GET <path>/property?id=<id>

Instead of put the property ID as part of path, it is added as a query parameter. The reason is that the ID may contain path character, for example "/", which will confuse the service. On the other hand, using URL encoded query parameter has better interoperability.

The payload of response will include the property.

Query of properties

This REST API is used to return a list of properties that matches scopes specified in the request.

GET <path>/query?<scopes with acceptable values>

For example

GET /REST/query?region=us-east-1&region=us-west-2&env=prod&appId=movieservice&appId=

The query looks for properties that matches the following criteria:

• application is "movieservice" or nonexistent
• region is either us-east-1 or us-west-2
• environment is prod

Here is an example of response payload:

[
{"propertyId": "1234567",
 "key": "foo", "value": "fooAndBar", 
 "description": "this property is used to determine behavior of foo and bar",
 "scopes": {"region":"us-east-1", "env":"prod"}, 
 "operationalAttributes":{"lastUpdatedBy": "allen", "source":"asgard", "ticket": "JIRA234567"},
 "lastUpdated":"2013-03-05T20:12:02.000Z"
},
// other properties
]

Deletion

DELETE <path>/property?id=<id>&<operational attributes>

There is no response payload. The response code indicates the result.

[floer32]

+1

[martijnburger]

+1

[newdha]

+1

[rgallardo-netflix]

The team has no bandwidth to try to implement this.