datastore: turn keys into arrays

After some discussion with @shesek, and my own usage, we agreed that a more comprehensive interface, which explicitly supports grouping, is desirable. Thus keys are now arrays, with the semantic that a key is either a parent or has a value, never both. For convenience in the JSON schema, we always return them as arrays, though we accept simple strings as arguments. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
ElementsProject · Aug 25, 2021 · fe86c11 · fe86c11
1 parent 533571a
commit fe86c11
Show file tree

Hide file tree

Showing 18 changed files with 561 additions and 319 deletions.
diff --git a/common/jsonrpc_errors.h b/common/jsonrpc_errors.h
@@ -92,6 +92,8 @@ static const errcode_t DATASTORE_DEL_WRONG_GENERATION = 1201;
 static const errcode_t DATASTORE_UPDATE_ALREADY_EXISTS = 1202;
 static const errcode_t DATASTORE_UPDATE_DOES_NOT_EXIST = 1203;
 static const errcode_t DATASTORE_UPDATE_WRONG_GENERATION = 1204;
+static const errcode_t DATASTORE_UPDATE_HAS_CHILDREN = 1205;
+static const errcode_t DATASTORE_UPDATE_NO_CHILDREN = 1206;
 
 /* Errors from wait* commands */
 static const errcode_t WAIT_TIMEOUT = 2000;

diff --git a/doc/lightning-datastore.7 b/doc/lightning-datastore.7
diff --git a/doc/lightning-datastore.7.md b/doc/lightning-datastore.7.md
@@ -12,8 +12,11 @@ DESCRIPTION
 The **datastore** RPC command allows plugins to store data in the
 c-lightning database, for later retrieval.
 
-There can only be one entry for each *key*, so prefixing with the
-plugin name (e.g. `summary.`) is recommended.
+*key* is an array of values (though a single value is treated as a
+one-element array), to form a heirarchy.  Using the first element of
+the key as the plugin name (e.g. `[ "summary" ]`) is recommended.
+A key can either have children or a value, never both: parents are
+created and removed automatically.
 
 *mode* is one of "must-create" (default, fails it it already exists),
 "must-replace" (fails it it doesn't already exist),
@@ -31,16 +34,19 @@ RETURN VALUE
 
 [comment]: # (GENERATE-FROM-SCHEMA-START)
 On success, an object is returned, containing:
-- **key** (string): The key which has been added to the datastore
-- **generation** (u64): The number of times this has been updated
-- **hex** (hex): The hex data which has been added to the datastore
+- **key** (array of strings):
+  - Part of the key added to the datastore
+- **generation** (u64, optional): The number of times this has been updated
+- **hex** (hex, optional): The hex data which has been added to the datastore
 - **string** (string, optional): The data as a string, if it's valid utf-8
 [comment]: # (GENERATE-FROM-SCHEMA-END)
 
 The following error codes may occur:
 - 1202: The key already exists (and mode said it must not)
 - 1203: The key does not exist (and mode said it must)
 - 1204: The generation was wrong (and generation was specified)
+- 1205: The key has children already.
+- 1206: One of the parents already exists with a value.
 - -32602: invalid parameters
 
 AUTHOR
@@ -58,4 +64,4 @@ RESOURCES
 
 Main web site: <https://github.com/ElementsProject/lightning>
 
-[comment]: # ( SHA256STAMP:0867f9910b75ef66e640a92aad55dbab7ce0b3278fd1fb200f91c2a1a6164409)
+[comment]: # ( SHA256STAMP:a66ad377a86e479704a3f5db06cffbd54bcd34fc9d36724649ace7b1f1e16bce)
diff --git a/doc/lightning-deldatastore.7 b/doc/lightning-deldatastore.7
diff --git a/doc/lightning-deldatastore.7.md b/doc/lightning-deldatastore.7.md
@@ -20,9 +20,10 @@ RETURN VALUE
 
 [comment]: # (GENERATE-FROM-SCHEMA-START)
 On success, an object is returned, containing:
-- **key** (string): The key which has been removed from the datastore
-- **generation** (u64): The number of times this has been updated
-- **hex** (hex): The hex data which has removed from the datastore
+- **key** (array of strings):
+  - Part of the key added to the datastore
+- **generation** (u64, optional): The number of times this has been updated
+- **hex** (hex, optional): The hex data which has removed from the datastore
 - **string** (string, optional): The data as a string, if it's valid utf-8
 [comment]: # (GENERATE-FROM-SCHEMA-END)
 
@@ -46,4 +47,4 @@ RESOURCES
 
 Main web site: <https://github.com/ElementsProject/lightning>
 
-[comment]: # ( SHA256STAMP:ca2b7b8f45b3ecd6332978599c803e38c4f80945119a777cb8ae346cbf063b10)
+[comment]: # ( SHA256STAMP:cd6f944965165b0e276da493592f9decb15046150367e06ff3e5b5547517d4b9)
diff --git a/doc/lightning-listdatastore.7 b/doc/lightning-listdatastore.7
diff --git a/doc/lightning-listdatastore.7.md b/doc/lightning-listdatastore.7.md
@@ -12,17 +12,18 @@ DESCRIPTION
 The **listdatastore** RPC command allows plugins to fetch data which was
 stored in the c-lightning database.
 
-All entries are returned in *key* isn't present; if *key* is present,
-zero or one entries are returned.
+All immediate children of the *key* (or root children) are returned:
+a *key* with children won't have a *hex* or *generation* entry.
 
 RETURN VALUE
 ------------
 
 [comment]: # (GENERATE-FROM-SCHEMA-START)
 On success, an object containing **datastore** is returned.  It is an array of objects, where each object contains:
-- **key** (string): The key which from the datastore
-- **generation** (u64): The number of times this has been updated
-- **hex** (hex): The hex data from the datastore
+- **key** (array of strings):
+  - Part of the key added to the datastore
+- **generation** (u64, optional): The number of times this has been updated
+- **hex** (hex, optional): The hex data from the datastore
 - **string** (string, optional): The data as a string, if it's valid utf-8
 [comment]: # (GENERATE-FROM-SCHEMA-END)
 
@@ -44,4 +45,4 @@ RESOURCES
 
 Main web site: <https://github.com/ElementsProject/lightning>
 
-[comment]: # ( SHA256STAMP:a6503e3d2da8f9a35a0d461b5b93248f3fea306371ad62f98df613efea51959d)
+[comment]: # ( SHA256STAMP:ee29b53cad20c6dfe9e19a979816280cc9f778507e5638d1803418284125e4c1)
diff --git a/doc/schemas/datastore.schema.json b/doc/schemas/datastore.schema.json
@@ -2,12 +2,15 @@
     "$schema": "http://json-schema.org/draft-07/schema#",
     "type": "object",
     "additionalProperties": false,
-    "required": [ "key", "hex", "generation" ],
+    "required": [ "key" ],
     "properties": {
         "key": {
-            "type": "string",
-            "description": "The key which has been added to the datastore"
-        },
+	    "type": "array",
+	    "items": {
+		"type": "string",
+		"description": "Part of the key added to the datastore"
+	    }
+	},
         "generation": {
             "type": "u64",
             "description": "The number of times this has been updated"

diff --git a/doc/schemas/deldatastore.schema.json b/doc/schemas/deldatastore.schema.json
@@ -2,12 +2,15 @@
     "$schema": "http://json-schema.org/draft-07/schema#",
     "type": "object",
     "additionalProperties": false,
-    "required": [ "key", "hex", "generation" ],
+    "required": [ "key" ],
     "properties": {
         "key": {
-            "type": "string",
-            "description": "The key which has been removed from the datastore"
-        },
+	    "type": "array",
+	    "items": {
+		"type": "string",
+		"description": "Part of the key added to the datastore"
+	    }
+	},
         "generation": {
             "type": "u64",
             "description": "The number of times this has been updated"

diff --git a/doc/schemas/listdatastore.schema.json b/doc/schemas/listdatastore.schema.json
@@ -9,11 +9,14 @@
 	    "items": {
 		"type": "object",
 		"additionalProperties": false,
-		"required": [ "key", "hex", "generation" ],
+		"required": [ "key" ],
 		"properties": {
 		    "key": {
-			"type": "string",
-			"description": "The key which from the datastore"
+			"type": "array",
+			"items": {
+			    "type": "string",
+			    "description": "Part of the key added to the datastore"
+			}
 		    },
 		    "generation": {
 			"type": "u64",