-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathtodo.txt
139 lines (125 loc) · 6.48 KB
/
todo.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
#TODO
* id-auto-generate in wrapper: set up objmapper._db_id_generator
* try external-id-generator: e.g. https://github.com/sony/sonyflake
* XXX depend on system time being monotonous, i.e. only increasing
* try internal uuid6.uuid7 : single node
* for multinode, overwrite some 8 bits with unique node-id (2^8=256 nodes)
- either .uuid7 over last 8 rand_b bits , or .uuid8 over whole subsec_b (8-bit, lsb)
* XXX probably also depend on system time being monotonous
* xtdb/v2:
+ query-builder
+ dbclient
+ rework rpc/dbclients: per-dbclient-specific (and not independent sym/kw/immutables)
- transit+json :
+ source-wise, seems slow. try speed up ?
+ few changes, some more impactful than others. broken caching
+ fix caching
+ fix UTF-8-encoding
+ fix json-stream:decode-piece-by-piece/restarting-cache
+ incremental json a.k.a "streaming-json" whitespace-delimited-multiple-top-values
+ writer is struct->str! while read/decode is ok=struct->struct
- test speed+size with/without cache X with/without gzip, for repeating things, and for non-repeating
- tests of transit-thing i/o as used+hacked in dbclient
- json-LD ??
###########
- objmapper.. maybe common ??
- save:
- update-or-create = default
- these constraints are possible :
-- create-no-update = ensure not-exists + put
-- update-no-create = ensure exist + put
-- with version-of-obj-optimistic-lock = match-exists-same-version+put
-- links/FK = ensure those exists
- id=some-autoid-if-not-there
- datomic: id treated as temporary, used to link objs and report real-id
- xtdb: id is the-real-id
see https://github.com/sony/sonyflake
- >> unit-of-work
- - topology-sort etc.. not needed
- needs schema
- can auto-make constraints e.g. uniq/inexisting-obj or pre-existing obj/links/FKs
- attr-declare/processing/conversions:
- schema
? pydantic ?
? marshmallow ? similar to django/sqlalchemy/.. class-level attrs : serlz/deserlz
XXX see ..../maybev3/openapi-swg/fastapi-code-generator/ ... all that is around pydantic
- refs
- to-one
- to-many
- constraints:
- requiredness:
- transaction-function rejecting data if xyz missing
- xtdb2: assert-exists
- uniqueness:
~ datomic: unique/ident , composite indexes etc
- xtdb: tx-func.. https://docs.xtdb.com/resources/faq/#uniqueconstraint
- xtdb2: assert-not-exists
- see unit-of-work above for automatics
type-assurance choices:
-A attributes carry the type:
e.g. just eav( x, :account/name, 'abc' ) alone infers x is account
+ simplest, fastest, nothing extra
- rely on all attr-names being 2-level and correct
- having x, no way to guess what it is
-B id-attributes carry the type - add extra 2-clause
eav( x, :thing-type/id) ANDed to other filters
e.g. .where( eav( x, :account/id), #, infers x is account
eav( x, :account/name, 'abc' ), ...)
~ reuse available data/attr but
- intermediate obj, needs to also have such :typename/id
- extra 2-clause/condition per filter
- rely on attr-name+data being correct+available (requiredness ?)
+ not rely on other attr-naming - can be any or without namespaces
- having x, tricky/ambiguous/no-way to guess what it is - may have multiple xxx/id
-C special common attribute carry the type - add extra attr + 3-clause
eav( x, :meta/type, thing-type) ANDed to other filters
e.g. .where( eav( x, :meta/type, 'account'), #, infers x is account
eav( x, :account/name, 'abc' ), ...)
- extra attr/data per obj
- extra 3-clause/condition per filter
+ not rely on other attr-naming - can be any or without namespaces
+ having x, can know what it is ; non-ambiguous
-B2 system db-id-attr carry the type
+ no extra attr/data per obj
- only possible in xt ; datomic always makes its own ids
- 2 extra 3-clause/condition per filter
- checking via pred_startswith ???
+ having x, can know what it is - the_id.split(..)[0] ; non-ambiguous
???? undecided
++?? maybe C - most clear&non-ambiguous ; non-user objects (intermediate m2m tables, separate components, etc) also needs it
attr-naming: depending on above type-assurance choices...
> A -> mandatory namespace=thing-type
B -> only id has mandatory namespace=thing-type
C -> anything ; but every obj has the special type-attr with special namespace)
++++ go for A: most limiting but most clear which-is-what, allows for later choice
https://forum.datomic.com/t/attribute-naming-conventions/835/8
https://clojurians-log.clojureverse.org/datomic/2020-04-01
'model the relations (attrs) , not the entities (tables)'
#########
PLAIN ATTRS: text numeric=int/float/? datetime bool enum:as-str-or-kw? uuid:(for ids/refs)
attr-features: nullable/required many (ordered?) is_identity auto-fill?
constraints: unique indexable?
RELATIONS: ref=link=pointer inside parent
features: many: o2o-o2m-m2o-m2m separate/ component/ composite=embedded
#separate/ component/ composite-features:
# - +da -xt + auto-retrieve/delete with parent
# + + - change-independently-of-parent
# + + - ?v2 search-independently-of-parent: find component-with-characteristics
# + + - +v2 parent-searchable: find-parent-which-has-component-with-characteristics
# - - + change-only-via-parent
# - - + inside-parent-history
# - ~+ + o2o
# - ~+ + o2m
# + ~- - m2o
# ~+ ~- - m2m
## components are separate, hence the o2m-etc needs extra constraints (tx-funcs?) for one-ness
## as long as link/s is inside parent, it's not fully symmetric m2m, change-wise
#XXX seems anything searchable needs be separate, and only marked as "component", for auto-retrieve/delete-with-parent etc
# mongo can index/search on inner stuff ; xtdb does not ; datomic has all separated except tuples
# history: therefore = history of entity + histories of all components-mentioned-anywhere-and-anytime
#XXX XXX XXX XXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
v2:
* has nested inner stuff - indexable and queryable.. so composites becomes usable
* has tables i.e. explicit "types"/namespaces
* thus it is closer to a document-db
# vim:ts=4:sw=4:expandtab