-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path_notes.txt
312 lines (205 loc) · 11.9 KB
/
_notes.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
Giant Killing with Beanstalkd
https://www.sitepoint.com/giant-killing-with-beanstalkd/
Getting Started with Beanstalkd
https://lornajane.net/posts/2014/getting-started-with-beanstalkd
beanstalkd
https://beanstalkd.github.io/
https://github.com/beanstalkd/beanstalkd
Protocol
https://github.com/beanstalkd/beanstalkd/blob/master/doc/protocol.txt
https://github.com/iFaceless/beanstalkc-rust/blob/master/protocol.md
Tick Tock on Beanstalkd Message Queues
https://www.todaysoftmag.com/article/791/tick-tock-on-beanstalkd-message-queues
Simple and easy task queue - beanstalkd.
https://www.cnblogs.com/immaxfang/p/16503956.html - flow chart!!!
beanstalk - based on article above ++++
https://geraldo.dev/til/beanstalk.html +++++ !!!!!! the same flow chart
Scalable Work Queues with Beanstalk
https://www.igvita.com/2010/05/20/scalable-work-queues-with-beanstalk/
Client design (PHP)
https://github.com/pheanstalk/pheanstalk/blob/v5/doc/library-design.md
Manager - gather information about the server and its jobs
Publisher - publishing jobs
Subscriber - listening to tubes and reserving / managing the jobs in them
Commands
https://github.com/pheanstalk/pheanstalk/blob/v5/doc/library-design.md#commands
Client Libraries
https://github.com/beanstalkd/beanstalkd/wiki/Client-Libraries
GO -
https://github.com/beanstalkd/go-beanstalk - !!! Check implementation !!!
uses https://cs.opensource.google/go/go/+/refs/tags/go1.23.3:src/net/textproto/
https://github.com/getjackd/go-jackd Go latest???
https://github.com/getjackd/go-jackd/blob/master/main.go - !!! Check implementation !!!
https://github.com/iFaceless/beanstalkc-rust?tab=readme-ov-file
https://github.com/earl/beanstalkc PY
https://github.com/bergundy/evbsc
https://github.com/bergundy/libbeanstalkclient
https://github.com/deepfryed/beanstalk-client +++
https://github.com/pheanstalk/pheanstalk PHP
https://github.com/pheanstalk/pheanstalk/tree/v5 PHP
Beanstalkd vs Gearman: What are the differences?
## Key Differences Between Beanstalkd and Gearman
1. **Functionality**:
Beanstalkd is a simple, fast work queue
whereas Gearman is a distributed job queuing system.
Beanstalkd focuses on providing a straightforward queuing mechanism,
while Gearman allows for more complex distributed job processing scenarios.
2. **Protocol Support**:
Beanstalkd uses its own binary TCP protocol, making it lightweight and fast,
while Gearman supports multiple protocols like TCP, UDP, and SSL. This difference affects the compatibility and flexibility of the two systems in various network environments.
3. **Job Dependencies**:
Beanstalkd does not provide native support for job dependencies, meaning jobs cannot be set to run based on the completion of other jobs.
On the other hand, Gearman allows for job dependencies, enabling more intricate job workflows and processing logic.
4. **Ease of Use**:
Beanstalkd is known for its simplicity and ease of use with a minimalistic feature set, making it ideal for straightforward queuing tasks.
Gearman, being more feature-rich and complex, may require more time and effort to set up and configure for advanced job processing needs.
5. **Community Support**:
Gearman, being around longer and having a larger user base, has more extensive community support, documentation, and plugins available compared to Beanstalkd. This can be a significant factor to consider when choosing between the two systems.
6. **Scalability**:
While both Beanstalkd and Gearman support horizontal scalability by adding more servers,
Gearman's distributed nature allows for easier scaling across multiple machines and clusters. This scalability aspect can be crucial for applications with high processing demands.
In Summary, Beanstalkd and Gearman offer distinct functionalities and features, such as simplicity vs. complexity, protocol support, job dependencies, ease of use, community support, and scalability.
Alternatives
Gearman
Celery
NSQ - Go
Disque - => redis ?
Chronicle Queue - Java
https://taskqueues.com/
Packed struct too big / uninitialized
https://ziggit.dev/t/packed-struct-too-big-uninitialized/6752
Zig Playground
https://playground.zigtools.org/
Submodules
https://github.com/beanstalkd/beanstalkd
https://github.com/deepfryed/beanstalk-client
https://github.com/bergundy/evbsc
https://github.com/bergundy/libbeanstalkclient
git submodule add https://github.com/beanstalkd/beanstalkd src/submodules/beanstalkd
git submodule add https://github.com/deepfryed/beanstalk-client src/submodules/beanstalk-client
git submodule add https://github.com/bergundy/evbsc src/submodules/evbsc
git submodule add https://github.com/bergundy/libbeanstalkclient src/submodules/libbeanstalkclient
git rm src/submodules/TBD
Zig & JSON
https://www.openmymind.net/Reading-A-Json-Config-In-Zig/
https://www.reddit.com/r/Zig/comments/1e1pqrg/is_it_possible_to_parse_a_json_object_to_a_map/
https://www.reddit.com/r/Zig/comments/1bignpf/json_serialization_and_taggeddiscrimated_unions/
https://zigbin.io/651078
https://www.openmymind.net/Zigs-std-json-Parsed/
Translate-C
https://zig.guide/master/working-with-c/translate-c
https://fig.io/manual/zig/translate-c
Reading/writing strings from binary data
https://ziggit.dev/t/reading-writing-strings-from-binary-data/1159/6?u=g41797
Difference between struct level align(x) and field level align(x)
https://ziggit.dev/t/difference-between-struct-level-align-x-and-field-level-align-x/3082/1
Packed structs in Zig make bit/flag sets trivial
https://devlog.hexops.com/2022/packed-structs-in-zig/
Pointer arithmetic
https://github.com/andrewCodeDev/random_access
A Programmer-Friendly I/O Abstraction Over io_uring and kqueue
https://tigerbeetle.com/blog/a-friendly-abstraction-over-iouring-and-kqueue
Cross-platform event loop
https://github.com/mitchellh/libxev
What is io_uring?
https://unixism.net/loti/what_is_io_uring.html
https://github.com/saltzm/async_io_uring
https://github.com/Cloudef/zig-aio
https://developers.redhat.com/articles/2023/04/12/why-you-should-use-iouring-network-io
TCP Server in Zig - Part 1 - Single Threaded
https://www.openmymind.net/TCP-Server-In-Zig-Part-1-Single-Threaded/
TCP Server in Zig - Part 2 - Message Boundaries
https://www.openmymind.net/TCP-Server-In-Zig-Part-2-Message-Boundaries/
TCP Server in Zig - Part 3 - Minimizing Writes & Reads
https://www.openmymind.net/TCP-Server-In-Zig-Part-3-Minimizing-Writes-and-Reads/
TCP Server in Zig - Part 4 - Multithreading
https://www.openmymind.net/TCP-Server-In-Zig-Part-4-Multithreading/
An HTTP/1.1 server for zig
https://github.com/karlseguin/http.zig
Example of the parsing
https://github.com/karlseguin/http.zig/blob/63c317cdf86a04a56cc185a5ad789321d9c025c4/src/request.zig#L650
Server-side framework for Zig, relying heavily on dependency injection.
https://github.com/cztomsik/tokamak
Writing a task scheduler in Zig
https://www.openmymind.net/Writing-a-Task-Scheduler-in-Zig/
Length of zero terminated string
const c_string: [*:0]const u8 = "hello";
const length = std.mem.len(c_string);
std.debug.print("length {d} sentinel {d} \n", .{length, c_string[length]});
Output: length 5 sentinel 0
Generate docs
cd ./dgen
/home/g41797/Devs/doxygen-1.12.0/bin/doxygen
GitHub-Flavored Markdown
https://www.freecodecamp.org/news/github-flavored-markdown-syntax-examples/
GitHub Wiki
https://www.freecodecamp.org/news/what-is-github-wiki-and-how-do-you-use-it/
HTTP Zig repos
https://github.com/ducdetronquito/http
https://github.com/ducdetronquito/requestz/
https://github.com/ducdetronquito/h11
A I/O agnostic HTTP/1.1 parser and encoder for Zig.
https://github.com/truemedian/hzzp
An interpreter written in Zig for the book Crafting Interpreters
https://github.com/eikooc/lox-interpreter-zig
A zig library to manipulate gemini text files
https://github.com/ikskuh/zig-gemtext
Parser Combinators
https://devlog.hexops.com/2021/zig-parser-combinators-and-why-theyre-awesome/
A Http server written in Zig
https://github.com/frmdstryr/zhp
Go generic queue - beanstalkd, nats, etc
https://github.com/zeromicro/go-queue
Delayed queue
https://medium.com/codex/delayed-queue-a-message-queue-with-delay-feature-268c8647c2fd
Docker-compose
https://github.com/pheanstalk/pheanstalk/blob/v5/docker-compose.yml
Docker integration
https://github.com/g41797/queue-beanstalkd/tree/master/docker
CI
https://github.com/g41797/queue-beanstalkd/blob/master/.github/workflows/tests.yml
Comparing Beanstalkd, IronMQ and Amazon SQS
https://www.sitepoint.com/message-queues-comparing-beanstalkd-ironmq-amazon-sqs/
Glossary
When working with queues you may come across these terms:
Bury (a job) – puts a job in a failed state. The job cannot be reprocessed until it is manually kicked back into the queue. Not supported by IronMQ and SQS.
Consumer – see Worker.
Delay – defer a job from being sent to a worker for a predetermined amount of time.
Delete (a job) – see Dequeue.
Dequeue – marks a job as completed and removes it from the queue.
Enqueue – adds a job to a queue ready for a worker.
FIFO – describes the way jobs are handled in a queue as First In, First Out. This is the most common type of message queue.
FILO – describes the way jobs are handled in a queue as First In, Last Out.
Job – a deferred task in a queue containing metadata to identify what task is waiting to be processed. Akin to database rows.
Kick (a job) – returns a previously buried job to the queue ready for workers to pick up. Not supported by IronMQ and SQS.
Provider – a client which connects to the message server to create jobs.
Queue – a way to group similar jobs into a queue. Akin to database tables.
Reserve (a job) – delivers a job to a worker and locks it from being delivered to another worker.
Worker – a client which connects to the message server to reserve, delete and bury jobs. These perform the labour intensive part of the processing.
Terminology
Some words are used differently between Beanstalkd and Amazon SQS. There’s a quick list of translations:
Beanstalkd Amazon SQS IronMQ
Tube Queue Queue
Job Message Message
Job data Message body Message body
Put Send message POST
Reserve Receive message GET
Delete Delete message DELETE
TTR (time-to-run) Visibility timeout Timeout
Delay Delivery delay Delay
– Retention Period Expires in
basic-tcp-chat.zig
using net.StreamServer
https://gist.github.com/andrewrk/34c21bdc1600b0884a3ab9fa9aa485b8
Build System Tricks
https://ziggit.dev/t/build-system-tricks/3531
Check connection
sudo netstat --tcp --programs --numeric|grep 11300
Zig callbacks
https://ziggit.dev/t/how-to-use-callbacks-with-run-time-arguments/2449
https://stackoverflow.com/questions/74251650/how-to-pass-zig-function-pointers-to-other-functions
https://ziglang.org/documentation/master/#Functions
Pneumatic tube
https://en.wikipedia.org/wiki/Pneumatic_tube
Pneumatic Networks
http://douglas-self.com/MUSEUM/COMMS/pneumess/pneumess.htm