Skip to content

Commit

Permalink
Make major improvements to redbean and libraries
Browse files Browse the repository at this point in the history
The most exciting improvement is dynamic pages will soon be able to use
the executable itself as an object store. it required a heroic technique
for overcoming ETXTBSY restrictions which lets us open the executable in
read/write mode, which means (1) wa can restore the APE header, and (2)
we can potentially containerize redbean extension code so that modules
you download for your redbean online will only impact your redbean.

Here's a list of breaking changes to redbean:

- Remove /tool/net/ prefix from magic ZIP paths
- GetHeader() now returns NIL if header is absent

Here's a list of fixes and enhancements to redbean:

- Support 64-bit ZIP archives
- Record User-Agent header in logs
- Add twelve error handlers to accept()
- Display octal st_mode on listing page
- Show ZIP file comments on listing page
- Restore APE MZ header on redbean startup
- Track request count on redbean index page
- Report server uptime on redbean index page
- Don't bind server socket using SO_REUSEPORT
- Fix #151 where Lua LoadAsset() could free twice
- Report rusage accounting when workers exit w/ -vv
- Use ZIP iattr field as text/plain vs. binary hint
- Add ParseUrl() API for parsing things like a.href
- Add ParseParams() API for parsing HTTP POST bodies
- Add IsAcceptablePath() API for checking dots, etc.
- Add IsValidHttpToken() API for validating sane ASCII
- Add IsAcceptableHostPort() for validating HOST[:PORT]
- Send 400 response to HTTP/1.1 requests without a Host
- Send 403 response if ZIP or file isn't other readable
- Add virtual hosting that tries prepending Host to path
- Route requests based on Host in Request-URI if present
- Host routing will attempt to remove or add the www. prefix
- Sign-extend UNIX timestamps and don't adjust FileTime zone

Here's some of the improvements made to Cosmopolitan Libc:

- Fix ape.S indentation
- Improve consts.sh magnums
- Write pretty good URL parser
- Improve rusage accounting apis
- Bring mremap() closer to working
- Added ZIP APIs which will change
- Check for overflow in reallocarray()
- Remove overly fancy linkage in strerror()
- Fix GDB attach on crash w/ OpenBSD msyscall()
- Make sigqueue() portable to most UNIX distros
- Make integer serialization macros more elegant
- Bring back 34x tprecode8to16() performance boost
- Make malloc() more resilient to absurdly large sizes
  • Loading branch information
jart committed Apr 18, 2021
1 parent 69c5087 commit bf03b2e
Showing 307 changed files with 4,557 additions and 2,581 deletions.
78 changes: 39 additions & 39 deletions ape/ape.S
Original file line number Diff line number Diff line change
@@ -865,48 +865,48 @@ ape_macho:
@see "The Portable Executable File Format from Top to Bottom",
Randy Kath, Microsoft Developer Network Technology Group. */

// ┌14:Uniprocessor Machine ┌─────────────────────────┐
// │┌13:DLL │ PE File Characteristics │
// ││┌12:System ├─────────────────────────┤
// │││┌11:If Net Run From Swap │ r │ reserved │
// ││││┌10:If Removable Run From Swap │ d │ deprecated │
// │││││┌9:Debug Stripped │ D │ deprecated with │
// ││││││┌8:32bit Machine │ │ extreme prejudice │
// │││││││ ┌5:Large Address Aware └───┴─────────────────────┘
// │││││││ │ ┌1:Executable
// │││││││ │ │┌0:Relocs Stripped
// d│││││││dr│Ddd││
PEEXE = 0b0000001000100011

// ┌15:TERMINAL_SERVER_AWARE ┌─────────────────────────┐
// │┌14:GUARD_CF │ PE DLL Characteristics │
// ││┌13:WDM_DRIVER ├─────────────────────────┤
// │││┌12:APPCONTAINER │ r │ reserved │
// ││││┌11:NO_BIND └───┴─────────────────────┘
// │││││┌10:NO_SEH
// ││││││┌9:NO_ISOLATION
// │││││││┌8:NX_COMPAT
// ││││││││┌7:FORCE_INTEGRITY
// │││││││││┌6:DYNAMIC_BASE
// ││││││││││┌5:HIGH_ENTROPY_VA
// │││││││││││rrrrr
// ┌14:Uniprocessor Machine ┌─────────────────────────┐
// │┌13:DLL │ PE File Characteristics │
// ││┌12:System ├─────────────────────────┤
// │││┌11:If Net Run From Swap │ r │ reserved │
// ││││┌10:If Removable Run From Swap │ d │ deprecated │
// │││││┌9:Debug Stripped │ D │ deprecated with │
// ││││││┌8:32bit Machine │ │ extreme prejudice │
// │││││││ ┌5:Large Address Aware └───┴─────────────────────┘
// │││││││ │ ┌1:Executable
// │││││││ │ │┌0:Relocs Stripped
// d│││││││dr│Ddd││
PEEXE = 0b00000001000100011

// ┌15:TERMINAL_SERVER_AWARE ┌─────────────────────────┐
// │┌14:GUARD_CF │ PE DLL Characteristics │
// ││┌13:WDM_DRIVER ├─────────────────────────┤
// │││┌12:APPCONTAINER │ r │ reserved │
// ││││┌11:NO_BIND └───┴─────────────────────┘
// │││││┌10:NO_SEH
// ││││││┌9:NO_ISOLATION
// │││││││┌8:NX_COMPAT
// ││││││││┌7:FORCE_INTEGRITY
// │││││││││┌6:DYNAMIC_BASE
// ││││││││││┌5:HIGH_ENTROPY_VA
// │││││││││││rrrrr
DLLSTD = 0b0000000100100000
DLLPIE = 0b0000000001100000
DLLEXE = DLLSTD

// ┌31:Writeable ┌─────────────────────────┐
// │┌30:Readable │ PE Section Flags │
// ││┌29:Executable ├─────────────────────────┤
// │││┌28:Shareable │ o │ for object files │
// ││││┌27:Unpageable │ r │ reserved │
// │││││┌26:Uncacheable └───┴─────────────────────┘
// ││││││┌25:Discardable
// │││││││┌24:Contains Extended Relocations
// ││││││││ ┌15:Contains Global Pointer (GP) Relative Data
// ││││││││ │ ┌7:Contains Uninitialized Data
// ││││││││ │ │┌6:Contains Initialized Data
// ││││││││ o │ ││┌5:Contains Code
// ││││││││┌┴─┐rrrr│ ooror│││rorrr
// ┌31:Writeable ┌─────────────────────────┐
// │┌30:Readable │ PE Section Flags │
// ││┌29:Executable ├─────────────────────────┤
// │││┌28:Shareable │ o │ for object files │
// ││││┌27:Unpageable │ r │ reserved │
// │││││┌26:Uncacheable └───┴─────────────────────┘
// ││││││┌25:Discardable
// │││││││┌24:Contains Extended Relocations
// ││││││││ ┌15:Contains Global Pointer (GP) Relative Data
// ││││││││ │ ┌7:Contains Uninitialized Data
// ││││││││ │ │┌6:Contains Initialized Data
// ││││││││ o │ ││┌5:Contains Code
// ││││││││┌┴─┐rrrr│ ooror│││rorrr
PETEXT = 0b01110000000000000000000001100000
PEDATA = 0b11000000000000000000000011000000
PEIMPS = 0b11000000000000000000000001000000
@@ -1437,7 +1437,7 @@ long: push $GDT_LONG_DATA
.endfn long

/* ▄▄▒▀▀▀▀▒▒▄
█████▓▓▄░░░░ ▒▒▄
█████▓▓▄░░░░ ▒▒▄
▐█▓▓█▓▄█████▓░ ▀▒▄
▓█▓▓▓▓▓▓▓▓▓█▓ ░▀▒░
▀▀▓█▓▓▓▓▓▓█▓ ░▀▒▄▄▒▄▄▄▒▒▒▀▀▀▀▀▀▀▀▀▀▀▀▀▀▒▒▄▒
Binary file modified build/bootstrap/package.com
Binary file not shown.
Binary file modified build/bootstrap/zipobj.com
Binary file not shown.
30 changes: 17 additions & 13 deletions libc/alg/bisect.internal.h
Original file line number Diff line number Diff line change
@@ -6,19 +6,23 @@ COSMOPOLITAN_C_START_
forceinline void *bisect(const void *k, const void *data, size_t n, size_t size,
int cmp(const void *a, const void *b, void *arg),
void *arg) {
int dir;
const char *p, *pos;
p = data;
while (n > 0) {
pos = p + size * (n / 2);
dir = cmp(k, pos, arg);
if (dir < 0) {
n /= 2;
} else if (dir > 0) {
p = pos + size;
n -= n / 2 + 1;
} else {
return (void *)pos;
int c;
const char *p;
ssize_t m, l, r;
if (n) {
l = 0;
r = n - 1;
p = data;
while (l <= r) {
m = (l + r) >> 1;
c = cmp(k, p + m * size, arg);
if (c > 0) {
l = m + 1;
} else if (c < 0) {
r = m - 1;
} else {
return p + m * size;
}
}
}
return NULL;
4 changes: 2 additions & 2 deletions libc/bits/bitreverse16.c
Original file line number Diff line number Diff line change
@@ -21,6 +21,6 @@
/**
* Reverses bits in 16-bit word.
*/
uint16_t(bitreverse16)(uint16_t x) {
return kReverseBits[x & 0x00FF] << 8 | kReverseBits[(x & 0xFF00) >> 8];
int bitreverse16(int x) {
return BITREVERSE16(x);
}
8 changes: 4 additions & 4 deletions libc/bits/bitreverse32.c
Original file line number Diff line number Diff line change
@@ -22,10 +22,10 @@
/**
* Reverses bits in 32-bit word.
*/
uint32_t(bitreverse32)(uint32_t x) {
uint32_t bitreverse32(uint32_t x) {
x = bswap_32(x);
x = ((x & 0xaaaaaaaa) >> 1) | ((x & 0x55555555) << 1);
x = ((x & 0xcccccccc) >> 2) | ((x & 0x33333333) << 2);
x = ((x & 0xf0f0f0f0) >> 4) | ((x & 0x0f0f0f0f) << 4);
x = (x & 0xaaaaaaaa) >> 1 | (x & 0x55555555) << 1;
x = (x & 0xcccccccc) >> 2 | (x & 0x33333333) << 2;
x = (x & 0xf0f0f0f0) >> 4 | (x & 0x0f0f0f0f) << 4;
return x;
}
6 changes: 3 additions & 3 deletions libc/bits/bitreverse64.c
Original file line number Diff line number Diff line change
@@ -24,8 +24,8 @@
*/
uint64_t bitreverse64(uint64_t x) {
x = bswap_64(x);
x = ((x & 0xaaaaaaaaaaaaaaaa) >> 1) | ((x & 0x5555555555555555) << 1);
x = ((x & 0xcccccccccccccccc) >> 2) | ((x & 0x3333333333333333) << 2);
x = ((x & 0xf0f0f0f0f0f0f0f0) >> 4) | ((x & 0x0f0f0f0f0f0f0f0f) << 4);
x = (x & 0xaaaaaaaaaaaaaaaa) >> 1 | (x & 0x5555555555555555) << 1;
x = (x & 0xcccccccccccccccc) >> 2 | (x & 0x3333333333333333) << 2;
x = (x & 0xf0f0f0f0f0f0f0f0) >> 4 | (x & 0x0f0f0f0f0f0f0f0f) << 4;
return x;
}
4 changes: 2 additions & 2 deletions libc/bits/bitreverse8.c
Original file line number Diff line number Diff line change
@@ -21,6 +21,6 @@
/**
* Reverses bits in 8-bit word.
*/
uint8_t(bitreverse8)(uint8_t x) {
return kReverseBits[x];
int bitreverse8(int x) {
return BITREVERSE8(x);
}
198 changes: 98 additions & 100 deletions libc/bits/bits.h
Original file line number Diff line number Diff line change
@@ -13,8 +13,8 @@ extern const uint8_t kReverseBits[256];

uint32_t gray(uint32_t) pureconst;
uint32_t ungray(uint32_t) pureconst;
uint8_t bitreverse8(uint8_t) libcesque pureconst;
uint16_t bitreverse16(uint16_t) libcesque pureconst;
int bitreverse8(int) libcesque pureconst;
int bitreverse16(int) libcesque pureconst;
uint32_t bitreverse32(uint32_t) libcesque pureconst;
uint64_t bitreverse64(uint64_t) libcesque pureconst;
unsigned long roundup2pow(unsigned long) libcesque pureconst;
@@ -31,106 +31,104 @@ intptr_t atomic_store(void *, intptr_t, size_t);
│ cosmopolitan § bits » no assembly required ─╬─│┼
╚────────────────────────────────────────────────────────────────────────────│*/

#define bitreverse8(X) (kReverseBits[(uint8_t)(X)])
#define bitreverse16(X) \
((uint16_t)kReverseBits[(uint8_t)(X)] << 010 | \
kReverseBits[((uint16_t)(X) >> 010) & 0xff])
#define BITREVERSE8(X) (kReverseBits[255 & (X)])
#define BITREVERSE16(X) \
(kReverseBits[0x00FF & (X)] << 8 | kReverseBits[(0xFF00 & (X)) >> 8])

#define READ16LE(S) \
((uint16_t)((unsigned char *)(S))[1] << 010 | \
(uint16_t)((unsigned char *)(S))[0] << 000)
#define READ32LE(S) \
((uint32_t)((unsigned char *)(S))[3] << 030 | \
(uint32_t)((unsigned char *)(S))[2] << 020 | \
(uint32_t)((unsigned char *)(S))[1] << 010 | \
(uint32_t)((unsigned char *)(S))[0] << 000)
#define READ64LE(S) \
((uint64_t)((unsigned char *)(S))[7] << 070 | \
(uint64_t)((unsigned char *)(S))[6] << 060 | \
(uint64_t)((unsigned char *)(S))[5] << 050 | \
(uint64_t)((unsigned char *)(S))[4] << 040 | \
(uint64_t)((unsigned char *)(S))[3] << 030 | \
(uint64_t)((unsigned char *)(S))[2] << 020 | \
(uint64_t)((unsigned char *)(S))[1] << 010 | \
(uint64_t)((unsigned char *)(S))[0] << 000)

#define READ16BE(S) \
((uint16_t)((unsigned char *)(S))[0] << 010 | \
(uint16_t)((unsigned char *)(S))[1] << 000)
#define READ32BE(S) \
((uint32_t)((unsigned char *)(S))[0] << 030 | \
(uint32_t)((unsigned char *)(S))[1] << 020 | \
(uint32_t)((unsigned char *)(S))[2] << 010 | \
(uint32_t)((unsigned char *)(S))[3] << 000)
#define READ64BE(S) \
((uint64_t)((unsigned char *)(S))[0] << 070 | \
(uint64_t)((unsigned char *)(S))[1] << 060 | \
(uint64_t)((unsigned char *)(S))[2] << 050 | \
(uint64_t)((unsigned char *)(S))[3] << 040 | \
(uint64_t)((unsigned char *)(S))[4] << 030 | \
(uint64_t)((unsigned char *)(S))[5] << 020 | \
(uint64_t)((unsigned char *)(S))[6] << 010 | \
(uint64_t)((unsigned char *)(S))[7] << 000)

#define WRITE16LE(P, V) \
do { \
uint8_t *Ple = (uint8_t *)(P); \
uint16_t Vle = (V); \
Ple[0] = (uint8_t)(Vle >> 000); \
Ple[1] = (uint8_t)(Vle >> 010); \
} while (0)
#define WRITE32LE(P, V) \
do { \
uint8_t *Ple = (uint8_t *)(P); \
uint32_t Vle = (V); \
Ple[0] = (uint8_t)(Vle >> 000); \
Ple[1] = (uint8_t)(Vle >> 010); \
Ple[2] = (uint8_t)(Vle >> 020); \
Ple[3] = (uint8_t)(Vle >> 030); \
} while (0)
#define WRITE64LE(P, V) \
do { \
uint8_t *Ple = (uint8_t *)(P); \
uint64_t Vle = (V); \
Ple[0] = (uint8_t)(Vle >> 000); \
Ple[1] = (uint8_t)(Vle >> 010); \
Ple[2] = (uint8_t)(Vle >> 020); \
Ple[3] = (uint8_t)(Vle >> 030); \
Ple[4] = (uint8_t)(Vle >> 040); \
Ple[5] = (uint8_t)(Vle >> 050); \
Ple[6] = (uint8_t)(Vle >> 060); \
Ple[7] = (uint8_t)(Vle >> 070); \
} while (0)
#ifdef __STRICT_ANSI__
#define READ16LE(S) ((255 & (S)[1]) << 8 | (255 & (S)[0]))
#define READ16BE(S) ((255 & (S)[0]) << 8 | (255 & (S)[1]))
#define READ32LE(S) \
((uint32_t)(255 & (S)[3]) << 030 | (uint32_t)(255 & (S)[2]) << 020 | \
(uint32_t)(255 & (S)[1]) << 010 | (uint32_t)(255 & (S)[0]) << 000)
#define READ32BE(S) \
((uint32_t)(255 & (S)[0]) << 030 | (uint32_t)(255 & (S)[1]) << 020 | \
(uint32_t)(255 & (S)[2]) << 010 | (uint32_t)(255 & (S)[3]) << 000)
#define READ64LE(S) \
((uint64_t)(255 & (S)[7]) << 070 | (uint64_t)(255 & (S)[6]) << 060 | \
(uint64_t)(255 & (S)[5]) << 050 | (uint64_t)(255 & (S)[4]) << 040 | \
(uint64_t)(255 & (S)[3]) << 030 | (uint64_t)(255 & (S)[2]) << 020 | \
(uint64_t)(255 & (S)[1]) << 010 | (uint64_t)(255 & (S)[0]) << 000)
#define READ64BE(S) \
((uint64_t)(255 & (S)[0]) << 070 | (uint64_t)(255 & (S)[1]) << 060 | \
(uint64_t)(255 & (S)[2]) << 050 | (uint64_t)(255 & (S)[3]) << 040 | \
(uint64_t)(255 & (S)[4]) << 030 | (uint64_t)(255 & (S)[5]) << 020 | \
(uint64_t)(255 & (S)[6]) << 010 | (uint64_t)(255 & (S)[7]) << 000)
#else /* gcc needs help knowing above are mov if s isn't a variable */
#define READ16LE(S) \
({ \
const uint8_t *Ptr = (const uint8_t *)(S); \
Ptr[1] << 8 | Ptr[0]; \
})
#define READ16BE(S) \
({ \
const uint8_t *Ptr = (const uint8_t *)(S); \
Ptr[0] << 8 | Ptr[1]; \
})
#define READ32LE(S) \
({ \
const uint8_t *Ptr = (const uint8_t *)(S); \
((uint32_t)Ptr[3] << 030 | (uint32_t)Ptr[2] << 020 | \
(uint32_t)Ptr[1] << 010 | (uint32_t)Ptr[0] << 000); \
})
#define READ32BE(S) \
({ \
const uint8_t *Ptr = (const uint8_t *)(S); \
((uint32_t)Ptr[0] << 030 | (uint32_t)Ptr[1] << 020 | \
(uint32_t)Ptr[2] << 010 | (uint32_t)Ptr[3] << 000); \
})
#define READ64LE(S) \
({ \
const uint8_t *Ptr = (const uint8_t *)(S); \
((uint64_t)Ptr[7] << 070 | (uint64_t)Ptr[6] << 060 | \
(uint64_t)Ptr[5] << 050 | (uint64_t)Ptr[4] << 040 | \
(uint64_t)Ptr[3] << 030 | (uint64_t)Ptr[2] << 020 | \
(uint64_t)Ptr[1] << 010 | (uint64_t)Ptr[0] << 000); \
})
#define READ64BE(S) \
({ \
const uint8_t *Ptr = (const uint8_t *)(S); \
((uint64_t)Ptr[0] << 070 | (uint64_t)Ptr[1] << 060 | \
(uint64_t)Ptr[2] << 050 | (uint64_t)Ptr[3] << 040 | \
(uint64_t)Ptr[4] << 030 | (uint64_t)Ptr[5] << 020 | \
(uint64_t)Ptr[6] << 010 | (uint64_t)Ptr[7] << 000); \
})
#endif

#define WRITE16BE(P, V) \
do { \
uint8_t *Ple = (uint8_t *)(P); \
uint16_t Vle = (V); \
Ple[1] = (uint8_t)(Vle >> 000); \
Ple[0] = (uint8_t)(Vle >> 010); \
} while (0)
#define WRITE32BE(P, V) \
do { \
uint8_t *Ple = (uint8_t *)(P); \
uint32_t Vle = (V); \
Ple[3] = (uint8_t)(Vle >> 000); \
Ple[2] = (uint8_t)(Vle >> 010); \
Ple[1] = (uint8_t)(Vle >> 020); \
Ple[0] = (uint8_t)(Vle >> 030); \
} while (0)
#define WRITE64BE(P, V) \
do { \
uint8_t *Ple = (uint8_t *)(P); \
uint64_t Vle = (V); \
Ple[7] = (uint8_t)(Vle >> 000); \
Ple[6] = (uint8_t)(Vle >> 010); \
Ple[5] = (uint8_t)(Vle >> 020); \
Ple[4] = (uint8_t)(Vle >> 030); \
Ple[3] = (uint8_t)(Vle >> 040); \
Ple[2] = (uint8_t)(Vle >> 050); \
Ple[1] = (uint8_t)(Vle >> 060); \
Ple[0] = (uint8_t)(Vle >> 070); \
} while (0)
#define WRITE16LE(P, V) \
((P)[0] = (0x00000000000000FF & (V)) >> 000, \
(P)[1] = (0x000000000000FF00 & (V)) >> 010, (P) + 2)
#define WRITE16BE(P, V) \
((P)[0] = (0x000000000000FF00 & (V)) >> 010, \
(P)[1] = (0x00000000000000FF & (V)) >> 000, (P) + 2)
#define WRITE32LE(P, V) \
((P)[0] = (0x00000000000000FF & (V)) >> 000, \
(P)[1] = (0x000000000000FF00 & (V)) >> 010, \
(P)[2] = (0x0000000000FF0000 & (V)) >> 020, \
(P)[3] = (0x00000000FF000000 & (V)) >> 030, (P) + 4)
#define WRITE32BE(P, V) \
((P)[0] = (0x00000000FF000000 & (V)) >> 030, \
(P)[1] = (0x0000000000FF0000 & (V)) >> 020, \
(P)[2] = (0x000000000000FF00 & (V)) >> 010, \
(P)[3] = (0x00000000000000FF & (V)) >> 000, (P) + 4)
#define WRITE64LE(P, V) \
((P)[0] = (0x00000000000000FF & (V)) >> 000, \
(P)[1] = (0x000000000000FF00 & (V)) >> 010, \
(P)[2] = (0x0000000000FF0000 & (V)) >> 020, \
(P)[3] = (0x00000000FF000000 & (V)) >> 030, \
(P)[4] = (0x000000FF00000000 & (V)) >> 040, \
(P)[5] = (0x0000FF0000000000 & (V)) >> 050, \
(P)[6] = (0x00FF000000000000 & (V)) >> 060, \
(P)[7] = (0xFF00000000000000 & (V)) >> 070, (P) + 8)
#define WRITE64BE(P, V) \
((P)[0] = (0xFF00000000000000 & (V)) >> 070, \
(P)[1] = (0x00FF000000000000 & (V)) >> 060, \
(P)[2] = (0x0000FF0000000000 & (V)) >> 050, \
(P)[3] = (0x000000FF00000000 & (V)) >> 040, \
(P)[4] = (0x00000000FF000000 & (V)) >> 030, \
(P)[5] = (0x0000000000FF0000 & (V)) >> 020, \
(P)[6] = (0x000000000000FF00 & (V)) >> 010, \
(P)[7] = (0x00000000000000FF & (V)) >> 000, (P) + 8)

/*───────────────────────────────────────────────────────────────────────────│─╗
│ cosmopolitan § bits » some assembly required ─╬─│┼
Loading

0 comments on commit bf03b2e

Please sign in to comment.