panic serving 127.0.0.1:****: unaligned 64-bit atomic operation

[martnst]

Describe the bug
File Upload not working on Debian Installation.

I did install the the binary from https://github.com/tus/tusd/releases/download/v2.2.1/tusd_linux_386.tar.gz on a Debian server.

Attempting to upload file I get this error:

http: panic serving 127.0.0.1:43434: unaligned 64-bit atomic operation
goroutine 36 [running]:
net/http.(*conn).serve.func1()
\tnet/http/server.go:1868 +0xe0
panic({0x8c538e0, 0x8f80608})
\truntime/panic.go:920 +0x24c
runtime/internal/atomic.panicUnaligned()
\truntime/internal/atomic/unaligned.go:8 +0x2d
runtime/internal/atomic.Xadd64(0xb3091d4, 0x12)
\truntime/internal/atomic/atomic_386.s:125 +0x11
github.com/tus/tusd/v2/pkg/handler.(*bodyReader).Read(0xb3091c0, {0xb338000, 0x8000, 0x8000})
\tgithub.com/tus/tusd/v2/pkg/handler/body_reader.go:43 +0x9a
io.copyBuffer({0x8f811ac, 0xb27df98}, {0x8f81314, 0xb3091c0}, {0x0, 0x0, 0x0})
\tio/io.go:430 +0x1bf
io.Copy(...)
\tio/io.go:389
os.genericReadFrom(0xb27df98, {0x8f81314, 0xb3091c0})
\tos/file.go:161 +0x53
os.(*File).ReadFrom(0xb27df98, {0x8f81314, 0xb3091c0})
\tos/file.go:155 +0xda
io.copyBuffer({0x8f80f40, 0xb27df98}, {0x8f81314, 0xb3091c0}, {0x0, 0x0, 0x0})
\tio/io.go:416 +0x127
io.Copy(...)
\tio/io.go:389
github.com/tus/tusd/v2/pkg/filestore.(*fileUpload).WriteChunk(0xb1ab9f0, {0x8f8c5e0, 0xb1b3200}, 0x0, {0x8f81314, 0xb3091c0})
\tgithub.com/tus/tusd/v2/pkg/filestore/filestore.go:165 +0x90
github.com/tus/tusd/v2/pkg/handler.(*UnroutedHandler).writeChunk(0xb3ee000, 0xb1b3200, {0xcc, {0x0, 0x0}, 0xb3091a0}, {0x8f8c5a0, 0xb1ab9f0}, {{0xb1977e0, 0x20}, ...})
\tgithub.com/tus/tusd/v2/pkg/handler/unrouted_handler.go:874 +0x4d9
github.com/tus/tusd/v2/pkg/handler.(*UnroutedHandler).PatchFile(0xb3ee000, {0x8f8bc88, 0xb1c8320}, 0xb19ce80)
\tgithub.com/tus/tusd/v2/pkg/handler/unrouted_handler.go:770 +0x8b9
net/http.HandlerFunc.ServeHTTP(0xb04b2f0, {0x8f8bc88, 0xb1c8320}, 0xb19ce80)
\tnet/http/server.go:2136 +0x34
github.com/bmizerany/pat.(*PatternServeMux).ServeHTTP(0xb011500, {0x8f8bc88, 0xb1c8320}, 0xb19ce80)
\tgithub.com/bmizerany/pat@v0.0.0-20170815010413-6226ea591a40/mux.go:117 +0x172
github.com/tus/tusd/v2/pkg/handler.NewHandler.(*UnroutedHandler).Middleware.func1({0x8f8bc88, 0xb1c8320}, 0xb19ce00)
\tgithub.com/tus/tusd/v2/pkg/handler/unrouted_handler.go:252 +0x943
net/http.HandlerFunc.ServeHTTP(0xb011510, {0x8f8bc88, 0xb1c8320}, 0xb19ce00)
\tnet/http/server.go:2136 +0x34
github.com/tus/tusd/v2/cmd/tusd/cli.Serve.StripPrefix.func9({0x8f8bc88, 0xb1c8320}, 0xb19cd00)
\tnet/http/server.go:2179 +0x20b
net/http.HandlerFunc.ServeHTTP(0xb0ad398, {0x8f8bc88, 0xb1c8320}, 0xb19cd00)
\tnet/http/server.go:2136 +0x34
net/http.(*ServeMux).ServeHTTP(0xb03ad80, {0x8f8bc88, 0xb1c8320}, 0xb19cd00)
\tnet/http/server.go:2514 +0x15e
net/http.serverHandler.ServeHTTP({0xb3ee0a0}, {0x8f8bc88, 0xb1c8320}, 0xb19cd00)
\tnet/http/server.go:2938 +0x99
net/http.(*conn).serve(0xb30d740, {0x8f8c480, 0xb0ad680})
\tnet/http/server.go:2009 +0x66d
created by net/http.(*Server).Serve in goroutine 1
\tnet/http/server.go:3086 +0x47f

To Reproduce
Steps to reproduce the behavior:

1. Execute command '...'
2. Start upload '....'
3. See error

Expected behavior
Request should work.

Setup details
Please provide following details, if applicable to your situation:

• Operating System: Linux

  root@server ~ # cat /etc/os-release 
  PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
  NAME="Debian GNU/Linux"
  VERSION_ID="11"
  VERSION="11 (bullseye)"
  VERSION_CODENAME=bullseye
  ID=debian
  HOME_URL="https://www.debian.org/"
  SUPPORT_URL="https://www.debian.org/support"
  BUG_REPORT_URL="https://bugs.debian.org/"
  root@server ~ # uname -m
  x86_64

• Used tusd version: 2.2.1

  Version: v2.2.1
  Commit: f03aa9b4ddd69cf83c1fa3f74f1172c179be7bdc
  Date: Wed Dec 13 10:00:25 UTC 2023
  

• Used tusd configuration: (running as root)

  tusd -port 9000 -behind-proxy -upload-dir /tmp/tusd-files/

• Used tusd data storage: local disk store

  root@myserver ~ # ll /tmp/tusd-files
  total 48
  drwxr-xr-x   2 root root  4096 Dec 13 15:03 ./
  drwxrwxrwt 640 root root 32768 Dec 13 15:06 ../
  -rw-r--r--   1 root root     0 Dec 13 14:33 2e390ae1cdbe288bfa4d36c7ec711cf2
  -rw-r--r--   1 root root   414 Dec 13 14:33 2e390ae1cdbe288bfa4d36c7ec711cf2.info
  -rw-r--r--   1 root root     0 Dec 13 14:47 dec2d5978045c44811bb884827eb3582
  -rw-r--r--   1 root root   349 Dec 13 14:47 dec2d5978045c44811bb884827eb3582.info

• Used tus client library: curl:

  root@myserver ~ # curl 'http://0.0.0.0:9000/files/dec2d5978045c44811bb884827eb3582' -X PATCH -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:120.0) Gecko/20100101 Firefox/120.0' -H 'Accept: */*' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate, br' -H 'Tus-Resumable: 1.0.0' -H 'Upload-Offset: 0' -H 'Content-Type: application/offset+octet-stream'  -H 'Connection: keep-alive' -H 'Sec-Fetch-Dest: empty' -H 'Sec-Fetch-Mode: cors' -H 'Sec-Fetch-Site: same-site' --data-raw $'hello from martin\n'
  curl: (52) Empty reply from server

[martnst]

Here is how I installed it:

1. Create & enter a temporary working folder

   mkdir tmp-install-tusd && cd tmp-install-tusd

2. Download

   wget https://github.com/tus/tusd/releases/download/v2.2.0/tusd_linux_386.tar.gz

3. Verify integrity of the file

   curl -L -s https://github.com/tus/tusd/releases/download/v2.2.0/tusd_linux_386.tar.gz.sha256 | shasum -a 256 -c
   tusd_linux_386.tar.gz: OK

4. Unpack the archive

   tar xzf tusd_linux_386.tar.gz

5. Move the binary

   cp -v tusd_linux_386/tusd /usr/local/bin/
   'tusd_linux_386/tusd' -> '/usr/local/bin/tusd' 

6. Test the server

    tsud

   Output should look like this:

   2023/12/13 11:29:31.164283 Using '/root/data' as directory storage.
   2023/12/13 11:29:31.164608 Using 0.00MB as maximum size.
   2023/12/13 11:29:31.164635 Supported tus extensions: creation,creation-with-upload,termination,concatenation,creation-defer-length
   2023/12/13 11:29:31.164640 Using 0.0.0.0:8080 as address to listen.
   2023/12/13 11:29:31.164645 Using /files/ as the base path.
   2023/12/13 11:29:31.164678 Using /metrics as the metrics path.
   2023/12/13 11:29:31.164860 You can now upload files to: http://[::]:8080/files/
   

7. Remove the working folder

   cd .. && rm -rfv tmp-install-tusd

[Acconut]

Thanks for the detailed report, that's very helpful!

You seem to be on a 64-bit machine (x84_64) but downloaded a 32-bit build of tusd (386 architecture). Was there a particular reason why you downloaded tusd_linux_386.tar.gz and not tusd_linux_amd64.tar.gz?

I found similar reports where this happened when the Go code was compiled for 32-bit architecture, e.g. https://www.couchbase.com/forums/t/go-sdk-panics-with-unaligned-64-bit-atomic-operation-on-x86-64-ubuntu-server/34124/2

The panic is caused by

tusd/pkg/handler/body_reader.go

Line 43 in f03aa9b

atomic.AddInt64(&r.bytesCounter, int64(n))

We should be able to fix this by adjusting the position of the field that is written to like they did in blevesearch/zapx#148.

[martnst]

Ah right. Thanks a lot. That was a stupid mistake of mine. I am not working with linux on a daily basis.