Keeps disconnecting while backup

[thegabriele97]

Hi!
Today I am trying to configure on my home server a borg backup storage. So I installed borg on the server and Vorta on my PC. I configured Vorta to backup my entire home folder except some folders for a total of 10 GB of data and I started the backup. After a while (1 minute or so), an excetion occurs saying that the remote host closed the connection.

I configured the ssh service on the server side with TCPKeepAlive and different combinations of countmax and interval values. The backup stops always at the same file and I think it happens because its size is a little bit bigger then others (I see that stays a little bit on it before the exception occurs and some files before it are bigger too and requires a few seconds).

On the client side, .ssh/config contains:

Host <ip_server>
ServerAliveInterval 60
ServerAliveCountMax 5
TCPKeepAlive yes

This is the stack trace:

Running command /app/bin/borg create --list --progress --info --log-json --json --filter=AM -C lz4 --exclude-from /tmp/tmpnr354btv ssh://borg@192.168.1.133/mnt/nfs1_mp0_share/gabriele97/files/Backups/Borg/rogpopos::popos.rog.lan-2023-06-01-173630 /home/gabriele97
2023-06-01 17:36:58,053 - vorta.borg.borg_job - WARNING - Remote: Killed stale lock borg.lan@160901351925564.8272-0.
2023-06-01 17:36:58,284 - vorta.borg.borg_job - WARNING - Remote: Removed stale exclusive roster lock for host borg.lan@160901351925564 pid 8272 thread 0.
2023-06-01 17:36:58,286 - vorta.borg.borg_job - WARNING - Remote: Removed stale exclusive roster lock for host borg.lan@160901351925564 pid 8272 thread 0.
2023-06-01 17:36:58,603 - vorta.borg.borg_job - INFO - Creating archive at "ssh://borg@192.168.1.133/mnt/nfs1_mp0_share/gabriele97/files/Backups/Borg/rogpopos::popos.rog.lan-2023-06-01-173630"
2023-06-01 17:37:45,871 - vorta.borg.borg_job - ERROR - Local Exception
2023-06-01 17:37:45,872 - vorta.borg.borg_job - ERROR - Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 183, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 675, in do_create
    create_inner(archive, cache, fso)
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 608, in create_inner
    self._rec_walk(path=path, parent_fd=parent_fd, name=name,
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 823, in _rec_walk
    self._rec_walk(
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 823, in _rec_walk
    self._rec_walk(
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 823, in _rec_walk
    self._rec_walk(
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 786, in _rec_walk
    status = self._process_any(path=path, parent_fd=parent_fd, name=name, st=st, fso=fso, cache=cache,
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 688, in _process_any
    return fso.process_file(path=path, parent_fd=parent_fd, name=name, st=st, cache=cache)
  File "/app/lib/python3.10/site-packages/borg/archive.py", line 1451, in process_file
    self.process_file_chunks(item, cache, self.stats, self.show_progress, backup_io_iter(self.chunker.chunkify(None, fd)))
  File "/app/lib/python3.10/site-packages/borg/archive.py", line 1273, in process_file_chunks
    item.chunks.append(chunk_processor(chunk))
  File "/app/lib/python3.10/site-packages/borg/archive.py", line 1261, in chunk_processor
    chunk_entry = cache.add_chunk(chunk_id, data, stats, wait=False)
  File "/app/lib/python3.10/site-packages/borg/cache.py", line 951, in add_chunk
    self.repository.put(id, data, wait=wait)
  File "/app/lib/python3.10/site-packages/borg/remote.py", line 476, in do_rpc
    return self.call(f.__name__, named, **extra)
  File "/app/lib/python3.10/site-packages/borg/remote.py", line 711, in call
    for resp in self.call_many(cmd, [args], **kw):
  File "/app/lib/python3.10/site-packages/borg/remote.py", line 828, in call_many
    raise ConnectionClosed()
borg.remote.ConnectionClosed: Connection closed by remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 5213, in main
    exit_code = archiver.run(args)
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 5144, in run
    return set_ec(func(args))
  File "/app/lib/python3.10/site-packages/borg/archiver.py", line 168, in wrapper
    with repository:
  File "/app/lib/python3.10/site-packages/borg/remote.py", line 636, in __exit__
    self.rollback()
  File "/app/lib/python3.10/site-packages/borg/remote.py", line 476, in do_rpc
    return self.call(f.__name__, named, **extra)
  File "/app/lib/python3.10/site-packages/borg/remote.py", line 711, in call
    for resp in self.call_many(cmd, [args], **kw):
  File "/app/lib/python3.10/site-packages/borg/remote.py", line 781, in call_many
    send_buffer()  # Try to send data, as some cases (async_response) will never try to send data otherwise.
  File "/app/lib/python3.10/site-packages/borg/remote.py", line 721, in send_buffer
    written = self.ratelimit.write(self.stdin_fd, self.to_send.peek_front())
  File "/app/lib/python3.10/site-packages/borg/remote.py", line 417, in write
    written = os.write(fd, to_send)
BrokenPipeError: [Errno 32] Broken pipe

Platform: Linux popos.rog.lan 6.2.6-76060206-generic #202303130630~1683753207~22.04~77c1465 SMP PREEMPT_DYNAMIC Wed M x86_64
Linux: Unknown Linux  
Borg: 1.2.4  Python: CPython 3.10.6 msgpack: 1.0.4 fuse: pyfuse3 3.2.2 [pyfuse3,llfuse]
PID: 31  CWD: /home/gabriele97
sys.argv: ['/app/bin/borg', 'create', '--list', '--progress', '--info', '--log-json', '--json', '--filter=AM', '-C', 'lz4', '--exclude-from', '/tmp/tmpnr354btv', 'ssh://borg@192.168.1.133/mnt/nfs1_mp0_share/gabriele97/files/Backups/Borg/rogpopos::popos.rog.lan-2023-06-01-173630', '/home/gabriele97']
SSH_ORIGINAL_COMMAND: None

Any suggestion? Thanks!

[real-yfprojects]

Try adding the following to your servers ssh config (/etc/ssh/sshd_config):

# Server kills connection after 3*30 seconds without client response:
ClientAliveInterval 30
ClientAliveCountMax 3

[televisi]

what happen if you don't have access to the server /etc/ssh folder?
I'm with hetzner storage box plan and they don't allow you to access that folder.

Only the directory /home/ is writable on the Storage Box. 
To address directories or files on the Storage Box, we therefore recommend that you use relative paths

I guess, the only option for me is to set the config on the client side (~/.ssh/config)?

Thanks

[real-yfprojects]

I guess, the only option for me is to set the config on the client side (~/.ssh/config)?

I think the server will still close the connection to early when you only add it to the client side. I would try to contact the Hetzner support.