With podman remote client, podman build with a large files throws error: io: read/write on closed pipe

[ogipogi]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

With a remote connection (Azure CentOS VM) defined, invoked podman build with a large file of 1gb. After 90s I get an error. If I execute podman build with a smaller file (upload < 90s), it works without errors.

Steps to reproduce the issue:

1. Generate large file -> dd if=/dev/zero of=large-file-1gb.txt count=1024 bs=1048576

2. Create Dockerfile with:

FROM alpine:3.1
COPY large-file-1gb.txt .

3. Execute podman build -t test-large-container .

Describe the results you received:
Error: Post "http://d/v2.0.0/libpod/build?dockerfile=Dockerfile&forcerm=1&httpproxy=1&platform=darwin%2Famd64&rm=1&shmsize=67108864&t=test-large-container": io: read/write on closed pipe

Describe the results you expected:

podman build -t test-large-container .
STEP 1: FROM alpine:3.1
Resolved short name "alpine" to a recorded short-name alias (origin: /etc/containers/registries.conf.d/shortnames.conf)
Getting image source signatures
Copying blob sha256:0f253aa151d797195fd887f4d3bf2794bfead4f61f5ba1d455cab177fdcb0511
Copying config sha256:a1038a41fe2b75d8c53d0a4d22207e4e7f72e95a11da4d20424f0062b239b67f
Writing manifest to image destination
Storing signatures
STEP 2: COPY large-file-1gb.txt .
STEP 3: COMMIT test-large-container
Getting image source signatures
Copying blob sha256:d4c261b2e24885a63d2333a31d9a56d863f8f7bd89247afeb3543d7f1491b621
Copying blob sha256:cf511c6bb55c6ec4e9ac7e3a23bd340dda00d738cec9d238881ba078c40df0d4
Copying config sha256:f33856ce2910d470fda13597c0df339ec6592eac9fd466995edf75326e8a4058
Writing manifest to image destination
Storing signatures
--> f33856ce291
f33856ce2910d470fda13597c0df339ec6592eac9fd466995edf75326e8a4058

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Client:
Version:      2.2.1
API Version:  2.1.0
Go Version:   go1.15.5
Built:        Tue Dec  8 16:01:35 2020
OS/Arch:      darwin/amd64

Server:
Version:      2.2.1
API Version:  2.0.0
Go Version:   go1.14.7
Built:        Thu Dec 10 19:26:48 2020
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.18.0
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.0.21-1.el8.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.21, commit: f619ab8ef5f69bd40bb75ed64f3e1dace1815c22-dirty'
  cpus: 2
  distribution:
    distribution: '"centos"'
    version: "8"
  eventLogger: journald
  hostname: max-linux
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 4.18.0-193.28.1.el8_2.x86_64
  linkmode: dynamic
  memFree: 8720236544
  memTotal: 16813711360
  ociRuntime:
    name: runc
    package: runc-1.0.0-145.rc91.git24a3cf8.el8.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.2-dev'
  os: linux
  rootless: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.8-1.el8.x86_64
    version: |-
      slirp4netns version 1.1.8
      commit: d361001f495417b880f20329121e3aa431a8f90f
      libslirp: 4.3.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.4.3
  swapFree: 0
  swapTotal: 0
  uptime: 45h 56m 3.58s (Approximately 1.88 days)
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /home/max/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: vfs
  graphOptions: {}
  graphRoot: /home/max/.local/share/containers/storage
  graphStatus: {}
  imageStore:
    number: 3
  runRoot: /run/user/1000/containers
  volumePath: /home/max/.local/share/containers/storage/volumes
version:
  APIVersion: 2.1.0
  Built: 1607624808
  BuiltTime: Thu Dec 10 18:26:48 2020
  GitCommit: ""
  GoVersion: go1.14.7
  OsArch: linux/amd64
  Version: 2.2.1

Package info (e.g. output of rpm -q podman or apt list podman):

podman: stable 2.2.1 (bottled)
Tool for managing OCI containers and pods
https://podman.io/
/usr/local/Cellar/podman/2.2.1 (147 files, 37.3MB) *
  Poured from bottle on 2020-12-08 at 20:05:40
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Recent versions: No. Looked at the changelog and this issue is not included among the 2.2.x changes.
Troubleshooting: Yes, nothing there about this.

Additional environment details (AWS, VirtualBox, physical, etc.):
Client -> MacBook Pro 16 inch
Server -> Azure CentOS VM

[Luap99]

I assume you run the service with systemd. This sounds like this issue #8751

[ogipogi]

Yes, I run the service with systemd.
I followed the intructions, here:
https://github.com/containers/podman/tree/2d36c8bd4be9d6fd0f21056e7162edb238e248b5/contrib/systemd
It works now,
Thank you @Luap99!