Re-syncing new commits into an existing repository result in inaccessible content

[lubosmj]

Steps to reproduce the issue

Instantiate a new repository locally and publish it:

# on the host machine
ostree --repo=repo1 init --mode=archive
mkdir test
echo "test1" > test/test.txt
ostree --repo=repo1 commit --branch foo test
ostree summary -u --repo=repo1

# on the host machine
python3 -m http.server 8001

# on the pulp machine
pulp ostree repository create --name foo
pulp ostree remote create --name foo --url http://0.0.0.0:8001/repo1/
pulp ostree repository sync --name foo --remote foo
pulp ostree distribution create --name foo --base-path foo --repository foo
http http://localhost:5001/pulp/content/foo/summary

Add a new commit to the repository and re-sync it:

# on the host machine
echo "test2" > test/test2.txt
ostree --repo=repo1 commit --branch foo test
ostree summary -u --repo=repo1

# on the pulp machine
pulp ostree repository sync --name foo --remote foo
http http://localhost:5001/pulp/content/foo/summary

pulp [None]: pulpcore.content.handler:ERROR: Multiple (pass-through) matches for {b}/{p}
[2023-05-24 17:05:33 +0000] [1141] [ERROR] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib64/python3.8/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib64/python3.8/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib64/python3.8/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/local/lib/python3.8/site-packages/pulpcore/content/authentication.py", line 48, in authenticate
    return await handler(request)
  File "/usr/local/lib/python3.8/site-packages/pulpcore/content/handler.py", line 245, in stream_content
    return await self._match_and_stream(path, request)
  File "/usr/local/lib/python3.8/site-packages/pulpcore/content/handler.py", line 673, in _match_and_stream
    ca = await sync_to_async(get_contentartifact_blocking)()
  File "/usr/local/lib/python3.8/site-packages/asgiref/sync.py", line 448, in __call__
    ret = await asyncio.wait_for(future, timeout=None)
  File "/usr/lib64/python3.8/asyncio/tasks.py", line 455, in wait_for
    return await fut
  File "/usr/lib64/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/asgiref/sync.py", line 490, in thread_handler
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/pulpcore/content/handler.py", line 668, in get_contentartifact_blocking
    ca = ContentArtifact.objects.select_related(
  File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 439, in get
    raise self.model.MultipleObjectsReturned(
pulpcore.app.models.content.ContentArtifact.MultipleObjectsReturned: get() returned more than one ContentArtifact -- it returned 2!
127.0.0.1 [24/May/2023:17:05:33 +0000] "GET /pulp/content/foo/summary HTTP/1.0" 500 225 "-" "HTTPie/3.2.2"

The errors are raised for config and summary files.

[lubosmj]

The errors are raised only when the relative path of content persists but not its internals.

In [37]: from pulp_ostree.app.models import OstreeSummary

In [38]: OstreeSummary.objects.filter(pk__in=version.content)
Out[38]: <BulkTouchQuerySet [<OstreeSummary: pk=0030454d-aff4-4461-83a5-5128ea3f2fc4>, <OstreeSummary: pk=ca90d41e-cef4-4c52-91ca-b907f9fe43be>]>

In [39]: summary = OstreeSummary.objects.filter(pk__in=version.content).first()

In [40]: summary._artifacts.get().file
Out[40]: <FieldFile: artifact/83/4d8ab1a24732e18c8f2a93d7c7261f2fce7a685d3d076da11eeec28515c84f>

In [41]: summary._artifacts.get().file.read()
Out[41]: b'foo\x00\x00\x00\x00\x00v\x00\x00\x00\x00\x00\x00\x00\x86,\t3\x88O\xf4\xd8\x8a\x1f\xe9\x9b\xdbQ\x81\xcf\x126\x8e\xb7\xb10=`\x02;\x8a \xaa\x86\xc6oostree.commit.timestamp\x00\x00\x00\x00\x00dp\x9e@\x00t\x18#(\x04V\x00ostree.summary.last-modified\x00\x00\x00\x00\x00\x00\x00\x00dp\x9e\x80\x00t\x1d\x00\x00\x00\x00\x00ostree.summary.mode\x00\x00\x00\x00\x00archive-z2\x00\x00s\x14\x00\x00ostree.summary.tombstone-commits\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00b!\x00\x00\x00\x00ostree.summary.indexed-deltas\x00\x00\x00\x01\x00b\x1e+V\x84\xacW\x00'

In [42]: summary = OstreeSummary.objects.filter(pk__in=version.content).last()

In [43]: summary._artifacts.get().file.read()
Out[43]: b'foo\x00\x00\x00\x00\x00\x96\x00\x00\x00\x00\x00\x00\x00#*\x04"tR\xbb\x96\x83O\x14}\x06\x1d\x82\xcb\x0e\x94\xbf?\xb7)Mv\x06\x01\xc5\x1f\xdb\n$Qostree.commit.timestamp\x00\x00\x00\x00\x00dp\x9e\xd4\x00t\x18#(\x04V\x00ostree.summary.last-modified\x00\x00\x00\x00\x00\x00\x00\x00dp\x9e\xda\x00t\x1d\x00\x00\x00\x00\x00ostree.summary.mode\x00\x00\x00\x00\x00archive-z2\x00\x00s\x14\x00\x00ostree.summary.tombstone-commits\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00b!\x00\x00\x00\x00ostree.summary.indexed-deltas\x00\x00\x00\x01\x00b\x1e+V\x84\xacW\x00'

[lubosmj]

Old configs and summaries should be replaced/removed and not stored along with the new ones in the same repository version after every sync.