Spurious lockfile sync errors when writing to a known pip log file

[benjyw]

Reproduction:

$ rm -f pip.log lockfile.json
$ python -m pex.cli lock sync  --lock lockfile.json --pip-log pip.log "ansicolors==1.1.8" "cowsay==6.1"
pex: Preserving `pip download` log at pip.log
$ python -m pex.cli lock sync  --lock lockfile.json --pip-log pip.log "ansicolors==1.1.8" 
pex: Preserving `pip download` log at pip.log
Updates for lock generated by cp311-cp311-macosx_14_0_arm64:
  Deleted cowsay 6.1
Updates to lock input requirements:
  Deleted 'cowsay==6.1'
$ python -m pex.cli lock sync  --lock lockfile.json --pip-log pip.log "ansicolors==1.1.8" 
pex: Preserving `pip download` log at pip.log
Encountered 1 error updating lockfile.json:
1.) cp311-cp311-macosx_14_0_arm64: No distribution metadata found for cowsay==6.1.
Given distribution metadata for:
ansicolors==1.1.8
$ rm -f pip.log 
$ python -m pex.cli lock sync  --lock lockfile.json --pip-log pip.log "ansicolors==1.1.8" 
pex: Preserving `pip download` log at pip.log
No updates for lock generated by cp311-cp311-macosx_14_0_arm64.
$ 

This doesn't happen if --pip-log is not set.

My understanding of what is happening: when setting --pip-log pip.log pex appends to the file. So presumably pex is trying to parse lines from previous pip runs, and is confused when it sees data for requirements that aren't in the current set of input requirements.

[benjyw]

Is this just a matter of setting self._offset to the file length in Tailer before calling start()?

[jsirois]

No, its here:

pex/pex/resolver.py

Lines 79 to 96 in d94fa6a

	class PipLogManager(object):
	@classmethod
	def create(
	cls,
	log, # type: Optional[str]
	targets, # type: Sequence[Target]
	):
	# type: (...) -> PipLogManager
	log_by_target = {} # type: Dict[Target, str]
	if log and len(targets) == 1:
	log_by_target[targets[0]] = log
	elif log:
	log_dir = safe_mkdtemp(prefix="pex-pip-log.")
	log_by_target.update(
	(target, os.path.join(log_dir, "pip.{target}.log".format(target=target.id)))
	for target in targets
	)
	return cls(log=log, log_by_target=log_by_target)

pex/pex/resolver.py

Lines 113 to 119 in d94fa6a

	def finalize_log(self):
	# type: () -> None
	target_count = len(self._log_by_target)
	if target_count <= 1:
	return
	with safe_open(self.log, "a") as out_fp:

I handle the multi-target case correctly (say you sepcify 2 --complete-platforms), but not the single target common case.