Disconnect broadcaster in case of a segmentation error.

[j0sh]

This resolves the immediate issue of broadcast nodes hanging after OBS does not cleanly disconnect. The segmenter already has a built-in timeout which is typically triggered by idle RTMP ingest connections, so use that as a signal to kill the ingest connection.

[ericxtang]

1. Let's add the commit hash from LPMS.

2. Is there any other places we should close the RTMP connection? Feels almost like we should do this every time we are returning an error.

[j0sh]

Let's add the commit hash from LPMS.

It is already there (see c488b62), but prior to merging this, I'll update the commit message with the final commit hash after merging with LPMS, since fast-forward merges change the hash.

Is there any other places we should close the RTMP connection?

Returning a non-nil error from gotRTMPStreamHandler will break the incoming connection, so I think asynchronous goroutines/callbacks are what need to be checked. Reviewing the mediaserver.go code (maybe not the only relevant file), there are a couple other places where we asynchronously handle video-related data.

go-livepeer/server/mediaserver.go

Lines 299 to 302 in dabcc64

	if s.LivepeerNode.Eth != nil {
	//Create Transcode Job Onchain
	go s.LivepeerNode.CreateTranscodeJob(hlsStrmID, BroadcastJobVideoProfiles, BroadcastPrice)
	}

Once we have #429 in then this might not be relevant anymore. Also, given the OBS reconnect behavior, I'm not sure about the implications of re-trying the job submission every time it fails. (Although I should mention that this bit of code becomes more complex with the new networking system; see 482db44#diff-236217e7f473dfbde9134c6a391f3408L313)

go-livepeer/server/mediaserver.go

Lines 275 to 290 in dabcc64

	//Set up the transcode response callback
	s.LivepeerNode.VideoNetwork.ReceivedTranscodeResponse(string(hlsStrmID), func(result map[string]string) {
	//Parse through the results
	for strmID, tProfile := range result {
	vParams := ffmpeg.VideoProfileToVariantParams(ffmpeg.VideoProfileLookup[tProfile])
	pl, _ := m3u8.NewMediaPlaylist(stream.DefaultHLSStreamWin, stream.DefaultHLSStreamCap)
	variant := &m3u8.Variant{URI: fmt.Sprintf("%v.m3u8", strmID), Chunklist: pl, VariantParams: vParams}
	manifest.Append(variant.URI, variant.Chunklist, variant.VariantParams)
	}
	//Update the master playlist on the network
	if err = s.LivepeerNode.VideoNetwork.UpdateMasterPlaylist(string(mid), manifest); err != nil {
	glog.Errorf("Error updating master playlist on network: %v", err)
	return
	}
	})

If updating the master playlist fails, do we want to disconnect the incoming stream? (Right now, with the current basicnet implementation, it doesn't look like it can ever fail.) Since this isn't actually operating on the incoming/segmented stream, perhaps it's best to leave it alone? Although if we do add other types of non-recoverable errors from the transcoder later on, we could special-case a disconnect here.

Are there any other places that might need to be checked?

[j0sh]

Updated the LPMS commit to the final hash of 6466ec3 , 9e50061

[ericxtang]

Agree that masterPlaylist update failure shouldn't necessarily drop RTMP connection. 🚢

[j0sh]

@ericxtang Can we roll livepeer/lpms#82 into this change set?

[ericxtang]

@j0sh Yep!