O & B self-verification workflow #32
Comments
yondonfu
changed the title
Assuming signatures would be eventually sent for the case of blind inserts, a B has the recourse of stopping work with the O if no signatures are received after some period. The O has an incentive to retain business, so has little reason to withhold signatures - and any anticipated verification failures should be signaled somehow. There can also be other forms of social pressure, eg a community block list of Os that are clearly violating expectations. Depending on the confidence in the verification algorithm, Os might want to offer a lower price in exchange for withholding signatures. This allows them to avoid performing any extra computation for verification, and avoids the specter of slashing. |
That depends on the goal of O and in my opinion we should not make assumptions whether its goals are short-term or long-term and its behaviour rational or purely economical. This becomes especially true in a slashing context. An O might very well withhold signatures if those impose a slashing condition from which the economic penalty is greater than reward from the amount of business it can retain for the foreseeable future. A slash of 1-10% would (or should) equal the value of multiple weeks of work (and many transcoded streams). This value can not be compensated through retaining business from B for the lifetime of its stream. A slashing condition might also imply O to be selected less frequently for future jobs. From such perspective the decision of O to send or withhold signatures is a purely economic one and not rational.
From a broadcaster perspective, blindly inserting unsigned segments can lead to very unwanted content being displayed and without guarantee of receiving signatures there is no certainty of being able to penalize O, or even being able to provide irrefutable proof to the community that a certain O is misbehaving (perhaps possible through certificates using eth keys) . While that specific B might stop working with that specific O, other B's have no notion of O's misconduct. Verification is an option but that would imply having to verify the segment synchronously when no signature is received, otherwise there is no way to spot serious misconduct before insertion into the playlist, but that would by itself have latency implications. |
|
Is it feasible to consider that B would just transcode the segment itself in the case of receiving an This may be impractical from a video tech perspective though - if B doesn't have similar hardware to O, then the B-encoded-segment may not look like the other segments in the video. If its definitely not possible for B to transcode the segment, then I could see a scenario where it makes the decision to either trust the result from one of the redundant O's who also returned an unsigned segment - or to pay more to a centralized, identity based, business reputation backed node to transcode the segment and just trust the result. IE use the open network to get the lowest possible price, but when you get an unsigned segment pay the Livepeer Inc premium (if there is one) to get Livepeer to do it and trust the output. |
In theory that sounds great but if we are going by Livepeer Inc providing a premium service, then isn't there a conflict of interest here since we are also developing the network? And if it's not livepeer inc providing this service but a third party then there would still be a conflict of interest because it would be a entity likely vetted/chosen by livepeer inc, I think. |
|
It was just an example to indicate that there are trusted brand actors who may have higher costs associated with providing a high reliability service on the network. Not committing LP to running a premium service but just suggesting that there will likely be other actors who can win work at lower prices on the public network.
…________________________________
From: Nico Vergauwen <notifications@github.com>
Sent: Thursday, July 25, 2019 3:17 PM
To: livepeer/research
Cc: Doug Petkanics; Comment
Subject: Re: [livepeer/research] O & B self-verification workflow (#32)
but when you get an unsigned segment pay the Livepeer Inc premium (if there is one) to get Livepeer to do it and trust the output.
In theory that sounds great but if we are going by Livepeer Inc providing a premium service, then isn't there a conflict of interest here since we are also developing the network?
I'm not saying that we would abuse the conflict, but it exists.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub<#32?email_source=notifications&email_token=AAACRFIM5QDGVNNWRHXIXJTQBH355A5CNFSM4IFZ6OWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD22QA2Q#issuecomment-515178602>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAACRFIRRNZ4AJXUWKCON7TQBH355ANCNFSM4IFZ6OWA>.
|
|
I actually think it's the other way around: an O offering "trusted" transcoding without self-verification would be priced lower. There will be a risk premium baked into any slashable service, plus any added cost stemming from the overhead of checking. The implicit cost of that trust is the B needing to verify the segments itself, or accept the risk of tampering. A high-end trusted service may choose to charge its own premium, but I expect downward pricing pressure as more orchestrators gain a reputation for reliability. By the way, if we do opt for a "verification-less" approach, segments should still be signed; there just wouldn't be any penalty associated with doing the work incorrectly. In that case, malfeasance can be displayed for the world to see (along with associated signature), creating social pressure. |
The purpose of this issue is to organize discussion around orchestrator (O) and broadcaster (B) behavior in a workflow where orchestrators self-verify results before returning them to broadcasters. We assume that the verification method relies on a machine learning classifier that has a false negative rate > 0% (i.e. it is possible that the classifier labels a valid transcode result as an invalid transcode result). We also assume that O is penalized if it fails verification when B provides source & result data as well as signatures to a third party verifier. The goal of a self-verification workflow is to allow O to check if a result would fail verification (i.e. due to a false negative) and then change its behavior accordingly in order to avoid penalization when doing honest work.
Results without signatures?
A basic sketch of a self-verification workflow is as follows:
Candidates for X include:
InvalidResultErrorerror without the resultIt is worth thinking about B's possible behavior given these candidates for X i.e. is one option more helpful for B such that B can expect that behavior from O and if the expectation is not met B can consider O faulty.
Before we discuss option 1, it is worth establishing why O's signature over the result is important. The signature cryptographically links the result to O's ETH address which is associated with on-chain stake held by a contract. Thus, if B has O's signature, it can present an invalid result to a third party verifier to penalize O. Additionally, the signature authenticates the sender of the result such that B knows that the result was produced by the O it expects to receive results from (based on O's ETH address). We might not need to be as reliant on signatures for authentication of a result sender if B & O can use a TLS enabled connection with certificates self-signed using ETH keys.
The potential benefit of option 1 is that B could still use the result in its playlist. However, this is only a benefit if B has a way to determine whether to use the result or not. B shouldn't just outright use a result without a signature because B does not know that it can penalize O if the result is faulty. So, B would just be blindly inserting the result into its playlist without a recourse mechanism. B could also verify the result if it does not have a signature but if verification fails it would be left in the same position that it would be in if it didn't verify the result at all. At the moment, I'm skeptical of the utility of returning a result without a signature and option 2 might accomplish the same thing.
Retrying segments with other Os?
Regardless of whether O returns a result without a signature or a special
InvalidResultErrorerror, if B is not going to use the result in its playlist, what will B do instead? One possibility is that B will just re-submit the same source segment to a different O similar to what B would do if it encountered any other error. My concern here is whether other Os will similarly fail self-verification for the same source segment and then not return a signed result to B. The question to answer regarding the ML classifier based verification method is: "if 1 O fails self-verification for a given input, is it likely that all other Os will also fail self-verification for that same input?". If this answer to this question is yes, then given a false negative rate of X%, then B would expect to not receive signed results from any Os X% of the time (in a live context a segment might be dropped after a timeout if no signed results are received within that period of time). If the answer to this question is no, then re-submitting the source segment might be a reasonable solution.The text was updated successfully, but these errors were encountered: