Improve wraparound check parameter search

[junyechen1996]

The improvements come as:

• We now look for the smallest r_succ that satisfies soundness, then searches for alpha (if possible) that can satisfy ZK.
• Then combine with dimension to compute the number of proofs to satisfy the overall soundness.

Turns out with num_frac_bits = 24, neither Field56, nor Field48 can satisfy the field size requirement in wraparound checks.

[junyechen1996]

Even with r = 1500, we cannot find possible alpha that can satisfy 2^-100 ZK error for Field48 and Field56, when num_frac_bits = 24, but it's possible for num_frac_bits = 15,

|r         |r_succ    |l2_norm   |frac_bits |field     |-log2(eta)     |-log2(zk)      |-log2(sound)   |overhead  |alpha               |
|:---------|:---------|:---------|:---------|:---------|:--------------|:--------------|:--------------|:---------|:-------------------|
|1500      |971       |1         |15        |Field48   |3              |376            |99             |27000     |1.6920026163616637  |
|1500      |971       |1         |15        |Field56   |3              |376            |99             |27000     |1.6920026163616637  |
|1500      |971       |1         |15        |Field64   |3              |376            |99             |27000     |1.6920026163616637  |
|1500      |971       |1         |24        |Field64   |3              |376            |99             |40500     |1.6920026163616637  |
|1500      |971       |1         |15        |Field128  |3              |376            |99             |27000     |1.6920026163616637  |
|1500      |971       |1         |24        |Field128  |3              |376            |99             |40500     |1.6920026163616637  |

[cjpatton]

I haven't worked my way through the code all the way yet, but I have a lot of comments and wanted to pause here. Two high level things:

1. The code interleaves pretty printing with the actual logic for computing the parameters. It would be better if the parameters were computed first and stored in some data structure, then the data structure gets printted. That way the reader isn't distracted by irrelevant code.
2. Let's get rid of the code for generating CSV. Note that I had suggested in an earlier PR that this might be more useful than pretty printing: I meant that we should do one or the other, not both. Let's wait to write code until we need it. If you prefer pretty printing, let's go with that.

[junyechen1996]

I refactored the code more based on the feedbacks:

• I decoupled the search logic and printing logic, so now there are two functions that do the searching: search_wr_params and search_vdaf_params (not sure about the name of the second one, it's basically computing the number of proofs for FLP soundness), and for printing: display_wr_params and display_vdaf_params.
• I won't compute the max_alpha given Lemma 3.3. Instead I used a fixed max_alpha, e.g., 10, and then do the binary searching of alpha between min_alpha and max_alpha, and delegate the field size checking of each alpha to PineValid.__init__. This makes the program a little more inefficient, but a lot cleaner, since we can leave Lemma 3.3 logic in PineValid.__init__ completely.
• Removed bin_pmf, instead I binary search num_wr_successes. This still runs reasonably fast.
• Added more code comments.

I squashed commits, since reviewing the new diff on top of the initial diff might get really confusing.

[junyechen1996]

With num_frac_bits = 20, Field56, there are some interesting outputs (num_wr_successes < num_wr_checks):

Parameters to satisfy 2^-100 soundness and 2^-100 ZK error
Failed to find alpha in [0.8325546111576977, 10.0] with any combination of num_wr_checks and num_wr_successes, for user parameters l2_norm_bound = 1, num_frac_bits = 20, field = Field48.
Displaying feasible set of wraparound check and user parameters:
|l2_norm   |frac_bits |field     |num_wr_checks  |num_wr_successes    |alpha               |-log2(eta)     |-log2(zk)      |-log2(sound)   |overhead  |
|:---------|:---------|:---------|:--------------|:-------------------|:-------------------|:--------------|:--------------|:--------------|:---------|
|1         |15        |Field48   |100            |100                 |8.639207325093722   |106            |99             |100            |2100      |
|1         |15        |Field56   |100            |100                 |8.639207325093722   |106            |99             |100            |2100      |
|1         |20        |Field56   |150            |140                 |3.2676572925689347  |14             |100            |99             |3600      |
|1         |15        |Field64   |100            |100                 |8.639207325093722   |106            |99             |100            |2100      |
|1         |20        |Field64   |100            |100                 |8.639207325093722   |106            |99             |100            |2600      |
|1         |24        |Field64   |100            |100                 |8.639207325093722   |106            |99             |100            |3000      |
|1         |15        |Field128  |100            |100                 |8.639207325093722   |106            |99             |100            |2100      |
|1         |20        |Field128  |100            |100                 |8.639207325093722   |106            |99             |100            |2600      |
|1         |24        |Field128  |100            |100                 |8.639207325093722   |106            |99             |100            |3000      |

Displaying full VDAF parameters:
|l2_norm   |frac_bits |dimension |chunk_len |field     |proofs    |num_wr_checks  |num_wr_successes    |-log2(zk) |-log2(sound)        |
|:---------|:---------|:---------|:---------|:---------|:---------|:--------------|:-------------------|:---------|:-------------------|
|1         |15        |1000      |57        |Field48   |3         |100            |100                 |99        |99                  |
|1         |15        |10000     |110       |Field48   |3         |100            |100                 |99        |99                  |
|1         |15        |100000    |319       |Field48   |3         |100            |100                 |99        |99                  |
|1         |15        |1000      |57        |Field56   |3         |100            |100                 |99        |99                  |
|1         |15        |10000     |110       |Field56   |3         |100            |100                 |99        |99                  |
|1         |15        |100000    |319       |Field56   |3         |100            |100                 |99        |99                  |
|1         |20        |1000      |69        |Field56   |3         |150            |140                 |100       |99                  |
|1         |20        |10000     |117       |Field56   |3         |150            |140                 |100       |99                  |
|1         |20        |100000    |322       |Field56   |3         |150            |140                 |100       |99                  |
|1         |15        |1000      |57        |Field64   |2         |100            |100                 |99        |99                  |
|1         |15        |10000     |110       |Field64   |2         |100            |100                 |99        |99                  |
|1         |15        |100000    |319       |Field64   |2         |100            |100                 |99        |99                  |
|1         |20        |1000      |61        |Field64   |2         |100            |100                 |99        |99                  |
|1         |20        |10000     |113       |Field64   |2         |100            |100                 |99        |99                  |
|1         |20        |100000    |320       |Field64   |2         |100            |100                 |99        |99                  |
|1         |24        |1000      |64        |Field64   |2         |100            |100                 |99        |99                  |
|1         |24        |10000     |114       |Field64   |2         |100            |100                 |99        |99                  |
|1         |24        |100000    |321       |Field64   |2         |100            |100                 |99        |99                  |
|1         |15        |1000      |57        |Field128  |1         |100            |100                 |99        |99                  |
|1         |15        |10000     |110       |Field128  |1         |100            |100                 |99        |99                  |
|1         |15        |100000    |319       |Field128  |1         |100            |100                 |99        |99                  |
|1         |20        |1000      |61        |Field128  |1         |100            |100                 |99        |99                  |
|1         |20        |10000     |113       |Field128  |1         |100            |100                 |99        |99                  |
|1         |20        |100000    |320       |Field128  |1         |100            |100                 |99        |99                  |
|1         |24        |1000      |64        |Field128  |1         |100            |100                 |99        |99                  |
|1         |24        |10000     |114       |Field128  |1         |100            |100                 |99        |99                  |
|1         |24        |100000    |321       |Field128  |1         |100            |100                 |99        |99                  |

[cjpatton]

Huge improvement, thanks for the hard work. One blocking change request, which is to preoprly compute Field48.gf (likewise for Field56.gf).

[cjpatton]

Likewise below.

Suggested change

	gf = Field64.gf
	gf = GF(MODULUS)

[junyechen1996]

I tried that for Field48 for example. And it fails with this assertion:

val = Field48(2**15)
assert 2**15 == val.as_unsigned()

I checked the val property in Field becomes 0, after computing self.gf(val). Any idea?

[junyechen1996]

Side note: I don't think the initializer is performing any field operations, e.g. the computation of sq_norm_bound, etc.

[cjpatton]

Well this was a pretty fun bug!

sage: GF(2^48.previous_prime())
Finite Field in z47 of size 2^47
sage: GF((2^48).previous_prime())
Finite Field of size 281474976710597

It turns out that 2^48.previous_prime() is evaluated as 2^(48.previous_prime()) == 2^47. In fact, GF(2^47) is a finite field, just not a prime order finite field, which we need.

2**15 is represented as 0 in this field:

sage: f = GF(2^48.previous_prime())
sage: f(0)
0
sage: f(1)
1
sage: f(2)
0
sage: f(3)
1
sage: f(2**15)
0

[cjpatton]

GF(2^47) is known as an extension field. Its elements are actually represented as polynomials. I think what sage is doing is interpreting the input as the coefficient of a degree-0 polynomial in the field.

[cjpatton]

By the way, feel free to just not define gf ... that way the code fails noisily if we try to use it as a fully fledged field.

[junyechen1996]

Good catch, thanks! Not defining gf will not let me instantiate a field element from an integer though.

[junyechen1996]

It seemed to have changed the results too. It seems that for Field48 and above, we can just use a large alpha (e.g. 8.63), and for Field40, alpha is 1.90, num_wr_checks = 450, and num_wr_successes = 344.

[junyechen1996]

Open thread: #81 (comment). Not sure why after computing GF(Field48.MODULUS), initializing a Field from an integer always makes Field.val be 0.

[cjpatton]

Well this was a pretty fun bug!

sage: GF(2^48.previous_prime())
Finite Field in z47 of size 2^47
sage: GF((2^48).previous_prime())
Finite Field of size 281474976710597

It turns out that 2^48.previous_prime() is evaluated as 2^(48.previous_prime()) == 2^47. In fact, GF(2^47) is a finite field, just not a prime order finite field, which we need.

2**15 is represented as 0 in this field:

sage: f = GF(2^48.previous_prime())
sage: f(0)
0
sage: f(1)
1
sage: f(2)
0
sage: f(3)
1
sage: f(2**15)
0

[cjpatton]

GF(2^47) is known as an extension field. Its elements are actually represented as polynomials. I think what sage is doing is interpreting the input as the coefficient of a degree-0 polynomial in the field.

[junyechen1996]

Squashed.