Class-21.Rmd

---
title: ""
output:
  html_document:
    toc: true
    toc_float: true
    number_sections: true   
    collapsed: false
---

 <!--


rmarkdown::render_site("Class-21.Rmd")# build website
library(knitr)
rmarkdown::render_site()# build website

source("_build.R")
build("Class-21",bld="PAGE",docs=TRUE) # bld = PAGE,ENTIRE,SCRIPT

# PURL THIS SHIT & MOVE FILES TO DOCS
build("Class-21",bld="SCRIPT",docs=TRUE) # bld = PAGE,ENTIRE,SCRIPT

source("_build.R");build("Class-21",bld="PAGE",docs=TRUE)# bld = PAGE,ENTIRE,SCRIPT

-->


```{r echo=FALSE, out.width="100%"}
library(knitr)
include_graphics("media/banner-07.jpg")
rm(list=objects())
```


# Class 21: Monitoring, Information, and ARM {-}

# Class preliminaries

* Supplemental background reading for next class(es):
<!--
********** add some reading
-->
* Reading(s) for next time:
    * Conroy and Peterson Chapter 7
    * Conroy and Peterson Chapter 9
    * Moore, J. L., and M. C. Runge. 2012. Combining Structured Decision 
    Making and Value-of-Information Analyses to Identify Robust Management 
    Strategies. Conservation Biology 26:810-820.[PDF](pdfs/M234.pdf) 
    * Canessa, S., G. Guillera-Arroita, J. J. Lahoz-Monfort, D. M. 
    Southwell, D. P. Armstrong, I. Chadès, R. C. Lacy, and S. J. Converse. 
    2015. When do we need more data? A primer on calculating the value of 
    information for applied ecologists. Methods in Ecology and 
    Evolution:1219-1228. [PDF](pdfs/C262.pdf)
* Class project: 
    * Be developing your decision model
    * Final exam period-April 28th at 3pm.
* Link to class recording  [YouTube]()
* Today's R script [Class-21.R](scripts/Class-21.R)

## Class overview & objectives 

The objectives of this class are to:

1. Formally incorporate monitoring into decision making
2. Calculate the value of perfect and imperfect information
3. Formally use monitoring to learn

## Getting ready to go

A couple of networks we will be working with

* [Value of perfect information](nets/class-21/EVPI.neta)
* [Value of imperfect information](nets/class-21/EVII.neta)

Packages you may want to install to play along.

```{r}
#install.packages("msm")
#install.packages("MDPtoolbox")
```

# Value of information 

While we are trained to think about sampling and monitoring in terms of 
statistical power and more being better to reduce uncertainty it is 
difficult to evaluate how information and additional information applies 
to decision making. We have seen how sensitivity analyses can help us 
identify what components of our decision models are important that is 1 
part of the puzzle. Specifically we might be interested in how 
information improves decision making (note this is different than 
understanding) and just how much information do we need to collect to 
make a good decision. In some cases additional information does not do 
much for us. We can value information in terms of our utility (maps to 
objectives not understanding) and calculate the value of that 
information. There are 2 types of information we can value, perfect and 
imperfect and we will look into both of those. 



## Expected value of perfect information

The expected value of perfect information (EVPI) is the increase in the 
expected value of a decision if the 'true' value of a model component(s) 
or the relationship among components is known with certainty. Thus, it 
can be used, in part, to identify and rank potential variables for 
monitoring or additional data collection efforts. 

To illustrate the calculation of EVPI, consider a simple conservation 
decision model illustrated below where the decision to conduct a 
management action affects the degree of environmental disturbance which 
in turn, affects the future status of an animal population. The future 
population status also is affected by the current status of the 
population and takes two states, 'present' and 'absent'. 


```{r echo=FALSE, out.width="100%"}
library(knitr)
include_graphics("media/class-21/Picture1.png")
```


The value of the decision depends on the management action and the 
future population status. The _optimal_ decision and value when the 
probability of current species presence is 50% is 'low intensity' with 
an expected value of 9.65. 

If we had information from a space age sensor or a clairvoyant told us 
what the the species is present with a 100% probability (i.e., absolute 
certainty), the optimal decision is 'none' with a value of 18.25. If 
that same sensor or clairvoyant told us the species is absent with a 
100% probability, the optimal decision is 'high intensity' with a value 
of 5.45. 

Unfortunately, we do not have the space age sensor and none of you are 
clairvoyant as far as I can see, so we assign the probabilities of 
current species presence and absence in the model are 50%-50% (i.e., 
complete uncertainty), or we might use previous information estimate 
from an occupancy model. These are our best estimates of the probability 
that the species is present, so we use them to weight the expected 
values of the optimal decision under perfect information as follows: 


$$ 18.25 \cdot 0.5 + 5.45 \cdot 0.5 = 11.89 $$


This is the expected value of the decision if the population status was 
known with certainty. EVPI is calculated as the difference between the 
expected value with and without a knowing the current population status
calculated as:

 
$$11.89 - 9.65 = 2.20\text{.}$$


### More than 2 states

The preceding was a simple example of two discrete states (present or absent). Let’s 
assume that rather than presence and absence, we had the density of the 
species with a mean density ($\lambda$) of 10 and density was discretized into 
4 states: 0-8, 8-11, 11-20, and 20+ animals per unit area. The 
probabilities of each state can be calculated using the following R 
code: 


```{r}
lambda <- 10 
brks<-c(0,8,11,20,1000)
outcomes<-rpois(50000,lambda)
states<- cut(x=outcomes,
    breaks= brks,
    labels=c("0-8","8-11","11-20","20+"),
    include.lowest=TRUE)
counts<- table(states)
probs<- prop.table(counts)
probs
```


Assume that the expected values of the optimal decisions, given 
abundance was with 100% probability in states 1 through 4 (i.e., we used 
our space age sensor or clairvoyant) are 10, 15, 21, and 23, 
respectively. The weighted expected value of when we have perfect 
information is: 


```{r}
values<-c(10,15,21,23)
weightedValues<- values*probs
# EXPECTED VALUE WITH PERFECT INFORMATION
ev<- sum(weightedValues) 
```


Now if the the expected value for the optimal decision was 9.8 then the 
value of perfect information is $`r round(ev,2)`-9.8$ which is $`r 
round(ev,2)-9.8`$. 


## Expected value of imperfect information

Although the EVPI can be useful as a first approximation, it is usually 
not realistic to expect information to be perfect. To account for the 
error in measurements or models, requires the estimation of the expected 
value of imperfect information (EVII). EVII is considerably more 
complicated to estimate than EVPI. It requires an estimate of the 
expected outcome and the use of Bayes rule to calculate probabilities 
and expected values. 

We illustrate calculating EVII using the previous conservation decision 
model. Here the decision-maker is interested in evaluating the value of 
conducting sample surveys in the potential management area to detect the 
species. One source of imperfection in sample data that would affect the 
value of conducting studies is incomplete detection. We first need an 
estimate the probability of detecting the species give it was present. 
Assume that 10 samples are needed to detect the species with an 80% 
probability, given the species is present in the area. 

The probability of detecting the species if they are not present is, of 
course, zero. Assuming that there is a 50% chance that the species is 
present, the probability of detecting the species with 10 samples is 
0.8*0.5 = 0.4 and the probability of not detecting them is 1.0 - 0.4 = 
0.6. 

This new component is added to the model and is graphically displayed in 
an influence diagram with current species presence pointing into a 
sampling result node. 

If the species is detected, we know it is present. If the species is 
not detected, the species may be:

1. absent or  (true negative)
2. be present but was missed during sampling (false negative). 

To incorporate the possibility of falsely concluding the species was 
absent, we need to use Bayes rule to estimate the probability the 
species is present, given it was not detected, which requires four 
estimates: 

1. Prior probability of critter being present (0.5)
2. Prior probability of critter being absent (0.5)
3. Probability of detecting the critter given it is present (0.8)
4. Probability of detecting the critter given it is absent (0)


```{r}
# prior probability of presence 
priorPresent<- 0.5 
#prior probability of absence 
priorAbsent<- 1- priorPresent 
```

These numbers are our original values where we acknowledged we had total 
uncertainty (i.e., either outcome is equally likely). Now we formally incorporate
sampling or monitoring by adding a node that would represent the results of a field
effort. We make an assumption that the detection probability is 0.8, which may come
from some occupancy studies or the literature.


```{r}
#probability of detection given presence 
detectionProbability<-0.8 
#probability not detection given absence  (no false positives)
nonDetecionProbability <-1 
```


This is a simple case for the binomial but if abundance was used some
simulation would be in order to figure out what the probability of detecting
1 or more of the the critters given the critters were there. But for the sake of
coding here is how we tackle it here. If the probability of detection is 0.8 given
the critter is there the probability of not detecting the critter is 1-0.8. Let's 
confirm.


```{r}
# PROBABILITY OF DETECTION GIVEN PRESENT
detections<- rbinom(500000,1,detectionProbability)
counts<- table(detections)
probs<- prop.table(counts)
probs[1] # Present and not detected
probs[2] # Present and detected
```
<!--
Bayes Theorem

The core of Bayesian inference and Bayes approaches to updating 
information is Bayes’ Theorem. To understand Bayes’ Theorem (BT) we 
first have to understand what a conditional probability is. A 
conditional probability (or distribution) is simply the probability of 
an event y given that some other event $x$ occurs. As a simple example, 
$y$ could be the event of drawing a spade from a poker deck and x is the 
event of drawing a heart. Either of these events can occur by themselves 
with probability $p(y)$, $p(x)$ respectively (=1/4 each in this case). 
The rule for joint probabilities specifies that the probability of the 
events $y$ and $x$ jointly occurring is:

$$P(y \bigcap x) = p(y \textpipe x) p(x)$$

where p(y | x) is the conditional probability of y given x. For example, p(y | x) might be the
probability of drawing a spade if a heart has already been drawn (and not replaced in the deck: in
this case, 13/51 . Thus, the joint probability of drawing a heart or a spade is

P( y \bigcap x) = p(y \textpipe x) p(x) = \frac{13}{51}\cdot \frac{1}{4}

In the equation to numerator is the probability of a critter being 
present and not detected 

$$p(\theta|x)=\frac{p(x|theta)\cdot p(\theta)}{p(x)}\text{.}$$

in this example we can break out the the components as:

* $p(x|theta)$ the probability of detecting the critter given it is present
* $p(\theta)$ prior probability of critters being present
* $p(x)$ is the probability of x
-->

We are not going to go into depth on Bayes' theorem but it is a way to 
update probabilities given new information. The general form of Bayes' 
theorem is the the posterior probability of a critter begin absent given 
the species was not detected is equal to the probability of non 
detection times the prior of a critter being present divided by the 
probability of non detection times the prior of a critter being present 
plus the probability of non detection given absences times the prior 
probability of being absent. 


```{r}
post<-probs[1]*priorPresent/
    (probs[1]*priorPresent + nonDetecionProbability*priorAbsent) 
post 
```

The probability of absence given the species was not detected is the 
compliment $1- 0.1667 = 0.8333$. But let's make sure we can prove this.

```{r}
post<-nonDetecionProbability*priorAbsent/
    (probs[1]*priorPresent + nonDetecionProbability*priorAbsent) 
post 
```

If we are really interested we can pretty easily calculate the 
posterior probabilities for population statuses given sampling
outcomes. First we need to make a matrix for the outcomes of sampling
given the population status.

```{r}
Z<-matrix(c(0.8,0.2,0,1),
    ncol=2, 
    byrow=TRUE,
    dimnames=list(c("Present","Not present"),
     c("Detected","Not detected")))
Z
```
That looks exactly like the CPT in the netica model.

Now we are going to set up a matrix of prior probabilities
for whether the species is present or not.


```{r}
p<-matrix(c(0.5,0.5),ncol=2,nrow=2, byrow=TRUE,
    dimnames=list(c("Present","Not present")))
```

Now we can multiply the transpose of Z by p and then 
divide by the sum of the rows to get the posterior probabilities.


```{r} 
xx<-t(Z) *p
xx/rowSums(xx)
```

The values are then entered into the model as the conditional 
probabilities of species presence, given the sampling result not 
detected. Graphically, this is depicted in an influence diagram 
reversing the arrow from `Current popn status` to sampling results 
illustrated below. The next step is to estimate the value of the optimal 
decision for the each sampling result using the new model. Open the 
influence diagram with the sampling results node and examine the 
expected value of information when the sampling result is known with 
100% certainty. 


```{r echo=FALSE, out.width="100%"}
library(knitr)
include_graphics("media/class-21/evii.png")
```
 
 
For the sampling result, 'detected', the optimal decision in 'none' with 
a value of 18.25. The same value as present = 100%. When the sampling 
result is 'not detected', the optimal decision is 'high intensity' 
management action with a value of 6.83. Notice that this is different 
from the expected value of perfect information example, where we assumed 
absence was 100%. This is because there is a 16.7% chance that the 
species occurs given it was not detected, which changes the value of the 
optimal decision. 

The final step to estimate EVII the expected values are again weighted 
by the probability of obtaining that sampling result and sum the values 
as:
 
 
$$18.25 \cdot 0.4 + 6.83 \cdot 0.6 = 11.40\text{.}$$
 

This is the expected value of the decision if sampling is conducted. We 
subtract that value from the expected value if the species was not 
sampled and we get $11.40 - 9.65 = 1.75$ as the EVII. Notice that the 
EVII value of 1.75 is lower than the EVPI value of 2.20. This should 
always be the case; the value of imperfect information should always be 
smaller than that of perfect information. Of course, this value would be 
discounted by the actual cost of collecting samples. Assuming that the 
probability of detecting a species is related to the number of samples, 
the decision maker could examine how the EVII varies with sample size to 
determine the optimal level of sampling effort. 


### Evaluating sample size

Suppose there was an occupancy study done and the detection probability 
was estimated to be 0.23. We can use EVII to calculate the value of sampling 
a certain number of sites. In other words, how many sites do we need to sample
given the decision context?

Suppose we did 1, 2, 3, 5, 10, 20, 30, 50, 75, or 100. Given the capture 
probability we can figure out the the probability of detecting a critter 
given it is the management unit by simulation, or if you are clever you 
can string together the probabilities. But let's use simulation to hone 
our `R` skills and overall understanding. 


```{r}
pDetection<-0.23
sites<-c(1,2,3,5,10,20,30,50,75,100)
prob<- c()
for(i in 1:length(sites))
    {
    prob<- c(prob, 
    # proportion of reps with at least 1 site detected
    mean(rbinom(5000,sites[i],pDetection)>=1))
    }
probs<- cbind(sites,prob,1-prob)  
```

Now if we work through the netica model as we did before and change the probabilities
of detecting the critter we can calculate the EVII for each sample size.  The 
expected values given the number of sites sampled and $p=0.23$ are in the table below and
the bold values are maximums which will be used to calculate expected values adn then EVII.

<table style="border-collapse:collapse;border-spacing:0"><tr><th style="font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top"></th><th style="font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top" colspan="3">Detected</th><th style="font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top" colspan="3">Not detected</th><th style="font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top" colspan="3">Sampled</th></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">Sites</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">None</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">Low</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">High</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">None</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">Low</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">High</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">None</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">Low</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">High</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">1</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">8.37</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">8.67</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.06</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">2</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">7.257</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">7.702</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">8.536</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">3</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">6.24</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">6.82</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">8.05</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">4.49</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">5.3</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">7.22</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">10</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">1.86</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">3.01</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">5.97</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">20</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">0.85</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">2.14</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">5.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">30</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">0.76</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">2.06</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">5.45</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">0.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">2.05</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">5.45</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">0.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">2.05</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">5.45</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">100</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">17.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">13.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">0.75</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">2.05</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">5.45</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;font-weight:bold;text-align:center;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;text-align:center;vertical-align:top">9.6</td></tr></table>

Using the probability of detected or not detecting we can calculate the 
expected value 

<table style="border-collapse:collapse;border-spacing:0"><tr><th style="font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top" colspan="2">Detected</th><th style="font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top" colspan="2">Not detected</th><th style="font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top"></th><th style="font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top"></th><th style="font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top"></th></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">Probability</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">EV</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">Probability</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">EV</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">EV sampling</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">EV no sampling</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">EVII</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">11.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">88.6</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.06</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">10.12591</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">0.47591</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">20.4</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">79.6</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">8.536</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">10.517656</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">0.867656</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">27.1</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">72.9</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">8.05</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">10.8142</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">1.1642</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">36.4</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">63.6</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">7.22</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">11.23492</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">1.58492</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">46.6</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">53.4</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">5.97</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">11.69248</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">2.04248</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">49.7</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50.3</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">5.5</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">11.83675</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">2.18675</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">5.45</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">11.85</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">2.2</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">5.45</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">11.85</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">2.2</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">5.45</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">11.85</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">2.2</td></tr><tr><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">18.25</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">50</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">5.45</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">11.85</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">9.65</td><td style="font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:1px;overflow:hidden;word-break:normal;vertical-align:top">2.2</td></tr></table>

The results of calculating EVII for the detection probabilities above are 
specified in the `R` code below.

```{r}
evii<- c(0.48, 0.87, 1.16, 1.58, 2.04, 2.19, 2.20, 2.20, 2.20, 2.20) 
```

Recall the EVPI was 2.2. The values of EVII are equal to or less than EVPI which is
to be expected. but the neat thing is you can evaluate what more sampling gets you 
in terms of making a decision! Let's visualize this.

```{r}
plot(x=sites,y=evii,
    xlab="Number of sites to sample",
    ylab="Value of information",
    pch=19)
abline(h=2.2,lty=2)# Value of perfect information
```

Wow, more data does not improve our decision! Keep in mind this is different 
than statistical power, here the value of information is in context of the decision
and not making a type 1 or type 2 error. In terms of making a decision 20 sites is 
just as good as 100! That saves a manager quite a bit of effort.



# Introduction to Adaptive Resource Management

Here we will jump into some slides that begin to cover adaptive resource management.


<!--

## Amphibian model

### Current Number of Wetlands Node

#### Calculating EVPI


| Current # wetlands 	| Prior probability 	| Utility assuming perfect information 	| Weighted utility value^a^	|
|--------------------	|-------------------	|--------------------------------------	|-------------------------	|
| 0                  	| 0.673841          	| 2.78                                 	| 0.018733                	|
| 1                  	| 3.36921           	| 2.78                                 	| 0.093664                	|
| 2                  	| 8.42301           	| 2.95                                 	| 0.248479                	|
| 3                  	| 14.0384           	| 6.23                                 	| 0.874592                	|
| 4                  	| 17.5479           	| 10.43                                	| 1.830246                	|
| 5                  	| 17.5479           	| 14.86                                	| 2.607618                	|
| 6                  	| 14.6233           	| 19.45                                	| 2.844232                	|
| 7                  	| 10.4452           	| 24.78                                	| 2.588321                	|
| 8                  	| 6.52825           	| 28.53                                	| 1.86251                 	|
| 9                  	| 3.62681           	| 31.92                                	| 1.157678                	|
| 10                 	| 1.8134            	| 35                                   	| 0.63469                 	|
| 11                 	| 0.824275          	| 38.48                                	| 0.317181                	|
| 12                 	| 0.343448          	| 40.04                                	| 0.137517                	|
| 13                 	| 0.132095          	| 40.04                                	| 0.052891                	|
| 14                 	| 0.047177          	| 44.035                               	| 0.020774                	|
| 15                 	| 0.015726          	| 45.96^b^                             	| 0.007227                	|

^a^ $\frac{\text{Prior probability}}{100} \cdot \text{Utility assuming perfect information}$  
^b^ See Figure 1 for how this value is obtained


Expected value = 15.29635(sum of last column)

Expected value of perfect information= 15.29635 - 15.2859 = 0.010452 
(not too much value in perfect information, this is supported by the 
sensitivity analysis where the utility is relatively insensitive to 
changes in the current number of wetlands 



```{r echo=FALSE, out.width="100%"}
library(knitr)
include_graphics("media/class-21/amphib-evi.png")
```

Figure 1. Utility value (31.58) assuming that the current number of 
wetlands is known with certainty to be 15. 



### Calculating EVII

To calculate EVII we need to add a node that represents information 
gained by sampling. So in this case I am going to add a node that 
represents going out and estimating the current number of wetlands. 
However this sampling underestimates the 'true' number of wetlands 
by 1 (See Ampib_EVII.neta attached). 


#### Methods:

1. Click on Current number of wetlands node, right click and select 
'copy' 
2. Rename the node to be est_wetlands with a title `Estimated number 
of wetlands` 
3. Draw an arrow from the `Current number of wetlands node` to the 
estimated `number of wetlands node` (Figure 2) 
4. Enter the equation: `p (est_wetland |Cur_wetland ) = 
NormalDist(est_wetland,Cur_wetland,0.5)`. This is saying that given the 
current number of wetlands sampling will be a bit fuzzy assuming a 
normal distribution around the current ("true" number of wetlands) 
5. Apply the equation to the table:  table->equation to table
6. Click okay and compile the network
7. Set the estimated number of wetlands to 0 and get the maximum utility 
and repeat for all the states in the node. 
8. Calculate EVII as in Table 2.

```{r echo=FALSE, out.width="100%"}
library(knitr)
include_graphics("media/class-21/amphib-evii.png")
```

Figure 2. Illustration of adding a sampling node to the current number of wetlands.


Table 2. Calculation of expected value of imperfect information (EVII).

| Estimated # of wetlands 	| Prior probability 	| Utility assuming imperfect information 	| Weighted utility value^a^ 	|
|-------------------------	|-------------------	|----------------------------------------	|-------------------------	|
| 0                       	| 0.954314          	| 2.78                                   	| 0.02653                 	|
| 1                       	| 3.63146           	| 2.78                                   	| 0.100955                	|
| 2                       	| 8.48327           	| 3.52                                   	| 0.298611                	|
| 3                       	| 13.8123           	| 6.59                                   	| 0.910231                	|
| 4                       	| 17.1712           	| 10.55                                  	| 1.811562                	|
| 5                       	| 17.2338           	| 14.79                                  	| 2.548879                	|
| 6                       	| 14.4885           	| 19.27                                  	| 2.791934                	|
| 7                       	| 10.4731           	| 24.23                                  	| 2.537632                	|
| 8                       	| 6.63725           	| 28.1                                   	| 1.865067                	|
| 9                       	| 3.74369           	| 31.45                                  	| 1.177391                	|
| 10                      	| 1.90201           	| 34.53                                  	| 0.656764                	|
| 11                      	| 0.878941          	| 37.77                                  	| 0.331976                	|
| 12                      	| 0.372443          	| 39.67                                  	| 0.147748                	|
| 13                      	| 0.145708          	| 40.04                                  	| 0.058341                	|
| 14                      	| 0.053144          	| 42.86                                  	| 0.022778                	|
| 15                      	| 0.018905          	| 45.44^b^                                 	| 0.008591                	|

^a^ [Prior probability]/100•[Utility assuming imperfect information]
^b^ See Figure 3 to see how this number as obtained.


Expected value of imperfect information= 15.29499 - 15.2859 = 0.009088 
(EVII is less than the EVPI of 0.010452; not too much value in imperfect 
information, this is supported by the sensitivity analysis where the 
utility is relatively insensitive to changes in the current number of 
wetlands). Sampling had better be really cheap/easy to be worthwhile in 
terms of the decision! 


# Adaptive resource management


## Calculating posterior probabilities (model weights) using monitoring data

```{r}
est.flow.mod_decrease.power <- 30
est.predation.mod_decrease.power <- 15
est.flow.mod_control.flatheads <- 15
est.predation.mod_control.flatheads <- 25
```

## Model weights

```{r}
Flow <- 0.5
Predation <- 0.5
```
## Expected value

```{r}

Decrease.power.generation<- Flow*est.flow.mod_decrease.power + Predation*est.predation.mod_decrease.power
Control.flatheads <- Flow*est.flow.mod_control.flatheads + Predation*est.predation.mod_control.flatheads

Decrease.power.generation
Control.flatheads
```



```{r}
## observed abundance after management action Decrease.power.generation
obs<- 21

## conditional Likelihoods for Decrease.power.generation action
flow.like<-dpois(obs,est.flow.mod_decrease.power)
predation.like<-dpois(obs,est.predation.mod_decrease.power)
```


### Posterior probabilities

```{r}
## used Bayes rule to calculate posterior probabilities
Flow.post<-flow.like*Flow/(flow.like*Flow + predation.like*Predation)
Predation.post<- predation.like*Predation/(flow.like*Flow + predation.like*Predation)

# New model weights 
Flow.post
Predation.post
```

### Updating weights

```{r}
## These posteriors become the weights for the next time step
Flow <- Flow.post
Predation <- Predation.post

## expected value
Decrease.power.generation<- Flow*est.flow.mod_decrease.power + Predation*est.predation.mod_decrease.power
Control.flatheads <- Flow*est.flow.mod_control.flatheads + Predation*est.predation.mod_control.flatheads

Decrease.power.generation
Control.flatheads

## observed abundance after management action Control.flatheads
obs<- 18

## conditional Likelihoods for Control.flatheads
flow.like<-dpois(obs,est.flow.mod_control.flatheads)
predation.like<-dpois(obs,est.predation.mod_control.flatheads)


## used Bayes rule to calculate posterior probabilities
Flow.post<-flow.like*Flow/(flow.like*Flow + predation.like*Predation)
Predation.post<- predation.like*Predation/(flow.like*Flow + predation.like*Predation)

# New model weights, these will become the prior for the next time step
Flow.post
Predation.post
```







# Population dynamics & decisions

## Overview

### Using stochastic dynamic programming


```{r,message=FALSE,warning=FALSE}
### Library needed for truncated normal
library(msm)
### library needed for SDP
library(MDPtoolbox)

rm(list=ls())
```

```{r}

## we will create transition matrices by simulating 
## harvest and population dynamics note that the way it is set up
## indicates that the harvest decision was based on the spring population
## before reproduction Also note that we are not keeping track of the number of 
## ponds to aid in decision making

combo<-merge(c(1:15),c(0:4*0.1))
colnames(combo)<-c("N_t","H_t")

attach(combo)

H_t<- rep(H_t,500)
N_t<- rep(N_t,500)
## choose harvest mortality model
har_type<-'AMH'
R_t<- 16
P_t <- 2
P_t_N=max(0.0001,-2.76+0.391*P_t+0.233*R_t)# KEEP FROM GOING NEGATIVE (equation 10 (Anderson 1975))  
#  young production to be added to fall population (equation 2 (Anderson 1975))
Y_t  = 1/((1/(12.48*P_t^0.851))+(0.519/N_t)) 

Y_t <- rtnorm(length(Y_t),Y_t,Y_t*0.3,lower = 1)

  # Fall population at time t (equation 5 (Anderson 1975))
  F_t	= (0.92*N_t) + Y_t 
  harvest<-ifelse(F_t < H_t*F_t,F_t, H_t*F_t)# harvest at time t (need to keep track of this)
  
  # AMH: ADDITIVE MORTALITY
  survival_adult_amh<- (1-0.27*exp(2.08*H_t))
  survival_young_amh<- (1-0.40*exp(0.67*H_t))
  
  # CMH: COMPENSATORY MORTALITY
  survival_adult_cmh<- ifelse(H_t<0.25,0.57,(0.57-1.2*(H_t-0.25)))
  survival_young_cmh<-  ifelse(H_t<0.25,0.5,(0.5-1*(H_t-0.25)))
  
  # Pop size after spring migration
  # AMH: ADDITIVE MORTALITY
  N.t1.amh<- N_t*survival_adult_amh + Y_t*survival_young_amh
  
  # CMH: COMPENSATORY MORTALITY
  N.t1.cmh<- N_t*survival_adult_cmh + Y_t*survival_young_cmh
  
  ## Weight and add model specific population size estimates
  harvest_type<- ifelse(har_type=='AMH', 1, 0)   
  N_t_N<- N.t1.amh*harvest_type + N.t1.cmh*(1-harvest_type)
  
  
# Discretize population sizes into 3 states
# Could do more classes this is just to simplify
Initial_N<-floor(N_t/5.5)
### prevents new pop states from arising
N_t_N <- ifelse(N_t_N < 1, 1, N_t_N)
End_N <-ifelse(floor(N_t_N/5.5)> 2, 2,floor(N_t_N/5.5)) 

## create a table of transition frequencies that will be turned into 
## state transition probabilities one for each decision alternative
TM<- table(Initial_N,End_N,H_t)

### These are now transition matrices one for each  harvest decision alternative
TM_1 <- prop.table(TM[,,1],1)
TM_2 <- prop.table(TM[,,2],1)
TM_3 <- prop.table(TM[,,3],1)
TM_4 <- prop.table(TM[,,4],1)
TM_5 <- prop.table(TM[,,5],1)


## calculate the average (expected) return for each population
## state / decision alternative combination
Return<-tapply(harvest,list(Initial_N,H_t), mean)


# Set up arrays for solving 
P <- array(0, c(3,3,5))
P[,,1] <- TM_1
P[,,2] <- TM_2
P[,,3] <- TM_3
P[,,4] <- TM_4
P[,,5] <- TM_5
R <- Return


### now find optimal state dependent harvest 
mdp_policy_iteration(P, R, discount=.99999)


  
```

-->