Skip to content

Commit

Permalink
Add matrix calculation of covariance update. #2
Browse files Browse the repository at this point in the history
  • Loading branch information
@aaronsheldon
aaronsheldon committed Apr 18, 2020
1 parent 40ff6ba commit 96e8240
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 43 deletions.
8 changes: 4 additions & 4 deletions Manifest.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ version = "0.7.7"

[[ColorTypes]]
deps = ["FixedPointNumbers", "Random"]
git-tree-sha1 = "dc232130b48be37c2612d32a3196f37f3aa50e24"
git-tree-sha1 = "f746d4fc892fdf683b5c22064c8e99b2f5b990e7"
uuid = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
version = "0.10.1"
version = "0.10.2"

[[Colors]]
deps = ["ColorTypes", "FixedPointNumbers", "InteractiveUtils", "Reexport"]
Expand Down Expand Up @@ -379,9 +379,9 @@ version = "0.12.0"

[[Parsers]]
deps = ["Dates", "Test"]
git-tree-sha1 = "75d07cb840c300084634b4991761886d0d762724"
git-tree-sha1 = "f8f5d2d4b4b07342e5811d2b6428e45524e241df"
uuid = "69de0a69-1ddd-5017-9359-2bf0b02dc9f0"
version = "1.0.1"
version = "1.0.2"

[[Pkg]]
deps = ["Dates", "LibGit2", "Markdown", "Printf", "REPL", "Random", "SHA", "UUIDs"]
Expand Down
61 changes: 22 additions & 39 deletions src/vonFoersterHazards.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -96,27 +96,15 @@
"You will note that the transition probabilities out of a single state are a [multinomial distribution](https://en.wikipedia.org/wiki/Multinomial_distribution) so that the covariance of the transitions within a single birth cohort is given by the multinomial covariance matrix. \n",
"\n",
"$$\n",
"\\begin{array}{rcl}\n",
" \\mathbb{C}ov\\left[ N^{\\left(a, \\Delta t \\right)}_{i \\rightarrow j}, N^{\\left(a, \\Delta t \\right)}_{i \\rightarrow k} \\parallel N^{\\left(a, 0 \\right)}_i \\right]\n",
" & = &\n",
" \\begin{cases}\n",
" N^{\\left(a, 0 \\right)}_i \\mathbb{P}\\left[ T^{\\left(a\\right)}_{i \\rightarrow j} \\le \\Delta t \\right] \\left( 1 - \\mathbb{P}\\left[ T^{\\left(a\\right)}_{i \\rightarrow j} \\le \\Delta t \\right] \\right)\n",
" & j = k\\\\\\\\\n",
" N^{\\left(a, 0 \\right)}_i \\mathbb{P}\\left[ T^{\\left(a\\right)}_{i \\rightarrow j} \\le \\Delta t \\right] \\mathbb{P}\\left[ T^{\\left(a\\right)}_{i \\rightarrow k} \\le \\Delta t \\right]\n",
" & \\text{otherwise}\n",
" \\end{cases}\\\\\\\\\n",
" & = &\n",
" \\begin{cases}\n",
" N^{\\left(a, 0 \\right)}_i \\frac{\\left|h_{i \\rightarrow j}\\right|}{h_{i \\rightarrow i}} \\left( 1 - e^{-h_{i \\rightarrow i} \\Delta t} \\right) \\left( 1 - \\frac{\\left|h_{i \\rightarrow j}\\right|}{h_{i \\rightarrow i}} \\left( 1 - e^{-h_{i \\rightarrow i} \\Delta t} \\right)\\right)\n",
" & j = k\\\\\\\\\n",
" N^{\\left(a, 0 \\right)}_i \\frac{\\left|h_{i \\rightarrow k}\\right|}{h_{i \\rightarrow i}} e^{-h_{i \\rightarrow i} \\Delta t} \\left( 1 - e^{-h_{i \\rightarrow i} \\Delta t} \\right)\n",
" & i = j\\\\\\\\\n",
" N^{\\left(a, 0 \\right)}_i \\frac{\\left|h_{i \\rightarrow j}\\right|}{h_{i \\rightarrow i}} e^{-h_{i \\rightarrow i} \\Delta t} \\left( 1 - e^{-h_{i \\rightarrow i} \\Delta t} \\right)\n",
" & i = k \\\\\\\\\n",
" N^{\\left(a, 0 \\right)}_i \\frac{\\left|h_{i \\rightarrow j}\\right|}{h_{i \\rightarrow i}} \\frac{\\left|h_{i \\rightarrow k}\\right|}{h_{i \\rightarrow i}}\\left( 1 -e^{- h_{i \\rightarrow i} \\Delta t}\\right)^2\n",
" & \\text{otherwise}\n",
" \\end{cases}\n",
"\\end{array}\n",
"\\mathbb{C}ov\\left[ N^{\\left(a, \\Delta t \\right)}_{i \\rightarrow j}, N^{\\left(a, \\Delta t \\right)}_{k \\rightarrow l} \\parallel N^{\\left(a, 0 \\right)}_i \\right]\n",
"=\n",
"\\begin{cases}\n",
" 0 & i \\ne k\\\\\\\\\n",
" N^{\\left(a, 0 \\right)}_i \\mathbb{P}\\left[ T^{\\left(a\\right)}_{i \\rightarrow j} \\le \\Delta t \\right] \\left( 1 - \\mathbb{P}\\left[ T^{\\left(a\\right)}_{i \\rightarrow j} \\le \\Delta t \\right] \\right)\n",
" & j = l\\\\\\\\\n",
" N^{\\left(a, 0 \\right)}_i \\mathbb{P}\\left[ T^{\\left(a\\right)}_{i \\rightarrow j} \\le \\Delta t \\right] \\mathbb{P}\\left[ T^{\\left(a\\right)}_{i \\rightarrow l} \\le \\Delta t \\right]\n",
" & \\text{otherwise}\n",
"\\end{cases}\n",
"$$\n",
"\n",
"It follows from the [Levy-Khintchine](https://en.wikipedia.org/wiki/L%C3%A9vy_process#L%C3%A9vy%E2%80%93Khintchine_representation) characterization and the [law of total covariance](https://en.wikipedia.org/wiki/Law_of_total_covariance) that the covariance of the expectation of the density of states occupancy $\\vec{n}$ at time $\\Delta t$ within a birth cohort can be recursively calculated from the prior covariances and the covariances of the transitions into each states:\n",
Expand All @@ -125,31 +113,26 @@
"\\begin{array}{rcl}\n",
" \\mathbb{C}ov\\left[ N^{\\left(a, \\Delta t\\right)}_i , N^{\\left(a, \\Delta t\\right)}_j \\right]\n",
" & = &\n",
" \\begin{cases}\n",
" \\mathbb{E}\\left[\\mathbb{V}ar\\left[ N^{\\left(a,\\Delta t\\right)}_i \\parallel \\vec{N}^{\\left(a,0\\right)} \\right]\\right]\n",
" +\n",
" \\mathbb{V}ar \\left[ \\mathbb{E}\\left[ N^{\\left(a,\\Delta t\\right)}_i \\parallel \\vec{N}^{\\left(a,0\\right)} \\right] \\right]\n",
" & i=j \\\\\\\\\n",
" \\mathbb{E}\\left[\\mathbb{C}ov\\left[ N^{\\left(a,\\Delta t\\right)}_i , N^{\\left(a,\\Delta t\\right)}_j \\parallel \\vec{N}^{\\left(a,0\\right)} \\right]\\right]\n",
" +\n",
" \\mathbb{C}ov\\left[ \\mathbb{E}\\left[ N^{\\left(a,\\Delta t\\right)}_i \\parallel \\vec{N}^{\\left(a,0\\right)} \\right] , \\mathbb{E}\\left[ N^{\\left(a,\\Delta t\\right)}_j \\parallel \\vec{N}^{\\left(a,0\\right)} \\right] \\right]\n",
" & \\text{otherwise}\n",
" \\end{cases}\\\\\\\\\n",
" \\mathbb{C}ov\\left[ \\mathbb{E}\\left[ N^{\\left(a,\\Delta t\\right)}_i \\parallel \\vec{N}^{\\left(a,0\\right)} \\right] , \\mathbb{E}\\left[ N^{\\left(a,\\Delta t\\right)}_j \\parallel \\vec{N}^{\\left(a,0\\right)} \\right] \\right]\\\\\\\\\n",
" & = &\n",
" \\begin{cases}\n",
" \\displaystyle \\sum_k \\mathbb{E}\\left[N^{\\left(a,0\\right)}_k\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right] \\left(1-\\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right]\\right)\n",
" +\n",
" \\mathbb{V}ar\\left[N^{\\left(a,0\\right)}_k\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right]\n",
" & i=j \\\\\\\\\n",
" \\displaystyle \\sum_k \\mathbb{E}\\left[N^{\\left(a,0\\right)}_k\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow j}\\right]\n",
" +\n",
" \\sum_{k,l} \\mathbb{C}ov \\left[ N^{\\left(a,0\\right)}_k, N^{\\left(a,0\\right)}_l \\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{l \\rightarrow j}\\right]\n",
" & \\text{otherwise}\n",
" \\end{cases}\n",
" \\displaystyle \\sum_k \\mathbb{E}\\left[N^{\\left(a,0\\right)}_k\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right] \\left(1-\\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right]\\right)\n",
" +\n",
" \\sum_{k,l} \\mathbb{C}ov \\left[ N^{\\left(a,0\\right)}_k, N^{\\left(a,0\\right)}_l \\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{l \\rightarrow i}\\right]\n",
" & i=j \\\\\\\\\n",
" \\displaystyle \\sum_k \\mathbb{E}\\left[N^{\\left(a,0\\right)}_k\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow j}\\right]\n",
" +\n",
" \\sum_{k,l} \\mathbb{C}ov \\left[ N^{\\left(a,0\\right)}_k, N^{\\left(a,0\\right)}_l \\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{k \\rightarrow i}\\right] \\mathbb{P}\\left[T^{\\left(a\\right)}_{l \\rightarrow j}\\right]\n",
" & \\text{otherwise}\n",
" \\end{cases}\\\\\\\\\n",
" & = &\n",
" P \\left(diag\\left(\\vec{n}\\right) + \\Sigma^2 \\right) P^\\dagger + diag\\left(\\left( P - 2 P \\circ P \\right) \\vec{n} \\right)\n",
"\\end{array}\n",
"$$\n",
"\n",
"The first summand in the covariance is the update from the covariances of the current transition and the second summand is the pull forward of the previous covariances. This recursive updating of the covariances on each iteration is sufficiently complicated that it warrants a dedicated function."
"The first summand in the covariance is the update from the covariances of the current transition and the second summand is the pull forward of the previous covariances. The last equation is a summary of the matrix algebra assuming $P$ is the matrix of probabilities, $\\Sigma^2$ is the previous covariance matrix, and $\\vec{n}$ is the previous occupancies. This recursive updating of the covariances on each iteration is sufficiently complicated that it warrants a dedicated function."
]
},
{
Expand Down

0 comments on commit 96e8240

Please sign in to comment.