-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
399 lines (280 loc) · 29.1 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
<!DOCTYPE html>
<html>
<head>
<link rel="stylesheet" type="text/css" href="style.css">
<script src='https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-MML-AM_CHTML' async></script>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {inlineMath: [["$","$"],["\\(","\\)"]]}
});
</script>
<title>An Interactive Guide to the Wonderful World of Neural Networks</title>
</head>
<body>
<section id="content">
<h1>An Interactive Guide to the Wonderful World of Neural Networks</h1>
<p>Hello!</p>
<p>I gave myself the challenge to learn as much as I could about neural networks within a week. I've noitced process of understanding it could be easy and you could learn what I learned within one to two days of what took me a week, while only knowing high school maths.</p>
<aside>Other sources that helped:
<ol>
<li><a href="https://www.youtube.com/watch?v=bxe2T-V8XRs&list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU">Welch Lab's videos on YouTube</a></li>
<li><a href="https://www.youtube.com/watch?v=kNPGXgzxoHw">Solving XOR with a hidden layer</a></li>
</ol>
</aside>
<p>During my week I constantly reread <a href="http://neuralnetworksanddeeplearning.com">Michael Nielsen's first two chapters</a> and rewatched <a href="https://www.youtube.com/watch?v=aircAruvnKk&index=2&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi">3Blue1Brown's</a> explanation with the same book as source material. While their explanations are great, there were a few things that made the process a lot longer than it should've been. I hope with this post to be able to make the introduction a bit more gentle by comparing some of the different mathematical notation they use and by using visualized interactive neural nets!
<h2>So what are artificial neural networks?</h2>
<p>A machine learning model that is loosely inspired on the real thing -- a biological brain. Researchers
were trying to mathematically model the brain back in the sixties. Instead of recreating
an organism-like brain, they created a new technique to classify cats or dogs!</p>
<aside>This is a cat. Most trained neural networks agree. Some trained neural networks might even be scared as they can't infer whether this cat is hiding or sneak up on them.<img src="cat.jpg" width="100" height="67" alt="Photo by Mikhail Vasilyev on Unsplash"></aside>
<p>So like the real thing, the artificial version has:
<ol>
<li>neurons</li>
<li>and axons which link the neurons</li>
</ol>
<p>And they are used to predict stuff! Such as whether there is a cat in a picture.</p>
<p>A <em>trained</em> artificial neural network does this well and an untrained one does
this not well at all. So in order to know what artificial neural networks are, one really
needs to know how it (1) predicts and (2) how it is trained.</p>
<p>Training an artifical neural network is mathematically speaking the harder part. For now
we will start with the easier part: how does a neural network predict something?</p>
<h2>Showing how a neural network predicts by modeling them as logic gates</h2>
<p>Let's look at the smallest and simplest neural network possible. We are going to use raw values (non-normalized values) to make it even simpler. We'll tack on complexity as we go. For now, the idea is to show the whole process with the simplest of simplest 'networks' that I can think of. Once you understand the main ideas behind the most simple network, other things are much easier to understand.</p>
<p>The smallest network possible has two layers: one input layer and one output layer. The input layer contains 1 neuron and the output layer contains 1 neuron. Try to make the output neuron display a -5. Now try to make the output neuron display a -5 while \(x = 1\)</p>
<div id="playground-simplest-ann">
<canvas id="simplest-ann" width=400 height=350></canvas>
<svg class="neuron" style="position: absolute; left:15px; top:30px">
<use xlink:href="#neuron-blue"></use>
</svg>
<svg class="neuron" style="position: absolute; left:265px; top:30px">
<use xlink:href="#neuron-blue"></use>
</svg>
<input type="range" min="-5" max="5" value="3" step="0.25" class="slider" id="simplest-ann-slider-x">
<input type="range" min="-5" max="5" value="2" step="0.25" class="slider" id="simplest-ann-slider-wx">
<!-- <input type="range" min="-3" max="3" value="1" step="0.25" class="slider" id="simplest-ann-slider-wb"> -->
<div id="simplest-ann-input-layer">Input Layer</div>
<div id="simplest-ann-output-layer">Output Layer</div>
</div>
<p>Did you get it? Do you know what process is happening for it to produce a -5?</p>
<aside>\(a\) stands for <em>activation</em>.</aside>
<p>This architecture is able to model \(f(x) = wx\), so while it is not very useful, it is very simple to understand. More formally, the two sliders are determining the value of the input neuron \(x\) and the weight \(w\), which is between \(x\) and the output neuron \(a\).
My slider values were: \(w = -5\) and \(x = 1\).</p>
<h2>Code your own neural super simple neural network</h2>
We are going to code the feedforward function of our own neural network.
<iframe width="100%" height="300" src="https://jsfiddle.net/mettamage/603fwezu/9/embedded/js,result/dark/" allowfullscreen="allowfullscreen" allowpaymentrequest frameborder="0"></iframe>
<p>The goal is to implement the return statement in JSFiddle. Look at the model to see what it should be.</p>
<h2>Automatically training the weights</h2>
<aside>Later on the idea is for you to put code examples in your developer tools of your Chrome browser.</aside>
<p>The answer to the previous exercise was: <span style="background:#f1f1f1;"><span style="color:#800000; font-weight:bold; ">return</span> w * x</span>.
<p>Let's talk about how this neural network learns things automatically. So far, we trained our small humbe neural network by fumbling around with the parameters ourselves. We played with the weight \(w\) of \(x\). We trained them on one training example: \(x = 1\).</p>
<p>Before we talk about how neural networks automatically learn through example data, let's quickly look at how I would learn a new accent through imitating another person's accent from a YouTube video. Learning by example!</p>
<p>
I listen to how the person I want to imitate is pronouncing the word and then listen to how I am saying it. I then think about what the <em>difference</em> is, adjust the difference as much as I possibly can and then imitate again. When I try to imitate again, the difference between what I am doing and what I'm trying to imitate is hopefully smaller. I iterate on this process as much as possible until I do it good enough.</p>
<aside>See <a href="https://www.youtube.com/watch?v=VBQWZmqb33I">these people learning a British accent through this method</a>! At one point, the difference is not necessarily getting smaller.</aside>
<p>
With neural networks, the output neuron has a certain value. That certain value has to match a desired value. By analogy of the previous paragraph, it would make sense that a neural network needs to learn more when that output is nowhere close the desired value compared to when it is quite close.
</p>
<p>
More formally this means that there are three topics that need to be discussed:
<ol>
<li>Defining how to calculate the difference between what is desired and whatever the neural network is doing. This is called the loss or <strong>cost function</strong>.</li>
<li>An algorithm for updating the parameters of the network (in this case only the weight \(w\)) by finding the lowest value for the cost function. This is called <strong>gradient descent</strong></li>.
<li>A specialized form of gradient descent by constantly applying the chain rule. This is called <strong>backpropagation</strong>.</li>
</ol>
</p>
<h3>
Training a Network: Cost Functions
<span class="subtitle">Also known as: knowing how wrong your prediction is</span>
</h3>
<p>Let's start with defining the cost function. The difference between the actual output value of a neuron and the desired value is called the error. For example, if the output neuron \(a = 0.2\) and the desired value \(y = 1\), then the error \(e = 0.8 \). Therefore:
$$ e = (a - y) \textrm{ and } a = wx $$
</p>
<strong>Maybe graphical example here?</strong>
<!-- Maybe put in a network to have an error exercise. -->
<aside>I will soon explain why we need to calculate partial derivatives, for now, just take it for granted.</aside>
<p>
Now that we have the error defined, we can define a cost function. A cost function is simply the same thing as calculating the error, with the difference that it is easy to use for calculating derivatives (this is needed for the gradient descent part). The idea of the cost function is the same as calculating an error: <em>it should give us an indication of how much we're wrong</em>. Because of this, there is no one right formula for defining a cost function. It's a mathematicians playground really, they get to do whatever they want as long as it makes sense for (1) knowing how wrong you are and (2) finding a way to become less wrong. In other words, all formula's are intended to minimize some error that we find important to reduce.
This means that $$ C = (a - y) $$
can be seen as a cost function. It is not a very good one however, we need to calculate the partial derivative of it later and therefore a quadratic function will be more informative. Another one I've seen is this one:
$$ C = (y - a)^2 $$
</p>
<aside><a href="https://duckduckgo.com/?q=cute+baby+animals&t=hk&iax=images&ia=images">This always makes my day</a>, it's just too cute! :3</aside>
<p>
Intuitively this makes the most sense to look at if we are talking about finding the squared error. However, I prefer the following cost function because the partial derivative is cute and I like cute.
$$ C = \frac{1}{2} (y - a)^2 $$
</p>
<p>This is how the cost function looks graphically for the onlye one training example we have: \(x = 1\). The y-axis is the cost \(C\), the x-axis are possible values for the weight \(w\). The desired value has been set to \(y = -5\) in this graph.</p>
<canvas id="simplest-cost-function-graph1" width="535" height="350">
Canvas not supported in the browser, please install Chrome or Firefox.
</canvas>
<p>From this we can infer that the best weight that matches the desired value as closely as possible is -5 (normally y-values and w-values don't overlap but in this case they're both -5). Woo! It's working! We have an idea of how wrong we are for every weight value for \(w\)!
</p>
<p>
But how did we see that? Well, we use our eyes and see that \(w = -5\) is simply the lowest point. Computers need to do a similar thing. They somehow need to 'perceive' that \(w = -5\) is the lowest point. One way of doing this is with an algorithm called <em>gradient descent</em>. This is especially handy in a high dimensional space where it is impossible to plot the graph.</p>
<aside>Gradient descent informally means descending by partial derivative since gradient means partial derivative.</aside>
<p>
With having a cost function defined, we can use that function to calculate the error which helps us to automatically adjust the weight! Currently we are able to do that analytically but in a higher dimensional space we can't do this. One way of doing this is by using gradient descent. There are other algorithms, but gradient descent is a basic algorithm used in many machine learning techniques. So the knowledge transfers quite well to other methods.
</p>
<strong>End with a visualization of what they've learned, summarize and give an exercise of programming their cost function</strong>
<h3>
Training a Network: Gradient Descent
<span class="subtitle">Also known as: knowing how to automatically tune your weights so that you're almost always right</span>
</h3>
<aside>If you want to go downhill, you can also play the game <a href="http://sinerider.com">Sinerider</a>!</aside>
<p>Derivatives and partial derivatives tell us how a function changes when we give a very gentle nudge in either the positive or negative direction. Consider a parabola \(f(x) = x^2\) as our hypothetical cost function. Then \(f'(x) = 2x\). If you want to go 'downhill', then all you need to do is slowly subtract \(2x\) with a small value repeatedly. This small value is called the learning rate \(\eta\) (pronounced as eta), so finally you'll need to subtract
$$2x\eta$$
$$\textrm{as such: }x \leftarrow x - 2x\eta$$
Or in programming terms: <br /> <br />
<div style="text-align: center";><span style="background:#f1f1f1;"><span style="color:#55c">let</span> x<span style="color:#a05050; font-weight:bold; "> = </span> x - 2 * x * eta</span></div>
</p>
<p>
Another way to look at it is:
$$ x \leftarrow x - f'(x) \eta $$
</p>
<aside>I've wondered a for long time as to why this works. In my view: the derivative is the one stop shop answer for the question: what is the <em>fastest change</em> one can make when you have a very small difference for \(x\) such as \(x_1 = 3\) and \(x_2 = 3.0001\)? The answer is: if you can differentiate the function, then the derivative is the fastest change one can make. On your left, you can see this graphically.</aside>
<p>In the following example this is what we are going to see for the function \(f(x) = x^2\). It will be a very slowed down version of gradient descent. The new function that will be repeatedly recalculated is \(f(x) \leftarrow f(x - 2x\eta)\) and the learning rate \(\eta = 0.01\).</p>
<canvas id="cost-function-graph-gd" width="535" height="300">
Canvas not supported in the browser, please install Chrome or Firefox.
</canvas>
<button id="gradient-descent-example-play">Play!</button>
<input type="range" min="0" max="1" value="0.01" step="0.01" class="slider" id="eta">\(\eta\):
<span class="slider" id="eta-out">0.01</span>
<script>
function initSliders(){
let eta = document.getElementById('eta')
eta.addEventListener('input', () => {
document.getElementById('eta-out').innerHTML = eta.value
})
}
initSliders();
</script>
<aside>Psst! Try and change \(\eta\) and see what happens. What happens for \(\eta = 0.3, \eta = 0.6, \eta = 0.99 \textrm{ and } \eta = 1 \textrm{?}\) </aside>
<p>
<strong>Give them a JSFiddle exercise to implement gradient descent. And then state the final paragraph</strong>
</p>
<p>So we can just use gradient descent on our previous cost function right? And then we're done! Well, no. It's mathematically a bit more complex because the function isn't as simple as \(x^2\), but if it were as simple as \(x^2\), then yes. Yes, indeed my friend. Unfortunately, that isn't the case. We need to move onward.</p>
<aside>
<!-- Aside of Training a Network: Backpropagation -->
If you don't know what partial derivatives are, they are very alike normal derivatives. The difference is: in a partial derivatives there are multiple variables in an equation, but you treat one variable as an actual variable and pretend that the other variables are constant. For example,
$$f(x, y) = x^2 + y^3$$
The partial derivate of f with respect to x is: $$\frac{\partial f}{\partial x} = 2x$$
The partial derivate of f with respect to y is: $$\frac{\partial f}{\partial y} = 3x^2$$
<br /> <br />
For a more complete but quick introduction, see <a href="https://www.khanacademy.org/math/multivariable-calculus/multivariable-derivatives/partial-derivatives/v/partial-derivatives-introduction">the video series of Khan Academy</a>.
</aside>
<h3>
Training a Network: Backpropagation
<span class="subtitle">Also known as: constantly applying the chain rule in order to perform gradient descent, but not telling anyone by giving it a fancy name</span>
</h3>
<p>
Fully written the cost function we use is:
$$C = \frac{1}{2}(y - a)^2$$
$$\textrm{for which: } a = wx$$
$$\textrm{such that: } C = \frac{1}{2}(y - wx)^2$$
Which looks alike \(x^2\) because something is squared, but it also has more variables in it with all kinds of meanings such as: \(w\), \(x\) and \(y\).
We want to find the partial derivative of \(C\) with respect to \(w\). Why with respect to \(w\)? Because that's the weight we want to automatically train!
To differentiatie this function it needs the chain rule because there are secretly 2 functions in need of differentiation.</p>
<p>The first one is the \(\frac{1}{2}(...)^2\) part which is written as the partial derivative of \(C\) with respect to \(a\) which is \(\frac{\partial C}{\partial a}\).
</p>
<p>
The second one is the \(wx\) part which is the partial derivative of \(a\) with respect to \(w\) which is \(\frac{\partial a}{\partial w}\).
</p>
<p>
Combined, it can be written as
$$\frac{\partial C}{\partial w} = \frac{\partial C}{\partial a}\frac{\partial a}{\partial w}$$
Let's take a look at both functions and then find the derivative.
</p>
<aside style="margin-top: 0;">
\(\frac{\partial C}{\partial y} = (a - y)\) is an application of the <a href="https://www.khanacademy.org/math/ap-calculus-ab/ab-differentiation-1-new/ab-2-5/v/power-rule">power rule</a> and <a href="https://www.khanacademy.org/math/ap-calculus-ab/ab-differentiation-2-new/ab-3-1a/v/chain-rule-introduction">chain rule</a> such that you need to differentiate:
\(\frac{1}{2}(y - a)^2 = \frac{1}{2}(...)^2 \textrm{ which is } 1*(...)^1 = (...) = (y - a)\textrm{.}\)
\(\textrm{And } y - a \textrm{ which is } {-1} \textrm{ since } {-a} \textrm{ is } {-1} \textrm{ when differentiated.}\)
Multiplying it as the chain rule states is: \(-1*(y-a) = (-y - a) = (a - y)\)
</aside>
<p>
$$C = \frac{1}{2}(y - a)^2, \textrm{ so } \frac{\partial C}{\partial a} = (a - y)$$
$$a = wx, \textrm{ and } \frac{\partial a}{\partial w} = x$$
$$\frac{\partial C}{\partial w} = \frac{\partial C}{\partial a}\frac{\partial a}{\partial w} = (a - y)x$$
</p>
<p>
And that is what backpropagation is: finding the partial derivative of \(C\) with respect to all the weights in a network (in this case one). This means you have calculated the gradient (partial derivative) for weight \(w\) by using the chain rule and now you can use the rest of the gradient descent algorithm in order to tune the weight.
</p>
<p>Here is a graphical example of our particular network for which we have one training example \(x=1\).</p>
<canvas id="simplest-cost-function-graph2" width="535" height="300">
Canvas not supported in the browser, please install Chrome or Firefox.
</canvas>
<button id="simplest-ann-cost-function-simplest-gd-play">Play!</button>
<input type="range" min="0" max="1" value="0.01" step="0.01" class="slider" id="simplest-ann-cost-function-eta">\(\eta\):
<span class="slider" id="simplest-ann-cost-function-eta-out">0.01</span>
<script>
function initSlidersSimplestCostFunction(){
let eta = document.getElementById('simplest-ann-cost-function-eta')
eta.addEventListener('input', () => {
document.getElementById('simplest-ann-cost-function-eta-out').innerHTML = eta.value
})
}
initSlidersSimplestCostFunction();
</script>
<br /><br /><br /><br /><br />
<p>It is important to note that I have omited many things about neural networks, but this is in essence the nitty gritty overview of how a neural network predicts and trains itself. In the next chapter I will create a network that has an activation function, multiple training examples (2, yes, a 100% increase) and a bias neuron.</p>
<strong>End with JSFiddle and perhaps an updated visualization of the network?</strong>
<!-- <p>
Backpropagation is an application of the Chain Rule to neural networks. -- see https://stackoverflow.com/questions/35765607/why-do-we-take-the-derivative-of-the-transfer-function-in-calculating-back-propa
</p> -->
<!-- Plotting cost function and gradient descent widget -- also need to include network -->
<!-- with id cost-function-graph needs to be put here -->
<!-- Backpropagation widget -->
</section>
<script type="text/javascript" src="simplest-ann.js"></script>
<script type="text/javascript" src="simplest-ann-cost-function.js"></script>
<script>
simplestCostFunctionGraph('simplest-cost-function-graph1')
simplestCostFunctionGraph('simplest-cost-function-graph2', 'simplest-ann-cost-function-eta')
</script>
<script type="text/javascript" src="gradient-descent-example.js"></script>
<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<defs>
<path d="M114.069576,127.088198 C120.256553,133.919134 124.82584,136.108127 135.722221,145.102941 L125.983549,141.420331 L128.584412,150.579654 C122.354806,142.184644 118.538821,138.269538 117.136457,138.834336 C116.255381,139.189186 117.639312,145.433337 117.303946,150.15313 C116.838636,156.701708 114.776409,162.215609 114.069576,161.971247 C112.853303,161.550764 114.825283,156.121101 112.511962,140.696837 C110.29236,125.897456 98.5308064,113.189229 95.5339512,112.452226 C93.5360478,111.960891 92.0028146,112.289283 90.9342517,113.437401 L86.619111,112.640625 C81.3125786,113.739875 78.5115345,115.589562 78.2159784,118.189688 C77.7726444,122.089876 69.1702365,125.524525 84.4199219,153.793945 C99.6696073,182.063365 81.0737195,153.53138 79.7724609,153.793945 C78.6375523,154.022945 85.3903633,175.797648 77.5380999,159.154297 C69.6858366,142.510946 67.1711512,117.867078 62.0644531,116.260742 C50.2421875,112.541992 43.7542611,123.727131 40.4047939,127.276597 C34.8395317,133.174164 31.267735,139.603441 29.6894037,146.564428 C29.6894037,138.064158 29.1145085,133.463165 27.9647179,132.76145 C26.8149273,132.059735 21.9530994,133.474549 13.3792341,137.005893 C17.7746591,129.213258 28.8359686,128.513585 37.3706625,122.168098 C39.2812919,120.747558 48.5407665,110.759903 46.1729327,107.853523 C41.9230411,102.637026 3.62060547,113.622803 8.2890625,109.291016 C12.9575195,104.959229 15.3289604,103.702036 15.388916,101.578857 C15.4488716,99.4556786 -0.391417928,88.2653107 6.79857859,90.7597656 C14.1396702,93.3066406 17.1907735,99.8548399 27.9647179,99.2226562 C35.0572738,98.8064858 42.7011719,95.1938 42.7011719,90.7597656 C42.7011719,85.7280589 34.7394,83.5414874 21.8876953,78.9975586 C17.7760992,77.543837 10.6408453,74.5523387 0.481933594,70.0230637 C6.90190852,68.3787687 10.1751833,67.1525154 10.3017578,66.3443038 C10.5568288,64.7156085 9.64513228,61.8304255 6.79857859,57.0016113 C8.67869991,57.150704 13.286714,61.5344746 14.1396702,62.8008022 C15.2594462,64.4632593 15.7729955,66.5009977 16.4126804,67.3714821 C17.5396263,68.9050324 21.1524829,76.8640194 27.9647179,75.4282227 C32.5062079,74.4710248 35.3518481,73.0923947 36.5016387,71.2923325 L40.5144283,67.6897812 L45.3626661,68.0938025 C49.7588653,62.4257578 51.9966629,58.2909435 52.0760589,55.6893598 C52.374824,45.8996625 46.1729327,36.7338304 46.1729327,29.5888358 C46.1729327,24.8255062 48.0529339,18.9456252 51.8129363,11.9491931 C51.022536,15.6189476 51.022536,17.856376 51.8129363,18.6614784 C52.6033367,19.4665808 54.7999071,19.4665808 58.4026478,18.6614784 C54.6407345,20.3797318 52.4441641,21.6069364 51.8129363,22.3430922 C50.8660948,23.4473258 51.9461558,49.1102999 54.9738325,54.4675135 C56.9922837,58.0389893 61.4007278,60.42646 68.1991648,61.6299257 L72.0356829,59.3997831 L76.1372095,60.6310488 C81.5543176,58.8653214 85.0843728,57.2073174 86.727375,55.6570368 C92.3257937,50.3745606 86.619111,26.0247059 86.619111,22.3430922 C86.619111,19.888683 83.9127407,14.4873891 78.5,6.13921043 L89.0920515,11.9491931 L98.4884031,0 C93.814324,10.7494621 91.4772845,18.8305701 91.4772845,24.243324 C91.4772845,25.7642817 90.4940425,34.6811102 91.4772845,42.9451021 C92.7525825,53.6637778 96.1757808,64.0812439 97.3606073,64.0696415 C98.5454338,64.0580392 107.425057,56.405833 110.147461,54.4675135 C114.002822,51.7225416 116.776466,43.6992389 121.62344,33.2644238 L119.140906,48.5127253 L131.885903,44.7300209 C124.342144,48.2231141 117.683454,52.3136443 111.909834,57.0016113 C106.136213,61.6895783 98.961649,65.2823277 100.353503,69.4276916 C101.745356,73.5730555 130.624765,63.1910255 134.233792,61.5267859 C137.842818,59.8625462 137.638489,54.1278028 137.842818,54.4899889 C138.99395,56.5304429 136.799042,66.2528136 137.842818,66.3443038 C139.071868,66.4520338 146.759979,61.3700713 149.877932,59.2113843 L139.955336,69.7062524 C140.087413,70.3630252 140.730601,70.8186164 141.884901,71.0730259 C143.039201,71.3274353 148.69839,73.8690466 153.912477,74.3905765 C149.65257,75.2558647 116.086849,69.9611563 112.773547,77.9567556 C109.460245,85.9523548 105.810778,91.6950521 114.847007,96.8552507 C123.883236,102.015449 133.48202,102.139733 138.31219,100.716505 C141.532304,99.7676872 144.151881,96.0658098 148.529429,86.9306037 L145.041686,104.116704 L154.594266,114.20255 C148.931194,110.649349 144.399639,108.470207 140.999603,107.665124 C139.099081,107.215106 123.423156,98.823104 111.909834,106.834541 C100.396511,114.845979 108.297844,120.715728 114.069576,127.088198 Z" id="shape"></path>
<g id="face" transform="translate(38.000000, 55.000000)">
<g id="eye-left" transform="translate(12.333333, 27.461905)">
<ellipse id="Oval-4" fill="#000000" cx="7.04761905" cy="6.95238095" rx="7.04761905" ry="6.95238095"></ellipse>
<ellipse id="Oval" fill="#FFFFFF" cx="8.45714286" cy="5.90952381" rx="2.46666667" ry="2.43333333"></ellipse>
<ellipse id="Oval-Copy" fill="#FFFFFF" cx="3.7" cy="10.602381" rx="1" ry="1"></ellipse>
</g>
<g id="eye-right" transform="translate(47.571429, 27.461905)">
<ellipse id="Oval-4" fill="#000000" cx="7.04761905" cy="6.95238095" rx="7.04761905" ry="6.95238095"></ellipse>
<ellipse id="Oval" fill="#FFFFFF" cx="8.45714286" cy="5.90952381" rx="2.46666667" ry="2.43333333"></ellipse>
<ellipse id="Oval-Copy" fill="#FFFFFF" cx="3.7" cy="10.602381" rx="1" ry="1"></ellipse>
</g>
<path d="M31.7251256,44.4952381 C32.4229185,47.0444444 34.18121,48.3190476 37,48.3190476 C39.81879,48.3190476 41.6228497,47.0444444 42.4121791,44.4952381" id="mouth" stroke="#000000" stroke-width="2"></path>
</g>
<g id="neuron-blue">
<use fill-rule="evenodd" xlink:href="#shape" fill="#9ADBFF"></use>
<use fill="black" fill-opacity="1" filter="url(#filter-blue)" xlink:href="#shape"></use>
<use id="after-effect" stroke="#000000" stroke-width="1" xlink:href="#shape" fill="none"></use>
<use xlink:href="#face" fill="none" />
</g>
<filter x="-1.9%" y="-1.8%" width="103.9%" height="103.6%" filterUnits="objectBoundingBox" id="filter-blue">
<feMorphology radius="2" operator="erode" in="SourceAlpha" result="shadowSpreadInner1"></feMorphology>
<feGaussianBlur stdDeviation="1" in="shadowSpreadInner1" result="shadowBlurInner1"></feGaussianBlur>
<feOffset dx="0" dy="1" in="shadowBlurInner1" result="shadowOffsetInner1"></feOffset>
<feComposite in="shadowOffsetInner1" in2="SourceAlpha" operator="arithmetic" k2="-1" k3="1" result="shadowInnerInner1"></feComposite>
<feColorMatrix values="0 0 0 0 0.373992488 0 0 0 0 0.586802329 0 0 0 0 0.704666241 0 0 0 1 0" type="matrix" in="shadowInnerInner1"></feColorMatrix>
</filter>
<filter x="-2.6%" y="-3.5%" width="106.2%" height="105.3%" filterUnits="objectBoundingBox" id="filter-purple">
<feMorphology radius="2" operator="erode" in="SourceAlpha" result="shadowSpreadInner1"></feMorphology>
<feGaussianBlur stdDeviation="1" in="shadowSpreadInner1" result="shadowBlurInner1"></feGaussianBlur>
<feOffset dx="0" dy="1" in="shadowBlurInner1" result="shadowOffsetInner1"></feOffset>
<feComposite in="shadowOffsetInner1" in2="SourceAlpha" operator="arithmetic" k2="-1" k3="1" result="shadowInnerInner1"></feComposite>
<feColorMatrix values="0 0 0 0 0.469173898 0 0 0 0 0.413586505 0 0 0 0 0.709077381 0 0 0 1 0" type="matrix" in="shadowInnerInner1"></feColorMatrix>
</filter>
<filter x="-2.6%" y="-3.5%" width="106.2%" height="105.3%" filterUnits="objectBoundingBox" id="filter-pink">
<feMorphology radius="2" operator="erode" in="SourceAlpha" result="shadowSpreadInner1"></feMorphology>
<feGaussianBlur stdDeviation="1" in="shadowSpreadInner1" result="shadowBlurInner1"></feGaussianBlur>
<feOffset dx="0" dy="1" in="shadowBlurInner1" result="shadowOffsetInner1"></feOffset>
<feComposite in="shadowOffsetInner1" in2="SourceAlpha" operator="arithmetic" k2="-1" k3="1" result="shadowInnerInner1"></feComposite>
<feColorMatrix values="0 0 0 0 1 0 0 0 0 0.838541667 0 0 0 0 0.968849072 0 0 0 1 0" type="matrix" in="shadowInnerInner1"></feColorMatrix>
</filter>
</defs>
</svg>
</body>
</html>