atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Jesse Builds Software</title>
  
  <subtitle>Jesse&#39;s thoughts on building quality software.</subtitle>
  <link href="https://jessemcdowell.ca/atom.xml" rel="self"/>
  
  <link href="https://jessemcdowell.ca/"/>
  <updated>2024-12-02T22:53:53.000Z</updated>
  <id>https://jessemcdowell.ca/</id>
  
  <author>
    <name>Jesse McDowell</name>
    
  </author>
  
  <generator uri="https://hexo.io/">Hexo</generator>
  
  <entry>
    <title>Encoding Time Series Data</title>
    <link href="https://jessemcdowell.ca/2024/12/Encoding-Time-Series-Data/"/>
    <id>https://jessemcdowell.ca/2024/12/Encoding-Time-Series-Data/</id>
    <published>2024-12-02T18:29:50.000Z</published>
    <updated>2024-12-02T22:53:53.000Z</updated>
    
    <content type="html"><![CDATA[<p>I spent just shy of 9 years working on a suite of products that stored and processed environmental sensor data. This kind of data is naturally stored and processed as a time series. A time series is a set of data points, each with a time and value, and often more. They are common for sensor data, but can be used for pretty much anything that is measured over time such as prices, performance metrics, and analytical data.</p><p>There is a lot of subtlety to working with time series data, especially if you are concerned with scientific accuracy. One surprising challenge, which I’ll be discussing in this post, is storing and transmitting the data efficiently.</p><p>At my previous company, the data we dealt with was relatively low frequency (typically a reading every 5-15 minutes), but some of the data sets went back into the 1800s! Although the data is simple in structure, the size can get surprisingly big when you need to move around millions of points.</p><h2 id="Test-application"><a href="#Test-application" class="headerlink" title="Test application"></a>Test application</h2><p>I built a simple app to measure the transmission size of the data in the different variations discussed in this post. You don’t need to look at it to follow along, but you might find it interesting. The source code is available here: <a href="https://github.com/jessemcdowell/time-series-encoding-test">https://github.com/jessemcdowell/time-series-encoding-test</a></p><p>The test application is written in TypeScript, and uses <a href="https://nodejs.org/">node</a>, <a href="https://expressjs.com/">Express</a>, and <a href="https://axios-http.com/">Axios</a>. It took some fiddling to be able to control compression, and to measure the transmission size (as opposed to the total size of the data, which would normally be more important). The numbers below also include other transmission overhead like headers and chunk encoding. This will skew the numbers a bit, but the results should still be more than good enough to illustrate the differences between approaches.</p><h2 id="In-application"><a href="#In-application" class="headerlink" title="In application"></a>In application</h2><p>A typical way to store a point in application memory is as an array of simple structures&#x2F;objects. With an 8-byte integer for the timestamp, and an 8-byte floating point for the value, you can represent each point with 16 bytes. This will have enough precision for most use cases.</p><p>For one million points packed in an efficient array, this would consume 16 million bytes of memory. I’m going to use this as a baseline for evaluating other options.</p><p>This is an efficient, convenient way to handle the data in an application, but chances are you will want to write it to disk, or return it from a web service. It is possible to send this binary representation over the wire, but there are some drawbacks to consider. I’ll come back to this.</p><h2 id="JSON"><a href="#JSON" class="headerlink" title="JSON"></a>JSON</h2><p>JSON is a common way to transmit data between server and client or server and server. A simple representation might look like the following:</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="punctuation">[</span></span><br><span class="line">    <span class="punctuation">&#123;</span></span><br><span class="line">        <span class="attr">&quot;time&quot;</span><span class="punctuation">:</span> <span class="string">&quot;2024-01-01T00:00:00.000Z&quot;</span><span class="punctuation">,</span></span><br><span class="line">        <span class="attr">&quot;value&quot;</span><span class="punctuation">:</span> <span class="number">0.54188</span></span><br><span class="line">    <span class="punctuation">&#125;</span><span class="punctuation">,</span></span><br><span class="line">    <span class="punctuation">&#123;</span></span><br><span class="line">        <span class="attr">&quot;time&quot;</span><span class="punctuation">:</span> <span class="string">&quot;2024-01-01T01:00:00.000Z&quot;</span><span class="punctuation">,</span></span><br><span class="line">        <span class="attr">&quot;value&quot;</span><span class="punctuation">:</span> <span class="number">0.32624</span></span><br><span class="line">    <span class="punctuation">&#125;</span><span class="punctuation">,</span></span><br><span class="line">    <span class="punctuation">&#123;</span></span><br><span class="line">        <span class="attr">&quot;time&quot;</span><span class="punctuation">:</span> <span class="string">&quot;2024-01-01T02:00:00.000Z&quot;</span><span class="punctuation">,</span></span><br><span class="line">        <span class="attr">&quot;value&quot;</span><span class="punctuation">:</span> <span class="number">0.76939</span></span><br><span class="line">    <span class="punctuation">&#125;</span></span><br><span class="line"><span class="punctuation">]</span></span><br></pre></td></tr></table></figure><p>This is just three points, and you can see that it’s quite a few bytes. If you exclude the whitespace (included above for readability), it’s still 157 bytes with each point taking 52 bytes. This means 1 million points takes about 52 million bytes (just over 51MB)!</p><p>Here’s how it breaks down:</p><ul><li>19 bytes - object wrapper, commas, and property names</li><li>26 bytes - time string and its quotes</li><li>7 bytes - the value</li></ul><p>52 bytes is a lot more than the 16 we need in application memory. This is more than 200% overhead! It also has lower precision than the in memory version can support. It is possible to use a less precise timestamp format, or to use smaller names for the properties (like <code>t</code> and <code>v</code>). Those would help a bit, but we can do better.</p><h2 id="CSV"><a href="#CSV" class="headerlink" title="CSV"></a>CSV</h2><p>CSV has a lot less wrapping characters, and doesn’t need to repeat the property names for each point. Here are the same values as the JSON example:</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">time,value</span><br><span class="line">2024-01-01T00:00:00.000Z,0.54188</span><br><span class="line">2024-01-01T01:00:00.000Z,0.32624</span><br><span class="line">2024-01-01T02:00:00.000Z,0.76939</span><br></pre></td></tr></table></figure><p>The total is 109 bytes (or 112 with CRLF line endings). The header is just 11 bytes, and can be omitted if desired. Each point is 33 bytes, which is better than the 52 of JSON, but more than double that of our 16 bytes in memory (106% overhead). 1 million points takes just under 36 million bytes.</p><p>There are other advantages to CSV as well. It’s easy to produce, easy to parse, and can handle multiple values or other types of data for each timestamp easily. It’s more readable in debugging tools, and can  be consumed easily in many applications, more easily than JSON. It has capacity for extensibility via adding columns, though it can’t naturally handle structured data the same way JSON can.</p><h2 id="Custom-binary-format"><a href="#Custom-binary-format" class="headerlink" title="Custom binary format"></a>Custom binary format</h2><p>A time series can be stored or transmitted in a binary format very similar to the way it would be stored in application memory. You can encode it however you want, but my simple implementation takes 16 million bytes which is considerably better than both text formats discussed above. There is some added complexity to this approach, however. In particular, you will need your own versioning strategy if you ever need to change the structure.</p><p>You may also have some difficulty getting your web server and client to transmit and receive the data correctly. An easy way to get around this is to re-pack the binary data into something like Base64 and send that through a normal string property. This is especially helpful if you have a bunch of different time series or other related data you want to return in the same response. The drawback to this approach is that Base64 adds about 33% overhead, so the same million points from before would take 21.3 million bytes instead.</p><h2 id="Unpacking-precise-timestamps-in-JavaScript"><a href="#Unpacking-precise-timestamps-in-JavaScript" class="headerlink" title="Unpacking precise timestamps in JavaScript"></a>Unpacking precise timestamps in JavaScript</h2><p>Another bit of complexity you might encounter with binary data, or any kind of data really, is the inherent limitation of JavaScript when dealing with 64-bit integers. For most applications this probably won’t matter. At my previous company, however, it was a significant and memorable challenge. Some of our features required quoting exact timestamps back to the server, and even a single millisecond of lost precision broke things.</p><p>Another challenge was different timestamp representations on the client and server. Our server (C# and C++ based) used a single 64-bit integer to represent 100-nanosecond intervals. Our client (JavaScript) used two number properties (one to store seconds, and a second to store nanoseconds). When unpacking the binary data, every timestamp had to be divided by 10000 for seconds, and the remainder multiplied by 100 for nanoseconds. Doing this with floating points in native JavaScript was very slow. It took some frustrating research and experimentation, but I got the performance gains I needed using a bit of Web Assembly.</p><h2 id="Standardized-binary-format"><a href="#Standardized-binary-format" class="headerlink" title="Standardized binary format"></a>Standardized binary format</h2><p>Another option is to use a common binary format to transmit the data. <a href="https://msgpack.org/">MsgPack</a> is very similar to JSON, but a bit smaller and faster. It also has the same drawback with time series data: it includes the key names with every single point. A MsgPack equivalent of the million points take around 28 million bytes. This is 76% overhead compared to the 16 million bytes in memory.</p><p><a href="https://protobuf.dev/">Protobuf</a> (the binary protocol used in gRPC) does not include keys in the data, but it is harder to use. You have to define a schema that is kept is sync between your client and server. The same million points takes around 14 million bytes in Protobuf. This is a modest 11% less than the custom binary format, but with more safeguards and flexibility.</p><h2 id="Compressed-binary"><a href="#Compressed-binary" class="headerlink" title="Compressed binary"></a>Compressed binary</h2><p>Gzip compression is an easy way to reduce the size of the data being transmitted. Even better, this is a feature you probably already have enabled on your web servers and clients.</p><p>I tested a few configurations of compressed binary data to see how they performed. One interesting discovery was that the compression was noticeably better if I arranged the data in sets (<code>time1-time2-time3-value1-value2-value3</code> instead of pairs such as <code>time1-value1-time2-value2-time3-value3</code>. I assume this is because of how gzip handles adjacent similarities. Here are the results for similar 1-million point blocks:</p><ul><li>Binary in any arrangement with no compression: 16 million bytes</li><li>Binary pairs (<code>time-value-time-value</code>) with gzip: 10.3 million bytes</li><li>Binary sets (<code>time-time-value-value</code>) with gzip: 8.7 million bytes</li><li>Binary pairs, encoded in Base64, with gzip: 10.8 million bytes</li><li>Binary sets, encoded in Base64, with gzip: 10.1 million bytes</li></ul><p>I also tested MsgPack and Protobuf with gzip. Even though they are arranged in pairs, they performed slightly better:</p><ul><li>MsgPack: 9.8 million bytes</li><li>Protobuf: 8.2 million bytes</li></ul><p>One drawback to sets over pairs is that you would need all the points available to send them in sequence. If you had a very large amount of points and wanted to stream them incrementally, you would need to develop your own chunk-capable format. This is also a drawback of MsgPack which writes the size of each array at its start. I don’t know if Protobuf has this limitation too.</p><h2 id="Compressed-CSV-and-JSON"><a href="#Compressed-CSV-and-JSON" class="headerlink" title="Compressed CSV and JSON"></a>Compressed CSV and JSON</h2><p>Compression is usually pretty effective wherever you have lots of repeating sequences, and that is certainly the case with JSON and CSV time series data. The same millions points packed in JSON took 7.5 million bytes when gzipped. That’s a compression factor over 7 times as compared to the plain JSON, and more than 2x better than straight binary!</p><p>A million points of CSV compressed to 6.9 million bytes, which is even better. The compression factor is not as impressive, but the overall size is smaller. This is the smallest of all the options in this post.</p><h2 id="Downsampling"><a href="#Downsampling" class="headerlink" title="Downsampling"></a>Downsampling</h2><p>Another great way to reduce the size of a time series is to send fewer points. Downsampling (sometimes called Decimation) is a technique that excludes some or most of the points, and it can have a tremendous impact on the size of the data and the performance of your applications.</p><p>If you use this technique for data that is being graphed, the user might not even be able to notice. For example, If you are displaying a year of data in a spacious 1200-pixel wide area, you only have 3.3 pixels of width per day. If you have a point per hour, most of those 24 points won’t change the image at all and can be safely excluded. If you had a point every minute, you can exclude almost all the points!</p><p>There are a few ways to do this. The best technique will depend on your use case and the correct interpolation method for the data. I’m not going to get into those in this post. The result of downsampling is still a time series, so you should be able to use any of the techniques already discussed to transmit the downsampled data efficiently.</p><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>Time series data can be larger than you expect, but there are lots of ways to store and transmit them more efficiently. For most projects, I would start with CSV compressed by gzip, and use downsampling where possible. For cases with more strenuous requirements, there are plenty of other options to consider.</p><p>My testing and discussion were purely focused on the size of the data. Fewer bytes will generally lead to better performance, but in some situations, serialization speed or memory utilization might be more important. If that matters to you, you should <a href="https://pragmaticpotato.com/services/">work with an architect</a> to find the best approach.</p><!--Test results 2024-10-02:<figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">json, no compression:         53 624 980</span><br><span class="line">json, gzip:                    7 507 865</span><br><span class="line">csv, no compression:          34 701 387</span><br><span class="line">csv, gzip:                     6 890 209</span><br><span class="line">msgpack, no compression:      28 121 167</span><br><span class="line">msgpack, gzip:                 9 909 851</span><br><span class="line">protobuf, no compression:     14 251 672</span><br><span class="line">protobuf, gzip:                8 049 274</span><br><span class="line">binary pairs, no compression: 16 000 000</span><br><span class="line">binary pairs, gzip:           10 281 562</span><br><span class="line">binary sets, gzip:             8 546 006</span><br><span class="line">base64 pairs, no compression: 21 333 336</span><br><span class="line">base64 pairs, gzip:           10 878 917</span><br><span class="line">base64 sets, gzip:            10 070 165</span><br></pre></td></tr></table></figure><p>–&gt;</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I spent just shy of 9 years working on a suite of products that stored and processed environmental sensor data. This kind of data is natu</summary>
      
    
    
    
    
    <category term="web" scheme="https://jessemcdowell.ca/tags/web/"/>
    
    <category term="performance" scheme="https://jessemcdowell.ca/tags/performance/"/>
    
    <category term="nodejs" scheme="https://jessemcdowell.ca/tags/nodejs/"/>
    
    <category term="typescript" scheme="https://jessemcdowell.ca/tags/typescript/"/>
    
  </entry>
  
  <entry>
    <title>Abandoning Household Organization</title>
    <link href="https://jessemcdowell.ca/2024/11/Abandoning-Household-Organization/"/>
    <id>https://jessemcdowell.ca/2024/11/Abandoning-Household-Organization/</id>
    <published>2024-11-04T15:02:12.000Z</published>
    <updated>2024-11-04T15:02:12.841Z</updated>
    
    <content type="html"><![CDATA[<p>I have been trying to improve the organization of my household for years. My wife and I have been using shared Google calendars for over a decade, but there are plenty of issues we could still improve. It was only minor challenges when it was just the two of us, but once we had a child, the systems we had started to show their limits.</p><p>I quit my job with the dream of building my own product, but I started without any particular ideas about what to build. It didn’t take long before I was considering working on this exact problem. Not only was it something I would benefit from, but it was also something I would find interesting, and could enjoy building and maintaining by myself.</p><p>My first urge was to start coding. I had a clear idea of what I wanted, and my head was spinning with potential designs. However, years of experience have taught me the importance of starting with solid research, so I started there instead.</p><p>The first thing I discovered was that there are already several products specifically for this problem. I tried a few out, and while they were not all fantastic, a couple of them were pretty good. There were some differences, but they mostly all had the same core features: a shared calendar, a shared task list, and a shared shopping list.</p><p>Many had been around for years. Some were free. The fact that none of them seemed to be particularly successful was my first hint that there was a problem.</p><p>My next step was putting together a survey and sending it out via my social networks. I got 67 responses. It’s certainly not enough for any kind of scientific validity, but it was enough to notice some trends. I found many people who had the same kinds of problems I did, and most wanted to use technology to help. Desire is normally a good thing for an entrepreneur to find, but there had to be a reason that quality products that had existed for years weren’t working.</p><p>In the survey, I also asked participants if they would participate in a one-on-one virtual interview in exchange for a gift card. Most agreed. I selected a handful, and set up some meetings.</p><p>One of the people I spoke with had used and abandoned two of the best products I’d tried. I asked him about it, and his answers are the reason I decided to abandon this problem. He thought technology was a great idea, but he couldn’t make his partner use it. He tried using it on his own, but it didn’t help, it just became another chore for him to manage.</p><p>I had originally approached this problem like business software, but there is a fundamental difference between a workplace and a family: it’s easier to motivate an employee than a member of your family. It’s not enough to make something that appeals to one member of the family, a successful app would have to be intrinsically motivating to everyone in the household. This is an extremely high bar to clear.</p><p>If I take a holistic view of household organization, I don’t think another piece of software is the most effective solution. A shared calendar is the main thing a family needs, and there are some good free options. If you need more, there are a few free apps already. If you want to stay offline, you can use a whiteboard, or even a piece of paper hung on the wall. All that leaves for me is to show people how to use these things, but writing a self-help book or teaching a class isn’t what I want to work on.</p><p>The interviews weren’t a complete waste, however. Some of the people I talked to had very organized households, and those conversations were insightful. Here were a few ideas that stood out to me:</p><ul><li>Having a weekly planning ritual makes sure everyone knows what’s happening, and creates an opportunity to discuss issues. This seems to be especially helpful for larger households, households with multiple generations, and&#x2F;or households with children old enough to contribute.</li><li>Planning week by week (as opposed to day-by-day) simplifies the process and makes family routines more predictable.</li><li>Having a calendar that everyone can see, be it through software or posted on the wall, gives everyone the same view of the week.</li><li>Assigning chores to specific people, and making them accountable for those chores, helps to make sure that those chores get done.</li></ul><p>The exercise of researching the market, talking to people, and learning about the problem was interesting and enjoyable. It cost me some time and a bit of money, but I have no regrets. Maybe some of it will help with my next idea.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I have been trying to improve the organization of my household for years. My wife and I have been using shared Google calendars for over </summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="household organization" scheme="https://jessemcdowell.ca/tags/household-organization/"/>
    
  </entry>
  
  <entry>
    <title>Exporting to Google Sheets API from C#</title>
    <link href="https://jessemcdowell.ca/2024/10/Exporting-to-Google-Sheets-API-from-CSharp/"/>
    <id>https://jessemcdowell.ca/2024/10/Exporting-to-Google-Sheets-API-from-CSharp/</id>
    <published>2024-10-06T21:35:38.000Z</published>
    <updated>2024-10-06T21:36:43.972Z</updated>
    
    <content type="html"><![CDATA[<p>I recently did a project for a client that needed a way to share a bit of live data with an external organization. He originally asked for a simple website with authentication and an API. I could have built this, but for the frequency and volumes of data they were using, I suggested a simpler approach: a Google Sheet and a recurring job that updates it.</p><p>Any online spreadsheet is a great way to share a small amount of data. Users can filter, sort, make formulas, export the data, or even write their own integrations. It’s also possible (depending on how changes are handled) to manually annotate or override data that’s been exported. Implementing all of this in a custom web page is possible, but some of it is pretty tricky.</p><p>The project was a success, and I’m happy with the results. Working with the Google Sheets API was not as straightforward as I had expected though. It was hard to find documentation, or even many examples showing how to use it. I’m not sure why this is. In this post I’ll share some of the things I learned.</p><h2 id="Google-Apis-Sheets-v4-Package"><a href="#Google-Apis-Sheets-v4-Package" class="headerlink" title="Google.Apis.Sheets.v4 Package"></a>Google.Apis.Sheets.v4 Package</h2><p>The <a href="https://www.nuget.org/packages/Google.Apis.Sheets.v4/">Google.Apis.Sheets.v4 package</a> is the official .NET client library for the Google Sheets API. You can install it with NuGet.</p><p>It has <a href="https://googleapis.dev/dotnet/Google.Apis.Sheets.v4/latest/api/Google.Apis.Sheets.v4.html">official documentation</a>, but I found it to be almost useless. I recommend the <a href="https://developers.google.com/sheets/api/reference/rest">Google Sheets API documentation</a> instead. It has much more information about how the API works and what the arguments mean. Methods and properties in .NET client library match the Rest API documentation exactly, I assume this is a result of the client library being auto-generated.</p><p>My first tip is to make sure you’re using the exact name for an API endpoint. For example, these two APIs, although seemingly similar, take different arguments, and behave totally differently:</p><ul><li><code>service.Spreadsheets.BatchUpdate()</code></li><li><code>service.Spreadsheets.Values.BatchUpdate()</code></li></ul><p>Another thing to be aware of is that the methods above don’t actually call the service, they create a request object. They almost look like they can be used in a fluent style, but this is only possible for the simplest get calls. I found it much easier to assign the request object into a variable so that I could set its properties directly. For example:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> range = <span class="string">&quot;!A1:A3&quot;</span>;</span><br><span class="line"><span class="keyword">var</span> change = <span class="keyword">new</span> ValueRange</span><br><span class="line">&#123;</span><br><span class="line">    Range = range,</span><br><span class="line">    MajorDimension = <span class="string">&quot;ROWS&quot;</span>,</span><br><span class="line">    Values = [[<span class="string">&quot;one&quot;</span>, <span class="string">&quot;two&quot;</span>, <span class="string">&quot;three&quot;</span>]]</span><br><span class="line">&#125;;</span><br><span class="line"><span class="keyword">var</span> request = service.Spreadsheets.Values.Update(change, <span class="string">&quot;SHEET_ID&quot;</span>, range);</span><br><span class="line">request.ValueInputOption = SpreadsheetsResource.ValuesResource.UpdateRequest.ValueInputOptionEnum.RAW;</span><br><span class="line"><span class="keyword">var</span> result = <span class="keyword">await</span> request.ExecuteAsync();</span><br></pre></td></tr></table></figure><p>Notice how I passed an array of arrays to the <code>Values</code> property? The API uses lists of lists for most values. This is necessary when the data includes multiple rows, but a bit annoying when updating single rows or cells. Fortunately it’s pretty easy to make an inline Array in modern C#, and an Array implements <code>IList</code>.</p><p>Each request object does have a <code>Configure()</code> method that allows you to set properties inline, but I found it even more verbose than working with the request object directly.</p><h2 id="Authentication"><a href="#Authentication" class="headerlink" title="Authentication"></a>Authentication</h2><p>There are <a href="https://developers.google.com/workspace/guides/create-credentials">a bunch of ways to authenticate to the Google APIs</a>, but I found the choices overwhelming. For my purposes (a background service that needs to authenticate as a single user without a human present) a <a href="https://cloud.google.com/iam/docs/service-account-overview">service account</a> seemed like the best choice.</p><p>You can generate a service account in <a href="https://console.cloud.google.com/">the Google Cloud Console</a>. You’ll need a project. Also make sure to enable the Google Sheets API for your project. The UI in the console changes all the time, so you’ll be better off looking up instructions in the official documentation.</p><p>Once the account is created, you can download the JSON key file, and use this in your client code like this:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> credential = GoogleCredential</span><br><span class="line">    .FromJson(<span class="string">&quot;&lt;your JSON here&gt;&quot;</span>) <span class="comment">// or use .FromFile() and pass the path</span></span><br><span class="line">    .CreateScoped(SheetsService.ScopeConstants.Spreadsheets);</span><br><span class="line"></span><br><span class="line"><span class="keyword">using</span> <span class="keyword">var</span> service = <span class="keyword">new</span> SheetsService(<span class="keyword">new</span> BaseClientService.Initializer</span><br><span class="line">    &#123;</span><br><span class="line">        HttpClientInitializer = credential</span><br><span class="line">    &#125;);</span><br></pre></td></tr></table></figure><p>When retrieving this file, also take note of the service account email address (the property <code>client_email</code> in the JSON key file). You will need to share any google documents with this email address for the account to have access to them.</p><h2 id="Spreadsheet-Sheet-and-Range-Identifiers"><a href="#Spreadsheet-Sheet-and-Range-Identifiers" class="headerlink" title="Spreadsheet, Sheet, and Range Identifiers"></a>Spreadsheet, Sheet, and Range Identifiers</h2><p>There are three kinds of identifiers you need to be aware of (<a href="https://developers.google.com/sheets/api/guides/concepts">documentation here</a>):</p><ul><li><strong>Spreadsheet</strong> - This is the top-level document identifier. You’ll find it in the URL for the Google Sheets Spreadsheet.</li><li><strong>Sheet</strong> - This is the tab within the Spreadsheet, also identified by the gid in the URL. It’s usually passed as the first part of the range (before the <code>!</code>). I’ve found that an empty string refers to the first (auto-generated) sheet in the document (gid 0), or you can use the name of the sheet. I haven’t had success passing the gid (instead of the name) even though the documentation says it’s supported.</li><li><strong>Range</strong> - Most of the API calls require a range. This refers to a rectangular group of Cells, and there are two ways to specify them:<ul><li><strong>A1 notation</strong> is the most familiar and most supported. For example <code>Sheet1!A1:D1</code> will refer to the first four Cells in the first Row of the Sheet named “Sheet1”.</li><li><strong>R1C1 notation</strong> is another syntax that can be used with some api methods. The above example would look like <code>Sheet1!R1C1:R1C4</code> in R1C1 notation. It can be pretty handy when doing computed updates because you don’t have to convert the column index to a letter&#x2F;letters. Unfortunately, it doesn’t seem to be supported by all the API methods.</li></ul></li></ul><p>If you need to compute the column letter(s) for a given index, you can use this code:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">private</span> <span class="keyword">static</span> <span class="built_in">string</span> <span class="title">GetColumnNameFromIndex</span>(<span class="params"><span class="built_in">int</span> columnIndex</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">const</span> <span class="built_in">int</span> numberOfLetters = <span class="number">26</span>;</span><br><span class="line">    <span class="keyword">if</span> (columnIndex &gt;= numberOfLetters)</span><br><span class="line">        <span class="keyword">return</span> GetColumnNameFromIndex(columnIndex / numberOfLetters - <span class="number">1</span>) +</span><br><span class="line">               GetColumnNameFromIndex(columnIndex % numberOfLetters);</span><br><span class="line"></span><br><span class="line">    <span class="keyword">const</span> <span class="built_in">char</span> firstColumnLetter = <span class="string">&#x27;A&#x27;</span>;</span><br><span class="line">    <span class="keyword">return</span> ((<span class="built_in">char</span>)(firstColumnLetter + columnIndex)).ToString();</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><h2 id="Update-Types"><a href="#Update-Types" class="headerlink" title="Update Types"></a>Update Types</h2><p>There are three kinds of changes you can make to a sheet:</p><ul><li><strong>Update</strong> - this is the simplest, allowing you to update a single range of cells. Use these sparingly. Batch updates will be much more efficient where they’re possible.</li><li><strong>Batch Update</strong> - this method takes an array of range update blocks and processes them all together.</li><li><strong>Append</strong> - This method allows you to stick rows on the end of the document. It requires a range, but it’s okay to use row 1. It will automatically stick the new values at the end of the sheet for you.</li></ul><h2 id="ETags"><a href="#ETags" class="headerlink" title="ETags"></a>ETags</h2><p>All change requests and get responses include an ETag property. In an ideal world, you would include the ETag from your get request when you send an update so that any changes that have occurred since are taken into account. Unfortunately, this property never seems to be set during get requests, and I couldn’t figure out how to make it work.</p><p>In all of my testing, the APIs were responding quickly enough that it likely wouldn’t matter. If you have a much longer process between gets and updates, you may need to figure this out. Please comment below if you do!</p><h2 id="Performance-Update-Size"><a href="#Performance-Update-Size" class="headerlink" title="Performance &#x2F; Update Size"></a>Performance &#x2F; Update Size</h2><p>I couldn’t find any documentation on the limits of the API, so I did some rough testing instead. I found that a batch update with 10,000 cell updates in it (each as a separate change in the request) took only 1.9 seconds. This goes way beyond what my client could ever need.</p><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>Google sheets are a great way to share data from an automated process. Although it took a little while to figure out how to read and write data at all, once I had it working, the Google Sheets API worked great. Even better, my client was thrilled with the results.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I recently did a project for a client that needed a way to share a bit of live data with an external organization. He originally asked fo</summary>
      
    
    
    
    
    <category term=".net" scheme="https://jessemcdowell.ca/tags/net/"/>
    
    <category term="c#" scheme="https://jessemcdowell.ca/tags/c/"/>
    
    <category term="consulting" scheme="https://jessemcdowell.ca/tags/consulting/"/>
    
  </entry>
  
  <entry>
    <title>Using WeeChat and weechat-android on Windows 11</title>
    <link href="https://jessemcdowell.ca/2024/09/Using-WeeChat-on-Windows-11/"/>
    <id>https://jessemcdowell.ca/2024/09/Using-WeeChat-on-Windows-11/</id>
    <published>2024-09-06T00:43:38.000Z</published>
    <updated>2024-10-06T09:25:25.000Z</updated>
    
    <content type="html"><![CDATA[<p>Internet Relay Chat (IRC) is an old text-based chat system that, while seemingly in decline, still has many users around the world. It’s not as convenient as modern alternatives, but for those of us that are familiar with it, and especially those of us that have friends there, it’s still a very viable way to communicate.</p><p>There are a number of IRC clients available. On windows, <a href="https://www.mirc.com/">mIRC</a> is popular and easy to set up, but it shows its age. <a href="https://www.irccloud.com/">IRCCloud</a> is very convenient, especially if you want to use a mobile device, but it lacks some common features, and you are limited to two servers on the free account. There are other options, of course, but they all have limitations.</p><p>If you like doing things the hard way, you can use <a href="https://weechat.org/">WeeChat</a> like me. It’s powerful, has a bunch of plugins, and importantly, is still maintained. It also has a relay protocol that allows remote clients (like one on your phone) to read and respond to messages when you need to step away. It is, however, a terminal application, so it can take some getting used to.</p><p>WeeChat is built for Linux, but it is still very possible to run it on modern Windows. I’m confident you can get it working with the following steps.</p><h2 id="Running-WeeChat-in-a-Podman-Container"><a href="#Running-WeeChat-in-a-Podman-Container" class="headerlink" title="Running WeeChat in a Podman Container"></a>Running WeeChat in a Podman Container</h2><p>The best method I’ve found for running WeeChat is inside a Podman container. <a href="https://podman.io/">Podman</a> is a container engine similar to Docker, but free on Windows. Setting it up to run WeeChat is rather easy:</p><ol><li>(Recommended, but not required): Install <a href="https://github.com/microsoft/terminal">Windows Terminal</a>:<br><code>winget install Microsoft.WindowsTerminal</code></li><li>Download and install the Podman CLI from <a href="https://github.com/containers/podman/releases">the Podman Releases page</a>. See <a href="https://github.com/containers/podman/blob/main/docs/tutorials/podman-for-windows.md">the Podman installation instructions</a> for more info.</li><li>Create a directory for storing WeeChat configuration and logs. I use <code>$HOME/OneDrive/WeeChat</code> in the examples here, but you can use any location you like</li><li>Run the following in Windows Terminal (PowerShell). Or better yet, save it as a script somewhere in your path for easy use later: <figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">if</span> ((podman machine info | <span class="built_in">Out-String</span>).Contains(<span class="string">&#x27;machinestate: Stopped&#x27;</span>))</span><br><span class="line">&#123;</span><br><span class="line">    podman machine <span class="built_in">start</span></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">podman run <span class="literal">--rm</span> <span class="literal">-it</span> <span class="literal">--pull</span> newer <span class="literal">--env</span> <span class="string">&#x27;WEECHAT_HOME=/home/user/weechat&#x27;</span> <span class="literal">-v</span> <span class="variable">$HOME</span>/OneDrive/WeeChat:/home/user/weechat <span class="literal">--tz</span> local weechat/weechat:la<span class="built_in">test-alpine</span> weechat @args</span><br></pre></td></tr></table></figure></li></ol><p>And that’s it. You should now be running WeeChat.</p><p>Some notes about the startup script:</p><ul><li>The first block of the script runs the <code>podman machine start</code> command for you automatically if needed. This saves you a step the first time you run podman after each reboot of your computer</li><li>The container is being re-created and deleted every time it’s run (because of <code>podman run</code> and <code>--rm</code>), and new images are automatically downloaded (because of <code>--pull newer</code>). You could create a container and start it each time, but continually recreating it this way ensures that you get updates automatically when new images are published</li><li>the left half of <code>-v</code> specifies where data will be saved on your computer (outside the container). This is important, because everything else inside the container will be reset every time you run this script</li><li>The <code>--env</code> command forces WeeChat (via a supported environment variable) to write everything (configuration and logs) into a single folder inside the container which allows the file mapping with a single <code>-v</code></li><li>(Added 2024-10) <code>weechat @args</code> at the end of the command line allows the PowerShell script to pass arguments through to WeeChat. I have been using this to sometimes send the <code>--no-connect</code> option when I want to prevent auto-connect or auto-join.</li></ul><h3 id="Why-not-use-WSL-directly"><a href="#Why-not-use-WSL-directly" class="headerlink" title="Why not use WSL directly"></a>Why not use WSL directly</h3><p>It is absolutely possible to run WeeChat in WSL, but I prefer the Podman approach because:</p><ul><li>it’s easier to expose the WeeChat Relay port - I found it difficult to share a port for an app in WSL to my local network</li><li>you get updates to WeeChat automatically without running <code>apt update; apt upgrade</code></li><li>you are certain no state is being secretly left behind inside the WSL environment</li><li>you don’t have to worry about breaking WeeChat if you break WSL, and don’t lose anything if you decide to reset it</li><li>it uses less space. The normal Ubuntu image used by WSL is pretty big, which is wasteful if you aren’t using it for anything else</li></ul><h2 id="Using-WeeChat-Relay-and-a-mobile-client"><a href="#Using-WeeChat-Relay-and-a-mobile-client" class="headerlink" title="Using WeeChat Relay and a mobile client"></a>Using WeeChat Relay and a mobile client</h2><p>One of the main reasons I went with WeeChat was the WeeChat Relay protocol and clients like <a href="https://github.com/ubergeek42/weechat-android">weechat-android</a>. This allows you to keep WeeChat running on your main computer, but access it remotely to check for or respond to messages. I find this handy when I need to do a chore or take a break. Because IRC depends on persistent connections, it is hard to use it (reliably) on a mobile device directly. And because WeeChat is exposing the relay, people don’t even know that you’re switching devices.</p><p>You can set up the relay service with the following steps. There are also <a href="https://weechat.org/files/doc/weechat/stable/weechat_user.en.html#relay">detailed instructions in the documentation</a> if you have specific questions.</p><ol><li>Make up a password</li><li>Use the following commands in WeeChat to store the password, and use it for your relay server:<figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">/secure set relay new_password</span><br><span class="line">/set relay.network.password &quot;$&#123;sec.data.relay&#125;&quot;</span><br></pre></td></tr></table></figure></li><li>Use the following command in WeeChat to enable WeeChat Relay:<figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">/relay add weechat 9500</span><br></pre></td></tr></table></figure></li><li>Run the following in PowerShell (as Administrator) to allow WeeChat Relay through the Windows firewall, and to proxy any traffic from the local network to the relay in the container:<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">New-NetFirewallRule</span> <span class="literal">-DisplayName</span> <span class="string">&quot;WeeChat Relay&quot;</span> <span class="literal">-Direction</span> Inbound <span class="literal">-Protocol</span> TCP <span class="literal">-LocalPort</span> <span class="number">9500</span> <span class="literal">-Action</span> Allow</span><br><span class="line"></span><br><span class="line">netsh interface portproxy <span class="built_in">set</span> v4tov4 listenport=<span class="number">9500</span> listenaddress=<span class="number">0.0</span>.<span class="number">0.0</span> connectport=<span class="number">9500</span> connectaddress=<span class="number">127.0</span>.<span class="number">0.1</span></span><br></pre></td></tr></table></figure></li><li>Add the following to the <code>podman run</code> command to expose the relay port from the container: <code>-p 9500:9500</code>. The complete command will now be:<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">podman run <span class="literal">--rm</span> <span class="literal">-it</span> <span class="literal">--pull</span> newer <span class="literal">--env</span> <span class="string">&#x27;WEECHAT_HOME=/home/user/weechat&#x27;</span> <span class="literal">-v</span> <span class="variable">$HOME</span>/OneDrive/WeeChat:/home/user/weechat <span class="literal">--tz</span> local <span class="literal">-p</span> <span class="number">9500</span>:<span class="number">9500</span> weechat/weechat:la<span class="built_in">test-alpine</span> weechat @args</span><br></pre></td></tr></table></figure></li><li>Install <a href="https://play.google.com/store/apps/details?id=com.ubergeek42.WeechatAndroid.dev">weechat-android</a> on your Android device. I haven’t found an iOS equivalent. If you have one you’d recommend, please post it in the comments.</li><li>Configure your client. You can use the following settings in weechat-android:<ul><li>Connect type: Plain Connection</li><li>Relay host: IP address of the computer running WeeChat (inside your network)</li><li>Relay port: 9500</li><li>Relay password: The password you created earlier</li></ul></li></ol><p>As long as WeeChat is running on your computer, and the client is connected to the same network, you should now be able to connect to the relay.</p><p>If you want to access WeeChat relay from outside your home, you’ll have to set up some sort of vpn to your home network. I do not suggest opening this port to the internet directly without setting up TLS encryption first.</p><h2 id="Setting-up-WeeChat-Relay-with-TLS"><a href="#Setting-up-WeeChat-Relay-with-TLS" class="headerlink" title="Setting up WeeChat Relay with TLS"></a>Setting up WeeChat Relay with TLS</h2><p>WeeChat supports TLS for the relay protocol, which makes it much safer for connections going over the internet. However, it requires a DNS record that points to your WeeChat instance, and a self-signed certificate that matches the name. This appears to be a security limitation in Android itself, so you’ll have to deal with it if you want TLS.</p><p>For my own purposes, I stuck an A record in one of my domains that points to the local fixed IP address of my main computer. If you’re also using the relay protocol over your home network, you could use a dynamic DNS service like <a href="https://www.duckdns.org/">Duck DNS</a> that allows you to specify the IP address manually. Dynamic DNS will also be helpful if you are opening the port to the internet from your router.</p><p>Once you have DNS figured out, generating a self-signed certificate can be done relatively easily.</p><ol><li>Run the following in PowerShell to start an interactive shell in a temporary container:<figure class="highlight powershell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">podman run <span class="literal">--rm</span> <span class="literal">-it</span> <span class="literal">-v</span> <span class="variable">$HOME</span>/OneDrive/WeeChat:/weechat <span class="literal">--entrypoint</span> sh alpine/openssl</span><br></pre></td></tr></table></figure><ul><li>Note: Make sure the directory mapping (on the left) matches the location on your host machine where you store your WeeChat data.</li></ul></li><li>Run the following with the DNS name you set up:<figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">export</span> HOSTNAME=yourweechatrelay.duckdns.org</span><br></pre></td></tr></table></figure></li><li>Run the following to generate a self-signed certificate in the WeeChat directory:<figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">mkdir</span> -p weechat/tls</span><br><span class="line">openssl req -x509 -newkey rsa:4096 -nodes \</span><br><span class="line">    -keyout weechat/tls/relay.pem -out weechat/tls/relay.pem \</span><br><span class="line">    -days 3650 -subj <span class="string">&quot;/CN=<span class="variable">$HOSTNAME</span>&quot;</span> -addext <span class="string">&quot;subjectAltName=DNS:<span class="variable">$HOSTNAME</span>&quot;</span></span><br></pre></td></tr></table></figure></li><li>Run <code>exit</code> to stop the container</li><li>Run the following in WeeChat to replace the relay server with a TLS-enabled relay server<figure class="highlight text"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">/relay del weechat</span><br><span class="line">/set relay.network.tls_cert_key &quot;$&#123;weechat_config_dir&#125;/tls/relay.pem&quot;</span><br><span class="line">/relay add ssl.weechat 9500</span><br></pre></td></tr></table></figure></li></ol><p>From weechat-android, you can change the Connect type to “WeeChat SSL”. The next time you connect, you’ll be asked to verify the certificate. After this all communication between WeeChat and your client should be encrypted.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Internet Relay Chat (IRC) is an old text-based chat system that, while seemingly in decline, still has many users around the world. It’s </summary>
      
    
    
    
    
    <category term="howto" scheme="https://jessemcdowell.ca/tags/howto/"/>
    
  </entry>
  
  <entry>
    <title>Rocketbook vs Whiteboard</title>
    <link href="https://jessemcdowell.ca/2024/08/Rocketbook-vs-Whiteboard/"/>
    <id>https://jessemcdowell.ca/2024/08/Rocketbook-vs-Whiteboard/</id>
    <published>2024-08-07T05:24:33.000Z</published>
    <updated>2024-08-07T05:24:33.158Z</updated>
    
    <content type="html"><![CDATA[<p>I have always liked using whiteboards. They are a great tool for communicating, but also great for exploring thoughts and stimulating creativity. There are computerized equivalents, and they have their advantages, but I’ve never found them as effective because I get distracted too easily. I can spend more time fiddling with the lines and colours than working on my actual problem.</p><p>I like whiteboards so much that I bought a stack of them, and an artist’s portfolio bag to carry them around. I imagined popping out a few drawings at a meeting, or handing boards around for a creative session. I still love the idea, but in more than twenty years I’ve never even taken them out of the house.</p><p>These days I use a <a href="https://getrocketbook.com/">Rocketbook</a> for most of my creative explorations. I received one as a gift at work a few years ago, and have been in love with it ever since. It looks and performs like a normal notebook, but it’s easy to scan the pages, and the pages can be reused after erasing them with a wet cloth. This is achievable because of some clever design and a specially optimized app.</p><p>If I’m working at home, or travelling, the Rocketbook is very convenient. It’s also good when I’m working in an office and need to sketch something. More than just replacing a whiteboard, it can be helpful for notes. Entering notes in a laptop is usually better, but if I have a couple of quick notes, or don’t have a laptop handy, the Rocketbook will do the job.</p><p>A whiteboard is still the best tool for an in-person meeting. Having the drawings out in the open where anyone can grab a pen and add to it is very convenient. Online meetings that involve collaboration don’t work very well with a whiteboard, or a Rocketbook for that matter. In these situations I prefer a virtual whiteboard. It does, however, also require some preparation before the meeting to prevent wasting precious time in the meeting.</p><p>If you’re interested in a Rocketbook for yourself, I suggest looking at the Rocketbook Fusion first. It has a mix of lined pages for notes and grid pages for drawings. I like the executive size for its portability. The only hitch is that you need a hard surface to write effectively. This isn’t a problem at a desk, but you may want to get a hard cover if you need to take notes on the move. If you’re doing a lot of design work at a desk, a letter-sized Rocketbook Matrix may be a better choice.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I have always liked using whiteboards. They are a great tool for communicating, but also great for exploring thoughts and stimulating cre</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="gadgets" scheme="https://jessemcdowell.ca/tags/gadgets/"/>
    
    <category term="reviews" scheme="https://jessemcdowell.ca/tags/reviews/"/>
    
  </entry>
  
  <entry>
    <title>The Pragmatic Potato Tech Stack</title>
    <link href="https://jessemcdowell.ca/2024/06/Pragmatic-Potato-Tech-Stack/"/>
    <id>https://jessemcdowell.ca/2024/06/Pragmatic-Potato-Tech-Stack/</id>
    <published>2024-06-24T19:02:43.000Z</published>
    <updated>2024-08-07T05:04:17.818Z</updated>
    
    <content type="html"><![CDATA[<p>I <a href="/2023/11/Announcing-Pragmatic-Potato-Software/">recently launched my new company</a>, <a href="https://pragmaticpotato.com/">Pragmatic Potato Software Inc</a>. The creation of a company itself is pretty easy, but setting up everything you need to do business can become overwhelming quickly. There are a lot of compelling options available, each promising the moon. It’s not that simple though.</p><p>I’m going to be writing about the technology stack I’m using to run my company, and why I made the choices I did. There is a lot more to a company than its technology, but I’m not an expert in setting those up.</p><p>I made these choices for my company, keeping in mind my needs and values. I would probably recommend something similar for another small company, but it would depend. I didn’t perform an exhaustive search for each decision. Most options are going to be good enough, and even if I make a poor choice, I can replace most of these with a few hours of work while my company is small. I did put some effort into it though, and I’m happy to share that with you.</p><p>Here is some more of the context that guided my decisions:</p><ul><li>Pragmatic Potato Software is a small (single employee) software architecture and development consulting company based in Burnaby, Canada.</li><li>I want to minimize the time I’m spending managing infrastructure because I’d rather spend that time helping customers and potentially earning income.</li><li>I am pretty technically capable, and can handle setting up and hosting my own infrastructure. I also don’t need to worry much about usability: I am happy to use a code editor when I want to update the text on my website.</li><li>I have modest expectations for traffic and customer interactions, so I should be able to use free or low-cost services for most things. I would like to keep my costs down.</li></ul><h2 id="Domain-Registration-Cloudflare-and-Porkbun"><a href="#Domain-Registration-Cloudflare-and-Porkbun" class="headerlink" title="Domain Registration: Cloudflare and Porkbun"></a>Domain Registration: Cloudflare and Porkbun</h2><p>For any kind of web presence, you need a domain name. Of all the registrars I’ve used, my favourite is <a href="https://www.cloudflare.com/products/registrar/">Cloudflare Registrar</a>. They famously don’t make any profit on their domain offering, so it is also the best price you can get. I used them for the <code>pragmaticpotato.com</code> domain.</p><p>Because I have a Canadian company, I also registered <code>pragmaticpotato.ca</code>, and have it set up to redirect to the .com site. At the time of this writing Cloudflare can’t register .ca domains, so I am using Porkbun for that one.</p><h2 id="Website-Hugo-static-site-hosted-on-Cloudflare-Pages"><a href="#Website-Hugo-static-site-hosted-on-Cloudflare-Pages" class="headerlink" title="Website: Hugo static site hosted on Cloudflare Pages"></a>Website: Hugo static site hosted on Cloudflare Pages</h2><p><a href="/2023/11/This-Blog-Hexo-generated-static-site-hosted-on-GitHub-Pages/">I’ve written before about my preference for static websites</a>. They are the simplest thing that can possibly work, and that’s often a good starting point. I can’t imagine anything I’d want to do on this website that’d require a dynamic back end, so this should be good enough for a long time.</p><p>I originally planned to use Hexo for the website because I already use it for this blog, but I had trouble finding a template that I liked for a business landing page. I ended up using <a href="https://gohugo.io/">Hugo</a>. It is very similar to Hexo, but I found several templatse that I liked.</p><p>Having used both, I now prefer Hugo. One feature I appreciate is its ability to extend templates through partials rather than modifying the templates directly. This allows some pretty direct customizations, and should make it easier to take updates in the future. I used this functionality to tune the page metadata for search engines and social links. In the template I chose the templates was broken up into several partials to make these sorts of customizations trivial.</p><p>The hardest part of putting the website together was writing the text, and I went through several iterations to get it just the way I wanted. Then my friends told me it was garbage, so I rewrote most of it again!</p><p>I’m using <a href="https://pages.cloudflare.com/">Cloudflare Pages</a> to host the content. It’s very similar to GitHub Pages (which I use for this blog), but I like that it’s a bit more configurable. I also found the build system a little easier to set up. I liked it enough that I will probably end up moving this blog over at some point too.</p><h2 id="Contact-Form-FabForm"><a href="#Contact-Form-FabForm" class="headerlink" title="Contact Form: FabForm"></a>Contact Form: FabForm</h2><p>I’m using <a href="https://fabform.io/">FabForm</a> for the contact form on the website. It is a simple service that was trivial to set up. I also find the one-time pricing much more reasonable than the higher (over time) monthly fees of other popular options. Using this allows me to accept messages on the website without needing a dynamic back-end.</p><h2 id="Stock-Photography-Unsplash-and-Squoosh"><a href="#Stock-Photography-Unsplash-and-Squoosh" class="headerlink" title="Stock Photography: Unsplash and Squoosh"></a>Stock Photography: Unsplash and Squoosh</h2><p>I used a few stock photos on the website to make it feel more engaging. <a href="https://unsplash.com/">Unsplash</a> has a massive library of free, high-resolution photos taken by professionals. I made a point of including an attribution page, but that isn’t necessary with their licence. I used <a href="https://squoosh.app/">Squoosh</a> to resize and compress the images for efficient load times.</p><p>I also hired a professional photographer to take some pictures of me to make the website more personal. This was a particularly fun process, and I’m thrilled with the results. The photographer was very reasonable. My new clothing was the most expensive part.</p><h2 id="Website-Analytics-Cloudflare-Web-Analytics"><a href="#Website-Analytics-Cloudflare-Web-Analytics" class="headerlink" title="Website Analytics: Cloudflare Web Analytics"></a>Website Analytics: Cloudflare Web Analytics</h2><p>I am using <a href="https://www.cloudflare.com/web-analytics/">Cloudflare Website Analytics</a> for the website, mostly because it was easily available. I also appreciate that they collect minimal information, and in a way that doesn’t require popups to comply with modern privacy laws.</p><h2 id="Email-Zoho-Workplace"><a href="#Email-Zoho-Workplace" class="headerlink" title="Email: Zoho Workplace"></a>Email: Zoho Workplace</h2><p>Microsoft 365 and Google Workspace are the two big players for email and office application, and I did consider each of them. I ended up going with <a href="https://www.zoho.com/workplace/">Zoho Workplace</a> because it is much cheaper and comes bundled with more stuff.</p><p>Setting it up was an easy process. Even setting up email for my custom domain was easy. I had it all up and working in less than an hour.</p><p>I don’t recommend trying to run your own email server, even for medium-sized businesses. Email used to be an open system that anyone could participate in, but with all the spam and especially with the spam prevention mechanisms today, it can be difficult to keep your emails out of spam boxes.</p><h2 id="Document-Editing-Storage-Zoho-Workplace-Evernote"><a href="#Document-Editing-Storage-Zoho-Workplace-Evernote" class="headerlink" title="Document Editing &amp; Storage: Zoho Workplace &amp; Evernote"></a>Document Editing &amp; Storage: Zoho Workplace &amp; Evernote</h2><p>I’m gradually moving my (few) stored files over to <a href="https://www.zoho.com/workdrive/">Zoho WorkDrive</a>. I was using a little bit of Google and a bit of Microsoft, but ultimately found their free options too limiting. The final straw for me was editing a contract. I spent a couple of hours trying to get the numbered lists and cross-section links right in the free online Microsoft Word and Google Docs before giving up. Zoho Writer has limitations too, but it proved the easiest to use.</p><p>Almost all of my work notes and planning content is in <a href="https://evernote.com/">Evernote</a>, which I’ve been using for many years. It’s not great for some kinds of content (like sheets), but it is extremely searchable, which makes it fast for pretty much all other information.</p><h2 id="Task-Management-Trello-Evernote"><a href="#Task-Management-Trello-Evernote" class="headerlink" title="Task Management: Trello &amp; Evernote"></a>Task Management: Trello &amp; Evernote</h2><p>I’ve been using <a href="https://trello.com/">Trello</a> for years to organize myself. I use it primarily to track the things I’m working on in the short term. It’s especially easy to slap cards into a list and put them in a sensible order. I mix personal and professional tasks in the same list to reduce friction as I go through the day.</p><p>For my longer term backlog, I’m using notes in <a href="https://evernote.com/">Evernote</a>. It isn’t a great tool for project management, but it’s good enough for now.</p><h2 id="Video-Conferencing-Zoho-Meeting"><a href="#Video-Conferencing-Zoho-Meeting" class="headerlink" title="Video Conferencing: Zoho Meeting"></a>Video Conferencing: Zoho Meeting</h2><p>Because I’m already using Zoho Workplace, I get <a href="https://www.zoho.com/meeting/">Zoho Meeting</a> included, and it works great. It looks and performs a lot like Zoom, but I don’t have to pay extra for it. It’s not quite as smooth as Google Meet (which has a particularly streamlined experience), but it has a few more features that I sometimes want.</p><h2 id="Appointment-Scheduling-Zoho-Bookings"><a href="#Appointment-Scheduling-Zoho-Bookings" class="headerlink" title="Appointment Scheduling: Zoho Bookings"></a>Appointment Scheduling: Zoho Bookings</h2><p>Zoho also has <a href="https://www.zoho.com/bookings/">Zoho Bookings</a> which allows customers to schedule meetings with me themselves. This can save a lot of emailing when someone needs to talk to me. I have it integrated with my Google calendar (which I use for my personal and family schedules), so it can automatically avoid conflicts with my other obligations.</p><h2 id="Bookkeeping-Outsourced"><a href="#Bookkeeping-Outsourced" class="headerlink" title="Bookkeeping: Outsourced"></a>Bookkeeping: Outsourced</h2><p>Another key part of a business is keeping appropriate financial records. I spent some time looking into software for managing this, but I have decided so far not to bother. My accountant is happy to take care of this for me, and they can do it far faster than I can. Even though I’m the kind of person that likes to keep records up to date, and to see the numbers for myself, I would rather spend that time helping customers instead.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I &lt;a href=&quot;/2023/11/Announcing-Pragmatic-Potato-Software/&quot;&gt;recently launched my new company&lt;/a&gt;, &lt;a href=&quot;https://pragmaticpotato.com/&quot;&gt;P</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="architecture" scheme="https://jessemcdowell.ca/tags/architecture/"/>
    
    <category term="reviews" scheme="https://jessemcdowell.ca/tags/reviews/"/>
    
    <category term="pragmatic potato" scheme="https://jessemcdowell.ca/tags/pragmatic-potato/"/>
    
  </entry>
  
  <entry>
    <title>Cross-Cutting Concerns - Ten Approaches</title>
    <link href="https://jessemcdowell.ca/2024/05/Cross-Cutting-Concerns/"/>
    <id>https://jessemcdowell.ca/2024/05/Cross-Cutting-Concerns/</id>
    <published>2024-05-27T20:18:08.000Z</published>
    <updated>2024-08-07T05:04:17.817Z</updated>
    
    <content type="html"><![CDATA[<p>One often (and ironically) repeated rule in programming is: don’t repeat yourself. We repeat it so much we even have an abbreviation: DRY. There are good reasons for this advice. Duplicating and modifying code can be a quick and easy way to get a feature done, but it can also lead to problems over time. It’s harder to understand code when the valuable logic is mixed with reams of low-value boilerplate. Subtle differences can also sneak in, leading to inconsistent behaviour across the application. Things get even more difficult if you want to make a change.</p><p>Cross-cutting concerns are the requirements that span multiple features. This includes things such as error handling, logging, profiling, security policy, and so on. We often want these things to be consistent, and we occasionally want to make systematic adjustments to them. Unfortunately, because of their nature, they can be challenging to centralize.</p><p>There are a bunch of tools and techniques for dealing with cross-cutting concerns, and I’ve made a list of them. I recommend thinking carefully about which approach(es) you want to use. It may be challenging to switch approaches once one has become widely used in your code base.</p><p>Once you abstract away some of these big requirements, you start to think about similar techniques every time you repeat a pattern. This is a deep rabbit hole to go down, but I can’t say it’s entirely wrong to plunge the depths. Things like classes that report changes, hash functions, even writing property getters and setters can be convenient to automate. This kind of code is often not fun to write, and because we often put little thought into it, honest mistakes can creep in. Worse, it’s eve more tedious to test these sorts of things, and the bugs they cause can be tough ones to reproduce and find.</p><p>I am focusing on the broader-scoped cross-cutting concerns in this post, but the same tools and techniques can be applied to many places where you need to implement the same behaviour across multiple bits of code.</p><h2 id="Aspect-Oriented-Programming-AOP-tools"><a href="#Aspect-Oriented-Programming-AOP-tools" class="headerlink" title="Aspect Oriented Programming (AOP) tools"></a>Aspect Oriented Programming (AOP) tools</h2><p>Aspect Oriented Programming is an approach that tackles cross-cutting concerns head on. The way they work varies by framework, but essentially they inject behaviour by either wrapping functions or modifying compiled code. You attach behaviours either using language features (ex: Attributes, Annotations), or through pattern matching (ex: namespace, class name, function name, function signature). There are multiple tools available in many common languages, so you should have no trouble finding something for your project.</p><p>When they work, these tools can be magical. Slapping a security attribute on a function is far easier than interrogating some security service and writing your own error messages. It also helps us keep our code cleaner: the account balance class does need to be secure, but security isn’t really a core part of the logic, and it’s nice to keep them untangled.</p><p>The problem with this approach is that it can be a little too magical. A developer joining the team could find it difficult to understand why some behaviour is happening where there is no apparent code causing it. It is also not always straightforward to debug this kind of code when it doesn’t work (but some tools perform better than others). It can be even more confusing when some shared logic is executed where it’s not supposed to.</p><p>If you’re going this route, there are two flavours to consider: some tools modify the compiled code, and others wrap objects at runtime (often in cooperation with your dependency injection system).</p><p>Wrapping objects at runtime is helpful because they can sometimes wrap code you didn’t write. They may also make it possible to dynamically add or change behaviours, either via configuration files or your own code. For example, you could inject a policy that wraps all database calls with performance counters, or do this only in a test environment.</p><p>Working at (or near) compile time can be a good choice too. Behaviour inserted this way has the benefit of access to the internal bits of your classes, making more powerful policies possible.</p><h2 id="Extract-into-a-helper-function"><a href="#Extract-into-a-helper-function" class="headerlink" title="Extract into a helper function"></a>Extract into a helper function</h2><p>If you don’t want magical code, you can still centralize repeated logic with your own helper functions. This gives you one place to make changes, and is easier for new developers to follow. For logic that needs to wrap your code (ex: profiling), you can write a helper function that takes a lambda or delegate. For example (implementation simplified for illustration):</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">Main</span>()</span></span><br><span class="line">&#123;</span><br><span class="line">    Profile(<span class="string">&quot;Application Startup&quot;</span>, () =&gt;</span><br><span class="line">    &#123;</span><br><span class="line">        <span class="comment">// startup code here</span></span><br><span class="line">    &#125;);</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">Profile</span>(<span class="params"><span class="built_in">string</span> description, Action profiledFunction</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">var</span> executionTime = Stopwatch.StartNew();</span><br><span class="line">    <span class="keyword">try</span></span><br><span class="line">    &#123;</span><br><span class="line">        profiledFunction();</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">finally</span></span><br><span class="line">    &#123;</span><br><span class="line">        Log(<span class="string">$&quot;<span class="subst">&#123;description&#125;</span> took <span class="subst">&#123;executionTime.ElapsedMilliseconds&#125;</span>ms&quot;</span>);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>The difficulty with this approach is that it can be deceptively challenging to design a good helper function. The implementation can be changed easily so long as you don’t need to change the function signature. For example, if you are writing your own logging helper, what arguments do you require? Do you want the class name? Do you want a stack trace? Do you want to support string formatting inline? Every additional argument adds more effort for developers using the helper function, however, if you don’t require enough information, you may not be able to make that desired change to your centralized logic without making expensive sweeping changes anyway.</p><p>This technique may be best for cases where you have a bit of duplication in a handful of areas. If your helper function is being called from hundreds or thousands of places, you may want a technique that is a bit more robust.</p><h2 id="Use-an-established-library"><a href="#Use-an-established-library" class="headerlink" title="Use an established library"></a>Use an established library</h2><p>Picking a library and using it can be a good approach for common concerns. Logging is an example where this makes sense. There are lots of fantastic, well-established libraries available. They have mature APIs, tested and refined in thousands of apps. Every framework I’ve seen has been configurable for most common use cases. Many also support plugins for more specialized needs.</p><p>Logging is also the kind of thing that developers want to interact with: it can be helpful to throw a bit of extra information into the logs when you’re trying to track down an elusive bug. For comparison, if you are wrapping your code with logging using an aspect oriented framework, you may not have a natural way to pass specific information in to a log message.</p><p>Compared to aspect oriented programming, however, you do have to invoke the library whenever you want to use it. If you wanted to log all calls to your service layer, you would have to call your logging framework for all your service methods (or use one of the other automatic techniques in this list).</p><p>Another drawback to this approach is that you can become stuck with the library you choose because the cost of replacing it will be prohibitive. This may or may not be the kind of thing you have to worry about though. I have seen more than zero teams switch logging frameworks in an established project, but I’ve ever seen an example where it was necessary.</p><h2 id="Use-a-common-interface-library"><a href="#Use-a-common-interface-library" class="headerlink" title="Use a common interface library"></a>Use a common interface library</h2><p>If you want the stability of a mature API, but really might need the ability to change your implementation, you could look for a common interface intermediary. There are lots of examples of community-maintained interface projects, and even some built into base frameworks. With these, most of your code gets implemented referencing the common interface, but the actual implementation is wired up through a plugin. Java’s <a href="https://commons.apache.org/proper/commons-logging/">Apache Commons Logging</a> is a great example of this.</p><p>There are also more advanced interfaces that do more than wrap another implementation. <a href="https://opentelemetry.io/">OpenTelemetry</a> is a fantastic library for recording telemetry, traces, and logs. Its greatest benefit is that you can add one or more plugins to expose this data to your various monitoring tools such as a privately hosted Prometheus system, or a commercial observability platform.</p><p>A common interface is a particularly useful tool when they are so widely used that common libraries start to use them too. For example, parts of .Net Core are now instrumented using Open Telemetry, so you can pick up valuable data in many helpful areas just by adding a plugin.</p><p>When choosing one of these interfaces, make sure you’re picking something that’s widely used in your component ecosystem. If you are forced to use two (either because you have components that use difference ones, or you decide to change), you might be able to make a simple bridge to stick them together. It is far nicer to pick the right library the first time though.</p><p>Another thing to be careful about is that these abstractions can sometimes be complicated. Using an abstract authorization interface could have a lot of bells and whistles you don’t need, or aren’t even supported in your implementation. The more complicated the interface is, the more likely you’ll find diverging usage patterns across your code, and they may not universally work. This erodes some of the benefits of using a common library in the first place.</p><h2 id="Put-it-in-a-base-class"><a href="#Put-it-in-a-base-class" class="headerlink" title="Put it in a base class"></a>Put it in a base class</h2><p>Inheritance is the classic object-oriented way to share logic, and it can be very effective. Features like abstracts and virtual functions make it easy to deal with exceptions, which many approaches to centralizing logic have trouble with. It’s also quite helpful having the compiler to enforce the rules for you, making it safer to change the structure in the future.</p><p>The problem with inheritance is that where some is good, more is definitely not better. It is a powerful tool, but you can hit its limits pretty fast. Most languages only support single inheritance, and the ones that support multiple inheritance regret it. This means that you can only inject one set of behaviours into a class. If you want logging on some classes, but logging and authorization on others, you might need multiple layers of base classes to pile the features on or off. If you have a handfull of concerns, you could find yourself making lots of base classes to pick and choose where they go. This makes this technique generally unsuitable for the classic cross-cutting concerns. Another limitation of inheritance is sometimes it isn’t available to you: you might need your classes to derive from another type to participate in some other system.</p><p>Another challenge with using base classes for shared code appears when you have dependencies. You either need to include your dependencies in every constructor, or do some tricky stuff with statics to gain access to them. For example, if you are using a base class for authorization, you may need to pass a reference to your authorization service from every implementation of the base class. If you ever wanted to make a change that required a new dependency, you’d have to go and change the constructor for every implementation. This isn’t fun if it’s widely used.</p><p>This approach can, however, work well for small-scale repeating concerns. One example that comes to mind is as a foundation for a plugin architecture. Abstract methods are a great way to ensure implementations provide the entry points you require. If you go this route, I suggest keeping your utilities out of the base class to avoid versioning problems.</p><h2 id="Put-the-logic-higher-or-lower-in-the-stack"><a href="#Put-the-logic-higher-or-lower-in-the-stack" class="headerlink" title="Put the logic higher or lower in the stack"></a>Put the logic higher or lower in the stack</h2><p>Sometimes you can find a single place higher or lower in the stack to insert some shared logic. For example, for a common requirement to log errors in a web service, your web framework might support adding some code that intercepts every request. This code can check for and log errors before responses are returned. I generally prefer this kind of approach for error handling: logging general errors all over the place takes a lot of effort. It’s much nicer when you can let most errors bubble up the stack and know that the application will record and report them correctly.</p><p>You can also sometimes use this technique lower in the stack. For example, you might be able to wrap a database connection to add some generic profiling logic for your database interactions.</p><h2 id="Use-code-generation-tools"><a href="#Use-code-generation-tools" class="headerlink" title="Use code generation tools"></a>Use code generation tools</h2><p>Some advanced IDEs have tools for generating code. This makes it easy to insert the exact same code over and over. It also makes it possible to share the patterns across a team.</p><p>The code that’s generated will be duplicated. This will make some kinds of changes to the pattern harder, but for some cross-cutting concerns that may not be as important. Even so, if the code is identical, you may be able to get pretty far into a sweeping change with carefully crafted compare-and-replace commands.</p><p>This approach may be best in places where you want some common code, but you can’t use one of the other more automatic approaches for whatever reason.</p><h2 id="Generate-boilerplate-with-AI-powered-tools"><a href="#Generate-boilerplate-with-AI-powered-tools" class="headerlink" title="Generate boilerplate with AI-powered tools"></a>Generate boilerplate with AI-powered tools</h2><p>We can quickly generate code with one of the new AI-powered tools, and they should have no trouble generating boilerplate for us along with everything else. This could seem like an attractive option for dealing with cross-cutting concerns, but I am skeptical.</p><p>Generative AIs can generate boilerplate, but they will be making it up every time they write code. You would need to provide appropriate instructions for the kinds of boilerplate you want, and remembering to do this is not that different from adding it yourself every time anyway. Even if you do, there is no guarantee that the generated code will be consistent. It may be mostly cosmetic differences, but the threat of a functional difference is always looming.</p><p>If you ever needed to change the pattern across your code base, you would have all the duplication, but subtle differences (even cosmetic ones) could make more automatic approaches impossible. Scanning all your code manually to change your error handling approach would not be fun.</p><p>It’s hard to say how these tools will perform in the future. Maybe they’ll get much better at including required boilerplate in a particular way. For now, I recommend avoiding this approach to this particular problem.</p><h2 id="Use-automated-tests-to-enforce-consistency"><a href="#Use-automated-tests-to-enforce-consistency" class="headerlink" title="Use automated tests to enforce consistency"></a>Use automated tests to enforce consistency</h2><p>Sometimes there is no good way to share the implementation of a cross-cutting concern. In these cases, it may make sense to write an automated test that makes sure that the policy is present. It may sound a bit daunting, but it can actually be pretty easy to write a test that finds every class in your project and checks it. You may be able to use reflection to find and call methods. For some scenarios, it may be easier to write a test that looks at the source files directly.</p><p>Some concerns are so important that it may be worthwhile to write these sweeping tests even if you can implement the implementation automatically using one of the other techniques in this list. Authorization is a great example: you might want to test that all of your service classes fail with the right error if no permissions are present.</p><p>When writing these kinds of tests, I find it helpful to put a list of exceptions in a constant nearby. For example, you would want your login service to work for a user that hasn’t logged in yet. There may not be any other exceptions to the rule, but you can easily add them to the list if they appear.</p><h2 id="Do-nothing"><a href="#Do-nothing" class="headerlink" title="Do nothing"></a>Do nothing</h2><p>Depending on the dynamics of your team and the kind of concerns your dealing with, it may not be possible or even worthwhile to use one of these techniques. For example, an open source project may not want to enforce strict coding requirements on unpaid volunteers. Automatic handling of a concern may be too complicated for most contributors to understand, or make it too difficult to set up a development environment.</p><p>So what happens in this worst case scenario? You get inconsistent implementation of a requirement, inconsistent implementations when the requirement is implemented, and it becomes harder to change the behaviour across the system. On the other hand, you don’t have the overhead of these approaches either. You don’t have any magic code, or any unexpected tricks hidden in middleware.</p><p>For something like a prototype, or for a small internal project that’s unlikely to grow into something bigger, maybe this is good enough.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;One often (and ironically) repeated rule in programming is: don’t repeat yourself. We repeat it so much we even have an abbreviation: DRY</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="architecture" scheme="https://jessemcdowell.ca/tags/architecture/"/>
    
  </entry>
  
  <entry>
    <title>Writing Code for Better Reviews</title>
    <link href="https://jessemcdowell.ca/2024/04/Writing-Code-for-Better-Reviews/"/>
    <id>https://jessemcdowell.ca/2024/04/Writing-Code-for-Better-Reviews/</id>
    <published>2024-04-24T21:37:05.000Z</published>
    <updated>2024-08-07T05:04:17.816Z</updated>
    
    <content type="html"><![CDATA[<p>I believe code reviews are a high-value activity (which <a href="/2024/01/Optimal-Code-Reviews/">I’ve written about before</a>), but they take time and slow down your development process. With a few simple tricks, you can make it easier for reviewers to understand your changes, allowing them to give you better feedback faster. Not only does this save everyone time, but it also improves the quality of your code.</p><h2 id="Make-your-intentions-understood"><a href="#Make-your-intentions-understood" class="headerlink" title="Make your intentions understood"></a>Make your intentions understood</h2><p>Good commit messages and review titles are important. It may be (and in fact, should be) obvious from your code change what you’re trying to do, but a good message is still important. There are lots of cases where someone needs to scan the change list, and good messages make this a lot easier. It also helps your reviewer understand what you’re doing.</p><p>The key to a good commit message is describing your intention. “Added some code” may be correct, but it’s not helpful. “Check for invalid email addresses” explains why you added the code. Your audience is a future developer trying to figure out why a feature broke, and what they care about is getting a quick picture of what you could have changed.</p><p>A little humour can be fun sometimes, but be careful with this. Small tasteful jokes are fine, but sarcasm or jokes that obscure your meaning will just slow others down.</p><h2 id="Group-changes-by-intention-one-intention-at-a-time"><a href="#Group-changes-by-intention-one-intention-at-a-time" class="headerlink" title="Group changes by intention, one intention at a time"></a>Group changes by intention, one intention at a time</h2><p>I tend to meander through changes as opposed to attacking them linearly, and I think this is pretty common. I run into small things while I’m wandering through the code, and I want to fix them right away. I could take a note and come back later, but this sometimes feels like moving backwards. The compromise that I normally go with is fixing the side-thing anyway, but presenting it separately.</p><p>If the change is very small (ex: a typo in a private method name - the fix will only affect a couple lines of code in one file) it may be okay to tuck it in. If it’s only a few changes but spread across a few files, putting it in its own commit is better.</p><p>The worst offender is the sweeping change. No matter how much a typo in a widely used function name might bother, adding a couple of hundred lines of noise to the review is absolutely going to slow it down. A new, separate review is a great way to present these. It’s a lot easier to verify simple changes if they are the only kind of change present. With Git and GitHub, it’s pretty easy to rebase a single sweeping change commit into its own branch and put it up for for review immediately.</p><p>Some “simple” fixes are not as simple as they seem, and could introduce a bug. If there is any doubt at all, it may be worth separating these changes so that they get separate scrutiny in the test and documentation pipelines. This usually means creating a new issue in your issue tracker. If they’re small enough, and if your tools allow it, you might still be able to include the change (in its own commit) in your same code review.</p><h2 id="Break-up-large-changes-into-multiple-reviews"><a href="#Break-up-large-changes-into-multiple-reviews" class="headerlink" title="Break up large changes into multiple reviews"></a>Break up large changes into multiple reviews</h2><p>Sometimes a single change is big enough that you may want to break it into smaller pieces. For most cases, multiple commits may be enough, but for the largest changes you may want multiple reviews. This can be especially kind for a reviewer, or helpful if a change spans multiple areas of expertise.</p><p>One common and sensible rule is that every review should be able to build and pass tests before it is merged. That needs to be the first consideration when breaking a change down into smaller reviews. It can be temping to break up reviews by folder, or application layer, or some other simple delineation, but it will make it harder for reviewers to understand what is happening.</p><h2 id="Avoid-high-noise-low-value-changes"><a href="#Avoid-high-noise-low-value-changes" class="headerlink" title="Avoid high-noise, low-value changes"></a>Avoid high-noise, low-value changes</h2><p>Reordering code, rearranging imports, or running automatic reformatting functions can create a lot of changed lines in a review tool. Worse, it can sometimes be mistaken for removal and re-addition of code, making it harder to notice any changes made at the same time. This also makes it harder to track the true history for your lines of code. Sometimes there is enough value to justify the noise, but make the decision consciously.</p><h2 id="Establish-shared-hygiene-to-reduce-noise"><a href="#Establish-shared-hygiene-to-reduce-noise" class="headerlink" title="Establish shared hygiene to reduce noise"></a>Establish shared hygiene to reduce noise</h2><p>Using an explicit <code>.editorconfig</code> is an easy starting point. Automatic reformatting tools can save a lot of time. The noise should be minimal if everyone follows the same rules.</p><p>There are some other practices that can help reduce noise:</p><ul><li>use trailing commas in lists (if your language supports it) - this prevents the last line from changing when a new line is added</li><li>keep large lists in alphabetical order - this makes it easier to find code, and prevents the temptation to regroup and reorder things</li><li>keep code files small (ex: one class per file)</li><li>keep auto-generated code in separate files from custom code (ex: partial classes)</li></ul><h2 id="Rebase-with-care"><a href="#Rebase-with-care" class="headerlink" title="Rebase with care"></a>Rebase with care</h2><p>Avoid rebasing when a review is underway. Sometimes a rebase is necessary, but it can disrupt the change history that reviewers are in the middle of processing. GitHub can sometimes be pretty bad with this. It’s okay to rebase before anyone has looked at your review. If you have no choice, see if you can delay the rebasing until all feedback has been addressed and accepted.</p><h2 id="Review-your-work-before-asking-others"><a href="#Review-your-work-before-asking-others" class="headerlink" title="Review your work before asking others"></a>Review your work before asking others</h2><p>Quickly scan your changes before sharing them with others. I have caught innumerable typos or accidental changes this way. You could leave them in the review, but it wastes time for your reviewers, and it makes you look sloppy. It can also add an unnecessary iteration to the review process if there was no other feedback.</p><h2 id="Understand-your-review-tool"><a href="#Understand-your-review-tool" class="headerlink" title="Understand your review tool"></a>Understand your review tool</h2><p>There can be subtle differences in the reviewing experience depending on the tools and process your team is using. Take time to understand how your tool works, and make sure you follow the processes your peers prefer.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I believe code reviews are a high-value activity (which &lt;a href=&quot;/2024/01/Optimal-Code-Reviews/&quot;&gt;I’ve written about before&lt;/a&gt;), but they</summary>
      
    
    
    
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
    <category term="quality" scheme="https://jessemcdowell.ca/tags/quality/"/>
    
    <category term="code reviews" scheme="https://jessemcdowell.ca/tags/code-reviews/"/>
    
  </entry>
  
  <entry>
    <title>The True Cost of Dependencies</title>
    <link href="https://jessemcdowell.ca/2024/03/The-True-Cost-of-Dependencies/"/>
    <id>https://jessemcdowell.ca/2024/03/The-True-Cost-of-Dependencies/</id>
    <published>2024-03-27T22:35:10.000Z</published>
    <updated>2024-08-07T05:04:17.815Z</updated>
    
    <content type="html"><![CDATA[<p>I used to use Getform for a contact form on <a href="https://pragmaticpotato.com/">my consulting company website</a>. I recently received an email from them announcing that their free tier was dropping from 50 submissions per month to a lifetime limit of 25. This makes it useless for anything more than a trial, and their lowest tier is a more expensive than other similar options.</p><p>I’m not here to complain about companies taking back free offerings. I don’t like the change, and I wish they’d given me more than three days of notice, but they are a businesses, and businesses need to make money. It is a good reminder though: even if something is free to use, it still takes time and effort to integrate, to maintain, and you may occasionally need to throw it out and find a replacement.</p><p>Dependencies are inescapable in this business. We write programs using frameworks, written and compiled in computer languages, executing in operating systems that run on chips and boards designed and manufactured by others. It would be insane to build bespoke implementations of these for most purposes. Fortunately, these lowest level components are designed to minimize breaking changes.</p><p>Most software relies on a lot more dependencies, and that is often a sensible choice. There are costs to building things yourself even when they are simple: design, testing, and upkeep. Upgrading components to get security updates is a pain, but making your own vulnerabilities that never get detected or fixed has drawbacks too.</p><p>So if there are drawbacks to depending on third parties and drawbacks to building everything yourself, what should you do? It depends! Either way: Before adding a dependency, consider its maturity, stability, and how actively its supported. If the choice is hard to change (ex: a logging framework that will be referenced throughout your code), check again, and harder! Before building it yourself, consider the true cost, including testing and maintenance.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I used to use Getform for a contact form on &lt;a href=&quot;https://pragmaticpotato.com/&quot;&gt;my consulting company website&lt;/a&gt;. I recently received</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="devops" scheme="https://jessemcdowell.ca/tags/devops/"/>
    
    <category term="quality" scheme="https://jessemcdowell.ca/tags/quality/"/>
    
    <category term="architecture" scheme="https://jessemcdowell.ca/tags/architecture/"/>
    
  </entry>
  
  <entry>
    <title>Being Honest and Positive at Work</title>
    <link href="https://jessemcdowell.ca/2024/02/Being-Honest-and-Positive-at-Work/"/>
    <id>https://jessemcdowell.ca/2024/02/Being-Honest-and-Positive-at-Work/</id>
    <published>2024-02-28T19:37:11.000Z</published>
    <updated>2024-08-07T05:04:17.815Z</updated>
    
    <content type="html"><![CDATA[<p>Two things that are important to me are being honest and treating everyone with dignity. However, some situations make it difficult to do both at the same time. It is possible, though. It takes effort, creativity, and a bit of trust, but it gets easier with practice.</p><p>When I talk about being honest, I actually mean being candid. Candor means both being honest and also communicating in good faith. Many fantasy authors have written about characters who never lie but are also fundamentally dishonest. Candor eliminates this loophole. I push myself to communicate everything that feels important, even if it’s uncomfortable or to my own disadvantage.</p><p>Why would I do this? Honestly, I don’t know. I learned about candor from a business book, and the idea of it resonated so strongly with me that I decided to make it part of what I am. It has been a difficult road, but it has also been good for me. Saying the difficult can sting in the moment, but it builds trust over time. Trust is very important when working in a team. It’s helped bring attention to problems. It’s also helped me to strengthen the important friendships in my life.</p><blockquote><p>Being honest may not get you a lot of friends but it’ll always get you the right ones.<br>– John Lennon</p></blockquote><p>Being honest is hardest when emotions are involved. Sometimes I am worried about hurting feelings, or sometimes I am frustrated or upset with the way things have gone. In these moments it is easier, and in our culture sometimes even expected, that we say something polite and positive even if we feel otherwise. Even if it’s well-intentioned, this approach diminishes trust and allows problems to fester. Even worse, these hard feelings can amplify the longer we keep them unvoiced.</p><p>Blurting out our first thoughts is not a winning strategy either. Care needs to be taken to make sure what we say is honest, but also polite.</p><p>The way I do this is to re-frame my opinion so that I see the situation in a constructive way. I’m not pretending that I feel that way, I convince myself to feel that way. This is essential to being candid and professional at the same time. If this sounds hard, you’re right, it is. But I believe it is absolutely worth the effort.</p><p>Here are some questions I can ask myself to find another opinion:</p><ul><li>Am I misunderstanding the intentions of the other person?</li><li>Is the other person pushing beyond their comfort zone?</li><li>Is the failure a result of a system that allows mistakes to go undiscovered?</li><li>Is it possible the other person has personal circumstances that are influencing their capacity?</li><li>Are my feelings a result of my own personal baggage?</li></ul><p>With practice, this gets faster and easier, but sometimes it’s harder. Stepping away for a bit can help. Talking about it with someone else can help. Or sometimes I need to consider the source of the feelings and what that says about me. There are also ways to express being upset or disappointed sincerely that are less offensive.</p><p>Sometimes feedback is best given privately. I had a boss who often said, “Give praise in public and criticism in private.” I think this is good advice for the most part. People can get defensive and upset if they feel like they’re being shamed. That will not improve the working environment. If you are giving feedback, make it as constructive as you can. Focus on actions that happened and their consequences, giving concrete examples whenever you can. Generalizations and judgements about the other person should be avoided. A lot has been written on this subject by people who are much better at it than me. If you are in a management position, I highly recommend taking the time to study and practice these skills.</p><p>There are times when feedback goes sideways and tempers can come out. This is a good time to take a break; 30 minutes for the nervous system to calm down is great. Make sure to talk in private, and talk in person if you can. It isn’t fun when things get to this point, but it can happen when people care about what they’re working on. It isn’t fun, but it is a good sign.</p><p>If things go really wrong, make the effort to apologize once the dust has settled, and try to move through it. Repair also builds trust and improves relationships, so even if you get it wrong sometimes, don’t lose hope.</p><p>There are unfortunate times when being honest isn’t possible. I try very hard not to keep things secret just because it’s easier, but sometimes there is no choice.  One example that comes to mind is hiring and firing decisions. These situations can be hard, but they are unavoidable when you get to a certain level of influence in an organization.</p><p>Operating this way at work and in life has been difficult, but it has also been very freeing. I don’t have to carry the baggage of the truths I haven’t shared, and I attract the kind of people who want honest feedback. It has helped surface more problems and make my life and my work better. I have no intention of changing.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Two things that are important to me are being honest and treating everyone with dignity. However, some situations make it difficult to do</summary>
      
    
    
    
    
    <category term="leadership" scheme="https://jessemcdowell.ca/tags/leadership/"/>
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
  </entry>
  
  <entry>
    <title>My Experience with the Dvorak Keyboard Layout</title>
    <link href="https://jessemcdowell.ca/2024/01/My-Experience-with-the-Dvorak-Keyboard-Layout/"/>
    <id>https://jessemcdowell.ca/2024/01/My-Experience-with-the-Dvorak-Keyboard-Layout/</id>
    <published>2024-01-31T17:57:13.000Z</published>
    <updated>2024-08-07T05:04:17.814Z</updated>
    
    <content type="html"><![CDATA[<p>I’ve been happily using the Dvorak keyboard layout for more than 25 years. I switched because I thought typing faster would be a benefit in my career. I can type faster, and it also reduces the strain on my wrists. It does have some drawbacks though. If you had asked me before writing this post I would have told you that I love it, but now I’m not so sure.</p><h2 id="How-I-switched"><a href="#How-I-switched" class="headerlink" title="How I switched"></a>How I switched</h2><p>It seems pretty reckless in retrospect, but I was young then, and it seemed like a good idea. So one day I decided to go for it. I changed the keyboard setting and dropped suddenly from about 60 words per minute to less than 1.</p><p>If that wasn’t jarring enough, I decided to also break my hunt-and-peck typing habit at the same time. I had an older mechanical keyboard with a uniform key size, so I was able to re-arrange the key covers. I moved all of them into random positions except F and J because of their little bumps that are essential for touch typing. This way I had no choice but to memorize the new positions. It was disruptive and frustrating, but it was also very effective.</p><p>I was up to a workable speed after a couple of days. I could match my QWERTY speed after a couple of weeks.</p><p>Once I was comfortably using the new layout, I moved all the key covers to their positions in the Dvorak layout, including F and J. A dot of epoxy on the new starting keys (U and J) allowed me to keep touch typing.</p><h2 id="Faster-typing-reduced-wrist-strain"><a href="#Faster-typing-reduced-wrist-strain" class="headerlink" title="Faster typing &amp; reduced wrist strain"></a>Faster typing &amp; reduced wrist strain</h2><p>I can type faster with the Dvorak layout, and considerably faster than most of my peers. I don’t measure often, but I have recorded myself at 160 words per minute. I can only approach these speeds when I’ve been writing a lot of text for several days in a row, which hasn’t happened very often. It turns out that typing fast doesn’t actually help me much at all.</p><p>Almost all of my typing as a programmer is in short spurts. An email here, a chat message there. Writing source code is not much faster on Dvorak because of all the symbol characters used. I am not a slow programmer, but typing speed only becomes a limiting factor when I’m writing the kind of low-value boilerplate code I should be avoiding in the first place.</p><p>The most helpful benefit I got from switching was learning to touch type, but that doesn’t require a switch to the Dvorak layout.</p><p>The biggest benefit I get from the layout itself is reduced wrist strain. The more efficient layout means my fingers don’t need to move as much. It’s hard to quantify the improvement, but even as an invincible teenager, I noticed it immediately.</p><h2 id="Availability-of-Dvorak-layouts"><a href="#Availability-of-Dvorak-layouts" class="headerlink" title="Availability of Dvorak layouts"></a>Availability of Dvorak layouts</h2><p>When it comes to using a computer, the Dvorak layout has been implemented pretty much everywhere. Even in the early 90s, it was readily available on Windows, Linux, and Apple systems.</p><p>There are also keyboards that can do the mapping at the physical level. The one I tried had a switch that alternated between QWERTY and Dvorak. The computer it was attached to would have no idea when you were using Dvorak. Although these eliminate some mapping problems, I don’t find them that helpful. I also don’t want to carry a special keyboard around wherever I want to type.</p><p>I have tried printing the Dvorak layout onto my keyboards, but this has also been impractical. I have yet to find an ink or paint that can withstand more than a month of typing. I don’t look at the keys when typing, so even if I did, it wouldn’t help me. It might help a stranger using my computer, but it would still be excruciatingly difficult for them. If someone else needs to use my computer the first thing they should do is change the layout back to QWERTY.</p><p>I don’t bother with any of this nowadays. I have a normal QWERTY keyboard on my laptop, using just the Dvorak setting in the OS.</p><h2 id="QWERTY-is-inescapable"><a href="#QWERTY-is-inescapable" class="headerlink" title="QWERTY is inescapable"></a>QWERTY is inescapable</h2><p>There are keyboards everywhere nowadays, physical and virtual, and almost all are in the QWERTY layout. I have one on my phone, one on my TV’s menu, one on my gaming console, and even one on my printer. Most of these devices have no way to change the layout.</p><p>I occasionally want to use a computer that isn’t my own, such as when I’m pair programming, or helping my dad with his computer. I could change the layout in these cases, but it’s often too much of a nuisance.</p><p>For these reasons, I had to re-learn how to type on QWERTY shortly after switching to Dvorak. I’m unsure how or why, but I developed an ability to use both depending on where I’m looking - QWERTY if I’m looking at the keyboard or Dvorak if I’m looking away. It’s as if I have two different sets of muscle memory for typing and my eyes control which is engaged. This has been a benefit, but it causes some weird side effects too.</p><p>The most bizarre is that I find it nearly impossible to use shortcut keys one-handed because I have trouble finding the Drovak letter if I need to look at the keys. Apple computers have a layout called “Dvorak - QWERTY ⌘” that switches the layout back to QWERTY for command sequences. It solves the problem, but it isn’t available in Windows.</p><p>I use the QWERTY layout on touch screens like my phone for the same reason, even though the Dvorak layout is often available. It was even worse back when I was using a Microsoft Surface; because I sometimes had a physical keyboard, I had to change the layout back and forth whenever I needed the touch keyboard.</p><h2 id="Cut-Copy-and-Paste"><a href="#Cut-Copy-and-Paste" class="headerlink" title="Cut, Copy, and Paste"></a>Cut, Copy, and Paste</h2><p>One of the most infuriating challenges with the Dvorak layout has to be the shortcut keys for Cut, Copy, and Paste. X, C and V are excellent keys for QWERTY users because you can press them easily using only your left hand. This is especially great if you use your mouse with your right hand. On the Dvorak layout, C is where the QWERTY I is, and V is where the QWERTY period key is. This is a significant disadvantage.</p><p>I can switch back to QWERTY temporarily if I have a lot of pasting to do, but I’ll invariably need to modify a couple of words here or change the punctuation there. Switching back and forth constantly doesn’t end up being easier.</p><h2 id="Login-screens-and-shared-sessions"><a href="#Login-screens-and-shared-sessions" class="headerlink" title="Login screens and shared sessions"></a>Login screens and shared sessions</h2><p>Login screens on shared computers can be a real pain. It was an advantage at first - it was a fantastic way to stop my sister from using my computer when we lived with our parents - but it is much less useful in the workplace.</p><p>The behaviour is different in every operating system and often inconsistent. For example, if you lock your session in Windows, the unlock screen will use the keyboard layout of the session you locked. If you reboot your computer instead, the login screen will use the system default.</p><p>Most modern operating systems don’t require typing a username when logging in. This means the only characters you type are masked password characters, so you often can’t tell when the layout is wrong. Newer operating systems show the layout on the login screen, but the indicator is too subtle, even when I know to look for it. I often don’t check the layout until after one or two failed attempts. In a strict workplace, it doesn’t take very many before an account gets locked.</p><p>Another variation of this crops up when connecting to remote computers or virtual machines. Every technology can be different, but most transmit the hardware key codes rather than the ASCII character values. This means you must change the OS-level layout setting on the system you’re connecting to.</p><p>Windows Remote Desktop (RDP) sets the layout for you when you start a session. It takes the value from the session where you start the RDP client, which is usually helpful. However, it doesn’t change automatically when you connect to an existing session, nor does it switch back to the system default when you leave a session idle. If I’m sharing an account with someone (for example, a test server on a local network; sharing accounts in production is bad!), I could get either keyboard layout when I connect. I am used to dealing with this, but it has wasted a lot of time for my colleagues when they’ve intercepted my old sessions.</p><h2 id="Video-games"><a href="#Video-games" class="headerlink" title="Video games"></a>Video games</h2><p>Some video games suffer from a similar problem. Games from the olden days tended to work on hardware key codes, but newer web-based games usually use ASCII codes. A and D for left and right is universal, but it sucks on a Dvorak layout. The A is in the same place, but the D is over in the same position as the QWERTY H. Don’t even get me started on W and S. The best way to deal with this is to change the keyboard mappings in the game, but I usually switch the keyboard layout back to QWERTY because it’s easier.</p><p>Video games with in-game text chat are the most difficult. I need to be in QWERTY mode to play, then switch back to Dvorak mode to write a message. To make matters worse, the newer language&#x2F;layout switcher (Windows + Space) thrusts you out of full-screen games because it has a dialog window. Some games handle both gracefully (using ASCII codes for text and key codes for gameplay), but it’s not universal.</p><h2 id="Japanese-IME-and-the-Dvorak-layout"><a href="#Japanese-IME-and-the-Dvorak-layout" class="headerlink" title="Japanese IME and the Dvorak layout"></a>Japanese IME and the Dvorak layout</h2><p>Another unusual drawback appeared when I started learning Japanese. Japanese mixes 4 alphabets: the Latin alphabet, two phonetic alphabets with 46 characters each, and about 100,000 characters imported from traditional Chinese. A keyboard with every possible character would be obviously impractical, so text is written phonetically and converted as you go. This is done with an IME (Input Method Editor), an operating system feature that intercepts keystrokes and translates them into the appropriate characters.</p><p>Because the IME is a kind of keyboard itself, it replaces the keyboard mapping. This means there is no built-in way to use the Dvorak layout with Japanese writing. It used to be possible to change a registry key to replace the keyboard layout, but this option seems to no longer work with the updated Japanese IME that ships with Windows 11. There are instructions available to use the old IME, but who knows how long that’ll be available.</p><p>Typing Japanese via the QWERTY layout is painful for me. The IME is constantly popping up suggestions, and since I can’t touch type on the QWERTY layout, I have to keep looking up and down to use it properly. This is no longer much of a problem for me since I’ve stopped practicing Japanese, but it would be if I ever picked it back up. This is one of the few reasons I would consider a physical Dvorak keyboard.</p><h2 id="Final-thoughts"><a href="#Final-thoughts" class="headerlink" title="Final thoughts"></a>Final thoughts</h2><p>I am surprised by what I’ve written. I had imagined a small blurb about how I switched and then re-sharing the Japanese IME registry hack (which I’ve since discovered no longer works). I did not realize I had so much to say about my experiences! Would I recommend the Dvorak layout to others? No, I would not, and it surprises me that I say that. I can’t even think of a situation where the benefits would be important enough to justify the disadvantages.</p><p>Would I recommend Dvorak users switch back to QWERTY? I feel a sadness I don’t understand when I consider the answer for myself, but I think the answer is yes.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I’ve been happily using the Dvorak keyboard layout for more than 25 years. I switched because I thought typing faster would be a benefit </summary>
      
    
    
    
    
    <category term="gadgets" scheme="https://jessemcdowell.ca/tags/gadgets/"/>
    
    <category term="reviews" scheme="https://jessemcdowell.ca/tags/reviews/"/>
    
  </entry>
  
  <entry>
    <title>Optimal Code Reviews</title>
    <link href="https://jessemcdowell.ca/2024/01/Optimal-Code-Reviews/"/>
    <id>https://jessemcdowell.ca/2024/01/Optimal-Code-Reviews/</id>
    <published>2024-01-04T22:11:32.000Z</published>
    <updated>2024-01-31T08:00:00.000Z</updated>
    
    <content type="html"><![CDATA[<p>I am a strong advocate for code reviews. They have many advantages including improving code quality and team communication. On the other hand, they take a lot of time and add yet another delay to the development pipeline. With the wrong team culture, they can create hard feelings and discourage honest collaboration. This is a heavy price to pay, and yet, I’ve seen many teams that do them without ever talking about how or why.</p><p>With a bit of intentionality and the right support systems, code reviews can become a catalyst for positive change that are absolutely worth the time we put into them.</p><h2 id="Why-we-do-code-reviews"><a href="#Why-we-do-code-reviews" class="headerlink" title="Why we do code reviews"></a>Why we do code reviews</h2><p>There are a lot of reasons for code reviews, but the one I find the most valuable is that they promote shared ownership of the code. Sharing code forces teams to get aligned on architecture and coding practices. This gives you more flexibility to distribute work, reducing bottlenecks and guarding business continuity. The developers who work on your critical systems will appreciate the ability to take vacations as well.</p><p>Another advantage is that reviews improve code quality simply by being part of the development process. Developers can be lazy people (ask me how I know), but we get a little extra incentive to tidy things up when we know someone else will be looking. Since we spend much more time reading our code than writing it, this will improve efficiency in the long run.</p><p>Reviews can also be a more efficient way to onboard developers, especially for the kinds of knowledge that are hard to articulate. For a small team with little planned growth or churn, writing a huge body of documentation may not be a great use of time. Of course, learning through review feedback doesn’t always feel very nice when you’re on the receiving end, but this is another reason why it’s valuable. Making it normal to receive constructive feedback without shame is essential if you want a team where members push themselves and grow.</p><h2 id="First-automate-everything-you-effectively-can"><a href="#First-automate-everything-you-effectively-can" class="headerlink" title="First, automate everything you (effectively) can"></a>First, automate everything you (effectively) can</h2><p>One simple way to reduce the time you spend on code reviews is to automate as much as you can. Some things require a human touch, but some other things are much easier for computers.</p><p>Compilers are typically very easy to automate, and it can take humans lots of time to detect compilation errors. If you automate only the compilation of any proposed changes, you can prevent a lot of build breaks and wasted time. This is helpful even with a team of one.</p><p>You should be running your unit tests automatically as well. Not only will this make sure any new tests pass, but it’ll prevent failing tests from getting merged back to the main codebase, which can in turn disrupt other developers.</p><p>Longer-running automation tests or e2e tests may be too slow or expensive to run on every proposed change, but you should consider the tradeoffs. Even the most well-intentioned team can find it irresistible to ignore a failing test when a tight deadline is looming. Once you have a couple of tests that fail regularly, it doesn’t take long before confidence disappears and decay sets in.</p><p>There are lots of tools for checking basic code formatting and whitespace. These are also good to automate and should be set up early in any project. I have seen so much time wasted in reviews talking about white space and proper nesting. Let the machines give feedback in minutes instead of wasting developer time on it. Make sure you have an <code>.editorconfig</code> that matches the rules you’re checking so that your IDEs get most code correct before the review starts.</p><h2 id="What-to-look-for-when-reviewing-code"><a href="#What-to-look-for-when-reviewing-code" class="headerlink" title="What to look for when reviewing code"></a>What to look for when reviewing code</h2><p>I try to focus mostly on readability and maintainability. Do I understand the intention of the change? If I needed to change this code in 6 months would I be able to find it and figure out what it’s doing? Do the names being used describe what they are?</p><p>I don’t check if the change meets its requirements unless I happen to notice it. Code is not a great level to test functionality at, and I expect the author to test this themselves, and a further manual test to be done once the change is in a build. I do check that any commit messages adequately describe what the change is.</p><p>I also like to check that a good number of tests have been written, and any complicated or risky code has sufficient coverage. If I’m honest though, I don’t typically check the contents of test methods. I find most tests difficult to read, and I expect the developer to have checked that they work already.</p><h2 id="Self-editing-and-maintaining-a-positive-culture"><a href="#Self-editing-and-maintaining-a-positive-culture" class="headerlink" title="Self-editing and maintaining a positive culture"></a>Self-editing and maintaining a positive culture</h2><p>Before I post a code review comment I like to ask myself a few questions:</p><ul><li>Is the change important? Have I communicated that?</li><li>Is it clear what needs to be changed?</li><li>Is it reasonable to expect the author to make the change?</li></ul><p>At this point in my career, I find that a lot of my code review comments are more instinctual than formed logically. I don’t think it’s a bad thing to operate this way, but instincts can drift or misfire sometimes. Even if something feels wrong it might not be a problem, or it might not be important. I use my checklist to validate my instincts. It’s not uncommon for me to add a bunch of comments while doing a review, then go back and delete some of them before hitting the send button.</p><p>Sometimes I have concerns about the underlying approach or the broader scope of work the change is a part of. In these cases, I check myself carefully to make sure my feedback is important. If I don’t want to be a part of every decision made by the team, I have to trust the team to make them without me. Big changes are also more expensive, and the code review step is pretty late in the process. Sometimes the stakes are high enough though.</p><p>As with all business communication, it is important to be mindful of tone in code reviews. I always try to assume positive intent and try to reflect that in my comments. If I have a lot of feedback or serious feedback, I might speak with the author directly. A quick face-to-face or video chat makes it easier to diffuse hard feelings.</p><p>Another thing I avoid is the “I would have done it like…” pattern. At best it’s a poor justification for a change, at worst it implies superiority. It is much better to use “I suggest doing X because Y”.</p><h2 id="Suggesting-alternatives"><a href="#Suggesting-alternatives" class="headerlink" title="Suggesting alternatives"></a>Suggesting alternatives</h2><p>The easiest way to ensure a comment is reasonable is to provide an alternative myself. For example, I won’t leave a comment about something’s name unless I can offer a better name, and give a good reason why the old name doesn’t make sense.</p><p>I also like to use the diff feature in GitHub comments to show the actual change if it’s reasonable to do so. This often doesn’t work for naming (because there will be references that also have to be changed), but it’s great for little stuff like typos in user-facing strings. It is also sometimes the easiest way to communicate an improved algorithm or propose new function signatures.</p><h2 id="Number-of-reviewers"><a href="#Number-of-reviewers" class="headerlink" title="Number of reviewers"></a>Number of reviewers</h2><p>Sometimes it’s important to get lots of eyes on a critical change. Sometimes it’s not important but people do it anyway. Regardless of the reason, it can be especially painful to get code approved when multiple reviewers have to approve. In these circumstances, and especially when I’m late to the review, I temper my feedback further than usual so that the code review won’t go around in circles forever.</p><p>The worst is when two reviewers disagree on the best approach and argue about it in the code review. Whenever this happens I ask the two reviewers to talk to each other directly to resolve the issue. It can be beneficial for the author to join in as well.</p><p>I generally recommend just one reviewer for most code reviews as this gets the most benefit with the least cost. I also suggest making sure the whole team is doing reviews. You could assign them all to the most senior member of the team, but doing reviews is a valuable teaching experience and not all code requires an expert opinion. If a code review does require an expert opinion, I prefer to ask those experts to focus on specific areas but let someone else handle the broader review. Having new team members join code reviews is also an easy way to expose them to team practices and any work in progress.</p><h2 id="Don’t-say-anything-if-you-don’t-need-to"><a href="#Don’t-say-anything-if-you-don’t-need-to" class="headerlink" title="Don’t say anything if you don’t need to"></a>Don’t say anything if you don’t need to</h2><p>Sometimes the code is good enough, especially if the change is small and clear. If it’s good enough, resist the urge to nitpick and mark it as reviewed.</p><h2 id="Accepting-feedback-on-my-feedback"><a href="#Accepting-feedback-on-my-feedback" class="headerlink" title="Accepting feedback on my feedback"></a>Accepting feedback on my feedback</h2><p>When someone puts comments on the code I’ve put up for review, I take those comments seriously. Sometimes I don’t agree, but I put in extra effort to make sure I maintain an appreciative and curious tone. Even bad feedback is often grounded in good reasons. It might be a hint that another other part of the change isn’t clear. If I believe the comment is an error, I respectfully point it out and provide references if I can.</p><p>When I’m reviewing, I often make suggestions without testing them. I also go from memory, and sometimes I make mistakes. When someone challenges a comment of mine, I take it just as seriously. This isn’t a sign of disrespect, it’s just the nature of working on hard problems in a team.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I am a strong advocate for code reviews. They have many advantages including improving code quality and team communication. On the other </summary>
      
    
    
    
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
    <category term="quality" scheme="https://jessemcdowell.ca/tags/quality/"/>
    
    <category term="code reviews" scheme="https://jessemcdowell.ca/tags/code-reviews/"/>
    
  </entry>
  
  <entry>
    <title>Switching from Google Podcasts to Spotify</title>
    <link href="https://jessemcdowell.ca/2023/12/Switching-from-Google-Podcasts-to-Spotify/"/>
    <id>https://jessemcdowell.ca/2023/12/Switching-from-Google-Podcasts-to-Spotify/</id>
    <published>2023-12-14T18:17:07.000Z</published>
    <updated>2024-08-07T05:04:17.813Z</updated>
    
    <content type="html"><![CDATA[<p>Back in September <a href="https://blog.youtube/news-and-events/podcast-destination-on-youtube-music/">Google announced that Google Podcasts will be shut down</a>. It is supposed to remain available until April of 2024 and be available for exporting data until June. I chose not to wait.</p><p>I have been using Spotify happily for music for years now, so I decided to give it a try. I didn’t do any research or look for other options. I had some confidence that it would be a reasonable option however because Spotify has been putting a lot of effort and money into its podcast offering.</p><h2 id="Is-Spotify-a-good-replacement-for-Google-Podcasts"><a href="#Is-Spotify-a-good-replacement-for-Google-Podcasts" class="headerlink" title="Is Spotify a good replacement for Google Podcasts"></a>Is Spotify a good replacement for Google Podcasts</h2><p>Short version:</p><p>If you want something that will work on a wide variety of devices, has a huge library of content, and is likely to be around for a while, then Spotify is a good choice. If you want a simple replacement that behaves the same as Google Podcasts, you may have better results elsewhere.</p><p>Long version:</p><p>In terms of the availability of the app on multiple devices, availability of podcasts, and sound quality, Spotify performs excellently. They are a huge company with lots of content, and everything works great.</p><p>I can listen to podcasts on my phone, on my computer, in my car (via Android Auto), or on my kitchen speaker (via my Google Nest Mini). I can even listen to part of an episode on any of these devices and finish it on another device. It has remembered my position every time, something which was often not the case with Google Podcasts.</p><p>The biggest disadvantage has been the disruption of my listening workflows.</p><p>When I wanted to listen to a podcast on Google Podcasts, I would open the app, pick an item out of the Queue, and play it. When it finished, it would usually (depending on how I started it) start on the podcasts at the top of my Queue. If I wanted to listen to a few in a row without touching my phone (for example, while working in the garden), I could pick the next couple podcasts I wanted to hear and put them at the top of the list. Spotify’s “Your Episodes” playlist does not allow custom ordering, but I have found another way to do it.</p><p>The other annoying tidbit is that using the Spotify app or a Podcast app isn’t enough to specify what I want to hear. If I was in the middle of a podcast when I get into my car, I’ll be listening to a podcast there too. It’s not a big deal on the phone, but it can be pretty annoying to change this through the Android Auto interface.</p><h2 id="How-to-migrate-your-data-to-Spotify"><a href="#How-to-migrate-your-data-to-Spotify" class="headerlink" title="How to migrate your data to Spotify"></a>How to migrate your data to Spotify</h2><p>It took me less than 30 minutes to migrate my subscriptions and queue manually. There is supposed to be an export added to Google Podcasts, but I assume it’s not yet available because I couldn’t find it. I also never found an import in Spotify, so maybe it wouldn’t matter.</p><p>To do it manually, just search for and click “Follow” on any podcasts you had subscribed to. Next, search for and click the plus button on any episodes that were in your queue. I suggest using a computer for this, or at least two different devices. If you didn’t already know: <a href="https://podcasts.google.com/">Google Podcasts has a web version</a> that gives you access to everything you’ll need.</p><h2 id="My-Spotify-Podcast-workflow"><a href="#My-Spotify-Podcast-workflow" class="headerlink" title="My Spotify Podcast workflow"></a>My Spotify Podcast workflow</h2><p>When I want to play an episode, I use “Add to Queue” instead of the play button. I can then use the next track button to switch from my current song to the podcast. This makes it easy to queue up a couple of specific podcasts, and makes the player switch automatically back to music when they are all finished. If I want to switch immediately back to music, the next track button will save my place and move on.</p><p>I don’t like to play podcasts directly from the “Your Episodes” playlist because you can’t control the order and it will jump automatically to whatever comes next. Playing from within each podcast’s page is okay if you want it to go from one episode to the next.</p><p>I’ve pinned the “Your Episodes” playlist, which makes it easier to start an episode. In the mobile version, this playlist has a settings menu. Setting “Remove played episodes” to “After playing” will make this list behave the most like Google Podcasts’ queue.</p><p>I occasionally look at the “New Episodes” playlist and save any episodes that interest me. This playlist only appears in the mobile version, but it is extremely helpful because it shows the latest episodes from all the podcasts I follow. I don’t pin this playlist however, since Spotify is so stingy with its pins.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Back in September &lt;a href=&quot;https://blog.youtube/news-and-events/podcast-destination-on-youtube-music/&quot;&gt;Google announced that Google Podca</summary>
      
    
    
    
    
    <category term="reviews" scheme="https://jessemcdowell.ca/tags/reviews/"/>
    
    <category term="howto" scheme="https://jessemcdowell.ca/tags/howto/"/>
    
  </entry>
  
  <entry>
    <title>Announcing Pragmatic Potato Software</title>
    <link href="https://jessemcdowell.ca/2023/11/Announcing-Pragmatic-Potato-Software/"/>
    <id>https://jessemcdowell.ca/2023/11/Announcing-Pragmatic-Potato-Software/</id>
    <published>2023-11-30T22:07:52.000Z</published>
    <updated>2024-08-07T05:04:17.812Z</updated>
    
    <content type="html"><![CDATA[<p>I am very pleased to announce the launch of my new company, <a href="https://pragmaticpotato.com/">Pragmatic Potato Software</a>.</p><p>Pragmatic Potato is helping customers with software architecture and development. We can help with architectural investigations, software design, development process, interviewing, and even development. You can find out more about the kind of services we’re offering on our spiffy new website: <a href="https://pragmaticpotato.com/services/">pragmaticpotato.com&#x2F;services&#x2F;</a>.</p><p>If you’re interested, <a href="https://pragmaticpotato.com/contact/">contact me through my website</a>. Let’s have a quick conversation and see if we can help you.</p><p>Some of you may know that I’m also working on a product. I am keeping these efforts in stealth mode for now, but you can <a href="https://pragmaticpotato.com/products/">sign up to be notified</a> when I’m ready to say more.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I am very pleased to announce the launch of my new company, &lt;a href=&quot;https://pragmaticpotato.com/&quot;&gt;Pragmatic Potato Software&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;</summary>
      
    
    
    
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
    <category term="consulting" scheme="https://jessemcdowell.ca/tags/consulting/"/>
    
    <category term="pragmatic potato" scheme="https://jessemcdowell.ca/tags/pragmatic-potato/"/>
    
  </entry>
  
  <entry>
    <title>Using Architectural Decision Records</title>
    <link href="https://jessemcdowell.ca/2023/11/Using-Architectural-Decision-Records/"/>
    <id>https://jessemcdowell.ca/2023/11/Using-Architectural-Decision-Records/</id>
    <published>2023-11-22T01:51:51.000Z</published>
    <updated>2024-08-07T05:04:17.811Z</updated>
    
    <content type="html"><![CDATA[<p>Architectural Decision Records are a simple technique that promotes good architectural thinking and better collaboration. They don’t need to be big or complicated to be effective, but they do take some time, and are yet another step between setting goals and delivering value. If you use them in the right circumstances they can be a big help.</p><h2 id="What-is-an-Architectural-Decision-Record"><a href="#What-is-an-Architectural-Decision-Record" class="headerlink" title="What is an Architectural Decision Record?"></a>What is an Architectural Decision Record?</h2><p>An Architectural Decision Record (often abbreviated ADR) is just as the name implies: a document that describes an architectural decision. It can include various details, but should at a minimum describe the problem being solved and the solution that was chosen. It can be a simple text document with a page or two of text, or it can be longer with specific sections that are important to the team.</p><h2 id="Why-use-Architectural-Decision-Records"><a href="#Why-use-Architectural-Decision-Records" class="headerlink" title="Why use Architectural Decision Records?"></a>Why use Architectural Decision Records?</h2><p>The most obvious benefit of architectural decision records is that they record your important technical decisions. Years later, if you’re considering replacing Framework X, you can see the original analysis that went into its selection. There may have been good reasons your new candidate wasn’t chosen. Or more likely, the decision may have been based on assumptions that are now known to be invalid.</p><p>My favourite reason for using Architectural Decision Records is that they are an easy way to encourage good architectural habits. For example, forcing a developer to write down the problem they are trying to solve makes it harder to add technologies with no purpose (yes, this happens). Asking for alternatives considered makes sure that alternatives were actually considered. Even considering if an ADR is necessary is a valuable exercise.</p><p>The process of writing and reviewing ADRs also makes technical decisions more visible, and accessible to more of the team. With these simple guardrails in place, you can hand a decision off to an intermediate developer and know that your most senior developers will still be involved. Distributing work like this helps prevent bottlenecks, but it also gives everyone a chance to grow their skills.</p><p>Having a formal decision-making process is especially helpful for big teams where decisions can impact a lot of people. Or if a team has a partially involved architect. When I was an architect with responsibilities for multiple teams, I found this process to be an excellent way to stay in the loop, and a great entry point to get involved when needed.</p><h2 id="When-should-I-use-an-Architectural-Decision-Record-process"><a href="#When-should-I-use-an-Architectural-Decision-Record-process" class="headerlink" title="When should I use an Architectural Decision Record process?"></a>When should I use an Architectural Decision Record process?</h2><p>This is really two different-but-not-so-different questions: when should a team have an ADR process, and what kind of changes should require an ADR process? The answer to both is effectively the same thing: when is a decision architecturally significant?</p><p>The way I answer this question is to ask what the impact would be if the decision is wrong. If there is a low risk of serious consequences and I can easily change to a different solution, then I make a good-enough choice and move on.</p><p>Here is an incomplete list of reasons why a decision might be architecturally significant:</p><ul><li>A lot of code would need to be rewritten if the decision gets changed. For example, when selecting a:<ul><li>programming language</li><li>base framework</li><li>automated test framework</li></ul></li><li>Mistakes could have serious impacts. For example, if the project&#x2F;change has anything to do with:<ul><li>the privacy of sensitive data</li><li>security</li><li>financial transactions</li><li>life-critical systems</li><li>civic infrastructure</li></ul></li><li>The software is used by many different teams, and a change could impact many of them. For example: something exposed in the api of an open-source library or shared module used by multiple teams</li><li>The software could be used and maintained for many years or have a lot of different people working on it</li><li>The software is difficult to change if there is a mistake. For example: firmware</li></ul><p>If any of the above are true for your project generally, then implementing a formal ADR process could be helpful.</p><p>When you have an ADR process, you should also provide some guidance about when it needs to be used. For the same reasons, it makes sense to base it on the potential impact of a wrong decision. You can use an arbitrary replacement cost for your definition (for example: “more than a week of effort to replace”), or list potential impacts like privacy or security risks.</p><p>Even without a formal ADR process, individual contributors can also benefit from using one of these templates. Filling out a good template makes sure you truly understand your problem and that you don’t skip any important steps. This is helpful even without soliciting feedback. I use this technique myself sometimes. This is also effectively what <a href="/2023/08/My-Architectural-Report-Template/">my Architectural Report template</a> is.</p><h2 id="Can-I-introduce-an-Architectural-Decision-Record-process-to-an-existing-project"><a href="#Can-I-introduce-an-Architectural-Decision-Record-process-to-an-existing-project" class="headerlink" title="Can I introduce an Architectural Decision Record process to an existing project?"></a>Can I introduce an Architectural Decision Record process to an existing project?</h2><p>Yes, but it’s harder for the team to appreciate the value of it.</p><p>You get all the benefits of standardizing the decision-making process, but it’s hard to realize the value of documentation without a significant majority of your important decisions documented. Retroactively documenting decisions isn’t very practical either: without tremendous notes or exceptional memory, the results can easily lack sufficient value.</p><p>If you have a strong reason to improve the quality of future decisions, and the team believes in it, then you have a shot at adding an ADR process to an existing project. I wouldn’t recommend it otherwise.</p><h2 id="What-Architectural-Decision-Record-template-should-I-use"><a href="#What-Architectural-Decision-Record-template-should-I-use" class="headerlink" title="What Architectural Decision Record template should I use?"></a>What Architectural Decision Record template should I use?</h2><p>You can find several decent templates with a quick internet search. Joel Parker Henderson has helpfully posted <a href="https://github.com/joelparkerhenderson/architecture-decision-record/">a list of Architecture Decision Record templates</a> and some other helpful resources in GitHub, which I’ve used in the past.</p><p>My favourite template is the <a href="https://github.com/joelparkerhenderson/architecture-decision-record/tree/main/locales/en/templates/decision-record-template-by-michael-nygard">Decision record template by Michael Nygard</a>. I appreciate its simplicity as a starting point, and it’s trivial to add other sections as you need them.</p><p>The best template will depend ultimately on the benefits you want to get from your ADR process. For example, if you’re building life-critical systems you might include a section for safety testing measures to make sure they are considered with every decision. If your organization is concerned with copy-left licensing, you can include a section for any new dependencies and their licences.</p><h2 id="Where-should-I-store-Architectural-Decision-Records"><a href="#Where-should-I-store-Architectural-Decision-Records" class="headerlink" title="Where should I store Architectural Decision Records?"></a>Where should I store Architectural Decision Records?</h2><p>The best place to store these documents is the place your team will look for them. If you have a wiki where all your technical documentation goes, that is a good choice. Some people like to put these records into source control, and that’s a good choice too. Whatever location you choose, make sure it has good search facilities and some kind of change history. A mechanism for giving feedback in the document itself is nice as well.</p><p>One advantage to source control is that you can use your familiar code review tools and processes to manage them. One drawback to source control is that it encourages simple text formats, and that makes it harder to include diagrams or other rich content.</p><p>If you do put your ADRs into source control, think about which repository they belong in. Some people like to keep them in the same repository as the code they relate to, but this won’t work if your project spans multiple repositories. If you use branches heavily, it may be confusing to see the documents in different while working. When in doubt, create a new repository for your technical documentation.</p><p>Another detail to consider is the scope of the decisions being made. If your team maintains multiple pieces of software you might have different (and conflicting) decisions for each of them. Make sure it’s obvious what your documents apply to.</p><h2 id="Maintaining-Architectural-Decision-Records"><a href="#Maintaining-Architectural-Decision-Records" class="headerlink" title="Maintaining Architectural Decision Records"></a>Maintaining Architectural Decision Records</h2><p>Decision records can get stale with time. If you do start using them, remember to also occasionally go through and clean them up.</p><p>To help with this, many teams use an append-only approach. If a decision gets replaced, a new document will be created, and the old decision will get a note inserted marking it as deprecated. Decision records can be numbered to make cross-referencing them easier.</p><h2 id="Summary"><a href="#Summary" class="headerlink" title="Summary"></a>Summary</h2><p>Architectural Decision Records can improve your architectural thinking without much effort, and they have the added benefit of leaving useful documentation behind. If you add them early in a project, they can improve the quality of the decisions your team makes.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Architectural Decision Records are a simple technique that promotes good architectural thinking and better collaboration. They don’t need</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
    <category term="architecture" scheme="https://jessemcdowell.ca/tags/architecture/"/>
    
  </entry>
  
  <entry>
    <title>This Blog: Hexo-generated static site hosted on GitHub Pages</title>
    <link href="https://jessemcdowell.ca/2023/11/This-Blog-Hexo-generated-static-site-hosted-on-GitHub-Pages/"/>
    <id>https://jessemcdowell.ca/2023/11/This-Blog-Hexo-generated-static-site-hosted-on-GitHub-Pages/</id>
    <published>2023-11-01T17:34:32.000Z</published>
    <updated>2024-08-07T05:04:17.811Z</updated>
    
    <content type="html"><![CDATA[<p>A couple of years ago I switched this blog from a WordPress site hosted on GoDaddy to a statically generated site. The new setup is faster, more secure, and cheaper to operate. And it was easy to do!</p><h2 id="Static-vs-Dynamic-websites"><a href="#Static-vs-Dynamic-websites" class="headerlink" title="Static vs Dynamic websites"></a>Static vs Dynamic websites</h2><p><a href="https://wordpress.org/">WordPress</a> is a very popular blogging platform. You can use it hosted on WordPress.com, or any number of other web providers that offer it. Because it’s open source, you can also host it yourself easily. In my case, I had a virtual server running in GoDaddy where I maintained the installation myself.</p><p><a href="https://hexo.io/">Hexo</a> is a static website generator. When you want to publish changes, you run a command that causes it to spit out a bunch of static html, css, and so on that you can then host anywhere you like. These tools can store the raw content any way they like, but Hexo and all the other generators I’ve played with use a directory structure with simple markdown files.</p><p>Dynamic websites are generated by software running on the web server. Ignoring any caches or other optimizations, every web request triggers a call to the running application, which in turn makes database requests and invokes any loaded plugins before a page gets sent back to the browser. For medium-sized websites, this isn’t noticeable because the server will be constantly warmed up and ready to respond. For very small websites like mine that go idle, a fresh web request can take more than 30 seconds to respond while everything is initialized. This is not ideal.</p><p>Static websites have fantastic performance. The server only has to return the pre-computed content, and the providers that do this at scale do it very well. Even if I only get one hit a month, that page will still load very quickly, likely even faster than a warmed-up WordPress instance. It depends on the hosting platform, but static websites can typically also handle massive traffic spikes without any extra effort, and at a much lower cost. Your small self-hosted WordPress site, by comparison, could fall over when one of your posts goes viral.</p><p>Another benefit to static websites is their security. Because it’s just plain files, there is much less attack surface. As long as your account is secure, it should be impossible for an attacker to modify the content. Even if someone did, the content and site generator can be kept separate, so the site can easily be replaced with fresh, clean content. With a WordPress site, you have the running server, the WordPress software, and the database all running in the cloud. They are prone to bugs and attacks like any other cloud infrastructure, and they need to be updated regularly to keep them secure.</p><h2 id="Comments-and-contact-pages-on-a-static-website"><a href="#Comments-and-contact-pages-on-a-static-website" class="headerlink" title="Comments and contact pages on a static website"></a>Comments and contact pages on a static website</h2><p>The biggest drawback to static websites is that they are, well, static. There are a few tricks you can use, but some features are much more difficult to implement. For a simple blog like mine, this wasn’t a problem.</p><p>Comments are a common blog feature that is trivial for a dynamic site to implement. For this blog, I recently set up <a href="https://utteranc.es/">utterances</a>. It uses GitHub Issues (in a free public repository) like a database to store comments, and retrieves &#x2F; renders them on each page with a bit of client-side javascript. This way I don’t need to run <code>hexo generate</code> every time someone posts a comment.</p><p>A reader does need a GitHub account to add a comment with this system, but this seems like a reasonable limitation given my audience. There are other static-capable comment systems available, each with different styles and storage mechanisms. There are commercial offerings too, which may or may not be more expensive than using a hosted WordPress site for your blog to begin with.</p><p>I don’t have a contact form on this site, but I easily set one up on another static website using <a href="https://getform.io/">getform</a>. For the frequency of messages I expect there, it should also be free.</p><p>It gets unfortunately difficult to push much further than this on a static website. Some things can be done with client-side javascript or advanced application firewalls, but these can get complicated and&#x2F;or expensive pretty quickly. It may be worth it if you want the scalability advantages of a static site, but if you just want a cheap website with a couple of dynamic features, a hosted WordPress site may be your best bet.</p><h2 id="HTTPS-and-custom-domains"><a href="#HTTPS-and-custom-domains" class="headerlink" title="HTTPS and custom domains"></a>HTTPS and custom domains</h2><p>The main reason I left WordPress on GoDaddy was the desire to move my blog to HTTPS. Nowadays it’s not that hard to get a free certificate from <a href="https://letsencrypt.org/">Let’s Encrypt</a>, but the certificates don’t last very long, so it’s best to automate the process. At the time I changed,  automating this wasn’t possible with the package I had from GoDaddy. I could have changed to a more expensive virtual machine option or bought a proper certificate. I went with option 3 instead.</p><p>HTTPS requires a certificate, and the certificate has to contain the domain name of the website you’re using it for. That means that you need a custom certificate to go with your custom domain name. The good news is that many hosting services can procure certificates on your behalf, and some will do it for free.</p><p>GitHub Pages is one of these services, and it was easy to set up. There were a couple of steps necessary so they could validate that I had control of the domain, but now they just take care of it for me. They can also provide redirects from HTTP to HTTPS, and from www to the apex website. GitHub Pages has <a href="https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/about-custom-domains-and-github-pages">good documentation</a> if you want to set this up for yourself.</p><h2 id="Price-of-static-websites"><a href="#Price-of-static-websites" class="headerlink" title="Price of static websites"></a>Price of static websites</h2><p>Because GitHub Pages is free (for public repositories, at the time of this writing), the only thing I’m paying for is the domain name and the DNS service, both of which I get from AWS. The <code>.ca</code> domain costs $13 USD per year, and the Route53 hosted zone costs me $0.90 USD per month (the minimum possible since I don’t exceed the initial usage limits). This is much less than I was paying GoDaddy for a virtual server.</p><p>There are a few equivalents to GitHub Pages that are also free. There are also cheaper options than AWS for domain registration and DNS.</p><p>If you don’t want the hassle of setting this all up, there are lots of hosted WordPress solutions around, and other blogging&#x2F;website platforms too. Some of these are even free, but the free ones each have their own limitations. I haven’t searched exhaustively, but I don’t know of any free options that support custom domains and control over advertising.</p><h2 id="Creating-editing-and-publishing-posts"><a href="#Creating-editing-and-publishing-posts" class="headerlink" title="Creating, editing, and publishing posts"></a>Creating, editing, and publishing posts</h2><p>It can take a bit of effort to get a static site generator set up and configured with a nice theme, but things get easier once you have it all working. You still need to run a few commands in a terminal, which will be too much for some people. Even the mildly technically challenged should be able to keep themselves going with a few basic commands written down. This is what I use:</p><p>When I want to start a new post, I use the <code>hexo new</code> command:</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">hexo new draft &#x27;This Blog: Hexo-generated static site hosted on GitHub Pages&#x27;</span><br></pre></td></tr></table></figure><p>This command generates a markdown file in the draft folder with some initial “front-matter”. I can then edit the content in any text editor I wish. These days I use <a href="https://code.visualstudio.com/">VS Code</a>. Markdown is a very simple mostly-text format for making pages that is very easy to learn. Front-matter is a bit of data about the page that go at the top of the document such as the title, publish date, tags, categories, and so on. You can see <a href="https://github.com/hexojs/site/blob/7f465a267b7bee09c236bf67606cdb72b6fbb708/source/docs/writing.md?plain=1">an example of what this looks like</a> in the source for the Hexo documentation.</p><p>If I want to see how a page will look (for example: if I’m using some tricky markdown syntax), I use <code>hexo server --drafts</code>. This hosts a website on my computer that looks exactly like the blog will when I publish it. I also sometimes use commands like <code>hexo list tag --draft</code> to make sure I’m using the same tags across multiple posts.</p><p>Once I’m happy with a post, I run the <code>hexo publish</code> command:</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">hexo publish &#x27;This Blog: Hexo-generated static site hosted on GitHub Pages&#x27;</span><br></pre></td></tr></table></figure><p>The main thing this does is move the file and its attachments from the drafts folder into the posts folder. It also renames the file and updates the front-matter with the published date and time. You could do this manually, but the command is quick and reliable. Then I need to re-generate the static content for publishing:</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">hexo generate</span><br></pre></td></tr></table></figure><p>This goes through all the content files and configuration and generates everything needed for the website. The new post gets its own folder and html file, but it will also cause changes to the front page and a number of peripheral pages. The theme I’m using supports paging, so it also pushes the post at the bottom of the front page back to page 2, the bottom post from page 2 to page 3, and so on. There are also the tag, category, and archive pages that get updated similarly. It might seem like a lot, but I assure you, the software doesn’t mind.</p><p>After this, it’s a simple matter of uploading the content to my hosting provider. In the case of GitHub Pages, this means copying it to the website repository, commiting, and then pushing the branch. Nowadays I use a PowerShell script to generate, copy, commit, and push in a single step.</p><p>I use Git and GitHub to manage my hexo folders because I am very comfortable with these tools. This isn’t necessary though. You could just as easily store your hexo files on a cloud drive. You could store it on your local hard drive too, but I highly recommend using somewhere that has automatic backups and some form of change history.</p><p>Because I’m using drafts for incomplete posts, I commit and push any changes I’ve made regularly. The drafts aren’t included in the generated site, so I can keep a few sitting around while publishing other posts or changes. I originally went a little bonkers with a branch-per-post but found it made my process unnecessarily complicated.</p><p>You can simplify publishing your site further. Some static page providers can automatically run the generate step for you. I chose not to do this for my blog because I didn’t want my drafts and their edit history to be visible in the public repository (as a public repository is required to use Pages for free). There are other options that don’t have this limitation.</p><h2 id="Converting-from-WordPress-to-Hexo"><a href="#Converting-from-WordPress-to-Hexo" class="headerlink" title="Converting from WordPress to Hexo"></a>Converting from WordPress to Hexo</h2><p>Most of the work of migrating was achieved using a WordPress export file and the Hexo migration tool:</p><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">hexo migrate wordpress wordpress-export.xml --paragraph-fix --import-image original</span><br></pre></td></tr></table></figure><p>It took a bit of fiddling after this to get everything working perfectly. I had some issues with my code snippets (I was using a gist plugin in WordPress), and some of the attachment references didn’t work with my theme. The nice thing about working with markdown files is that it was really easy to search for patterns once I found them. With Git and branches, I could also quickly test fixes and revert them if they didn’t work.</p><p>I didn’t have a lot of content when I made the switch, but it only took me a few hours.</p><h2 id="Summary"><a href="#Summary" class="headerlink" title="Summary"></a>Summary</h2><p>Static websites can be cheaper, perform better, and require less maintenance than self-hosted options. They have their limitations, but for a simple blog like mine, it was an easy choice that I’m still happy with.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;A couple of years ago I switched this blog from a WordPress site hosted on GoDaddy to a statically generated site. The new setup is faste</summary>
      
    
    
    
    
    <category term="blogging" scheme="https://jessemcdowell.ca/tags/blogging/"/>
    
    <category term="web" scheme="https://jessemcdowell.ca/tags/web/"/>
    
    <category term="architecture" scheme="https://jessemcdowell.ca/tags/architecture/"/>
    
    <category term="reviews" scheme="https://jessemcdowell.ca/tags/reviews/"/>
    
    <category term="github" scheme="https://jessemcdowell.ca/tags/github/"/>
    
  </entry>
  
  <entry>
    <title>Review: Grammarly Premium</title>
    <link href="https://jessemcdowell.ca/2023/10/Review-Grammarly-Premium/"/>
    <id>https://jessemcdowell.ca/2023/10/Review-Grammarly-Premium/</id>
    <published>2023-10-19T15:50:37.000Z</published>
    <updated>2024-08-07T05:04:17.807Z</updated>
    
    <content type="html"><![CDATA[<p>I spend a fair bit of time working on this blog. More than I probably should given my unimpressive view metrics. It does help me to crystalize my thoughts and practice articulating points that are important to me, but I’m spending an awful lot of time doing it. Ultimately, I write because I enjoy writing. I just wish I could spend less time per post.</p><p>For a while there I was getting hammered by ads from <a href="https://www.grammarly.com/">Grammarly</a>, and in a fit of frustration, or maybe I was avoiding yet another hour of editing, I signed up. I didn’t start with the free trial, I went straight to Grammarly Premium expecting that it would be great.</p><p>Within an hour I had installed every app and plugin I could. A couple of days later most of them were removed or almost completely disabled. I cancelled my subscription before the first month was over.</p><p>I’m getting ahead of myself though. Let me first explain why I thought it would be so helpful.</p><p>The quickest posts I’ve written here have taken me a couple of hours where the more detailed ones can take 10 to 20 hours to complete. About 75% of that time is spent editing the posts. Given the amount of time I’m spending editing, I assumed a tool like Grammarly could help me free up some time.</p><p>I ran the numbers for a few recent posts to illustrate:</p><img src="/2023/10/Review-Grammarly-Premium/draft-time-vs-edit-time.svg" class="" title="Stacked bar chart showing times between 1.5 and 14 hours for a sampling of 6 recent posts"><p>It seemed like magic the first time I ran a blog post through Grammarly. The app filled up with suggestions, and many of them were helpful. Spelling mistakes, grammar errors, and little formatting mistakes were easy to spot and eliminate.</p><p>Here’s an example of a paragraph I edited one of the first times I used Grammarly on <a href="/2023/09/Case-of-the-Slow-Matchmaking-Routine/">a recent blog post</a>:</p><img src="/2023/10/Review-Grammarly-Premium/matchmaking-performance-grammarly-edits.png" class="" title="A comparison of a paragraph before and after editing with the differences highlighted."><p>Unfortunately, Grammarly made a lot of other suggestions too, and you can see some of them in the example above. It tried to eliminate passive voice and remove emphasis from important points. It frequently suggested replacements for words (to reduce duplication) that would ruin common industry expressions. Worst of all, it felt like it was removing my personal style from the blog. Every time I skipped a suggestion I also had this little nagging voice in my head suggesting that maybe it was right and my style of communicating was wrong.</p><p>Even if I skipped the tone suggestions and stuck purely to grammar and formatting errors, it didn’t speed up my editing process very much. I call it editing, but it might be more correct to call it rewriting. My first drafts have typos and grammatical errors, but they are also often disorganized and lacking a coherent point. AI is going to have to advance a lot further to fix that for me!</p><p>Grammarly is still helpful with annoying tasks like checking for “its” and “it’s”, and it’s definitely better than a standard spell checker for catching typos and grammatical errors. I don’t typically enjoy editing, but I enjoy this kind of mindless checking the least of all.</p><p>Having switched to the free version of Grammarly, I get far fewer suggestions about tone and word choice, which I find better. The app does still share those suggestions occasionally in an attempt to upsell me, but I don’t find it too intrusive.</p><p>I don’t use any of the apps or plugins now. I found them to be exceedingly intrusive. The Windows app added a giant green G button that covered text boxes, and a lot of times it was covering things I wanted to see. The VS Code plugin covered my rough drafts in red and blue lines, and my programmer muscle memory couldn’t just ignore them. The browser plugin was the least intrusive, but still too noisy and often not helpful.</p><p>Now I use Grammarly primarily via its web interface, and only as a final pass at the end of the editing process.</p><p>The technology is promising, and it has helped a bit with some of the more mechanical editing I do. Until it gets a lot better, though, I will only be using it for very specific tasks. I’d have to see some pretty major improvements before I’d consider Grammarly Premium again.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I spend a fair bit of time working on this blog. More than I probably should given my unimpressive view metrics. It does help me to cryst</summary>
      
    
    
    
    
    <category term="blogging" scheme="https://jessemcdowell.ca/tags/blogging/"/>
    
    <category term="reviews" scheme="https://jessemcdowell.ca/tags/reviews/"/>
    
  </entry>
  
  <entry>
    <title>Getting Unstuck Without a Rubber Duck</title>
    <link href="https://jessemcdowell.ca/2023/10/Getting-Unstuck-Without-a-Rubber-Duck/"/>
    <id>https://jessemcdowell.ca/2023/10/Getting-Unstuck-Without-a-Rubber-Duck/</id>
    <published>2023-10-04T01:03:27.000Z</published>
    <updated>2024-08-07T05:04:17.806Z</updated>
    
    <content type="html"><![CDATA[<p>Building software is a mostly creative endeavour, and as such, it sometimes resists progress. No matter how hard you try to push forward, if you are truly stuck, continuing on the same path is unlikely to work. Fortunately, there are a few different tricks you can use to get going again.</p><h2 id="Take-a-rest"><a href="#Take-a-rest" class="headerlink" title="Take a rest"></a>Take a rest</h2><p>The easiest way I’ve found to get back on track is to take a break. When in the office this was often making a cup of tea or eating a snack in the break room. It could have been a ten-minute walk around the block, but if I’m honest, I have rarely tried this.</p><p>Working from home, I typically load the dishwasher or throw a load of laundry in the wash. If I have time, I might do a bit of yoga or exercise.</p><p>My favourite is a quick shower which seems to reset thoughts and stimulate creative thinking. Rinsing and drying my face in a bathroom sink is not nearly as effective, but in an office, it may be better than nothing.</p><p>If I have the time, or if I’m particularly stuck, a night of sleep can do wonders. I almost always have something else just as urgent to work on, so I’ll usually switch to that for the rest of the day.</p><h2 id="Rubber-ducking-and-alternatives"><a href="#Rubber-ducking-and-alternatives" class="headerlink" title="Rubber ducking and alternatives"></a>Rubber ducking and alternatives</h2><p>Every now and then I get so thoroughly stuck that no amount of taking breaks is going to help. Or sometimes a problem is urgent enough that waiting a day to sleep would be irresponsible. That’s when I <a href="https://en.wikipedia.org/wiki/Rubber_duck_debugging">bust out my rubber duck</a>, at least metaphorically. I don’t actually have or want a rubber duck because there are several similar alternatives that work as well or better.</p><p>The one I use most often is to write an email asking an imaginary colleague for advice. If you treat it like a real work email, it forces you to rethink where you are from someone else’s perspective, and can often unlock some insights or surface some tidbits that got overlooked along the way. Writing an email also forces you to check your facts and go through your numbers again. This is harder when a rubber duck is waiting impatiently for you to finish your sentence. Another advantage is that you can use it in an environment where talking would be a nuisance to others.</p><p>Sometimes writing an email isn’t the best way to communicate a problem though. I have also drawn diagrams on whiteboards with similar results. There are lots of ways to communicate a problem, the key is that you make the effort to reframe it for the benefit of someone not directly involved.</p><p>If you are really really stuck, or your problem is really really important, it might be worthwhile to explain it to a real person. This can be a better first step if you’re working in areas you don’t know very well. It can also be helpful if you’re new to a team as it helps you demonstrate trust for your new peers and accelerates team building.</p><h2 id="Remove-all-distractions"><a href="#Remove-all-distractions" class="headerlink" title="Remove all distractions"></a>Remove all distractions</h2><p>It’s certainly not the first thing I want to try, but when a problem is particularly stressful or unpleasant, I can sometimes find myself working on other things on the side of my desk. It may be listening to and editing a playlist, or maintaining an idle chat with a friend. It could also be changing that load of laundry I threw in earlier as a small break. However I got there, if I’m really stuck, removing these extra things is another important step.</p><h2 id="Literally-changing-your-perspective"><a href="#Literally-changing-your-perspective" class="headerlink" title="Literally changing your perspective"></a>Literally changing your perspective</h2><p>If I’ve been looking at a piece of code long enough, sometimes I need to change how I’m looking at it to really see it. An easy way to do this is to change the font, or sometimes even just the font size. Even if the content hasn’t changed, rearranging the letters a tiny bit seems to force my brain to process it like it’s something new.</p><p>If I’m feeling particularly drained or frustrated with a problem, I have also found it helpful to move to another place for a bit. This is easiest with a laptop, or you may be able to use Remote Desktop from an empty meeting room. In a pinch, I’ve taken a pad of paper to a coffee shop to see what happens and been pleasantly surprised.</p><h2 id="Confirm-the-problem-is-important"><a href="#Confirm-the-problem-is-important" class="headerlink" title="Confirm the problem is important"></a>Confirm the problem is important</h2><p>If you are really, really, really stuck on a problem, maybe there is a good reason. It may be beyond your current capability. It may also not be important enough to justify more effort. A quick chat with your manager shouldn’t hurt. If it is dangerous to ask your manager, you may want to find a new manager, but that’s <a href="/2023/06/Importance-of-Alignment/">a topic for another post</a>.</p><h2 id="Ask-for-help"><a href="#Ask-for-help" class="headerlink" title="Ask for help"></a>Ask for help</h2><p>If your problem is important enough, and you’ve made an effort and failed to get yourself unstuck, it’s not inappropriate to ask for help. Pair programming is an excellent technique for tackling complicated problems.</p><p>Searching or asking online for an answer could also help, but I’ve not personally had much luck here. Your results may vary.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Building software is a mostly creative endeavour, and as such, it sometimes resists progress. No matter how hard you try to push forward,</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
  </entry>
  
  <entry>
    <title>Choosing Powerful Names</title>
    <link href="https://jessemcdowell.ca/2023/09/Choosing-Powerful-Names/"/>
    <id>https://jessemcdowell.ca/2023/09/Choosing-Powerful-Names/</id>
    <published>2023-09-18T19:44:36.000Z</published>
    <updated>2024-08-07T05:04:17.805Z</updated>
    
    <content type="html"><![CDATA[<p>A junior developer needs to strengthen their technical skills to advance. An intermediate developer needs to strengthen their organizational skills to advance. Senior developers need to master these and also demonstrate that they can move multiple teams forward together. Influencing people (and especially developers) is no easy task, but a positive reputation can do a lot of the heavy lifting. One of the easiest ways to amplify your reputation is to put some extra effort in when choosing a name.</p><p>The name of the things you build is the first and most frequent way that your work will be represented to others. It is the very first of the first impressions. A catchy name can generate interest in you and bring more attention to your hard work. A name that shows your personality can make you more approachable and attract people who see the world like you do.</p><p>This is especially important in larger companies. There is so much more activity and so many other people trying to move forward that maximizing the impact of your first impressions really helps. I got into this habit when I worked in a big company, and I’ve carried it with me ever since. It has served me well.</p><p>The technique I use is a pretty standard design technique: get as many options as I (reasonably) can, then pick the best one. I often go through hundreds or thousands of candidates when choosing a name. When it’s important enough, I’ve gone through more.</p><p>Even though I have been through the process several times now, I sometimes find it overwhelming at the start. It goes by quickly once I get started, though, and I can usually find a pretty spectacular name for anything in a couple of hours.</p><p>To generate candidates, I like to list out all the things I want the name to convey. I have had good luck with lists of words in relevant domains. For example, when naming a new module for an environmental monitoring tool, I got a great name from a list of meteorological terms.</p><p>Once I have some good candidates, I check my finalists against a few important criteria. I want a name to:</p><ul><li>be relevant to the problem being solved</li><li>have a playful spirit</li><li>focus on the positive</li><li>be respectful of all cultures</li><li>be easy to say, spell, and pronounce</li><li>not be confusable with common programming keywords or industry terms</li></ul><p>Playful is a part of my personality; you can take or leave that, but I highly suggest staying positive. Even if it’s meant in fun, using negative imagery in your name can give people the wrong idea about you. The point of this exercise is to get more people interested, not to push people away.</p><p>Being respectful of all cultures is becoming more important as the internet continues connecting the whole world together. Some words can have unexpected or unpleasant connotations for people who live (or have lived) in other cultures. Some words commonly used in Western culture should be retired now. Names from nature, science, geography, or popular fiction are usually safe. References to religions, cultures, or historical figures or groups should be avoided.</p><p>After focusing on a name for hours, it’s not uncommon to develop tunnel vision. Before pasting a name all over my files and objects, I like to get an external perspective. Asking a boss or a peer can save a lot of time if your name isn’t quite as great as you thought.</p><p>Once the name is set, all that’s left is living up to it.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;A junior developer needs to strengthen their technical skills to advance. An intermediate developer needs to strengthen their organizatio</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
  </entry>
  
  <entry>
    <title>Case of the Slow Matchmaking Routine</title>
    <link href="https://jessemcdowell.ca/2023/09/Case-of-the-Slow-Matchmaking-Routine/"/>
    <id>https://jessemcdowell.ca/2023/09/Case-of-the-Slow-Matchmaking-Routine/</id>
    <published>2023-09-05T22:08:46.000Z</published>
    <updated>2024-08-07T05:04:17.805Z</updated>
    
    <content type="html"><![CDATA[<p>The most challenging bug I’ve ever fixed was a performance issue in a matchmaking routine. Matchmaking is the process of finding players to compete against each other in a video game. An excellent matchmaking algorithm doesn’t just stick players together randomly; it tries to make the game more fun by balancing power levels and preventing anyone from waiting too long for a match.</p><p>About six weeks before a game I was working on was scheduled to be feature complete, we discovered our routine couldn’t handle our load targets. The rate at which players were being removed from the matchmaking queue started dropping during load tests. Things got bad quickly once it fell below the rate at which we inserted them. Not only would this cause a bad user experience if we didn’t fix it, but it made it impossible for us to drive enough traffic to our game servers to test that they could handle the projected load. The company wasn’t going to release a game that could crash if it was successful, so we had to fix this issue, and we had to fix it quickly.</p><p>To make matters worse, we could only reproduce the bug in our production system. We had to simulate hundreds of thousands of users to hit it, which just wasn’t possible on a developer machine. Fortunately, our production system was available for testing. Unfortunately, a team on another continent was running the load tests. It took us a day to deploy a new version and a couple of days for them to run the test. With time zone differences expanding the handoffs, the fastest we could iterate was about once a week.</p><p><a href="/2023/04/How-to-Fix-a-Bug/">As I’ve described before</a>, I like to find a bug, understand it, and then fix it. In this case we couldn’t figure it out by looking at the code, even with many smart people trying. This is one of those rare cases where I had to try fixing a bug before understanding it.</p><p>The week of iteration time also meant I had about a week to review the results, consider multiple possibilities, and sprinkle logging and profiling code all over the place, so I did that, too.</p><p> I plugged away at it week after week, but the problem was stubbornly difficult to find. The failing load tests were visible to our management chain, so I also had a fair number of questions about when it would be fixed (which I couldn’t estimate), how we were fixing it, and what we would do if we couldn’t fix it. I was also helping the rest of the team with their deliverables, which had the same deadline, so it was stressful for everyone. Nevertheless, I stuck to my process, narrowed down the location of the problem, and tested my fixes as best as possible in my development environment. Eventually, I figured it out.</p><p>The problem originated in a database query we used to find matches. It used a cartesian join, which takes exponentially longer to complete as more rows are added (<code>O(n²)</code> complexity). At some queue length, the processing time increases from milliseconds to seconds, and then it quickly increases from seconds to minutes. It also didn’t help that our contrived load test meant thousands of users were all attempting to make matches with the same power rating, thus making it harder to distinguish suitable matches.</p><p>After all the time it took to find the problem, it only took a day to fix it. I replaced the algorithm with one that processed matches linearly, giving it <code>O(n)</code> complexity.</p><p>About a week later, I got an email confirming that the issue was fixed. Since the report came in overnight, it was the first thing I read in the morning. It was exhilarating, but the day is still bittersweet in my memory. About an hour later, I attended a last-minute team-wide meeting where we were all let go. The game was never released.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;The most challenging bug I’ve ever fixed was a performance issue in a matchmaking routine. Matchmaking is the process of finding players </summary>
      
    
    
    
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="war stories" scheme="https://jessemcdowell.ca/tags/war-stories/"/>
    
    <category term="video games" scheme="https://jessemcdowell.ca/tags/video-games/"/>
    
  </entry>
  
  <entry>
    <title>Regarding Test Coverage Targets</title>
    <link href="https://jessemcdowell.ca/2023/08/Regarding-Test-Coverage-Targets/"/>
    <id>https://jessemcdowell.ca/2023/08/Regarding-Test-Coverage-Targets/</id>
    <published>2023-08-21T18:28:52.000Z</published>
    <updated>2024-08-07T05:04:17.804Z</updated>
    
    <content type="html"><![CDATA[<p>Unit tests are undeniably a good thing, but you only realize the full benefits of them when you have enough tests that you can make changes with confidence. If you can make a change, run your tests, and be comfortable enough to ship your changes, then you and your team can get work done much faster. More drastic changes to the shared code become feasible. Life gets better.</p><p>It makes sense then that teams want to ensure that code is sufficiently covered with tests. Nobody wants to count tests every time they review a PR, so tools are added that check it automatically. It’s then a small step to set a coverage target, and suddenly you have a machine checking every PR for tests. This all makes sense to me, and it was my first instinct too. I don’t recommend this approach any more.</p><p>The problem with test coverage tools is that they can’t (at least, can’t yet) measure the quality or value of a test. They instead measure the quantity of code that the tests exercise. This can encourage a misplaced focus on building lots of low-value tests. For example, consider the following piece of relatively standard web service code:</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> GetResult <span class="title function_">get</span><span class="params">(GetRequest request)</span> &#123;</span><br><span class="line">    <span class="keyword">return</span> service.get(request);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>Here is a standard test for this function:</p><figure class="highlight java"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">@Test</span></span><br><span class="line"><span class="keyword">public</span> <span class="keyword">void</span> <span class="title function_">getCallBusinessLayerGetAndReturnResult</span><span class="params">()</span> &#123;</span><br><span class="line">    <span class="type">var</span> <span class="variable">request</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">GetRequest</span>();</span><br><span class="line">    <span class="type">var</span> <span class="variable">expected</span> <span class="operator">=</span> <span class="keyword">new</span> <span class="title class_">GetResult</span>();</span><br><span class="line">    when(mockBusiness.get(request)).thenReturn(expected);</span><br><span class="line"></span><br><span class="line">    <span class="type">var</span> <span class="variable">actual</span> <span class="operator">=</span> service.get(request);</span><br><span class="line"></span><br><span class="line">    verify(mockBusiness).get(request);</span><br><span class="line">    assertEquals(expected, actual);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>And now imagine thousands of tests like this, all testing very similar functions.</p><p>The unit test is testing the expected behavior of the service function, but what are the chances of there being a bug in a function this simple? I think it’s far more likely that the test would be written incorrectly than the service function.</p><p>The worst case scenario would be if all these service functions and all of these tests were generated by copy-paste and modification. Then it is much more likely that the wrong values get pasted into the test and the implementation at the same time. Unfortunately this kind of boilerplate code is almost always generated with copy and paste because it’s fast and and easy.</p><p>You could make an argument that you should avoid architectures that encourage lots of boring boilerplate code. I agree with that idea, but in my experience most teams are not mature enough to design systems that prevent it. It is easy to follow simple service-business-repository patterns blindly, and to be honest, for most software this is good enough.</p><p>Again: I do think unit tests are a good thing, and good test coverage is essential to get good value from them, but unit tests also have a cost. Unit tests often need to be changed when code is being changed. If you have lots of low-value tests testing lots of simple methods, you can quickly get overwhelmed trying to make non-trivial changes. Unit tests are supposed to make it safer to go faster… but poorly written tests can do the exact opposite too.</p><p>Of course there is an exception to every rule. If you are writing the software for my bank, or for medical equipment, or for self driving cars, please enforce 100% coverage and use several other tools to enure an extremely high quality. For most of us though, not all of the tests are actually worth the effort of writing them, and I don’t want to edit the coverage percentage every time I make a change.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Unit tests are undeniably a good thing, but you only realize the full benefits of them when you have enough tests that you can make chang</summary>
      
    
    
    
    
    <category term="unit testing" scheme="https://jessemcdowell.ca/tags/unit-testing/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
    <category term="quality" scheme="https://jessemcdowell.ca/tags/quality/"/>
    
    <category term="architecture" scheme="https://jessemcdowell.ca/tags/architecture/"/>
    
  </entry>
  
  <entry>
    <title>My Architectural Report Template</title>
    <link href="https://jessemcdowell.ca/2023/08/My-Architectural-Report-Template/"/>
    <id>https://jessemcdowell.ca/2023/08/My-Architectural-Report-Template/</id>
    <published>2023-08-08T19:45:13.000Z</published>
    <updated>2024-08-07T05:04:17.803Z</updated>
    
    <content type="html"><![CDATA[<p>As an architect I’ve been asked to answer a lot of hard questions. I used to waste time figuring out how to structure my answers, preventing me from getting into a good flow sooner. Now I have a simple template that is easy to use, easy to read, and saves me that wasted time up front.</p><p>This template works for simple reports that are only a couple of pages, but can easily be adjusted or expanded for more complicated or much larger documents.</p><p>The template has these sections:</p><ul><li>Purpose</li><li>Context</li><li>Recommendation</li><li>Paths Not Taken</li><li>Questions and Answers</li></ul><p>I typically start a new problem by creating a new document and sticking in these headings. Next, I work to understand and document the context of the problem. Once that’s complete, I can design and describe my answer.</p><h2 id="Surfacing-Context"><a href="#Surfacing-Context" class="headerlink" title="Surfacing Context"></a>Surfacing Context</h2><p>It can be tempting to jump straight into solutioning, however, and I cannot emphasize this point enough, you cannot design a good solution to a problem if you don’t understand the problem. I enjoy armchair architecture as much as anyone, but a good architect chooses the best solution for the circumstances, not just the first solution they thought of.</p><p>I always start with the first two sections of this document. This is usually only a page of text, but it really helps to frame and validate the problem from a business &#x2F; customer perspective before starting the technical design.</p><h3 id="Purpose-Section"><a href="#Purpose-Section" class="headerlink" title="Purpose Section"></a>Purpose Section</h3><p>This first section has just a few brief sentences. It starts by explaining the purpose of the document, and then the scope of the document, and then a description of who it’s written for. It can seem a bit superfluous, but these details are important to remember when writing the rest of the document. It can also help a reader’s understanding if they know the context in which the document was written.</p><h4 id="Example"><a href="#Example" class="headerlink" title="Example"></a>Example</h4><blockquote><p>This document recommends an approach to boiling the ocean. It is intended to be used during discussions with potential partners for the project. It is written for the CTO and the PMO.</p></blockquote><h3 id="Context-Section"><a href="#Context-Section" class="headerlink" title="Context Section"></a>Context Section</h3><p>This is the most important section, and sometimes the hardest thing to get right. I only put in a few nuggets of information, but they need to be important. They serve as a sales pitch for upstream stakeholders and simultaneously guide and rationalize the decisions in the following sections. Even if the context seems obvious it is good to write it out.</p><p>I write some bullet points to articulate the importance of the problem and any significant constraints to the design. I keep it to just a few items, ideally between 3 and 6. If there are too many it becomes harder to process, if there are too few I probably shouldn’t be spending my time on the problem.</p><p>I like to polish each point a bit, but I also avoid flowery or superfluous language. Neutral defensible facts laid out clearly are a powerful tool for building alignment. It’s also good to write these in a way that makes sense to stakeholders who are not technically focused.</p><p>When I’m faced with an unclear problem, I will sometimes stick together some context statements based on my best guesses and send them out for feedback. <a href="https://meta.wikimedia.org/wiki/Cunningham%27s_Law">Cunningham’s Law</a> applies to all of design - it’s much easier to get corrections to a bad answer than ask for an answer in a vacuum. There are more specific techniques to help extract and validate this kind of information, but they take a lot more time from stakeholders so I try to use them only when necessary.</p><h4 id="Example-1"><a href="#Example-1" class="headerlink" title="Example"></a>Example</h4><blockquote><ul><li>Over 80% of internationally traded goods travel by sea. Marine freight is exposed to many dangers including navigational challenges, extreme weather, and piracy.</li><li>The sea level has been steadily rising due to human-caused climate change. As the ocean continues to rise, estimates anticipate hundreds of millions to billions of people will be displaced unless additional measures are taken.</li><li>The earth is simultaneously affected by an international housing shortage and the continual loss of essential farm land.</li><li>Eliminating the world’s oceans can simplify or make new solutions available to all of the problems above, which could in turn create numerous opportunities for raising and generating capital.</li></ul></blockquote><h3 id="Document-Title"><a href="#Document-Title" class="headerlink" title="Document Title"></a>Document Title</h3><p>A descriptive title is an important part of any document. I don’t want to spend too long figuring this out up front, but I do make sure I have a good one before any document gets circulated. Once it gets sent out, changing the name can break links and make it harder for people to find.</p><p>I also like to include the date in the file name and title. These documents are snapshots of the best information available at the time they are written. Since this is usually before a project has started, they lose a lot of their value once things get underway. A date in the title makes it clear when the document is an archeological artifact.</p><p>I always use the <code>yyyy-mm</code> format for dates because it’s precise enough, and the metric system is a good thing.</p><p>I could update the document as the project goes along, but I don’t see much value in this. The document has served its purpose once the decision is accepted and things start. If we need some sort of onboarding guide or architectural documentation, the parts of this document that are valuable for that effort can be easily copied and updated as needed.</p><h4 id="Example-2"><a href="#Example-2" class="headerlink" title="Example"></a>Example</h4><blockquote><p>How to Boil the Ocean - 2023-08</p></blockquote><h3 id="Stakeholder-Check-in"><a href="#Stakeholder-Check-in" class="headerlink" title="Stakeholder Check-in"></a>Stakeholder Check-in</h3><p>Once I have the above sections completed I like to check in with my upstream stakeholders.</p><p>Sometimes this process can take a bit of time. Depending on the size and urgency of the problem I might start on the technical design while collecting feedback. Parts of the design could be invalidated if the context changes, but usually any effort will still have nuggets that can be reused.</p><p>When I don’t get any significant feedback it can be a sign that the problem is either not well understood or not important to anyone. I might then schedule some workshops to dig in further before continuing.</p><h2 id="Describing-a-Solution"><a href="#Describing-a-Solution" class="headerlink" title="Describing a Solution"></a>Describing a Solution</h2><p>There is a lot that goes into answering a technical question, and the way I go about it depends on a lot of factors.</p><p>Before I get too far into it, I like to think about how much I should be collaborating on the answer. Even if the answer seems obvious to me, it is sometimes valuable to involve the people who’ll be building the solution to ensure better buy-in once the project starts.</p><p>It’s also important to think about the acceptable amount of risk. If the cost of getting the solution wrong is huge, it may be necessary to do more thorough research or build and test some prototypes. If I’m just putting some rough estimates together to help plan the roadmap I will put in considerably less effort.</p><p>After all the thinking and validating work is done, I dump the best design into the document.</p><h3 id="Recommendation-Section"><a href="#Recommendation-Section" class="headerlink" title="Recommendation Section"></a>Recommendation Section</h3><p>This is where I describe the answer to the question. I sometimes change the heading depending on the purpose of the document. If I’m preparing a thorough design for a complex system I might include multiple sub-headings and a few diagrams. For documents used more for resource planning I might just provide a brief architectural summary and a table of high-level tasks with t-shirt resolution estimates.</p><p>There are lots of ways to communicate designs. I do spend some time thinking about how best to do this. If I can use more efficient tools that are suitable for the audience and purpose of the document I can produce less. Smaller documents are easier to read, and easier to update when the feedback comes in.</p><h4 id="Example-3"><a href="#Example-3" class="headerlink" title="Example"></a>Example</h4><blockquote><p>Use barges with nuclear fission reactors to boil the ocean water. As the sea level drops, the barges can be moved to ensure they stay submerged in water.</p><p>The steam generated by the reactors can also be used to generate electricity. Electricity not used for the operation of the project can be sold, adding an additional revenue stream.</p></blockquote><h3 id="Paths-Not-Taken-Section"><a href="#Paths-Not-Taken-Section" class="headerlink" title="Paths Not Taken Section"></a>Paths Not Taken Section</h3><p>I’ve found it is often helpful to list some of the other possibilities that were considered but not chosen. Not everyone cares about this, but it can answer a lot of “what about X” questions from some personality types. It can also be helpful if you look back at a document years later. It’s not the most important part of the document though, so I try not to put too much effort into it. Bullet points and rough notes are usually enough.</p><h4 id="Example-4"><a href="#Example-4" class="headerlink" title="Example"></a>Example</h4><blockquote><ul><li>Use a giant laser from space.<ul><li>The laser beams would be a hazard to air and space traffic.</li></ul></li><li>Accelerate global warming by burning a lot of oil.<ul><li>The required oil would make this approach significantly more expensive than other options.</li><li>This option could generate too much negative PR.</li></ul></li><li>Pump the water into on-land boiling stations.<ul><li>Cost of pumping would be significant.</li><li>Hoses and pumping stations would need to be continually added as the sea level drops.</li></ul></li><li>Use nuclear fusion reactors to generate heat.<ul><li>The technology is not mature enough at the time of this writing. If it does become feasible during the project we could update the design for any new barges being constructed.</li></ul></li></ul></blockquote><h3 id="Questions-and-Answers-Section"><a href="#Questions-and-Answers-Section" class="headerlink" title="Questions and Answers Section"></a>Questions and Answers Section</h3><p>There will often be some things that I want to communicate that don’t fit anywhere else in the document. A Question and Answers section is really versatile, and it’s an easy place to add more information when responding to feedback.</p><h4 id="Example-5"><a href="#Example-5" class="headerlink" title="Example"></a>Example</h4><blockquote><ul><li>Is boiling the ocean really a responsible thing to do?<ul><li>That is outside the scope of this document.</li></ul></li><li>How will the boiling stations deal with salt accumulation?<ul><li>This will need to be investigated further if we proceed with this project.</li></ul></li></ul></blockquote><h2 id="Customization"><a href="#Customization" class="headerlink" title="Customization"></a>Customization</h2><p>Just like in software development, I find it best to use the simplest document that can possibly do the job. The less I write, the easier the document is to read, and the more time I have to work on my next project.</p><p>If I don’t need one of the sections above I’ll remove it. I can also add new sections as I need them. I have added sections with estimate tables, rudimentary project plans, optional features, glossary of terminology, and so on.</p><h2 id="Template"><a href="#Template" class="headerlink" title="Template"></a>Template</h2><p>I’ve created a standalone page with the example from this post so you can use it as a template for your own work. You can find it under <a href="/resources/architectural-report-template/">Resources, Architectural Report Template</a>. If you do use it I suggest replacing the contents before sharing it to prevent any embarrassment.</p><h2 id="Acknowledgements"><a href="#Acknowledgements" class="headerlink" title="Acknowledgements"></a>Acknowledgements</h2><p>This template was inspired by the <a href="https://github.com/joelparkerhenderson/architecture-decision-record/blob/main/templates/decision-record-template-by-michael-nygard/index.md">Decision record template by Michael Nygard</a> which I also recommend for its intended purpose.</p><p>If you want to know more about how to write architectural reports, gather information, or validate designs, I highly recommend the book <a href="https://pragprog.com/titles/mkdsa/design-it/">Design It!: From Programmer to Software Architect</a> by Michael Keeling. I learned a lot from this book, and have referred to its contents many times since.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;As an architect I’ve been asked to answer a lot of hard questions. I used to waste time figuring out how to structure my answers, prevent</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="architecture" scheme="https://jessemcdowell.ca/tags/architecture/"/>
    
    <category term="resources" scheme="https://jessemcdowell.ca/tags/resources/"/>
    
  </entry>
  
  <entry>
    <title>Design by Dogma Antipattern</title>
    <link href="https://jessemcdowell.ca/2023/07/Design-by-Dogma-Antipattern/"/>
    <id>https://jessemcdowell.ca/2023/07/Design-by-Dogma-Antipattern/</id>
    <published>2023-07-30T20:41:17.000Z</published>
    <updated>2024-08-07T05:04:17.803Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>Always use a NoSQL database so your app can scale.</p></blockquote><p>NoSQL databases can be more scalable, but schema-on-read has other drawbacks. NoSQL databases are much less capable of transactional changes. Relationships are difficult or impossible. Designing schemas to be efficient is much harder, and requires more up-front knowledge about your problem. NoSQL databases are sometimes the right tool for the job, but they are not the right tool for every job.</p><blockquote><p>Monoliths are bad. Build Microservices.</p></blockquote><p>Microservices have advantages, but they are harder to build. They are harder to deploy. They require more tools, processes, and governance to keep them running and working together properly. If they aren’t carved up just right they can create cross-team dependencies and performance bottlenecks. All that being said, they are ane excellent tool for very large organizations to decouple and decompose development teams.</p><blockquote><p>Never use reflection because it’s slow.</p></blockquote><p>Reflection can be a heavy operation, but it can also give you extremely valuable information. If it does prove to be a performance problem, there are also other optimizations which can help, or at worst, you can try to restrict lookups to development and test environments. Stack traces in exceptions can be extremely helpful in finding bugs. Even if they do cost a bit to collect them, the developer time wasted figuring out bugs, and the unhappy customers waiting for fixes may cost you more.</p><hr><p>I have heard these kinds of absolute statements many times in my career. To be fair: building software is hard and there is a lot to learn, and simple statements are a comfortable simplification. The problem is that few absolutes exist.</p><p>It doesn’t help that so much information in our industry is presented this way. Blogs that pronounce something bad get clicks. Vendors don’t like to tell you their drawbacks. Everyone wants you to agree with them.</p><p>This kind of blind adherence to arbitrary rules is bad engineering. We should work to understand the advantages and disadvantages of our technical choices, and understand when it is important to get them right.</p><p>Since I started with a quote, I’ll end with another:</p><blockquote><p>When you believe in things that you don’t understand<br>Then you suffer<br>Superstition ain’t the way</p><p>– Superstition by Stevie Wonder</p></blockquote>]]></content>
    
    
      
      
    <summary type="html">&lt;blockquote&gt;
&lt;p&gt;Always use a NoSQL database so your app can scale.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;NoSQL databases can be more scalable, but schema-on-</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
  </entry>
  
  <entry>
    <title>Horizontal One-on-Ones and Talking Practice</title>
    <link href="https://jessemcdowell.ca/2023/07/Horizontal-One-on-Ones-and-Talking-Practice/"/>
    <id>https://jessemcdowell.ca/2023/07/Horizontal-One-on-Ones-and-Talking-Practice/</id>
    <published>2023-07-18T22:35:11.000Z</published>
    <updated>2024-08-07T05:04:17.802Z</updated>
    
    <content type="html"><![CDATA[<p>When I was promoted to the role of architect it was a new role in the organization. The stakeholders I had to work with were not used to talking to an architect, and weren’t sure what I did or when I should be involved in a conversation. I started using recurring one-on-one meetings with each stakeholder separately. It worked great. It’s also made me a much better communicator.</p><p>One of the first and most important lessons I learned as an architect is that you can’t design a good architecture without a good understanding of its requirements. You can design a system in a vacuum, it’s also much easier to do it this way, but it’s far less likely to serve the organization. Gathering, validating, and documenting technical requirements is tough work, but an essential part of being an architect.</p><p>The best way to discover needs and requirements is to talk to people who know them. There is quite a lot of ways to approach this, and a lot has been written on this subject. For me, I knew I needed to chat with my stakeholders early and often, and private meetings was the simplest technique that could possibly work.</p><p>In my first one-on-one meetings I explained my role, and more importantly, why my role would be beneficial to each of them. For the product manager I explained that if I was good at my job I would help the development team go faster (because of less rework) and to reduce the technical risks of higher value projects before they were scheduled. For the product owner and development leads I explained that I would find and encourage new technologies that helped the team go faster, and help to keep projects moving smoothly.</p><p>I didn’t have any difficulty getting time on a regular schedule. Even with a 3-month release cycle, there was always enough to fill a meeting every two weeks with each of my main stakeholders. I always came prepared with questions, and after a while my stakeholders starting bringing questions for me too.</p><p>I think I have pretty good communication skills, but when you need to explain the value of technical work to people who are primarily non-technical it can be a lot more difficult. It’s even more daunting because the people I was meeting with were far, far better presenters and public speakers than I am.</p><p>I started noticing that I was getting very similar questions from different stakeholders, and so I was able to re-use my answers. And with time, I started getting better at it. Regular conversations certainly helped, but talking about the same topics over and over seemed to make an even bigger difference. Every time I answered a similar question I had a chance to refine my best points and try out new ones.</p><p>I was fortunate to work in an environment with a lot of trust: I could speak my mind honestly, and I didn’t really need to be persuasive to make sure my ideas were heard. Even still, the better I got, the more confident I felt. Portraying confidence is important in a leadership position - if a leader looks worried it can generate more stress for those around them.</p><p>After a few years of these meetings, I am convinced that they were a big part of my success as an architect. I’m also convinced that they’ve helped me improve as a communicator.</p><p>Repeating similar conversation has been so valuable, that I’ve started having practice conversations just for the sake of it. Rough notes and research is still my preferred place to start, but this type of preparation starts to have diminishing returns after a while. If I really need to nail a conversation, a practice conversation is an efficient way to improve my messages. All it takes is a bit of time from a trusted colleague, or if that’s not possible, a rubber duck can do the job too.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;When I was promoted to the role of architect it was a new role in the organization. The stakeholders I had to work with were not used to </summary>
      
    
    
    
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
    <category term="architecture" scheme="https://jessemcdowell.ca/tags/architecture/"/>
    
  </entry>
  
  <entry>
    <title>Case of the Appearing Users</title>
    <link href="https://jessemcdowell.ca/2023/07/Case-of-the-Appearing-Users/"/>
    <id>https://jessemcdowell.ca/2023/07/Case-of-the-Appearing-Users/</id>
    <published>2023-07-10T20:18:09.000Z</published>
    <updated>2024-08-07T05:04:17.801Z</updated>
    
    <content type="html"><![CDATA[<p>A couple of years after solving <a href="/2023/05/Case-of-the-Disappearing-Users">The Case of The Disappearing Users</a>, I was assigned another high profile bug where new users were being spontaneously created. They were being generated without a name or any profile information, but still filling up space in lists and appearing on schedules. A couple of other developers had tried fixing it but had no luck, so it was assigned to me.</p><p>I went through my usual bag of tricks: searched recent changes, searched for insert statements, tried to create empty users manually (and couldn’t). Nothing worked, and it was looking pretty hopeless.</p><p>I knew I wasn’t going to make any progress if I couldn’t narrow down the cause. The problem was only occurring for one customer, but it was also only occurring around once a week. I ended up making a patched version of our application that regularly scanned the number of users and threw up a notification if a new one had appeared. I also added some custom logging in a few conspicuous areas, and sent it to the customer.</p><p>The issue appeared again, but this time we were able to collect the logs and analyze them. I was able to determine that the problem was occurring for a specific user, and I was able to figure out which application had caused it. The weird thing: this application didn’t have a feature for creating users! We were able to question the user, and it turned out they had a new computer, and it had a brand new mouse.</p><p>It’s important to understand that this application was build using Microsoft Access (hey, don’t judge). One of the best features of Access in the data binding, saving your from writing a lot of boilerplate code. All you have to do is write a query and bind the controls on the form to the desired fields.</p><p>One of the (many) challenges with Access is that it is sometimes a bit too clover. It could convert a select statement into insert &#x2F; update &#x2F; delete statements automatically. If you navigated the form past the last row of the query (even if your query only selected a single row) it would go into creation mode.</p><p>The form where the row was created had all the keyboard navigation shortcuts blocked, as was the standard practice, but the user was still able to trigger it. This is where the new mouse comes in. This problem occurred around the time when mice just started shipping with scroll wheels, and in Access it automatically triggered row navigation.</p><p>We weren’t able to disable this behavior, but I was able to rewrite the query and modify the form so that a new row couldn’t be generated any more. We also updated our installation instructions to ban mice with scroll wheels in case the same problem could be triggered in any of our hundreds of other forms.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;A couple of years after solving &lt;a href=&quot;/2023/05/Case-of-the-Disappearing-Users&quot;&gt;The Case of The Disappearing Users&lt;/a&gt;, I was assigned </summary>
      
    
    
    
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="war stories" scheme="https://jessemcdowell.ca/tags/war-stories/"/>
    
  </entry>
  
  <entry>
    <title>Automating Non-Non-Downtime Upgrades in Kubernetes with ArgoCD</title>
    <link href="https://jessemcdowell.ca/2023/06/Automating-Non-Non-Downtime-Upgrades-in-Kubernetes-with-ArgoCD/"/>
    <id>https://jessemcdowell.ca/2023/06/Automating-Non-Non-Downtime-Upgrades-in-Kubernetes-with-ArgoCD/</id>
    <published>2023-06-26T17:55:24.000Z</published>
    <updated>2024-08-07T05:04:17.801Z</updated>
    
    <content type="html"><![CDATA[<p>I recently worked on a project to move a complicated legacy application onto Kubernetes. It was quite an undertaking, but in the end we were successful. One of the biggest challenges was figuring out how to automate our legacy deployment process, one where the whole application has to be stopped completely for schema upgrades to run.</p><p>The normal “Kubernetes way” to upgrade an application is by changing the <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Deployment resource</a>. With its default <code>RollingUpdate</code> strategy it will delete a pod with the old definition, start a pod with the new definition, wait for it to be healthy, then repeat continuously until the change is fully applied.</p><p>This process wouldn’t work for us, and it was not obvious how we could automate one that did. Our application is tied to its schema version; new versions of the app won’t run on the old schema, old versions of the app can’t run on the new schema, and the schema migrator won’t start if it detects any running applications. We would have preferred to use a rolling update without downtime, but it wasn’t possible to make our application support this in our timelines. I expect it will eventually be implemented, but it will require several changes and a significant testing effort.</p><p>The process we wanted to automate was:</p><ol><li>Shut down the old version of the application</li><li>Run the schema migrator</li><li>Start the new version of the application</li></ol><p>Or putting it into Kubernetes terms:</p><ol><li>Delete all the pods</li><li>Run the schema migrator job</li><li>Create new pods (with new image tag)</li></ol><p>We tried a few different approaches, but the solution we ultimately chose was using <a href="https://argoproj.github.io/cd/">ArgoCD</a> with its <a href="https://argo-cd.readthedocs.io/en/stable/user-guide/sync-waves/">sync phases and waves feature</a>. There were a few unexpected challenges, but we were still happy with the results.</p><h2 id="What-is-ArgoCD-and-What-Are-Sync-Phases-and-Waves"><a href="#What-is-ArgoCD-and-What-Are-Sync-Phases-and-Waves" class="headerlink" title="What is ArgoCD and What Are Sync Phases and Waves?"></a>What is ArgoCD and What Are Sync Phases and Waves?</h2><p><a href="https://argoproj.github.io/cd/">ArgoCD</a> is a powerful open source tool that lets you deploy Helm charts to a Kubernetes cluster. The charts and their settings are pulled from a configurable source, in our case GitHub. This allowed us to store all our Kubernetes configuration as code. We wanted better visibility and consistency in our infrastructure and this tool makes that possible. Being able to add “GitOps” is an added bonus.</p><p>ArgoCD is based on a custom resource called an <a href="https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/#applications">Application</a>. An application represents a single installation of a Helm chart. Its resource includes a source (where to retrieve the chart), a destination (where to install the chart), and any parameters to apply to the chart. You can automate more complicated scenarios with charts containing Applications, resulting in Applications containing Applications. You can also use the <a href="https://argo-cd.readthedocs.io/en/stable/operator-manual/applicationset/">ApplicationSet</a> resource to generate multiple Application objects. We combine all of these to push out a lot of similar but not exactly the same applications in several environments.</p><p>ArgoCD periodically compares the source (code) and destination (cluster state). If there are any differences, the application gets marked as out of sync. ArgoCD can synchronize all the changes in an application automatically or with the press of a button. It also has a nice user interface that shows all the applications, their state, and some other useful information.</p><p>If your application can be deployed all at once as a simple Helm chart, ArgoCD can easily do this. For a more complicated deployment process like ours, we used the sync phases and waves feature; we added special annotations in a few our resource definitions to control the order ArgoCD applies their changes. It looks like this:</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">metadata:</span></span><br><span class="line">  <span class="attr">annotations:</span></span><br><span class="line">    <span class="attr">argocd.argoproj.io/hook:</span> <span class="string">PreSync</span></span><br><span class="line">    <span class="attr">argocd.argoproj.io/sync-wave:</span> <span class="string">&quot;5&quot;</span></span><br></pre></td></tr></table></figure><h2 id="How-We-Automated-our-Deployment-Process"><a href="#How-We-Automated-our-Deployment-Process" class="headerlink" title="How We Automated our Deployment Process"></a>How We Automated our Deployment Process</h2><p>Our process uses these steps:</p><ol><li><code>PreSync</code> &#x2F; <code>-1</code> - Create necessary secrets, service accounts, etc.</li><li><code>PreSync</code> &#x2F; <code>10</code> - Create job: run a script that sets the replica count to 0</li><li><code>PreSync</code> &#x2F; <code>20</code> - Create job: run the schema migrator docker image</li><li><code>Sync</code> (default) - Create the deployment, services, and everything else</li></ol><p>The <code>PreSync</code> &#x2F; <code>10</code> step ensures the application is stopped before continuing. It checks that the deployment exists so it won’t fail on the very first run, then it sets the replica count to 0. The pods get deleted pretty quickly after this change is applied.</p><p>The schema migrator job runs next. It can upgrade the schema of an existing database or create a new one if one doesn’t already exists. Once it completes, all the rest of the resources are created in the <code>Sync</code> step. The deployment resource sets the new application version and restores the replica count. Pods start getting created, and pretty soon we have a fully working application.</p><p>If the schema migrator job fails, Kubernetes will execute any retries as per the job definition. If the job can’t complete, the synchronization cycle stops and gets marked as failed in ArgoCD. A human can then make any necessary changes and trigger another synchronization cycle.</p><p>We ran this process hundreds of times in our development environment and several more in our production environments. The ArgoCD part of it always worked correctly. Since some of our applications had an installation per tenant, we also ran several in parallel with no issues. We did have a few deployments fail, but they were all caused by infrastructure issues or application bugs. That won’t be different from any other Kubernetes system.</p><h2 id="Branching-Strategy"><a href="#Branching-Strategy" class="headerlink" title="Branching Strategy"></a>Branching Strategy</h2><p>The GitOps approach to managing our environments brought some significant benefits, but it also made it more challenging to test changes to our charts. For example, some application changes would require add or dropping startup parameters in the pod definition. We had to be especially careful that it wouldn’t break an environment if the new chart was applied before the application version was updated. It was possible to deal with simple changes like this using conditional blocks in the Helm templates, but it got a lot more messy when we were updating community charts for our logging or monitoring infrastructure, or changing the shared ingress definitions.</p><p>To ensure we didn’t have any accidents we moved to a branching strategy. We now use three branches:</p><ul><li>develop - for our development environment. This is where most of our chart changes occur. It’s also where we test daily builds of our applications</li><li>staging - for our staging environment. We use this to test new charts and applications before a production release</li><li>production - for all of our production environments</li></ul><p>Changes to the charts get tested in the development environment. They then get merged to the staging branch just before a major release. As part of the production release cycle we merge the same changes from the staging branch to the production branch, making sure that only tested changes get deployed.</p><p>To ensure there is no configuration drift, we also occasionally merge changes from the staging and production branches back to the development branch.</p><p>We did find branching a bit difficult to use, especially for parts of our team that had less experience with Git. Even with this difficulty, we found the added safety worthwhile.</p><h2 id="The-Good-Our-Upgrade-Process"><a href="#The-Good-Our-Upgrade-Process" class="headerlink" title="The Good: Our Upgrade Process"></a>The Good: Our Upgrade Process</h2><p>Our upgrade run list was beautifully simple:</p><ol><li>Make sure the environment is healthy and all the Application resources are in a healthy state</li><li>Merge any changes from the previous branch to the target branch (ex: develop to staging, or staging to production).</li><li>For each installation, modify the version (docker image tag) in the appropriate configuration yaml files and merge those.</li><li>Find the applications marked as out of sync in ArgoCD and trigger synchronization cycles. Wait for them to finish.</li></ol><p>Once we started the synchronization cycle, ArgoCD would start applying the changes. A few minutes later the new version would start up and the web services would start responding to requests again.</p><h2 id="The-Bad-Scaling-Pods-Without-Downtime"><a href="#The-Bad-Scaling-Pods-Without-Downtime" class="headerlink" title="The Bad: Scaling Pods Without Downtime"></a>The Bad: Scaling Pods Without Downtime</h2><p>The biggest drawback of this approach was the inability to make minor configuration tweaks to our production system through the code. ArgoCD uses a synchronization cycle to apply changes from the source. This was great when we were changing the version of the application and the schema migrator needed to run, but it wasn’t so great when we needed to add a little memory or increase the replica count to keep everything working smoothly.</p><p>In these cases we had to make changes to the Kubernetes resources directly, bypassing ArgoCD. This meant the Application resources would be marked as out of sync until we made the same changes in the code. If we forgot this step, the changes would get stomped during the next synchronization cycle.</p><p>ArgoCD has a feature to ignore certain state differences in a resource. This is great when you’re using autoscaling or other Kubernetes automation features. We couldn’t use it though because it also sometimes prevented ArgoCD from applying the changes to raise the replica count from 0 in the <code>Sync</code> phase.</p><p>This is only an issue when deploying applications that need downtime while the chart is being applied. Many applications that are designed to run it Kubernetes will keep working throughout this process, and ArgoCD can handle this just fine.</p><h2 id="The-Ugly-Automatic-Synchronization"><a href="#The-Ugly-Automatic-Synchronization" class="headerlink" title="The Ugly: Automatic Synchronization"></a>The Ugly: Automatic Synchronization</h2><p>ArgoCD is capable of triggering synchronization cycles automatically when it detects changes. This is helpful if you want to ensure your cluster’s state is always identical to your committed code, which is the ultimate in a GitOps workflow. The drawback is that it also means a synchronization cycle can be triggered whenever changes are detected. Since our process involves downtime, we didn’t want this to happen unintentionally in our production environments.</p><p>The other problem we ran into with automatic synchronization was that it made it harder to test minor configuration changes in our development environment. If we added or removed a bit of memory to measure the impact, ArgoCD would quickly reset it back. We could add parameters to allow these things to be configured, but that increased the complexity of the charts and made them harder to read. It also meant we had to remember te remove the same changes again later.</p><p>The setting for automatic synchronization is specified on a per-application basis via the resource definition, so you can make this behavior optional for some of your applications. We decided to use manual synchronization in our staging and production environments for everything but the top-level charts. This allowed us to control when changes were applied.</p><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>Using ArgoCD for automating the more complicated upgrade process worked well for us, even with its challenges. I would recommend this solution to others.</p><p>Another strong reason to use ArgoCD is that it is an excellent tool to use even if you don’t need to control the synchronization process. It was a great platform for us to deploy newer Kubernetes-native applications, and it was convenient to use the same tool for everything. It also left us in a position where we could iterate gradually to a simpler deployment process with our legacy applications.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I recently worked on a project to move a complicated legacy application onto Kubernetes. It was quite an undertaking, but in the end we w</summary>
      
    
    
    
    
    <category term="devops" scheme="https://jessemcdowell.ca/tags/devops/"/>
    
    <category term="kubernetes" scheme="https://jessemcdowell.ca/tags/kubernetes/"/>
    
    <category term="argocd" scheme="https://jessemcdowell.ca/tags/argocd/"/>
    
  </entry>
  
  <entry>
    <title>Importance of Alignment</title>
    <link href="https://jessemcdowell.ca/2023/06/Importance-of-Alignment/"/>
    <id>https://jessemcdowell.ca/2023/06/Importance-of-Alignment/</id>
    <published>2023-06-12T20:56:44.000Z</published>
    <updated>2024-08-07T05:04:17.799Z</updated>
    
    <content type="html"><![CDATA[<p>Unless you work entirely alone, alignment is a big deal. When you are well aligned with your company’s goals you are a more valuable employee. When you are well aligned with your manager they can keep you on the right track and be an ally against obstacles. When you are well aligned with your peers you can keep each other focused on the most important work.</p><p>On the other hand, when you have poor alignment you can see all sorts of problems. When teams are misaligned they can undermine each other’s efforts. When you are misaligned with your manager you can find yourself being over managed or left out to dry when things get rough. When you are not aligned with the goals of your organization you miss opportunities to demonstrate your skills and advance your career.</p><p>Alignment is such an important part of being successful in an organization that it’s important to understand what it looks like and regularly assess how healthy it is. If it starts to degrade, you should work to fix it, or you will find it gets harder and harder to be successful.</p><p>Ensuring good alignment with your manager is the most important. A good manager should help you find work that is rewarding, encourages development, and furthers the company’s goals. This is a hard job, and even the best manager in the world can’t do this if they don’t know what you enjoy and what your goals are. Make sure you are talking regularly with your manager and you have a good relationship. I can’t overstate how important it is to have a manager you trust that is helping you grow.</p><p>A manager that is honest with you when you’re doing poorly is also important. It’s impossible to grow without pushing yourself, and when you tackle new challenges it’s almost certain that you will make mistakes. If you don’t get negative feedback when you need it you can end up wasting a lot of time and energy, and ultimately damage your reputation.</p><p>In Vancouver &#x2F; Canadian culture a lot of managers (myself included) find it difficult to deliver negative feedback, but the alternative is so much worse. A manager that doesn’t communicate honestly and candidly may seem like they are supporting you and agree with your direction. This is like wearing a fake seat belt; when things go bad you will have no protection.</p><p>If you have a manager that can’t give you guidance because they don’t understand what you’re doing, you need to make sure they at least agree with the choices you are making. Be very careful here though, as some managers will say they agree with you without really meaning it, especially when the choices are complicated. You have to have make sure they really understand what you are asking and the implications of it. If they aren’t invested in the decisions, they may not back them up when you need it.</p><p>Another challenging situation is when your manager is too far removed from your work to have an opinion about it. In these cases you have to depend on the other kinds of alignment and do your best to manage and sell yourself. This becomes more likely the higher up you go in an organization.</p><p>Good alignment with your organization’s goals is also important, but this is impossible without a culture that supports it. People at all levels of an organization have to make decisions every day, communicate with customers and partners, and are constantly representing the brand. You and everyone else should know what the organization is trying to achieve and how you are personally contributing to it. When everyone isn’t working in the same direction it is easier to have miscommunications and disputes between teams.</p><p>The best teams I’ve worked on update everyone with recurring company or department-level meetings. Even if some of the information isn’t immediately or personally relevant, it all still soaks into the subconscious. Even just the exercise of gathering and presenting to the broader audience is valuable; it forces team leaders to understand the value they provide and makes sure they are measuring and delivering it.</p><p>Some companies don’t see the value of keeping everyone informed. If that’s the case where you work, you will have to take matters into your own hands. At the very least, make sure you read the company emails. Ask questions when you can, and try to always understand the direction your company is going.</p><p>Even when you have good alignment with your manager and a clear understanding of the organization’s goals, you may sometimes find yourself under a manager that isn’t well aligned themselves. Even if you are a perfect employee, the success of your team will reflect on you, and the team won’t be seen as successful if it isn’t helping the organization. The teams that are doing the best from this point of view tend to have the best bonuses and the best growth opportunities.</p><p>It’s not always possible to control the team you work on, sometimes the only way to improve things is to leave your company entirely. Of course there are lots of factors that go into a decision like that… but if you don’t feel that you, your team, and your company are going in the same direction, or if you can’t tell, you should at least be aware that it could be hurting your opportunities to grow and advance.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Unless you work entirely alone, alignment is a big deal. When you are well aligned with your company’s goals you are a more valuable empl</summary>
      
    
    
    
    
    <category term="people management" scheme="https://jessemcdowell.ca/tags/people-management/"/>
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
  </entry>
  
  <entry>
    <title>Polyglot Unconference 2023</title>
    <link href="https://jessemcdowell.ca/2023/05/Polyglot-Unconference-2023/"/>
    <id>https://jessemcdowell.ca/2023/05/Polyglot-Unconference-2023/</id>
    <published>2023-06-01T05:22:23.000Z</published>
    <updated>2024-08-07T05:04:17.799Z</updated>
    
    <content type="html"><![CDATA[<p>I recently had the pleasure of attending the 2023 Polyglot Unconference in Vancouver, put on by the <a href="https://www.polyglotsoftware.com/">Polyglot Software Association</a>. I’ve been attending these for years. It is my favourite local conference.</p><p>An <a href="https://en.wikipedia.org/wiki/Unconference">unconference</a>, sometimes called an open spaces conference, is a participant-driven event where attendees choose the topics of discussion and provide the content themselves. They are meant to be open and inviting, and build interpersonal relationships. This year was no exception.</p><p>The event started with a brief introduction and some ground rules, and then attendees began pitching sessions. Anyone could pitch a session, and then organize it however they wanted. This year I pitched a session about how to start a software company. I got on stage, gave my name, and explained what I wanted to talk about.</p><p>After all the sessions were pitched, attendees voted on the sessions they want to attend. Organizers then assigned them to rooms. All the sessions can usually be accommodated, so the voting is only used to make sure that the number of interested attendees can fit in the rooms they are assigned to.</p><p>For my session, I used a <a href="https://en.wikipedia.org/wiki/Fishbowl_(conversation)">fishbowl format</a>. I put 5 chairs on the stage. Only people in chairs could ask or answer questions, but anyone in the audience could take a chair at any time. The group on the stage had to ensure that one chair was always empty. What you end up with is an intimated discussion with an audience.</p><p>A few experienced founders attended my session, and quite a few people who wanted to or had already started software companies were present too. I asked my questions, other people asked their questions, and we got a lot of great answers. I took 3 pages of notes that will absolutely be helpful in my endeavors.</p><p>The best part of this event is learning about what other companies in town are doing, what’s working for them, and where they’ve had problems. Traditional software conferences tend to have more vendor-sponsored presentations where everything is a sales pitch. These are valuable too, but the unconference is a better way to get a balanced opinion.</p><p>I can’t wait to attend again next year.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I recently had the pleasure of attending the 2023 Polyglot Unconference in Vancouver, put on by the &lt;a href=&quot;https://www.polyglotsoftware</summary>
      
    
    
    
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
    <category term="learning" scheme="https://jessemcdowell.ca/tags/learning/"/>
    
    <category term="community" scheme="https://jessemcdowell.ca/tags/community/"/>
    
  </entry>
  
  <entry>
    <title>Sustainable Errors</title>
    <link href="https://jessemcdowell.ca/2023/05/Sustainable-Errors/"/>
    <id>https://jessemcdowell.ca/2023/05/Sustainable-Errors/</id>
    <published>2023-05-29T16:57:05.000Z</published>
    <updated>2024-08-07T05:04:17.798Z</updated>
    
    <content type="html"><![CDATA[<p>Making a program work for the happy path is not always easy, but given enough time I believe pretty much anyone could do it. When a professional takes on the task however they will make it work for more than just the happy path, and do it with code that is easy to debug, and easy for others to understand and change. Since so much of what we end up dealing with are exceptional flows, we need a concise way to deal with them. Fortunately we have the aptly named exception pattern.</p><p>When the pattern is used well it is almost invisible, and yet we should be thinking about it all the time. Sample code and simple apps often show exception handling as rote boilerplate that writes out stack traces and swallows errors. This is not a good example to be setting.</p><p>Most modern languages have an exception type and a throw statement. I’ll be using C#&#x2F;.Net terminology for this post, but the same or similar terms and patterns exist in Java, JavaScript, TypeScript, Python, and many other languages.</p><p>Most scripting languages (shell script, Windows batch, and sometimes PowerShell) and some low level languages like C don’t have exceptions. In these cases you have to check return codes every time you call a function or an external application, and it sucks. You don’t have to look far to find scripts filled primarily with error handling code. For the rest of us, there is something much easier and better.</p><h2 id="The-Exception-Pattern"><a href="#The-Exception-Pattern" class="headerlink" title="The Exception Pattern"></a>The Exception Pattern</h2><p>Here is an example of the exception pattern being used well:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="built_in">string</span> <span class="title">GetMagicString</span>()</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">using</span> <span class="keyword">var</span> reader = <span class="keyword">new</span> StreamReader(<span class="string">&quot;magic.txt&quot;</span>);</span><br><span class="line">    <span class="keyword">var</span> magicString = reader.ReadLine();</span><br><span class="line">    <span class="keyword">if</span> (String.IsNullOrEmpty(magicString))</span><br><span class="line">        <span class="keyword">throw</span> <span class="keyword">new</span> InvalidOperationException(<span class="string">&quot;magic.txt did not contain any text&quot;</span>);</span><br><span class="line">    <span class="keyword">return</span> magicString;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>If you don’t already know how exceptions work you can read about them in <a href="https://learn.microsoft.com/en-us/dotnet/csharp/fundamentals/exceptions/">Microsoft’s documentation</a>.</p><p>You may have noticed that no exceptions are being handled in the function, which is the point. This depends on the built-in exception that will be thrown being good enough. This is because I expect the file to be created by the installer in normal deployments. I could wrap this in a try&#x2F;catch block and do something, but what would I do? If you were about to say “return null” then you lose 10 points. You should always favour exposing problems quickly and clearly rather than ignoring them or pushing them down the line.</p><p>If I couldn’t expect the file to exist, if say <code>magic.txt</code> was a special override file that was only created in certain circumstances, I would write the function like this:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">public</span> <span class="built_in">string</span> <span class="title">GetMagicString</span>()</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">try</span></span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">using</span> <span class="keyword">var</span> reader = <span class="keyword">new</span> StreamReader(<span class="string">&quot;magic.txt&quot;</span>);</span><br><span class="line">        <span class="keyword">var</span> magicString = reader.ReadLine();</span><br><span class="line">        <span class="keyword">if</span> (String.IsNullOrEmpty(magicString))</span><br><span class="line">            <span class="keyword">throw</span> <span class="keyword">new</span> InvalidOperationException(<span class="string">&quot;magic.txt did not contain any text&quot;</span>);</span><br><span class="line">        <span class="keyword">return</span> magicString;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">catch</span> (FileNotFoundException)</span><br><span class="line">    &#123;</span><br><span class="line">        <span class="keyword">return</span> DefaultMagicString;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>Notice that I am only catching the specific file not found exception. I could have caught any exception, but that has its own dangers. Another possible exception is the <code>UnauthorizedAccessException</code>. If that happens, I’d rather notify the user than silently ignore the file.</p><p>I added a bit of guard code that can throw the <code>InvalidOperationException</code>. Even if it’s an unlikely problem, I prefer to surface problems like this earlier. They typically save time later when debugging. I’m keeping this code in the second example even while I have access to a default value because I think an empty file is more likely a mistake than intentional. It could be intentional for some use case I don’t know about, but it is always easier to ease restrictions than it is to add new ones in a deployed application, so I’d rather be more strict early.</p><p>The <code>using var</code> declaration is a relatively new C# feature that generates an implicit using block. It causes <code>reader</code> to be disposed before the function exits, be it successfully or a because of a thrown exception. Because the stream is opening and potentially locking a file, we want to make sure it is cleaned up quickly. Cleaning up resources in all cases is something we should always be mindful of as well, but in most modern languages you can use features like <code>using</code> to make this task similarly invisible.</p><h2 id="Global-Error-Handling"><a href="#Global-Error-Handling" class="headerlink" title="Global Error Handling"></a>Global Error Handling</h2><p>So where do exceptions get handled? Generally you should have an error handler somewhere, as high up the stack as is reasonable. You only have to write it once in one place, and reducing repetition has a lot of advantages. Your error handler can do any fancy stuff you want it to, and you can change the behavior of your error handling without making sweeping changes across your application.</p><p>Most of our applications are iterating on some atomic unit of work. Web servers iterate over web requests. Background processors iterate over messages in a queue or scheduled tasks. It makes sense to implement error handlers at the level where these are invoked, especially if there is a good chance that another similar unit of work could succeed.</p><p>In a web api the error handler can usually be added as a wrapper around every request through some hook in the framework. Common implementations will log the exception and return a 500-level status code. Fancier implementations will return more nuanced codes depending on the type of the exception (like return a <code>400</code> for <code>ArgumentException</code> or anything derived from it). Your web framework of choice probably already does this for you.</p><p>Some errors, however, aren’t recoverable. For example, without complicated retry logic, many applications will be in a bad state if their startup logic fails somehow. You might have one global handler for all your startup code, but its behavior should probably be to output some useful diagnostic information and exit. There may be other errors that indicate poor application health beyond startup, but this is often pretty challenging to deal with reliably.</p><p>.Net also has a few special exceptions that can’t be caught normally. Most exceptions mean that the operation you attempted failed, but these uncatchables indicate that the runtime environment could now be in a bad state. The dreaded <code>StackOverflowException</code> is an example of this. In these cases exiting and getting restarted is the only safe thing to do. The framework is going to do that regardless of your error handling so you don’t need to add special logic for it.</p><h2 id="Throwing-Good-Exceptions"><a href="#Throwing-Good-Exceptions" class="headerlink" title="Throwing Good Exceptions"></a>Throwing Good Exceptions</h2><p>The best thing to do when you detect unexpected conditions is often throwing an exception. The exception message should clearly describe what the problem is. Remember that you or one of your peers will be reading this message about 6 months from now when a test fails or a bug gets reported. Sometimes I like to include advice about fixing the problem in the exception, but use this sparingly; it’s much less helpful when the advice has become outdated and sends users on wild goose chases.</p><p>It sometimes makes sense to create a new exception type for your error cases. I can’t remember where I once read that exceptions should have the same level of abstraction as the interface that throws them… It’s not terrible advice, but I don’t think it’s worth the trouble for most cases. If you’re building a framework it might make sense. If you’re checking for an empty file after reading its contents it may not.</p><p>One case where I do create custom exceptions is when I’m writing a unit test. For really trivial stuff (like argument null) you probably don’t need a unit test, but for anything that should be tested, use a custom exception. If you’re asserting on any exception being thrown your test could be marked as passing when it should be failing. If you derive your custom exception from a built-in or another more common exception, your callers won’t normally need to handle your specific type, but they could if they wanted to.</p><p>If you are not creating your own type, try to use a specific and appropriate exception type, either built-in or custom. <code>InvalidOperationException</code> and <code>ArgumentException</code> give better hints the developer in the future than a simple <code>Exception</code>.</p><p>It’s also a good practice to include the inner exception whenever you generate a new exception in response to a failure lower in the stack. These details can be very helpful if the new exception is ever thrown unexpectedly.</p><h2 id="What-About-Performance"><a href="#What-About-Performance" class="headerlink" title="What About Performance?"></a>What About Performance?</h2><p>One common argument against throwing exceptions is that it causes a performance hit. It’s true, it does takes time to capture the stack trace, and allocating a new object on the heap isn’t free. It depends on your circumstance of course, but I think in 99% of scenarios this cost is too miniscule to matter, and certainly far cheaper than the wasted time of developers who can’t find bugs.</p><p>If your exceptional case is something you expect to hit often inside a tight loop though, it may actually matter. The <code>TryFunction</code> pattern is the common alternative in .Net. The drawback is that because your return value is typically boolean you can’t easily add new failure cases as safely as you can with exceptions. You could include more information in another output parameter, but that can also impact your callers whenever the list changes. It also means every caller needs to check the function response, or even worse, they can forget to and the app can continue running in a weird state without realizing the call failed.</p><h2 id="What-About-Leaking-Security-Sensitive-Information"><a href="#What-About-Leaking-Security-Sensitive-Information" class="headerlink" title="What About Leaking Security Sensitive Information?"></a>What About Leaking Security Sensitive Information?</h2><p>Another argument I’ve heard against good error handling is to prevent leaking information to attackers, but I think this is mostly bad advice too. For any software that is distributed, a malicious user can easily find tools to look inside and see how it’s wired. For server-side code you should at least share the error details with yourself in your own log files. If you need to withhold information from potential hackers then do that in your global error handler.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Making a program work for the happy path is not always easy, but given enough time I believe pretty much anyone could do it. When a profe</summary>
      
    
    
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term=".net" scheme="https://jessemcdowell.ca/tags/net/"/>
    
    <category term="c#" scheme="https://jessemcdowell.ca/tags/c/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
    <category term="quality" scheme="https://jessemcdowell.ca/tags/quality/"/>
    
  </entry>
  
  <entry>
    <title>Is the Bug Fun?</title>
    <link href="https://jessemcdowell.ca/2023/05/Is-the-Bug-Fun/"/>
    <id>https://jessemcdowell.ca/2023/05/Is-the-Bug-Fun/</id>
    <published>2023-05-15T17:47:52.000Z</published>
    <updated>2024-08-07T05:04:17.797Z</updated>
    
    <content type="html"><![CDATA[<p>There are many things about producing video games that are surprising, but one of the weirdest has to be the approach to bugs. Like any piece of software, bugs are found through testing or user reports, triaged, then assigned to developers. Unlike normal business software they also ask the question, “is the bug fun?”</p><p>There are plenty of unintended features (bugs) in games that became beloved. Attack combos were an accident in Street Fighter II, but they became so popular that they are a part of basically every fighting game now. Rocket jumps are another example. The internet is full of examples.</p><p>Sometimes very glitchy games can be fun too, especially for a certain audience. Speed runners sometimes use glitches to lower their times. People love games for all sorts of reasons beyond just beating them and getting high scores. At the end of the day the goal of a game is to entertain more than be correct.</p><p>The next time a bug comes across your desk, maybe ask yourself if fixing it would make your app less fun.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;There are many things about producing video games that are surprising, but one of the weirdest has to be the approach to bugs. Like any p</summary>
      
    
    
    
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="quality" scheme="https://jessemcdowell.ca/tags/quality/"/>
    
    <category term="bugs" scheme="https://jessemcdowell.ca/tags/bugs/"/>
    
    <category term="video games" scheme="https://jessemcdowell.ca/tags/video-games/"/>
    
  </entry>
  
  <entry>
    <title>Case of the Disappearing Users</title>
    <link href="https://jessemcdowell.ca/2023/05/Case-of-the-Disappearing-Users/"/>
    <id>https://jessemcdowell.ca/2023/05/Case-of-the-Disappearing-Users/</id>
    <published>2023-05-01T15:26:25.000Z</published>
    <updated>2024-08-07T05:04:17.797Z</updated>
    
    <content type="html"><![CDATA[<p>Many years ago I worked on a program that had a serious problem: the users in one customer’s system were getting deleted periodically. When a user was deleted, any data linked with them was also deleted. We could restore the data from backups, but it was a difficult process, and having a system that loses data wasn’t great for our reputation, so we wanted to resolve it quickly. Our VP of development tried to find the issue first, but after a day without any progress he assigned the issue to me.</p><p>I asked some questions and tried my usual tricks. There wasn’t an error message or anything helpful in the logs. There didn’t seem to be any obvious place to start, so I did something crazy: I searched the entire codebase for the world <code>DELETE</code>.</p><p>It seemed likely to me that the user was getting deleted by the application, so there had to be a delete statement somewhere that was the cause.</p><p>One of the biases I’ve noticed in myself is that I tend to drastically over-estimate the time it will take to perform mundane tasks. In reality, even if there were a few thousand delete statements in the code (and there weren’t), it would still be possible to review them all in an afternoon. Using a good IDE that has a preview for search results and a bit of care with search terms it was possible to get through them even faster.</p><p>I found the offending delete statement. The bug was actually a feature!</p><p>The customer had re-installed the software at some point and couldn’t find their licence key. The software wouldn’t work without a licence unless it was put into “demonstration mode.” Demonstration mode caused users to be deleted after the trial period ended.</p><p>We issued them a new licence key and changed the demonstration mode feature so it wouldn’t be so destructive if it ever got turned on in the future.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Many years ago I worked on a program that had a serious problem: the users in one customer’s system were getting deleted periodically. Wh</summary>
      
    
    
    
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="war stories" scheme="https://jessemcdowell.ca/tags/war-stories/"/>
    
  </entry>
  
  <entry>
    <title>How to Fix a Bug</title>
    <link href="https://jessemcdowell.ca/2023/04/How-to-Fix-a-Bug/"/>
    <id>https://jessemcdowell.ca/2023/04/How-to-Fix-a-Bug/</id>
    <published>2023-04-24T15:50:00.000Z</published>
    <updated>2024-08-07T05:04:17.796Z</updated>
    
    <content type="html"><![CDATA[<p>Building applications can be tricky, and it’s inevitable that mistakes will be made. As a result, we programmers spend a lot of time fixing bugs. Sometimes they are easy, but sometimes they can be pretty tough to figure out.</p><p>I’ve fixed a lot of bugs in my career, and to be honest with you, I usually enjoy the process. These days I am typically assigned the super urgent bugs that nobody else can figure out, and I kind of like it that way. I don’t get me wrong, I don’t like the bugs being there, but I enjoy being helpful and figuring out tough problems. I also think my successes have helped improve my reputation which is always a good thing.</p><p>This can take a lot of hard work, but I’ve made the job considerably easier through a set of tricks and a process I’ve developed over my career. I never thought of these as particularly special, or as a secret weapon, but I realize now that they aren’t really taught. They are important though, so I wrote them down.</p><p>This is the process I generally follow when I’m approaching a bug. I will obviously not go through a formal process for the super simple and obvious ones, but for the toughest nuts, all the steps here will help me get it over the finish line.</p><h2 id="Step-1-Read-the-Bug-Report"><a href="#Step-1-Read-the-Bug-Report" class="headerlink" title="Step 1: Read the Bug Report"></a>Step 1: Read the Bug Report</h2><p>When a bug report comes across my desk, I read it and make sure that I understand it. Fixing a bug properly takes time and carries its own risks, so I want to be sure I’m fixing the right bug.</p><p>If I can’t understand the bug report, this is also a good place to stop and request more information. I wrote <a href="/2023/03/How-to-Report-a-Bug/">another post about writing good bug reports</a>.</p><h2 id="Step-2-Reproduce-the-Problem"><a href="#Step-2-Reproduce-the-Problem" class="headerlink" title="Step 2: Reproduce the Problem"></a>Step 2: Reproduce the Problem</h2><p>The most important step to fixing a bug is reproducing it. Even if the problem seems obvious, even if I am feeling particularly lazy, I force myself to do it anyway. If I can’t, I have no way to test my fix later. It also means I won’t be able to use a debugger to figure out what is happening when it fails.</p><p>Sometimes it’s possible to write an automated test before finding the problem, and if I can, I will. It makes debugging and testing considerably faster. In complicated software this can even be faster than starting up all the required pieces to test an issue manually.</p><p>It can be difficult to reproduce an intermittent issue, but I still put in the effort for the same reasons. I’ve sometimes had luck wrapping unit tests in a for loop, but it won’t help if the causes are environmental.</p><p>There have been rare occasions when I’ve had to attempt fixing a bug before I can reproduce it, but I will only let this happen in extreme circumstances. When doing this kind of thing I also make sure to communicate it clearly with everyone involved.</p><h2 id="Step-3-Find-the-Problem"><a href="#Step-3-Find-the-Problem" class="headerlink" title="Step 3: Find the Problem"></a>Step 3: Find the Problem</h2><p>This can be the most infuriating part of fixing bugs, but it is essential. There are a bunch of techniques that can work here. For this post I’ll stick to the tricks I use the most.</p><p>The first thing I do is read the error details again. If there is an error message in the bug report it can carry a lot of clues. It’s tempting to skim over it, and I have made this mistake plenty of times myself. Now I make sure I not only read the error message, but also understand it. This has saved me a lot of time. System errors can be especially helpful since they tend to be specific. Stack traces are also incredibly valuable. If I don’t understand what an error code means I look it up.</p><p>In the absence of a stack trace, I like to narrow down where the problem is occurring. I start by visualizing all the steps through the system that are involved in the malfunctioning operation, then I choose some point in the middle. Using my trusty debugger, I test if it’s failed at that point. I continue along in a binary search pattern until I have narrowed down the problem to a specific spot.</p><p>For example: if I’m fixing a bug in a web application where a user’s name is getting saved incorrectly I can start by checking the web request from my browser’s developer tool. If the request is wrong, I know the bug is on the client side. If the request is correct, the bug will be somewhere in the server. Assuming it’s in the server, I might set a breakpoint between my business layer and repository layer to narrow it down further. Continuing in this way I can find the exact location of the bug quickly and reliably.</p><p>It works for more than just software problems too. I’ve used the same approach to diagnose load balancer problems, problems with components in Kubernetes, computer hardware, even electrical wiring.</p><p>In some cases it’s easier to figure out what change introduced the bug instead. I will use the same kind of binary search pattern but checking out commits between releases. A quick build time and a unit test makes this a lot less painful. I’ve never had a bug where it was a good fit, but you can also try the <code>git bisect</code> command to automate the process fully.</p><p>If I ever get stuck in my investigation, I go back to the beginning and re-check all my assumptions. Did I read the bug right? Was the process in a healthy state when it occurred? Am I looking at the right version of the code? Did the feature ever work? Did I misclassify a success or failure when I did the binary search? Even if all my assumptions were correct, going through the problem again can sometimes spark new theories for investigation. This is also where I start when someone else asks me for help with their bugs.</p><p>If all else fails, or even if it hasn’t, searching on the internet can help. This is especially true for third party components or services. Be careful though, I am finding internet resources to be increasingly less helpful but your results may vary. Even though I’ve wasted a lot of time following red herrings from random internet threads, this is still sometimes the best option available.</p><h2 id="Step-4-Test-My-Theory"><a href="#Step-4-Test-My-Theory" class="headerlink" title="Step 4: Test My Theory"></a>Step 4: Test My Theory</h2><p>I find it easy to jump to conclusions when I’m debugging, but my experience has taught me to approach bugs with a scientific kind of skepticism. Once I am pretty sure I know what the problem is, if it’s not a trivial change, I like to isolate it and prove to myself that I understand it.</p><p>I will write an automated test that reproduces the problem if at all possible. It might seem quicker to fix the bug first, but starting with the test will make the process simpler. Writing a failing test proves that you understand the bug, and it also proves that the test will fail if you don’t successfully fix it.</p><p>If you can’t write a test because that isn’t your team’s practice then you have my sympathies. On the bright side, you’ll be getting a lot more experience fixing bugs!</p><h2 id="Step-5-Fix-the-Problem"><a href="#Step-5-Fix-the-Problem" class="headerlink" title="Step 5: Fix the Problem"></a>Step 5: Fix the Problem</h2><p>I will remove some bad code and &#x2F; or put some more good code in.</p><h2 id="Step-6-Look-for-Similar-Bugs"><a href="#Step-6-Look-for-Similar-Bugs" class="headerlink" title="Step 6: Look for Similar Bugs"></a>Step 6: Look for Similar Bugs</h2><p>Sometimes a bug indicates a pattern of bugs. Before I fling my fix back out into the world I like to do a little research to see if the same mistake has been made in other places.</p><p>For example: if I was fixing a bug caused by a query operator that isn’t supported by an older database server version, I can do a quick search to see if the operator was used anywhere else. Since I’ve already figured out how to fix it, I can fix them all at the same time and eliminate a whole bunch of bugs.</p><h2 id="Step-7-Test-My-Fix"><a href="#Step-7-Test-My-Fix" class="headerlink" title="Step 7: Test My Fix"></a>Step 7: Test My Fix</h2><p>Since I almost always have automated tests, I can check if I’ve fixed the problem pretty easily.</p><p>I’ll usually do a manual test as well to make sure I really have fixed the issue. Sometimes a bug has more than one cause, or is repeated in more than one place. To be honest, I often find this step tedious, so I have to remind myself why it’s important. A lot of time can get wasted if there was some aspect I missed. Also, if I’m going to put my name on a fix, I want people to be able to depend on it actually being fixed.</p><h2 id="Step-8-Understand-the-Bug’s-Impact"><a href="#Step-8-Understand-the-Bug’s-Impact" class="headerlink" title="Step 8: Understand the Bug’s Impact"></a>Step 8: Understand the Bug’s Impact</h2><p>It depends on the bug, but if there is some damage left behind, I make sure to consider its impact. Sometimes this takes a bit of experimentation, but it is important. Sometimes a cleanup script is necessary, or sometimes manual steps can be provided to correct the issue. At the very least I want to make sure I can tell my stakeholders what the impact was.</p><h2 id="Step-9-Bug-Retrospective-Post-mortem"><a href="#Step-9-Bug-Retrospective-Post-mortem" class="headerlink" title="Step 9: Bug Retrospective (Post-mortem)"></a>Step 9: Bug Retrospective (Post-mortem)</h2><p>Once in a while I take a bit of time to reflect on the bugs I’ve encountered. How did the bug escape in the first place? Is it likely that similar bugs will be introduced again? Can I introduce tools or change processes to make this class of bug less likely to occur?</p><p>Some organizations have a formal post mortem process for impactful issues. This is a great way to ensure a team is learning from its mistakes. I have introduced this process in a few of my teams and highly recommend it.</p><p>Even for bugs with less impact it can be worth spending a bit of time thinking about this. It’s not always feasible to prevent some types of bugs, but as craftspeople we should be trying!</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Building applications can be tricky, and it’s inevitable that mistakes will be made. As a result, we programmers spend a lot of time fixi</summary>
      
    
    
    
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
    <category term="quality" scheme="https://jessemcdowell.ca/tags/quality/"/>
    
    <category term="bugs" scheme="https://jessemcdowell.ca/tags/bugs/"/>
    
  </entry>
  
  <entry>
    <title>How to Report a Bug</title>
    <link href="https://jessemcdowell.ca/2023/03/How-to-Report-a-Bug/"/>
    <id>https://jessemcdowell.ca/2023/03/How-to-Report-a-Bug/</id>
    <published>2023-03-11T05:40:00.000Z</published>
    <updated>2023-03-12T02:03:00.000Z</updated>
    
    <content type="html"><![CDATA[<p>Nobody likes bugs, least of all programmers. No matter how hard we try to catch them early, some will always escape into circulation. Until computers are smart enough to do what we meant instead of what we said, users are going to keep finding bugs, and we’re going to keep fixing them.</p><p>Before a bug is fixed, it needs to be reported. Unfortunately it’s not uncommon to receive incomplete reports. We can spend a lot of time hunting and making guesses, and sometimes that’s enough, but if we can’t figure out the problem it’s pretty hard to fix it. This can be especially unfortunate when the stakes are high, and oddly, this is when it also seems to be the most common.</p><p>So what does a bad bug report look like?</p><blockquote><p>The website crashed.</p></blockquote><p>I get this kind of report now and then, even from experienced members of our support teams. I almost always need to send it back to get more information.</p><p>If I got this report instead, I wouldn’t (usually) need more information:</p><blockquote><p>When I open the store website I get a 502 Bad Gateway error.</p></blockquote><p>When I’m trying to understand a bug, I need to know four things:</p><ul><li>How did you encounter the bug? (Steps to Reproduce)</li><li>What happened? (Actual)</li><li>What should have happened? (Expected)</li><li>Where did you encounter the bug? (Location)</li></ul><p>Here is another example of a bug that would be hard to figure out:</p><blockquote><p>I can’t delete a user.</p></blockquote><p>It only answers one of the questions (Expected). I might be able to guess what the user tried to do (Steps to Reproduce), but there could be multiple ways to do this. It doesn’t tell me what application was being used (Location) or what happened (Actual). Was there an error message? Did the application close unexpectedly? Did the computer explode in a ball of fire? I can help with most of these, but they all get approached differently.</p><p>If you want a gold star, this is a more formal way to report the same bug, and the format I typically use myself:</p><blockquote><p>Steps to Reproduce:<br>1. Go to the user list<br>2. Click a user<br>3. Click delete<br>    * Receive successfully deleted notification<br>4. Return to user list<br>Actual:<br>* User appears in the list<br>* When I click on the user again, I get the error “User not found”<br>Expected:<br>* User does not appear in the list<br>Location:<br>* Admin site: <code>https://&#123;someserver&#125;/admin/users</code></p></blockquote><h2 id="How-did-you-encounter-the-bug-Steps-to-Reproduce"><a href="#How-did-you-encounter-the-bug-Steps-to-Reproduce" class="headerlink" title="How did you encounter the bug? (Steps to Reproduce)"></a>How did you encounter the bug? (Steps to Reproduce)</h2><p>The first step to fixing a bug is trying to reproduce it. Exact steps can help a lot with this. Sometimes it isn’t enough, but it’s even more helpful when there are other factors involved.</p><h2 id="What-happened-Actual"><a href="#What-happened-Actual" class="headerlink" title="What happened? (Actual)"></a>What happened? (Actual)</h2><p>A detailed, unemotional description of what happened is the most important part of a bug report. If there in an error message, give us the exact text. It may not make sense to you, but this can tell us a lot about what is going on. If possible, copy and paste the exact text of the error. A screenshot can also be really helpful.</p><p>Expressions like “crashed” or “failed” or “doesn’t work” sound good, but they don’t tell us what happened. You don’t need to use technical words if you don’t know them, but you should avoid interpreting the information. For example “an error message appeared that said …” is much more helpful than “it failed”.</p><h2 id="What-should-have-happened-Expected"><a href="#What-should-have-happened-Expected" class="headerlink" title="What should have happened? (Expected)"></a>What should have happened? (Expected)</h2><p>Sometimes this is obvious. Generally when someone reports an error message we understand they expect not to get an error. Sometimes the expected behavior is not so obvious.</p><p>Keep in mind that programmers are trained in programming, and are usually not experts in the fields where our programs are being used. For example, if there’s a problem with the way tax is being calculated in an accounting application, the people looking at the bug may know less than you do on the subject. For technical bugs and miscalculations it can also be helpful to include how you determined the result you expected.</p><h2 id="Where-did-you-encounter-the-bug-Location"><a href="#Where-did-you-encounter-the-bug-Location" class="headerlink" title="Where did you encounter the bug? (Location)"></a>Where did you encounter the bug? (Location)</h2><p>It’s not always obvious, but sometimes companies have more than one piece of software, or more than one version in circulation.</p><p>If you’re using a website, include the URL. If it’s a visual bug, maybe include the browser you’re using, it’s version, and the operating system you’re using it on. If it’s a mobile app, include the kind of phone you have. If the app has a version number include that too.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Nobody likes bugs, least of all programmers. No matter how hard we try to catch them early, some will always escape into circulation. Unt</summary>
      
    
    
    
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
    <category term="quality" scheme="https://jessemcdowell.ca/tags/quality/"/>
    
    <category term="bugs" scheme="https://jessemcdowell.ca/tags/bugs/"/>
    
  </entry>
  
  <entry>
    <title>Breaking Past Senior Developer</title>
    <link href="https://jessemcdowell.ca/2021/01/breaking-past-senior-developer/"/>
    <id>https://jessemcdowell.ca/2021/01/breaking-past-senior-developer/</id>
    <published>2021-01-30T03:45:00.000Z</published>
    <updated>2021-01-30T07:01:00.000Z</updated>
    
    <content type="html"><![CDATA[<p>Developing software is an excellent career. Software has touched almost every aspect of our world, and its impact is always expanding. Many new things have become possible because of software, things that couldn’t have been dreamed of even ten years ago. The industry is continuing to expand. Tools are getting better. New opportunities are appearing everywhere… So why haven’t you gotten a promotion in ten years?</p><p>In the early days of my career, I got new responsibilities, promotions, and raises fairly regularly. It took a bit of luck, a lot of hard work, and a few years (but not very many years), to work my way up to a senior developer position. Senior means different things at different places, but eventually I got to a place where there was no easy next step, and I had a good number of peers in exactly the same position.</p><p>I can now proudly tell you that I have broken past the wall. About a year ago I achieved my goal and got promoted to the role of system architect. The following are the things that made the difference for me.</p><h2 id="Choose-the-career-path-you-want"><a href="#Choose-the-career-path-you-want" class="headerlink" title="Choose the career path you want"></a>Choose the career path you want</h2><p>There seems to be three distinct paths forward from senior developer:</p><ul><li>people and project management</li><li>technical leadership</li><li>focused technical expert</li></ul><p>The focused technical expert path only exists in companies that need experts focused in very specific areas such as hardware manufacturers or OS &#x2F; platform companies. If you are at the top of one of these specialties, you probably already work for the company where you’ll be getting promoted, and this post won’t really apply to you.</p><p>Both management and technical leadership are fine choices, but you should decide which is right for you. Here are some important differences to help you make your choice:</p><ul><li>Technical leadership positions are harder to get. There aren’t as many of them, and it seems they are more sought after.</li><li>Management positions will eventually strangle out all time for development. At some point you may not be able to go back.</li><li>Technical leadership positions require making harder technical choices based on research and experimentation.</li><li>Management positions require making harder business decisions based on company and customer priorities.</li></ul><p>Either way, they are both big changes from full-time development. You will be leaning heavily on your soft skills, spending more of your time in meetings and writing a lot of documents. You will typically have less control over the things you focus on, and your success will depend more heavily on the work of others.</p><p>If all this sounds awful to you, it is entirely fine to stay where you are. You can spice things up by changing projects or companies. You may want to scale up your impact some day, but until then, do what you love. The only unfortunate consequence is that you will likely be stuck in your current salary range.</p><h2 id="Develop-your-hard-skills-and-your-technical-breadth"><a href="#Develop-your-hard-skills-and-your-technical-breadth" class="headerlink" title="Develop your hard skills and your technical breadth"></a>Develop your hard skills and your technical breadth</h2><p>Both leadership and management positions will favour generalists because you’ll typically have responsibility for far more parts of your system. If you’re after a management position, a basic understanding of different problem areas will be enough. If you want to be a technical leader, make sure you have some success stories in a few areas. If you’ve only ever worked in the back end, for example, see if you can get some time working in the front of the house.</p><p>Technical leaders are going to need to be up to date on all the latest technologies and be able to discuss them intelligently. If learning and working with new things isn’t possible where you work, it may be time to consider changing companies. Contracting helped me expand my portfolio dramatically in just a few years, but it’s not a path I’d necessarily recommend to anyone.</p><h2 id="Find-the-right-company-and-manager-to-help-you-advance"><a href="#Find-the-right-company-and-manager-to-help-you-advance" class="headerlink" title="Find the right company and manager to help you advance"></a>Find the right company and manager to help you advance</h2><p>It’s always important to have a good relationship with your manager, but this is especially true if you want to advance your career. Make sure your manager knows what your goal is, and make sure they are supportive. A good manager will tell you what areas you need to improve, represent you when the right opportunity appears internally, or vouch for you if you find one outside.</p><p>Make sure your company has the role you want. Very small companies will probably not need a software architect, and may not have many openings for new managers. If you’re growing quickly it may appear in the future, but ask your manager. If you’re valued enough, they might even create the position for you when it makes sense.</p><p>If you’re in a place where advancing isn’t possible, or you aren’t getting the support you need, you may have to make the difficult choice and move on. When considering new roles, think about how they can help you work toward your goal. It’s also totally fine, maybe even advantageous, to tell a potential employer what you’re working toward, and ask if it’s possible with them.</p><h2 id="Develop-your-soft-skills"><a href="#Develop-your-soft-skills" class="headerlink" title="Develop your soft skills"></a>Develop your soft skills</h2><p>In your new role you’re going to need to juggle more priorities, build consensus with more people, and convince everyone that it’s all under control. You won’t need to be amazing at this on your first day, but you won’t get the job if they don’t think you can do it.</p><p>Here are some things to work on:</p><ul><li>Find your own personal organization system and get comfortable with it.</li><li>Study development process and team management. There are lots of great books, blogs, videos, and classes.</li><li>Give more presentations, and put effort into improving them.</li><li>If you aren’t already, see if you can become someone’s manager.</li><li>Try coaching &#x2F; mentoring someone who is struggling.</li><li>Run a study group or a book club.</li></ul><h2 id="Develop-your-personal-brand"><a href="#Develop-your-personal-brand" class="headerlink" title="Develop your personal brand"></a>Develop your personal brand</h2><p>I think a lot of us developers lack awareness of our personal brand, but it’s an important part of your career, and it’s importance increases the higher up you go. Make sure people want to work with you, and try to get some high-profile successes under your belt. These will all help you earn a promotion, and give you essential credibility once you get it.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;Developing software is an excellent career. Software has touched almost every aspect of our world, and its impact is always expanding. Ma</summary>
      
    
    
    
    
    <category term="people management" scheme="https://jessemcdowell.ca/tags/people-management/"/>
    
    <category term="leadership" scheme="https://jessemcdowell.ca/tags/leadership/"/>
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
  </entry>
  
  <entry>
    <title>Brewing Your Own Iced Tea</title>
    <link href="https://jessemcdowell.ca/2018/06/brewing-your-own-iced-tea/"/>
    <id>https://jessemcdowell.ca/2018/06/brewing-your-own-iced-tea/</id>
    <published>2018-06-14T04:29:59.000Z</published>
    <updated>2024-08-07T05:04:17.794Z</updated>
    
    <content type="html"><![CDATA[<p>There are few things more refreshing than a cold glass of good iced tea. I’ve tried iced tea from a lot of places, but the best I’ve had to this day is my own recipe. It’s so easy that I can’t in good conscience keep it secret. It’s also far cheaper than anything you can buy in a can or bottle, and a lot healthier because it doesn’t require any kind of sweetener. The only down side is that it takes a few hours to cool.</p><p>The short version: Make good tea with boiling water, then let it cool slowly before serving.</p><p>The long version:</p><p>Any decent black tea should work, but my favourite is loose Twinings Earl Grey (<a href="https://www.amazon.com/Twinings-Classics-Earl-Grey-Loose/dp/B0005ZXX1S/ref=sr_1_42?s=grocery&ie=UTF8&qid=1375601004&sr=1-42">Amazon.com</a>). The loose version tastes quite different than their tea bags, so do put in the effort to find it.</p><p>You will also need a fine strainer to remove the tea. I use a cloth tea sock (<a href="https://www.amazon.com/dp/B002U77176/ref=sxts_k2p-hero-vn_lp_3?pf_rd_m=ATVPDKIKX0DER&pf_rd_p=8011851592090061987&pd_rd_wg=wF4KQ&pf_rd_r=THGQ3HKM51EH9K3JW6Y2&pf_rd_s=desktop-sx-top-slot&pf_rd_t=301&pd_rd_i=B002U77176&pd_rd_w=e9sIM&pf_rd_i=tea+sock&pd_rd_r=63759cf2-6f82-4d78-840b-edb729e1d77b&ie=UTF8&qid=1527138358&sr=3">Amazon.com</a>).</p><p>Also, be careful that you brew the tea in a container that can handle boiling water. A heat-safe glass pitcher is nice, but a regular metal pot from your kitchen will do just as well. Once it’s down to room temperature, you can transfer it to any container you like.</p><p>I add 25 mL of loose tea for every 1 L of boiling water. Make sure the water is actually at a rolling boil. The hot tap on your water cooler is not going to work for this.</p><p>Set a timer for 5 minutes. At about 2 minutes, stir the tea. At 5 minutes, remove the tea strainer gently without stirring. The tea seems to produce the most flavour about two minutes in, and the most bitterness at the end of the steeping. Stirring only the once yields the smoothest flavour for me.</p><p>Let the tea cool slowly to room temperature, then put it in the fridge to cool the rest of the way. Adding ice while it’s hot can disturb the flavour, and dilutes the tea.</p><p>Once it’s cold, pour a glass, add ice if you like, and enjoy.</p><p>You can experiment with different teas. Green or white tea should work, but follow their proper brewing instructions. Only black tea should be brewed with boiling water. My second favourite mix is using a plain black orange pekoe, inserting a couple springs of fresh mint after removing the tea.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;There are few things more refreshing than a cold glass of good iced tea. I’ve tried iced tea from a lot of places, but the best I’ve had </summary>
      
    
    
    
    
    <category term="food" scheme="https://jessemcdowell.ca/tags/food/"/>
    
    <category term="tea" scheme="https://jessemcdowell.ca/tags/tea/"/>
    
    <category term="recipes" scheme="https://jessemcdowell.ca/tags/recipes/"/>
    
  </entry>
  
  <entry>
    <title>Infiltrating an Organization (or: Joining a New Team)</title>
    <link href="https://jessemcdowell.ca/2016/08/infiltrating-an-organization/"/>
    <id>https://jessemcdowell.ca/2016/08/infiltrating-an-organization/</id>
    <published>2016-09-01T05:23:42.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>It takes some time to integrate into a new team. I always feel like an outsider at first. As I build friendships and trust, I’m able to contribute with increasing effectiveness. Having noticed some patterns, I’ve been able to make the process faster and smoother using a few simple tricks.</p><p><a href="https://en.wikipedia.org/wiki/Tuckman%27s_stages_of_group_development">Tuckman’s Stages of Group Development</a> describe what happens when a team is formed. His theory has four stages: forming, storming, norming, and performing. As I stared writing this post, I noticed that the stages I was describing lined up fairly well with his. It’s important to note that I’m talking about joining an existing team, where he talks about a team being formed entirely from new people.</p><p>I will relate the stages I’ve noticed using his labels, but leave it as an exercise for the reader to compare to his more detailed descriptions.</p><h2 id="1-Forming"><a href="#1-Forming" class="headerlink" title="1) Forming"></a>1) Forming</h2><p>When I first join a team, everyone is as clueless about me as I am about them. Anything I can do to facilitate interaction here is valuable. I like to set out a couple trinkets on my desk, things that show my personality and interests. This creates openings for others to start conversations, which can help to break the ice. It can be awkward and uncomfortable as the “new guy”, but getting through this sooner really helps speed things up.</p><p>I also tend to ask a lot of questions while I’m getting my bearings. The hard part here is gathering information without judging or commenting too much. Honesty is important, but this early in the process, it can be more damaging than helpful. It’s easy to forget that things were built with constraints and assumptions I wasn’t there to experience.</p><p>That being said, looking things over with a fresh set of eyes can uncover many interesting things. So that I don’t lose track of them, I write every idea and observation down and revisit them later.</p><h2 id="2-Storming"><a href="#2-Storming" class="headerlink" title="2) Storming"></a>2) Storming</h2><p>After a month or two, I start getting used to what’s going on, and the problems the team is solving. The things that seemed strange before may now be routine, but some will still get in my way. My productivity improves, but I don’t feel like a full member of the team yet.</p><p>This is when I start going through my list of suggestions. I remove the items that no longer make sense, and then prioritize the remainder. I work through my list slowly, applying gentle pressure, trying to ask questions of various people on the team. Sometimes the questions receive insightful answers, other times they provoke good changes. Occasionally they uncover issues with team dynamics or lost political battles. All of these outcomes are valuable in different ways.</p><p>Essentially, I am trying to develop my voice. By waiting until I’ve been in the team a bit, I have a much firmer foundation to launch from. Being honest, and being able to disagree respectfully are necessary to fully contributing to the team. Just remember to take it slowly. If you come on too strong too suddenly you can alienate your coworkers.</p><p>With time, everyone gets used to me and my style. Once they realize that I can challenge opinions without judging their source, things become a lot smoother.</p><h2 id="3-Norming"><a href="#3-Norming" class="headerlink" title="3) Norming"></a>3) Norming</h2><p>After a while, I run out of questions. At this point, I will have a pretty good understanding of the architecture, the history and opinions that shaped it, and have a rough idea where people stand on the important issues. Others should have a good idea where I stand too.</p><p>This is when I start fine-tuning my personal processes, and trying to resolve anything that’s still holding me back. I expect to have a good relationship with my manager by this point. I would be bringing up more serious issues earlier, but now I want to start bringing up everything else. If our relationship strong enough, this is when I would try to fix that also.</p><p>Sometimes I never make it to this stage. I might be spending too much energy arguing, or feel like my contributions are not appreciated. If I don’t make it here within a few months, and have no clear path to improve things, I know it’s time to brush up my resume.</p><h2 id="4-Performing"><a href="#4-Performing" class="headerlink" title="4) Performing"></a>4) Performing</h2><p>With good working relationships, enough context about the problem space, and all my major issues resolved, I can now focus on getting some work done.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;It takes some time to integrate into a new team. I always feel like an outsider at first. As I build friendships and trust, I’m able to c</summary>
      
    
    
    
    
    <category term="leadership" scheme="https://jessemcdowell.ca/tags/leadership/"/>
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
  </entry>
  
  <entry>
    <title>Why I Only Drink Loose Tea</title>
    <link href="https://jessemcdowell.ca/2014/06/why-i-only-drink-loose-tea/"/>
    <id>https://jessemcdowell.ca/2014/06/why-i-only-drink-loose-tea/</id>
    <published>2014-06-01T22:47:35.000Z</published>
    <updated>2024-11-01T20:30:30.000Z</updated>
    
    <content type="html"><![CDATA[<p>When I was a child, I drank tea because my parents wouldn’t let me drink coffee. I would soak a tea bag in hot water until it made a dark, bitter liquid, then dump in milk and sugar until it was overly sweet, and mostly flavourless. I would sip it to fit in with adults, but I wouldn’t say that it was something I enjoyed.</p><p>As a young adult, I tried loose tea on the advice of a friend. It was a totally different drink. Black tea tasted rich and warming. Earl grey had a wonderfully soothing aroma. Green tea had a nourishing earthy taste that made me feel good when I drank it. More importantly, there was very little bitterness, so I could skip the milk and sugar, and enjoy the flavours even more.</p><p>The reason why loose tea tastes better is basic chemistry. Tea is damaged by exposure to air. The tea in tea bags is ground to a fine dust, then spread out into thin mesh pouches. The larger surface area accelerates the ageing process, robbing the flavour and aroma.</p><p>It may be true that tea bags are easier than loose tea, but with a little equipment they aren’t much easier. Most of the time, I use a simple strainer that sits in the cup like <a href="https://www.amazon.ca/s?k=finum+basket">this one</a> (<a href="https://www.amazon.ca/s?k=finum+basket">amazon.ca</a>, <a href="https://www.amazon.com/s?k=finum+basket">amazon.com</a>). Measure some loose tea into the basket, pour boiling water over it, and remove the basket once it’s finished steeping. The lid can be flipped to hold the wet basket, so you don’t even need to stay near the sink.</p><p>At work, or when I’m travelling, I use <a href="https://www.libretea.com/">a tea thermos with a built-in strainer made by a local company</a>. I put dark teas in the strainer part, steep with the container upside down, then flip and remove both lids to drink. It’s easy to use, and it makes it easy to carry hot tea around without spilling or making a mess.</p><p>Loose teas can be harder to find, but there are many great sources online. If there is a tea store in your neighbourhood, going in and selecting a few teas can be a lot of fun. Except for the very highest grades, most good teas aren’t too expensive, especially when compared to premium tea bags.</p><p>Once awakened, I started taking a lot of pleasure in hunting down and trying new teas and brewing equipment. I have been researching and experimenting for more than a decade now, and I still feel like I’ve barely scratched the surface.</p><p>Sometimes I am forced to drink tea from tea bags while I’m in a restaurant or visiting family. Some tea bags that are better than others, but all of them seem lacking compared to my own stash at home. It’s enough that I’ll never forget why I put in the extra effort to enjoy my cup of tea.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;When I was a child, I drank tea because my parents wouldn’t let me drink coffee. I would soak a tea bag in hot water until it made a dark</summary>
      
    
    
    
    
    <category term="food" scheme="https://jessemcdowell.ca/tags/food/"/>
    
    <category term="tea" scheme="https://jessemcdowell.ca/tags/tea/"/>
    
  </entry>
  
  <entry>
    <title>Controller Led Navigation in Angular</title>
    <link href="https://jessemcdowell.ca/2014/04/controller-led-navigation-in-angular/"/>
    <id>https://jessemcdowell.ca/2014/04/controller-led-navigation-in-angular/</id>
    <published>2014-04-01T22:59:35.000Z</published>
    <updated>2024-11-01T20:01:01.000Z</updated>
    
    <content type="html"><![CDATA[<p>I recently tried <a href="https://angularjs.org/">AngularJS</a> for a pet project. I watched <a href="https://www.youtube.com/watch?v=i9MHigUZKEM" title="a great tutorial">a great tutorial about the platform</a>, then dove in head first. You can see what I built here: <a href="https://jessemcdowell.ca/mysterysolver" title="Mystery Solver">MysterySolver</a></p><p>I enjoyed Angular. It was straightforward to use, and allowed me to bang out a lot of functionality without much cumbersome boilerplate code. Jasmine, the testing framework set up in the bootstrap source, was also pretty slick. I really liked how I could nest a bunch of test blocks inside of each other to reuse common setup code.</p><p>The only serious bump I ran into was getting multiple controllers to work together.</p><p>My goal was to build a wizard-style flow. A user enters a bit of info, hits a button, then enters more info. The answers in one step affect the questions in future steps, or might cause steps to be added or taken away. I wanted the controller to trigger the navigation, and I wanted to pass state when it did.</p><p>View-led navigation would have been easy: add a link whenever you like. A user clicks and the new controller is loaded. This kind of navigation is great for keeping controllers ignorant of each other. I suspect this approach is better for search engine indexers as well. It wasn’t ideal for me.</p><p>I searched the Internet and read a bunch of documentation hoping to find an easy answer. What I found was a bunch of other people asking the same questions. When I finally decided to build it myself, the solution was easier than I expected:</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&#x27;use strict&#x27;</span>;</span><br><span class="line"></span><br><span class="line"><span class="comment">// var module = angular.module(&#x27;...&#x27;, []);</span></span><br><span class="line"><span class="variable language_">module</span>.<span class="title function_">service</span>(<span class="string">&#x27;navigation&#x27;</span>, <span class="keyword">function</span>(<span class="params">$location</span>) &#123;</span><br><span class="line">  <span class="keyword">var</span> storage = <span class="literal">null</span>;</span><br><span class="line"></span><br><span class="line">  <span class="keyword">return</span> &#123;</span><br><span class="line">    <span class="attr">navigate</span>: <span class="keyword">function</span>(<span class="params">path, data</span>) &#123;</span><br><span class="line">      storage = &#123;</span><br><span class="line">        <span class="attr">path</span>: path,</span><br><span class="line">        <span class="attr">data</span>: data</span><br><span class="line">      &#125;;</span><br><span class="line">      $location.<span class="title function_">path</span>(path);</span><br><span class="line">    &#125;,</span><br><span class="line"></span><br><span class="line">    <span class="attr">getNavigationData</span>: <span class="keyword">function</span>(<span class="params"></span>) &#123;</span><br><span class="line">      <span class="keyword">if</span> ((storage == <span class="literal">null</span>) || (storage.<span class="property">path</span> != $location.<span class="title function_">path</span>())) &#123;</span><br><span class="line">        <span class="keyword">throw</span> <span class="string">&#x27;navigated without passing data&#x27;</span>;</span><br><span class="line">      &#125;</span><br><span class="line"></span><br><span class="line">      <span class="keyword">var</span> data = storage.<span class="property">data</span>;</span><br><span class="line">      storage = <span class="literal">null</span>;</span><br><span class="line">      <span class="keyword">return</span> data;</span><br><span class="line">    &#125;</span><br><span class="line">  &#125;;</span><br><span class="line">&#125;;</span><br></pre></td></tr></table></figure><p>To navigate, I used code like this:</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable language_">module</span>.<span class="title function_">controller</span>(<span class="string">&#x27;Page1Controller&#x27;</span>, [<span class="string">&#x27;$scope&#x27;</span>, <span class="string">&#x27;navigation&#x27;</span>, <span class="keyword">function</span>(<span class="params">$scope, navigation</span>) &#123;</span><br><span class="line">  $scope.<span class="property">navigate</span> = <span class="keyword">function</span> (<span class="params">navigationParameter</span>) &#123;</span><br><span class="line">      navigation.<span class="title function_">navigate</span>(<span class="string">&#x27;/Page2&#x27;</span>, navigationParameter);</span><br><span class="line">  &#125;;</span><br><span class="line">&#125;]);</span><br></pre></td></tr></table></figure><p>In the destination controller, I fetched the navigation parameter like this:</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="variable language_">module</span>.<span class="title function_">controller</span>(<span class="string">&#x27;Page2Controller&#x27;</span>, [<span class="string">&#x27;$scope&#x27;</span>, <span class="string">&#x27;navigation&#x27;</span>, <span class="keyword">function</span> (<span class="params">$scope, navigation</span>) &#123;</span><br><span class="line">  $scope.<span class="property">navigationParameter</span> = navigation.<span class="title function_">getNavigationData</span>();</span><br><span class="line">&#125;]);</span><br></pre></td></tr></table></figure><p>It was easy to test this code. Whenever I expected a controller to navigate, I checked the value of <code>$location.path()</code>. To pass navigation parameters into controllers, I just called the navigate method before the controller was created in the setup block.</p><p>Unfortunately this solution breaks the back button. Because the browser triggers backward navigation, the navigation parameter won’t be set when the controller tries to load. This wasn’t something I needed, so I left alone.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I recently tried &lt;a href=&quot;https://angularjs.org/&quot;&gt;AngularJS&lt;/a&gt; for a pet project. I watched &lt;a href=&quot;https://www.youtube.com/watch?v=i9M</summary>
      
    
    
    
    
    <category term="web" scheme="https://jessemcdowell.ca/tags/web/"/>
    
    <category term="angular" scheme="https://jessemcdowell.ca/tags/angular/"/>
    
    <category term="js" scheme="https://jessemcdowell.ca/tags/js/"/>
    
  </entry>
  
  <entry>
    <title>Reading Server Graphs: Connected Users</title>
    <link href="https://jessemcdowell.ca/2013/06/reading-server-graphs-connected-users/"/>
    <id>https://jessemcdowell.ca/2013/06/reading-server-graphs-connected-users/</id>
    <published>2013-06-15T05:30:44.000Z</published>
    <updated>2024-08-07T05:04:17.782Z</updated>
    
    <content type="html"><![CDATA[<p>I’ve spent the last several years working on multi-user server systems in two different companies. Both those companies had a giant monitor hanging off a wall showing a graph of connected users. It won’t give you detailed diagnostic information, but it is a good indicator for the health of your servers, and your product generally. If you learn to notice certain patterns in your user graph, it can also save you precious time when things go wrong.</p><h2 id="Assumptions"><a href="#Assumptions" class="headerlink" title="Assumptions"></a>Assumptions</h2><img src="/2013/06/reading-server-graphs-connected-users/NetworkDiagram.jpg" class="" title="A hand-drawn network diagram with a load balancer in front of three web nodes and a single backing database."><p>The patterns I’m describing assume a common architecture: you have some boxes in the front that have your traffic balanced between them, and some stuff behind them that’s not balanced. For the purposes of this discussion, it doesn’t matter if it’s one or more apps in the front, if you have a service tier, or what kind of data storage you use. Anything that’s redundant is going to be called distributed, and anything that’s not is going to be called central. You need some way to track user sessions, and the ability to detect disconnects within a few minutes. Some of these graphs also depend on a load balancer that’s configured to keep a single user session on the same distributed server.</p><h2 id="Healthy"><a href="#Healthy" class="headerlink" title="Healthy"></a>Healthy</h2><img src="/2013/06/reading-server-graphs-connected-users/Normal.png" class="" title="Two smooth sine waves being added to make a third smooth sine wave."><img src="/2013/06/reading-server-graphs-connected-users/NormalZoomed.png" class="" title="A zoomed in look at part of the total sine wave. The line is making a very smooth curve."><p>This is how a healthy system should look. I’m showing two views, one broken into a couple regions, and another just showing the total within a shorter time span. It’ll be a lot easier to show these patterns on the zoomed in graph, so I’ll use that as a base line for the following examples.</p><p>I was surprised when I first saw the smooth wave-like pattern a connected user graph makes. These examples are a pure sine wave because it was easy to produce, but it’s pretty close to what I’ve seen on real systems. The waves might get a little higher and wider on weekends, but it’s always a smooth line when things are normal.</p><p>The numbers and data shown are totally fabricated, and do not represent any of the systems I’ve worked on. My focus here is on the disruptions to the lines. I have observed all of the patterns I’m showing in real production environments, some of them numerous times.</p><h2 id="Central-Component-Malfunction-Failure"><a href="#Central-Component-Malfunction-Failure" class="headerlink" title="Central Component Malfunction &#x2F; Failure"></a>Central Component Malfunction &#x2F; Failure</h2><img src="/2013/06/reading-server-graphs-connected-users/CentralComponentMalfunction.png" class="" title="Line has sudden spikes up and down. The down spikes almost reach the 0 line."><p>This is the worst case for a server system, and as you can see, the results are drastic. You can tell it’s a central component because the number of connected users drops very close to zero. You’ll also note that I show the connected users shooting back above the norm. This happens when users try to reconnect once or multiple times when the system becomes unresponsive. This is a pattern you will notice during most malfunctions.</p><h2 id="Distributed-Component-Malfunction"><a href="#Distributed-Component-Malfunction" class="headerlink" title="Distributed Component Malfunction"></a>Distributed Component Malfunction</h2><img src="/2013/06/reading-server-graphs-connected-users/DistributedComponentMalfunction.png" class="" title="Line has sudden spikes up and down but they stay relatively close to the trend line."><p>This is a much more common occurrence in a server system; a server starts to malfunction without losing the ability to respond to network traffic. The load balancer doesn’t detect a failure, but users have serious trouble using the app. You will see a serious fluctuation in the graph as users start disconnecting and reconnecting, slowly getting pushed to new working servers. This is one of the reasons why it’s important to have sticky sessions on your load balancer.</p><h2 id="Distributed-Server-Failure"><a href="#Distributed-Server-Failure" class="headerlink" title="Distributed Server Failure"></a>Distributed Server Failure</h2><img src="/2013/06/reading-server-graphs-connected-users/DistributedServerFailure.png" class="" title="Line has a sudden drop down (a portion of it" alt="s load), a very small bump up, and then things return to normal."><p>When one of your distributed servers fails outright, you see a much more sudden gouge, but proportional to the number of servers you have. The graph returns to normal fairly quickly once the load balancer detects the bad server, and users finish logging back in.</p><h2 id="Application-Overloaded"><a href="#Application-Overloaded" class="headerlink" title="Application Overloaded"></a>Application Overloaded</h2><img src="/2013/06/reading-server-graphs-connected-users/ApplicationOverloaded.png" class="" title="At the peak of a curve the line goes from smooth to choppy with spikes downward from the trend line."><p>Performance limits can be distinguished because they get worse as the number of connecting users increases. This example shows a hard wall, but the severity you observe will depend on how your system is breaking down. The key indicator is the subtle twitching that gets progressively worse as the pressure builds. The deep downward spikes occur as various parts of the system start throwing large quantities of errors.</p><h2 id="Central-Component-Performance"><a href="#Central-Component-Performance" class="headerlink" title="Central Component Performance"></a>Central Component Performance</h2><img src="/2013/06/reading-server-graphs-connected-users/CentralComponentPerformance.png" class="" title="The line goes from smooth to choppy with spikes down higher than the spikes up. It is displayed off the peak of the normal curve."><p>This is what it looks like when a central component starts to have a performance issue. When it occurs off peak, it’s a good clue that some critical system is acting up. If it’s not obvious what’s wrong, here are a few things you can check: failed hard drives in your storage system, hardware errors in your system logs, unusual latency with a heavily-used APIs, or perhaps someone is running an ad-hoc query on the production database.</p><h2 id="Denial-of-Service-Attack"><a href="#Denial-of-Service-Attack" class="headerlink" title="Denial-of-Service Attack"></a>Denial-of-Service Attack</h2><img src="/2013/06/reading-server-graphs-connected-users/DosAttack.png" class="" title="The trend line suddenly drops near zero. There are a few spikes of retrying users before the line goes flat."><p>Denial of Service attacks are awful, and unfortunately effective things. They look different than network gear failures because attackers have trouble ramping up load generators quickly. Once they do get going though, your networking gear will usually start failing, and nothing will get through until they stop. Most DOS attacks are network-level, so you shouldn’t see increased activity or connections before or during the attack.</p><h2 id="Television-Ad"><a href="#Television-Ad" class="headerlink" title="Television Ad"></a>Television Ad</h2><img src="/2013/06/reading-server-graphs-connected-users/TvAd.png" class="" title="The trend line has a slight bump and then slowly trails back down to the trend line."><p>Advertising should increase your number of connected users. If your ad hits lot of people at the same time, such as a TV ad, you’ll see a bump like this. There will be a spike just as the ad airs, a bit of hang time, then it starts to trickle down to normal. The size of the bump will depend on the effectiveness and reach of your ad.</p><h2 id="Television-Event"><a href="#Television-Event" class="headerlink" title="Television Event"></a>Television Event</h2><img src="/2013/06/reading-server-graphs-connected-users/TvEvent.png" class="" title="The trend line takes a noticeable (but proportional) dip, then raises slightly above the trend line before returning to normal."><p>This is one of my favourite patterns. This is what happens if there is a big event that your audience is interested in. An example would be a sports site during the Super bowl. You see a dip while it airs, then you go back to normal when it ends. If people don’t like the event, the line might start returning to normal sooner.</p><h2 id="Monitoring-Failure"><a href="#Monitoring-Failure" class="headerlink" title="Monitoring Failure"></a>Monitoring Failure</h2><img src="/2013/06/reading-server-graphs-connected-users/MonitoringFailure.png" class="" title="The trend line falls instantly to 0 and stays there until it returns instantly to normal."><p>A flat line like this is almost never real, except maybe during deliberate maintenance windows. If it’s not obvious why you’re flat, you should check that your monitoring and graphing systems are collecting data correctly.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I’ve spent the last several years working on multi-user server systems in two different companies. Both those companies had a giant monit</summary>
      
    
    
    
    
    <category term="devops" scheme="https://jessemcdowell.ca/tags/devops/"/>
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="performance" scheme="https://jessemcdowell.ca/tags/performance/"/>
    
    <category term="quality" scheme="https://jessemcdowell.ca/tags/quality/"/>
    
  </entry>
  
  <entry>
    <title>InstallUtil and BadImageFormatException - Facepalm</title>
    <link href="https://jessemcdowell.ca/2012/04/installutil-and-badimageformatexception/"/>
    <id>https://jessemcdowell.ca/2012/04/installutil-and-badimageformatexception/</id>
    <published>2012-04-15T06:09:18.000Z</published>
    <updated>2024-08-07T05:04:17.782Z</updated>
    
    <content type="html"><![CDATA[<p>I had a frustrating issue at work this week: one that was easy to fix, but embarrassingly difficult to find. I came pretty close to giving up, which is not a solution I often explore, but in the end we figured it out and got everything working.</p><p>A member of our operations team was installing a Windows service I’d built to monitor some stuff in our production environment. I’ve made a few windows services in my day, and installed them many times on many machines. I’d even installed this one on my development machine with no issue. In our staging environment, however, this is what we got:</p><blockquote><p>C:\Install\TheService&gt;C:\Windows\Microsoft.NET\Framework64\v4.0.30319\InstallUtil.exe TheService.exe Microsoft (R) .NET Framework Installation utility Version 4.0.30319.1 Copyright (c) Microsoft Corporation. All rights reserved.</p><p>Exception occurred while initializing the installation: System.BadImageFormatException: Could not load file or assembly ‘file:&#x2F;&#x2F;&#x2F;C:\Monitoring\Service\TheService.exe’ or one of its dependencies. An attempt was made to load a program with an incorrect format.</p></blockquote><p>We checked the likely things: the framework version, the platform the app was built for, even re-copying the files in case they somehow got corrupted. When these didn’t work, we started trying more radical things: forcing all assemblies to 32 bit, even running the service as an executable to see if there was some error in the app.</p><p>In my defence, we are both experienced engineers, and I’m not the only person who missed it. Look closely at the command line we used:</p><blockquote><p>C:\Windows\Microsoft.NET\Framework<strong>64</strong>\v4.0.30319\InstallUtil.exe</p></blockquote><p>Long version: Service applications in Visual Studio 2010 are 32 bit by default, and this is a reasonable default for them to have. We were trying to install the 32 bit service with the 64 bit version of InstallUtil. InstallUtil loads the target assembly to access it’s installation instructions, but you can’t load a 32 bit assembly from a 64 bit application (or vice versa). If you try to, you get a BadImageFormatException.</p><p>Short version: Two numbers derailed my entire afternoon.</p><p>It would have been nice if the error message from InstallUtil was a little more specific, but I suppose this isn’t a common problem. At least I got a good reminder about the importance of checking the small details when the big ones aren’t bearing fruit.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I had a frustrating issue at work this week: one that was easy to fix, but embarrassingly difficult to find. I came pretty close to givin</summary>
      
    
    
    
    
    <category term="devops" scheme="https://jessemcdowell.ca/tags/devops/"/>
    
    <category term=".net" scheme="https://jessemcdowell.ca/tags/net/"/>
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
  </entry>
  
  <entry>
    <title>Choosing Priorities</title>
    <link href="https://jessemcdowell.ca/2012/03/choosing-priorities/"/>
    <id>https://jessemcdowell.ca/2012/03/choosing-priorities/</id>
    <published>2012-03-09T05:50:05.000Z</published>
    <updated>2024-11-01T20:01:01.000Z</updated>
    
    <content type="html"><![CDATA[<p>During the summer I started a difficult but important journey to reorganize my life. The first step was <a href="https://jessemcdowell.ca/2011/10/managing-priorities-outside-of-work" title="organizing my daily tasks">organizing my daily tasks</a>. I was successful, but becoming productive again has created new issues.</p><p>When you change from trying to do everything to doing what’s most important, you need to decide what important means. Figuring this out for myself has proven difficult. I still don’t have all the answers, but I am constantly making progress.</p><p>Deciding what isn’t important has been easier. I am abandoning Themis, my attempt at an open source project. I no longer have a need for it, and it will take a lot more effort to get it finished than I originally expected. I would be delighted if someone wanted to take it on, but there isn’t enough there for it to be likely.</p><p>Writing in this blog has also been pushed down the priority list. I intend to keep going, but instead of forcing out a steady pace of content, I’ll wait until I have ideas that I have a strong desire to share.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;During the summer I started a difficult but important journey to reorganize my life. The first step was &lt;a href=&quot;https://jessemcdowell.ca</summary>
      
    
    
    
    <category term="projects" scheme="https://jessemcdowell.ca/categories/projects/"/>
    
    <category term="Themis" scheme="https://jessemcdowell.ca/categories/projects/Themis/"/>
    
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
  </entry>
  
  <entry>
    <title>Sharpening The Saw</title>
    <link href="https://jessemcdowell.ca/2011/10/sharpening-the-saw/"/>
    <id>https://jessemcdowell.ca/2011/10/sharpening-the-saw/</id>
    <published>2011-10-25T01:49:26.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>One of the best things about working in the software business is the high rate of change. Not only do we work with machines that radically change in capability every few years, but new techniques and technologies are constantly appearing on the scene. It is a fascinating new frontier that never gets dull.</p><p>One of the worst things about working in the software business is the high rate of change. If you only focus on your job and stop paying attention, you could wake up one day unemployed with an obsolete skill set. Keeping up to date is a lot of work, and it becomes harder every year.</p><p>Some skills hold up better over time. Solid work habits are priceless, and can help you advance your career even where technical skills are lacking. Work management, software design, and coding practices are good subjects to study as well; they don’t age nearly as fast as specific technologies, and they can make a big impact on the quality of your work. Unless you become a manager however, you’re still going to need to study technical stuff.</p><p>The only way to really learn a tool is to use it, so study at home isn’t always enough. Some employers will deliberately favour new technologies to keep skills fresh and employees happy, but this is not the standard. If your company starts to lag behind industry trends, you might consider changing jobs while it’s still possible.</p><p>There are some tricks that help me stay up to date: I listen to podcasts while exercising or doing simple chores. Public transit is a great opportunity to read while commuting. A smart phone helps me squeeze in a few extra blogs and tweets when I’m waiting for something. This is primarily how I follow industry trends.</p><p>To learn specific technologies, I’ll spend an hour or two a few times a week reading or watching videos. I need to play with something to understand it, so I spend an odd evening banging away at some pet project.</p><p>My study used to be limited to an occasional book or seminar, but increasing my study time, and opening myself to podcasts, blogs, and twitter has made a huge difference in my career. I have more to contribute in design meetings. I have opinions on various technologies I haven’t even used. It takes more energy for sure, but I feel like I am a better developer for it, and I don’t regret it one bit.</p><p>This business is a challenging one, but exciting as well. If you are the kind of person that loves to learn, explore new technologies, and push yourself to constantly look at problems in new ways, then embrace the change and you will do well.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;One of the best things about working in the software business is the high rate of change. Not only do we work with machines that radicall</summary>
      
    
    
    
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
    <category term="learning" scheme="https://jessemcdowell.ca/tags/learning/"/>
    
    <category term="practices" scheme="https://jessemcdowell.ca/tags/practices/"/>
    
  </entry>
  
  <entry>
    <title>Managing Priorities Outside of Work</title>
    <link href="https://jessemcdowell.ca/2011/10/managing-priorities-outside-of-work/"/>
    <id>https://jessemcdowell.ca/2011/10/managing-priorities-outside-of-work/</id>
    <published>2011-10-07T02:55:07.000Z</published>
    <updated>2024-08-07T05:04:17.780Z</updated>
    
    <content type="html"><![CDATA[<p>I’ve been fortunate to take some serious down time during the summer. After working pretty much non-stop for ten years, things have piled up. I took some time to rest, then got to work on some personal projects that have been neglected far too long.</p><p>What I’ve learned, however, is that it takes more than time to get things done. After about a month, pretty much all I’d achieved was completing a couple video games, and putting all the things I wanted to do into one very long list.</p><p>My first thought was that I wasn’t focusing, so after listening to <a href="https://hanselminutes.com/268/personal-systems-of-organization-rey-bango-interviews-scott-hanselman">a podcast by Scott Hanselman about organization</a>, I decided to try the <a href="https://www.pomodorotechnique.com/">Pomodoro Technique</a>. This is a fairly simple system where you try to focus completely on a task for 25 minutes at a time, shutting out all distractions.</p><p>After a bit of Internet searching with no results, I started making my own pomodoro timer application. I had something working in a couple hours, barely holding myself back from adding sounds and animations. It would have been a lot faster, but I wasn’t completely focused on the task.</p><p>Armed with my timer, ready to experience my first burst of productivity, my wife came home from work. I told her excitedly how I’d built this magical thing, and how I’m going to start getting so much done. Then she asks me one simple question, “Why didn’t you use the timer from the kitchen?” This is when I realized that my problem was focus on a whole other level. I was doing work, but I wasn’t doing the right work.</p><p>This is a common problem in the software world: getting overwhelmed by a giant list of tasks, doing a little bit of everything, and completing nothing. I’ve experienced this first hand, and have been using effective tools for dealing with it for years, but I had never considered applying them to my personal life.</p><p>I tried a few things next, but the best help came from the book <a href="https://www.gettingresults.com/">Getting Results the Agile Way</a>, which was also mentioned in the podcast. It combines agile philosophy with some ideas about balancing life priorities and energy levels. I’m not committed to the entire process, but the parts I am trying have already made a big difference. By focusing on three goals a day, I’m now managing to plow through even dauntingly large tasks. Though I’m not doing everything on my list, I’m getting the most important things done, and because I feel that I’m making progress again it’s motivating me to work harder.</p><p>Taking the time off has been really good for me. Beyond organizing my personal priorities, I’ve also been organizing my main workspaces: my office, and my kitchen. You could say the experience is like rebooting a computer; it took a bit of time, but now I feel fresh and full of life, ready to tackle something new.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I’ve been fortunate to take some serious down time during the summer. After working pretty much non-stop for ten years, things have piled</summary>
      
    
    
    
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
  </entry>
  
  <entry>
    <title>Doubling Data for Performance Testing</title>
    <link href="https://jessemcdowell.ca/2011/08/doubling-data-for-performance-testing/"/>
    <id>https://jessemcdowell.ca/2011/08/doubling-data-for-performance-testing/</id>
    <published>2011-08-12T04:22:26.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p><strong>Or: The Most Impressive T-SQL Script I’ve Ever Written</strong></p><p>I was recently working on a new application. After three months in the field, users were starting to complain about performance issues. We had done some limited performance tuning for the first release, and more as part of the second release, but new issues were popping up as more data got entered into the system. We could have continued fixing issues as they came up, one release at a time, but we wanted to get ahead of the problem, and the client wanted to know that the system would remain usable without developer intervention for a few years at least.</p><p>The nature of the business was such that the rate of data entry and number of concurrent users shouldn’t change much over time. This meant that increasing the amount of data would be a good approximation of how the system would look in the future. How do you increase the amount of data in a normalized relational database? It would be difficult to enter realistic test data manually, and unrealistic test data can cause unrealistic test results. Bad results means we waste time fixing issues that wouldn’t appear, and never get a chance to detect issues that will.</p><p>I proposed creating a T-SQL script that would double all the existing data. Running it once would simulate six months of usage, running it again simulates a year, twice more and we’re simulating four years of usage. Once it was approved, it took me a couple days to finish, about half of that going to testing and debugging. The result is the most impressive T-SQL script I’ve ever written.</p><p>The script I wrote didn’t do the actual duplication, it combined hand-entered metadata with data from sys.tables and sys.columns to generate a much bigger script. Not only was it faster to write this way, but it made it easier to fix issues, and it allowed the script to be reused after columns were added to the database.</p><p>I had one table variable with a list of tables to copy, and another defining the relationships between foreign keys and their source tables. Most foreign keys were named the same in all tables that they appeared in, so a single mapping was often enough for all of them. Most of the mappings could be determined from the list of tables itself, so I only had a few other relationships and special column rules that had to be entered manually.</p><p>Another factor that helped was our use of guids for all primary keys. Because the can be determined before inserting a row, it was possible to generate the mapping from old to new key at the start of the script. I could also use a single insert statement for each table, and the order of execution only mattered where foreign key constraints existed.</p><p>The results were tremendous. We found a bunch of issues we wouldn’t have found otherwise, and had a fairly solid indication of how the application would behave years in the future.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;&lt;strong&gt;Or: The Most Impressive T-SQL Script I’ve Ever Written&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;I was recently working on a new application. After three m</summary>
      
    
    
    
    <category term="uncategorized" scheme="https://jessemcdowell.ca/categories/uncategorized/"/>
    
    
    <category term="database" scheme="https://jessemcdowell.ca/tags/database/"/>
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="performance" scheme="https://jessemcdowell.ca/tags/performance/"/>
    
  </entry>
  
  <entry>
    <title>The Critical Path</title>
    <link href="https://jessemcdowell.ca/2011/06/the-critical-path/"/>
    <id>https://jessemcdowell.ca/2011/06/the-critical-path/</id>
    <published>2011-06-25T16:58:08.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>For the last ten years, if you had asked, I would have told you with some conviction that a “critical path” is a kind of test script that tests the essential functionality of a system. If the app passes it’s critical path test, I would have said, it may not delight users, but it’s unlikely that they’ll storm your basement with torches and pitchforks.</p><p>It came up at work during a discussion about interview questions. My colleague was both proud and surprised to have been the only person interviewed for his position that had known what it really means.</p><p>A critical path is a project management term that refers to the chain of dependant tasks that will take the longest amount of time in a project plan. For example, if I were to make a project plan for my breakfast on Sunday morning, it would look like this:</p><img src="/2011/06/the-critical-path/BreakfastOnSunday.png" class="" title="The plan for my Sunday morning breakfast."><p>In this plan, making pancakes is the critical path. With an unlimited amount of people, and an unlimited amount of kitchen space, and all the resources and tools we would need, the fastest that we could produce my Sunday morning breakfast is the time it takes to make pancakes.</p><p>I can steam milk, flip pancakes, and toss a couple eggs in the pan all at the same time, but grinding coffee beans and mixing the pancake batter has to be done one at a time. This breakfast takes me just a few minutes longer than the critical path, so it’s not worth hiring a sous chef.</p><p>The time of a software projects is usually shaped more by the number of developers. We usually have a lot of features to deliver, and most of them are not directly dependant on each other. What you can see is a part at the beginning and end of a project where it’s hard to keep multiple people working, but that’s a whole other thing.</p><img src="/2011/06/the-critical-path/GenericSoftwareProject.png" class="" title="A generic project plan showing a chain of blocking tasks at the beginning and end of a project."><p>When a critical path is defining the time line for your project, it’s important to identify it; any step that gets delayed affects the entire chain. You should start these tasks as soon as possible, and do what you can to stop them from getting blocked.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;For the last ten years, if you had asked, I would have told you with some conviction that a “critical path” is a kind of test script that</summary>
      
    
    
    
    
    <category term="project management" scheme="https://jessemcdowell.ca/tags/project-management/"/>
    
  </entry>
  
  <entry>
    <title>Consulting - A Brave New World</title>
    <link href="https://jessemcdowell.ca/2011/05/consulting-a-brave-new-world/"/>
    <id>https://jessemcdowell.ca/2011/05/consulting-a-brave-new-world/</id>
    <published>2011-05-24T05:20:41.000Z</published>
    <updated>2024-11-01T20:30:30.000Z</updated>
    
    <content type="html"><![CDATA[<p>I’ve taken a big step in my career. I’ve become a software consultant.</p><p>The decision to leave my job of three and a half years wasn’t an easy one. I was working with some fantastic people, building the best software we could, but at the end of the day I wasn’t learning. The work was challenging, but the challenges were mostly non-technical. In this business, if you’re not moving forward you may as well lay down and die. I couldn’t change my situation, so I said my goodbyes and left.</p><p>Instead of settling in somewhere else for another long haul, I felt it was time for a change of pace. Now I get to hop from organization to organization working on new problems and broadening my skill set in the most efficient way possible. I might settle down if a good opportunity comes along, but for now I’m enjoying the freedom.</p><p>I’m currently working on a fun little contract, but I’ll be looking for my next opportunity soon.</p><p>If you want help designing or building awesome software that delights users, we should talk. I’d be happy to put my experience to work building you exactly what you need.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I’ve taken a big step in my career. I’ve become a software consultant.&lt;/p&gt;
&lt;p&gt;The decision to leave my job of three and a half years wasn</summary>
      
    
    
    
    
    <category term="career" scheme="https://jessemcdowell.ca/tags/career/"/>
    
    <category term="consulting" scheme="https://jessemcdowell.ca/tags/consulting/"/>
    
  </entry>
  
  <entry>
    <title>The Average Test</title>
    <link href="https://jessemcdowell.ca/2011/04/the-average-test/"/>
    <id>https://jessemcdowell.ca/2011/04/the-average-test/</id>
    <published>2011-04-30T23:08:30.000Z</published>
    <updated>2024-08-07T05:04:17.775Z</updated>
    
    <content type="html"><![CDATA[<p>I was conducting technical interviews recently for a senior developer position. I like to talk about a bunch of things, but there is one question I particularly enjoy asking each candidate. Write an implementation for this:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="built_in">int</span> <span class="title">Average</span>(<span class="params"><span class="built_in">int</span> a, <span class="built_in">int</span> b</span>)</span></span><br><span class="line">&#123;</span><br><span class="line"></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>It’s a great question because it’s a simple problem, and anyone that comes through the door should know the formula without having to ask. It’s interesting how people react to it sometimes, but I chalk it up to interview jitters. About half the people interviewed answer like this:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="built_in">int</span> <span class="title">SimpleAverage</span>(<span class="params"><span class="built_in">int</span> a, <span class="built_in">int</span> b</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">return</span> (a + b) / <span class="number">2</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>It’s a good first answer too, except that it doesn’t always work.</p><p>“What will be the result of your function if it’s called with the values Int32.MaxValue and Int32.MaxValue?”</p><p>The answer I want to hear is that it won’t work. Some people think (like I did), that it will throw a System.OverflowException, but it actually returns -1. The default for C# projects is to skip overflow checking for arithmetic.</p><p>This is another answer that I get from time to time:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="built_in">int</span> <span class="title">SystemMathAverage</span>(<span class="params"><span class="built_in">int</span> a, <span class="built_in">int</span> b</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">return</span> System.Math.Average(a, b);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>I respect this answer; it indicates that a developer wants to leave low-level details to the framework, which is a perfectly reasonable attitude when developing business-class software. Unfortunately there are two problems with this answer: it defeats the purpose of the test, and it doesn’t exist in the framework. There is the Linq average statement which can be used like this:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="built_in">int</span> <span class="title">LinqAverage</span>(<span class="params"><span class="built_in">int</span> a, <span class="built_in">int</span> b</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="built_in">int</span>[] inputArray = <span class="keyword">new</span> <span class="built_in">int</span>[] &#123; a, b &#125;;</span><br><span class="line">    <span class="keyword">return</span> (<span class="built_in">int</span>)inputArray.Average();</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>After the first attempt, I indicate that I want a function which will work correctly for all values of a and b. The way candidates tackle it tells you a lot about them as a developer. Here are some behaviours I’ve noticed:</p><ul><li>Very few candidates stop to consider a solution before answering.</li><li>About 3&#x2F;4 of candidates try to use an if statement.</li><li>About 1&#x2F;4 of candidates write the “if” on the board before deciding what to use it for.</li><li>Occasional candidates will challenge the necessity for handling high values.</li></ul><p>A clean correct answer is hard to pull off on the spot, especially under pressure. I’ve never seen it, and I wouldn’t be surprised if I never do. This doesn’t bother me either. The key thing I’m looking for is the ability to understand questions and discuss the source code. These skills are essential to working in a team.</p><p>How did I come up with such a brilliant interview question? I didn’t. Once upon a time in a company far far away, someone asked me the very same thing. I was pretty proud of my answer too, until I tested it. This was what I came up with:</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="built_in">int</span> <span class="title">JessesAverage</span>(<span class="params"><span class="built_in">int</span> a, <span class="built_in">int</span> b</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">return</span> (a / <span class="number">2</span>) + (b / <span class="number">2</span>) + (((a % <span class="number">2</span>) + (b % <span class="number">2</span>)) / <span class="number">2</span>);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>I wrote <a href="https://gist.github.com/949410" title="TestAverageMethod()">a test harness</a> in some spare time after the last round of interviews. Here’s how each of the answers measure up:</p><p>Results for SimpleAverage():<br> Near Zero: 121 of 121 passed.<br> End of Range: 0 of 3 passed.<br> Failed Values:</p><ul><li>SimpleAverage(2147483647, 2147483647) returns -1, but should return 2147483647</li><li>SimpleAverage(-2147483648, -2147483648) returns 0, but should return -2147483648</li><li>SimpleAverage(2147483647, 2147483645) returns -2, but should return 2147483646</li></ul><p>Results for LinqAverage():<br> Near Zero: 121 of 121 passed.<br> End of Range: 3 of 3 passed.</p><p>Results for JessesAverage():<br> Near Zero: 109 of 121 passed.<br> End of Range: 3 of 3 passed.<br> Failed Values:</p><ul><li>JessesAverage(-4, 1) returns -2, but should return -1</li><li>JessesAverage(-4, 3) returns -1, but should return 0</li><li>JessesAverage(-3, 4) returns 1, but should return 0</li><li>JessesAverage(-2, 1) returns -1, but should return 0</li><li>JessesAverage(-1, 2) returns 1, but should return 0</li><li>JessesAverage(-1, 4) returns 2, but should return 1</li><li>JessesAverage(1, -4) returns -2, but should return -1</li><li>JessesAverage(1, -2) returns -1, but should return 0</li><li>JessesAverage(2, -1) returns 1, but should return 0</li><li>JessesAverage(3, -4) returns -1, but should return 0</li><li>JessesAverage(4, -3) returns 1, but should return 0</li><li>JessesAverage(4, -1) returns 2, but should return 1</li></ul><p>This was the best answer I could come up with after some experimentation. I’m not sure why, but the need for type conversion bugs me.</p><figure class="highlight csharp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="built_in">int</span> <span class="title">ConversionAverage</span>(<span class="params"><span class="built_in">int</span> a, <span class="built_in">int</span> b</span>)</span></span><br><span class="line">&#123;</span><br><span class="line">    <span class="keyword">return</span> (Int32)(((Int64)a + (Int64)b) / <span class="number">2</span>);</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>Results for ConversionAverage():<br> Near Zero: 121 of 121 passed.<br> End of Range: 3 of 3 passed.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I was conducting technical interviews recently for a senior developer position. I like to talk about a bunch of things, but there is one </summary>
      
    
    
    
    
    <category term="c#" scheme="https://jessemcdowell.ca/tags/c/"/>
    
    <category term="interviewing" scheme="https://jessemcdowell.ca/tags/interviewing/"/>
    
  </entry>
  
  <entry>
    <title>Working Together and Having Fun</title>
    <link href="https://jessemcdowell.ca/2011/01/working-together-and-having-fun/"/>
    <id>https://jessemcdowell.ca/2011/01/working-together-and-having-fun/</id>
    <published>2011-01-30T19:46:45.000Z</published>
    <updated>2024-10-29T07:00:00.000Z</updated>
    
    <content type="html"><![CDATA[<p>We did one of our monthly releases at work this week. Releases can be stressful and frustrating, and take a lot of methodical preparation to get right. It can be thankless work too; the only time a user notices a release is when it goes badly. We do our releases early on a week day to minimize impact, so if anything does go wrong, there’s not many bodies around to help out. It’s not much fun, but it’s important work that needs to be done.</p><p>One thing that makes the experience considerably more enjoyable for me is the team of coworkers that come in to help. There’s a handful of us, each with our own responsibilities. I deploy the applications while someone else monitors the database and another person tests the system to make sure it’s running normally. We back each other up, help out where we can, and make decisions together when they need to be made quickly.</p><p>Pressure is never a desirable thing in a work environment, but one benefit is that it quickly builds trust between anyone facing it together. The people I work with are really fantastic: smart, dedicated, and fun. We come from different cultures, like different kinds of food, have different hobbies and different tastes in music, but we still find things to talk about, reasons to laugh and smile.</p><p>I’ve been trying ways to make releases more fun. We had a pot luck breakfast once, and a release soundtrack from team favourites another time. This time I made breakfast burritos, and as a joke, a doughnut salad topped with an espresso sauce. We played a few songs during the waiting periods, and had as much fun as anyone could that early in the morning.</p><img src="/2011/01/working-together-and-having-fun/DoughnutSalad.jpg" class="" title="A plate of chopped doughnuts topped with an espresso jelly."><p>Thanks to the people involved, this release went well despite a few hiccups.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;We did one of our monthly releases at work this week. Releases can be stressful and frustrating, and take a lot of methodical preparation</summary>
      
    
    
    
    
    <category term="devops" scheme="https://jessemcdowell.ca/tags/devops/"/>
    
    <category term="people management" scheme="https://jessemcdowell.ca/tags/people-management/"/>
    
    <category term="food" scheme="https://jessemcdowell.ca/tags/food/"/>
    
    <category term="leadership" scheme="https://jessemcdowell.ca/tags/leadership/"/>
    
  </entry>
  
  <entry>
    <title>When to Add an ORM Tool</title>
    <link href="https://jessemcdowell.ca/2011/01/when-to-add-an-orm-tool/"/>
    <id>https://jessemcdowell.ca/2011/01/when-to-add-an-orm-tool/</id>
    <published>2011-01-03T20:33:03.000Z</published>
    <updated>2024-08-07T05:04:17.772Z</updated>
    
    <content type="html"><![CDATA[<p>I’m working on the code that parses VCalendar data so that it can be processed. I’m copying the data I care about into a simple data structure that can represent a calendar request in any format. Any logic that interacts with calendar requests would use this internal structure. I want it to be simple, only having the stuff that I need, but I don’t want to completely re-invent the wheel either, so I will use the VCalendar format as a guideline.</p><p>I’ve used this pattern of processing an established or complex data format into a simple proprietary one before with good results. It allows you to isolate the complexities of a data format in the code that processes it, keeping the rest of your code clean. Like everything, there are cases where I wouldn’t use it, but for this kind of scenario it should work well.</p><p>Now that I’m building this neutral representation, my next thought is how it will be saved to the database. I don’t need a database at this stage of the project, so part of me wants to ignore the decision until later, but the whole point of the system is to persist and interact with these types, so it’s important that I get it right. If I make simple POCO classes now, and start writing a bunch of code that uses them, I might have to change a lot of code later if I want to switch to types generated by Entity Framework. I could write my own custom code to read and write my own types from the database, then I can use any type I want without restrictions, but it would be a waste to write this plumbing code myself when an ORM can do it faster and better.</p><p>Creating a table design now wouldn’t be a simple matter either. I don’t know enough about the needs of the scheduling service to know which parts of the VCalendar format to bring over. If I try to guess now I know I’ll bring over a bunch of stuff that I don’t need, but starting with a simple table and adding fields every time I need one is no fun either. Adding a database to a system is like attaching a ball and chain, and I want to wait until I can be sure I have my model correct before I do it.</p><p>I need to make a decision, so here it is: I’m going to keep building my own types, and try to keep them. When it comes time to hook up a database, I’ll play around with the new POCO support in Entity Framework 4, and if that fails I’ll try another ORM tool. I may need to change my model a bit to suit the ORM, but I’m hoping that it wont need to change much.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I’m working on the code that parses VCalendar data so that it can be processed. I’m copying the data I care about into a simple data stru</summary>
      
    
    
    
    <category term="projects" scheme="https://jessemcdowell.ca/categories/projects/"/>
    
    <category term="Themis" scheme="https://jessemcdowell.ca/categories/projects/Themis/"/>
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="database" scheme="https://jessemcdowell.ca/tags/database/"/>
    
    <category term="entity framework" scheme="https://jessemcdowell.ca/tags/entity-framework/"/>
    
    <category term=".net" scheme="https://jessemcdowell.ca/tags/net/"/>
    
  </entry>
  
  <entry>
    <title>Teaching IT</title>
    <link href="https://jessemcdowell.ca/2010/12/teaching-it/"/>
    <id>https://jessemcdowell.ca/2010/12/teaching-it/</id>
    <published>2010-12-25T06:28:12.000Z</published>
    <updated>2024-08-07T05:04:17.772Z</updated>
    
    <content type="html"><![CDATA[<p>My company runs a 24&#x2F;7 site with a substantial number of users and connections to partner systems all over the world. We do what we can to make the system fault tolerant, but problems can still appear at any time of day or night. Ideally we would have a technical support team that’s staffed around the clock, but that not in the cards for now.</p><p>We used to have a system where anyone that could fix a problem would get a text message, and whoever was closest to a computer would respond. It worked most of the time, but there were some issues. The team creating the tickets would often be unsure of which component was broken, so the issues would hit lots of people who would be unable to help, and it also exposes <a href="https://en.wikipedia.org/wiki/Diffusion_of_responsibility">a quirk of human behaviour</a> that makes us less likely to help when more people are available.</p><p>Thankfully the whole plan is being reworked. Now a single volunteer will carry a phone that gets all the support tickets. They will respond to issue within a reasonable amount of time, and if they can’t solve them themselves, find other people to help. The company has also added decent incentives to be on call, and is investing in tools to make the job easier.</p><p>As part of the preparation, I’ve been asked to do presentations and create supporting documentation for all of the first responders. I enjoy doing the presentations, and everyone seems to be benefiting from the content, but it’s a lot more work than you might expect. The group of people I’m working with is comprised mostly of heads-down developers. They’re smart people, and all wanting to learn, but most don’t have any IT experience.</p><p>My first session covered these topics:</p><ul><li><p>how to think and act like a seasoned IT pro</p></li><li><p>a brief description of our hosting environment, servers , and where the various applications are deployed</p></li><li><p>how to recognize network problems, hardware problems, and OS problems</p></li><li><p>the symptoms to expect when any of our key systems fail</p></li><li><p>how to safely restart our key systems</p></li></ul><p>It took me about an hour and a half to prepare, and two hours to present. I’ve spent about a day of effort writing it down, but would guess that I’ve covered less than half of the material I talked about. To be fair, the written material is laid out quite differently from the presentations; I am trying to capture a series of troubleshooting guides that can be used in an emergency.</p><p>Are you curious about the difference between a developer and an IT pro? The most significant difference I see is that we developers are used to an environment where we can test changes quickly and safely: change a line of code, hit F5, and see what happens. IT pros take the opposite approach because the cost of a bad change in a live production environment can be devastating. They need to be certain what any change will do before it’s made, and be prepared to roll it back if there’s any hint of trouble. It can be frustrating for a developer to work this way because it’s a serious effort to make even simple changes, but it’s this mentality that keeps the world’s servers running.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;My company runs a 24&amp;#x2F;7 site with a substantial number of users and connections to partner systems all over the world. We do what we </summary>
      
    
    
    
    
    <category term="devops" scheme="https://jessemcdowell.ca/tags/devops/"/>
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="people management" scheme="https://jessemcdowell.ca/tags/people-management/"/>
    
  </entry>
  
  <entry>
    <title>Invitations and the VCard Format</title>
    <link href="https://jessemcdowell.ca/2010/12/invitations-and-the-vcard-format/"/>
    <id>https://jessemcdowell.ca/2010/12/invitations-and-the-vcard-format/</id>
    <published>2010-12-16T20:59:22.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>My next goal for the Themis project is to parse an invitation from an email. I am starting with invitations generated by MS Outlook because that’s my target audience, but a peak inside of a Google Calendar invitation gives me hope that I’ll be able to support multiple calendars without much trouble.</p><p>Outlook invitations are sent in the VCalendar format, content type “text&#x2F;calendar”. The standard was published as <a href="https://www.w3.org/2002/12/cal/rfc2445.html">RFC 2445</a> in 1998. It describes a standard layout for calendar data in the VCard format, which is described in <a href="https://www.w3.org/2002/12/cal/rfc2425.html">RFC 2425</a>.</p><p>VCard is a simple text based format with a nested structure that’s similar to XML. It uses colons and semicolons instead of angle brackets, supports attributes, and has standard representations for several common data types. Here is an example of the same data represented in both formats (content from RFC 2425):</p><img src="/2010/12/invitations-and-the-vcard-format/vcard-vs-xml.png" class="" title="An example of VCard data beside an XML representation of the same data."><p>I looked around for a library to do the job,but decided to build my own. Writing this kind of code isn’t a good way to deliver business value, but it sure is fun. A bit of time with a white board and a couple of evenings coding, and I have a mostly functional VCard parser ready to go.</p><p>When I am setting out to build something like a parser, I like to consider the designs of other established libraries that perform a similar function. As I’ve already mentioned, VCard is a lot like XML in structure, so XML parsers are a great place to look for inspiration. I built a structure that loosely mimics the XmlDocument model, an object tree that holds the entire document in memory. It won’t perform as fast as using something like an XmlReader, but it makes it easier to handle variations in document format with polymorphism. Since the VCard documents I’m processing should never be too big, it won’t be much of a performance burden anyway.</p><p>The object structure that holds VCard data looks like this:</p><img src="/2010/12/invitations-and-the-vcard-format/VCardObjectModel1.jpg" class="A UML diagram showing the class structure for holding VCard data."><p>At first I wanted to have a property like Value, but there is no reliable way to know the type of a VCard value without knowing the structure of the document, and at this level I don’t want to be coupled to the structure of the document. Instead, I will be adding a series of methods to get and set the EscapedValue as a specific type.</p><p>The VCardSimpleValue class was a late addition to the model. I needed a way to hold parameter values (the equivalent of attributes in XML), but since they can’t have parameters of their own I didn’t want to pick up the Parameters collection. I also considered making a type separate from VCardValue for the parameters, but both these classes will need the ability to read and write the escaped value, and I don’t want to write that code twice.</p><p>I’ve added the first unit tests to the solution, checking the parser against a number of examples in the specifications and real snippets extracted from emails sent by Outlook. I’ve also added tests for a number of failure cases in the parser such as groups without an ending or values lines without a value delimiter.</p><p>My next addition will be parsers for the value types and a stub in the test harness that replies to emails with some info about the original request.</p><p>The code at the time of this post is available here: <a href="https://github.com/jessemcdowell/themis/tree/8e0c428df0e33145df0323d6e9ae617114dd0f85">https://github.com/jessemcdowell/themis/tree/8e0c428df0e33145df0323d6e9ae617114dd0f85</a></p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;My next goal for the Themis project is to parse an invitation from an email. I am starting with invitations generated by MS Outlook becau</summary>
      
    
    
    
    <category term="projects" scheme="https://jessemcdowell.ca/categories/projects/"/>
    
    <category term="Themis" scheme="https://jessemcdowell.ca/categories/projects/Themis/"/>
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
  </entry>
  
  <entry>
    <title>64-bit IIS vs. 32-bit Assemblies</title>
    <link href="https://jessemcdowell.ca/2010/11/64-bit-iis-vs-32-bit-assemblies/"/>
    <id>https://jessemcdowell.ca/2010/11/64-bit-iis-vs-32-bit-assemblies/</id>
    <published>2010-11-12T00:45:38.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>I found my first 64-bit bug at work. I was moving a windows service built for the ‘Any CPU’ to a 64-bit server. It started fine on the new server, and gave no indication of poor health in the logs, but one key function was malfunctioning. I’m not exactly sure what the cause is, but I know that the hash of any binary file was resulting in the same value. The service does some direct memory manipulation which is a likely culprit.</p><p>Fixing the problem was easy: Set all the assemblies to compile for the 32-bit platform, check it in, build it, ship it.</p><p>Except that my web service now looks like this:</p><p><em>Server Error in ‘&#x2F;MyApp’ Application.</em></p><p><em>Could not load file or assembly ‘MyCompany.MyApp.MyAssembly’ or one of its dependencies. An attempt was made to load a program with an incorrect format.</em></p><p>This application has two parts on the server: a windows service that processes emails, and a web service for interacting with them. The two parts share a database and a bunch of business logic, so they also share some assemblies. The windows service works great now that I’ve converted all the assemblies to 32-bit, but the web service wont start.</p><p>Although the error message is a bit misleading, it wasn’t that hard to find the problem. When IIS is running in 64-bit mode (as it tends to do on 64-bit servers), all of its application pools will be 64-bit, and a 64-bit application pool cant load a 32 bit assembly. These are all the ways I can think of to fix it:</p><ul><li>IIS 7 can run an application pool in 32-bit mode</li><li>IIS 6 can be configured to run in 32-bit mode, or you can run a second instance in 32-bit mode, though I don’t imagine both instances could share the same ports</li><li>Write 32-bit and 64-bit versions for any platform-specific operations using conditional compilation, then build two versions of your assemblies</li><li>Rewrite the malfunctioning parts of your app so you don’t have any platform-specific code</li><li>Compile most of your assemblies for ‘Any CPU’, but compile entry-point assemblies (such as my service executable) for 32-bit</li></ul><p>A configuration-based solution isn’t ideal based on the number and management practices of our environments. Finding and removing platform dependence would be the best technical solution, but it would be risky and time consuming. Switching some of my assemblies back to ‘Any CPU’ did the job with a minimal amount of impact.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I found my first 64-bit bug at work. I was moving a windows service built for the ‘Any CPU’ to a 64-bit server. It started fine on the ne</summary>
      
    
    
    
    
    <category term="devops" scheme="https://jessemcdowell.ca/tags/devops/"/>
    
    <category term=".net" scheme="https://jessemcdowell.ca/tags/net/"/>
    
    <category term="debugging" scheme="https://jessemcdowell.ca/tags/debugging/"/>
    
    <category term="web" scheme="https://jessemcdowell.ca/tags/web/"/>
    
  </entry>
  
  <entry>
    <title>SQLite vs. SQL CE 3.5 with Entity Framework 4</title>
    <link href="https://jessemcdowell.ca/2010/10/sqlite-vs-sql-ce-3-5-with-entity-framework-4/"/>
    <id>https://jessemcdowell.ca/2010/10/sqlite-vs-sql-ce-3-5-with-entity-framework-4/</id>
    <published>2010-11-01T05:19:23.000Z</published>
    <updated>2024-11-01T20:30:30.000Z</updated>
    
    <content type="html"><![CDATA[<p>My wife and I have been taking a Japanese class. I’m enjoying it quite a bit, even though learning languages has never been easy for me. I need to work hard to memorize all the new words, and that means lots and lots of practice.</p><p>My wife bought a pack of index cards. This is what a normal person would do. To me, it seems like defeat. Why use paper when there’s a computer in the room? While she’s at the store, I grab a white board and sketch out a data model.</p><p>I used WPF for a UI, two simple view models, and Entity Framework 4 to store the lists of characters, words, and phrases. I started out with SQL Compact Edition 3.5 as a database because that was the one that was there. That didn’t work out so well, so on the advice of a colleague I switched my app to SQLite.</p><p>This is what I found:</p><p>SQL CE 3.5</p><p>SQLite</p><p>Pros</p><ul><li><p>Can be created easily from the VS designer</p></li><li><p>Can view and modify data inside Visual Studio</p></li><li><p>Supports Identity Columns</p></li><li><p>Easy to deploy</p></li><li><p>Lots of good tools available for managing databases</p></li><li><p>Can be used from non .Net and even non Windows applications</p></li></ul><p>Cons</p><ul><li><p>Doesn’t support Identity Columns in EF</p></li><li><p>Requires extra installation on client computers</p></li><li><p>Generating tables from a model is painful</p></li><li><p>Many model changes require regenerating the database</p></li><li><p>Doesn’t map Integer type correctly</p></li><li><p>Doesn’t support foreign key constraints</p></li><li><p>Generating tables from a model is somewhat painful</p></li></ul><h1 id="SQL-Compact-Edition-3-5"><a href="#SQL-Compact-Edition-3-5" class="headerlink" title="SQL Compact Edition 3.5"></a>SQL Compact Edition 3.5</h1><p>Before I start dumping on SQL CE, I should say that I was trying the 3.5 version, not the fancy new 4.0 version. Maybe they’ve fixed all these issues.</p><p>Creating the model was easy, but as I went through a few iterations of the data model I quickly became frustrated with the inability to change the database to match the new model. I’m sure it’s possible to change it, but you have to write the scripts yourself and hope that you’ve named everything exactly right. Entity Framework created a handy database-generation script, but it won’t run directly on as SQL CE database because it doesn’t support the GO statement. Copy and pasting each section of the script is annoying, but still better than typing all the names manually.</p><p>The real shock was adding data from inside my app. Entity Framework can’t generate data in a SQL CE database if the primary key is an identity. The best workaround I could find was to use a regular old integer for a primary key and search the model for the highest key value every time an entity is created. Not fun at all.</p><h1 id="SQLite"><a href="#SQLite" class="headerlink" title="SQLite"></a>SQLite</h1><p>It took a bit of fooling around to switch my database to SQLite. Most of the creation script ran on the database, though the types had to be changed a bit to match the database. All integers in SQLite are actually Int64 in .Net, but the Entity Framework maps them to Int32 in the model. Modifying the mapping files manually is annoying, but not hard to figure out.</p><p>Once the tables were set up and working, it was easy to copy the data over. The database tool I’m using for SQLite can import CSV files, even the multi-byte Japanese characters came across correctly.</p><p>The only thing left is to study.</p><p><del>You can get a .Net library for SQLite here: <code>https://sqlite.phxsoftware.com/</code></del></p><p>It’s able to integrate with Visual Studio, even the Express version, but it didn’t work well for me. With a good database management tool you don’t need it. I’ve been using SQLite Administrator, which you can get here: <a href="https://sqliteadmin.orbmu2k.de/">https://sqliteadmin.orbmu2k.de/</a></p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;My wife and I have been taking a Japanese class. I’m enjoying it quite a bit, even though learning languages has never been easy for me. </summary>
      
    
    
    
    
    <category term="database" scheme="https://jessemcdowell.ca/tags/database/"/>
    
    <category term="entity framework" scheme="https://jessemcdowell.ca/tags/entity-framework/"/>
    
    <category term=".net" scheme="https://jessemcdowell.ca/tags/net/"/>
    
  </entry>
  
  <entry>
    <title>IE9 Try and Fail</title>
    <link href="https://jessemcdowell.ca/2010/09/ie9-try-and-fail/"/>
    <id>https://jessemcdowell.ca/2010/09/ie9-try-and-fail/</id>
    <published>2010-09-23T03:07:53.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>I decided to give IE9 a try. What’s not to like about a browser that’s supposed to be faster, cleaner, and support new HTML features?</p><p>They really nailed the faster and cleaner parts. I didn’t find the user interface in IE8 too busy, but seeing the improvement I’m happy for the extra space.</p><p>Plunking a site onto my task bar like it were an application is pretty awesome. There are a couple of websites that I’ll do this for without hesitation.I wonder how long it’ll take to see adoption of the magic html jump list task extensions. I’m not terribly optimistic myself; I’m hoping someone will find a way to add tasks and maybe an overlay to sites that don’t have them built in.</p><p>In spite of all of that, sadly, I had to remove it.</p><p>The most severe issue I found was with the Live Mesh Remote Desktop service. My mouse cursor kept flicking to the the busy cursors and back again a few times a second no matter what application I was using.Also, myCPU was running at about 26% where it normally hovers closer to 2%. Closing the browser doesn’t fix it either.</p><p>It took a bit of digging around in Resource Monitor, but I was able to find the problem. It wasn’t a single app running the CPU, it was dozens of processes starting and terminating every second. That explains the mouse cursor. The two apps in question were the Windows error reporting service and the Live Mesh Remote Desktop service.</p><p>The solution was easy: set the Mesh Remove Desktop service to disabled. You won’t be able to access the desktop, but using everything else won’t suck. I don’t use Mesh desktop sharing enough that it’s a problem for me, but anyone who does may want to try it’s new replacement.</p><p>I could have lived with the disabled desktop sharing if it were the only problem, but a few hideous rendering problems in Google Calendar, and WordPress failing to save my posts was a deal breaker. The compatibility mode doesn’t seem to be compatible enough, and tweaking the settings in the developer console (F12) is too much of a nuisance to do every day.</p><p>IE9 came off pretty easy once I figured out how to do it. It’s listed as an update, not as an application, so you need to click on the the “View installed updates” thingy of thePrograms and Features menu.</p><p>I hope they get the bugs worked out soon; it’s only been a day and I’m starting to miss it already.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I decided to give IE9 a try. What’s not to like about a browser that’s supposed to be faster, cleaner, and support new HTML features?&lt;/p&gt;</summary>
      
    
    
    
    <category term="uncategorized" scheme="https://jessemcdowell.ca/categories/uncategorized/"/>
    
    
    <category term="devops" scheme="https://jessemcdowell.ca/tags/devops/"/>
    
  </entry>
  
  <entry>
    <title>Themis: System Design</title>
    <link href="https://jessemcdowell.ca/2010/09/themis-system-design/"/>
    <id>https://jessemcdowell.ca/2010/09/themis-system-design/</id>
    <published>2010-09-15T05:29:40.000Z</published>
    <updated>2024-08-07T05:04:17.761Z</updated>
    
    <content type="html"><![CDATA[<p>I’m charging forward on the Shared Resource Schedule Service. There are a lot of things I’m getting into place before I start writing code. Don’t try to tell me that I’m not being agile either, you always need to design a little to write good code. The key to being agile is designing only as much as you need, and being prepared to change your mind later.</p><h1 id="The-Name"><a href="#The-Name" class="headerlink" title="The Name"></a>The Name</h1><p>The first order of business was to choose a name. It helps to know the name before you create a project. I’m also hoping to put the source in a public repository right away so that I can talk about the coding process while I build it.</p><p>Themis comes from Greek mythology. <a href="https://en.wikipedia.org/wiki/Themis">According toWikipedia</a>, she “is the embodiment of divine order, law, and custom.” That seems somewhat applicable to the subject at hand. I know thatGreek mythology isn’t too creative, but we can always change the name later if anyone has a better idea.</p><h1 id="Application-Structure"><a href="#Application-Structure" class="headerlink" title="Application Structure"></a>Application Structure</h1><img src="/2010/09/themis-system-design/Themis-Application-Architecture.jpg" class="" title="A diagram showing the two main applications and the database that they both reference."><p>This system should be very simple. It’ll have a windows service to check for invitation emails, a web page to view schedules and configure the service, and a database to hold all the information so that both apps can access it. For simplicity, I want to put most of the code into a single assembly.I want to the web UI and the service runner to be a thin veneer over the main assembly.</p><h1 id="Application-Design"><a href="#Application-Design" class="headerlink" title="Application Design"></a>Application Design</h1><p>Although this is just a home project I do want it to be good quality, even if my name weren’t going to be pasted into every source file with the license. To achieve this, I will be using unit tests in as many areas as I can add them; not only to catch my mistakes but to help others understand and maintain the system after I’m gone. To make unit testing easier, I want to use an<a href="https://en.wikipedia.org/wiki/Dependency_injection">IoC &#x2F; DI</a>coding style.</p><p>I’m going to use Entity Framework to keep the application code ignorant about the database. I don’t want to be tied to a particular database system, though I suspect most installations would use a sqlce database. I could use any ORM, but I’ve been wanting to try EF.</p><h1 id="Processing-Mail"><a href="#Processing-Mail" class="headerlink" title="Processing Mail"></a>Processing Mail</h1><p>Now that enough of the design (and some of the non-coding elements) have been straightened out, I can start to really focus on the first part of the project:Processing email.</p><img src="/2010/09/themis-system-design/Themis-Mail-Processing.jpg" class="" title="A sequence diagram showing the mail processing sequence."><p>I want to build the email handling logic as a group of small components,each responsible for a single aspect of the process. Aside from some shared types to hold email or request data, and the controller that knows how to orchestrate them, none of the components will know anything about each other. Even then the controller won’t know which types contain the actual implementation thanks to the IoC container.</p><p>By building each part in isolation I get a few good advantages. First, I’m able to test each component completely and separately from the rest of the system. Second, it becomes possible to replace any part of the chain with nothing more than a configuration change, which will be especially handy if anyone wants to fork the code. Third, I can reuse the individual components for other parts of the system, and they are likely to require less refactoring before they can be reused.</p><p>This is the new coding style: you build a system with complimentary building blocks loosely hooked together. You end up with a lot of classes, but you avoid the pain of complex inheritance trees.</p><h1 id="The-Next-Step"><a href="#The-Next-Step" class="headerlink" title="The Next Step"></a>The Next Step</h1><p>Now that I’ve figured out my approach, I see that my first step is too big for an afternoon. Instead, I’m going to tackle it in three smaller steps:</p><ol><li>Receive any email and send a simple response back.</li><li>Receive event invitations by email and send a human-readable response listing the details of the invitation.</li><li>Receive event invitations by email and send an acceptance email. (The original requirement.)</li></ol><p>All I have left is to make a good cup of tea, turn up the music,and bang some keys.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I’m charging forward on the Shared Resource Schedule Service. There are a lot of things I’m getting into place before I start writing cod</summary>
      
    
    
    
    <category term="projects" scheme="https://jessemcdowell.ca/categories/projects/"/>
    
    <category term="Themis" scheme="https://jessemcdowell.ca/categories/projects/Themis/"/>
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
    <category term="ioc" scheme="https://jessemcdowell.ca/tags/ioc/"/>
    
    <category term="unit testing" scheme="https://jessemcdowell.ca/tags/unit-testing/"/>
    
  </entry>
  
  <entry>
    <title>My First Open Source Project</title>
    <link href="https://jessemcdowell.ca/2010/09/my-first-open-source-project/"/>
    <id>https://jessemcdowell.ca/2010/09/my-first-open-source-project/</id>
    <published>2010-09-05T02:52:29.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>I’ve been spending time on the first part of the shared resource service project: “Receive event invitations by email and send an acceptance email”. I hadn’t considered the difficulty I would face receiving emails. This is the most important part of the application, and if I can’t make it work, there’s really no point to continue.</p><p>I don’t want to write my own POP3 library, I’ve done that before and it’s much more complicated than it appears. Buying a commercial component isn’t going to happen either. That left me with one option: To scour the Internet for an open source library.</p><p>Most of my career has been spent in companies that are paranoid about using third-party code. Finding something was easy enough (though I must say there are surprisingly few .Net POP3 libraries around), and testing a couple libraries didn’t take long at all. The most difficult part was figuring out how to use an open source library properly.</p><p>My favourite of the contenders is released under the GNU LGPL license. This shouldn’t impose any awkward restrictions. I have additional freedom because I’m planning to release my project as open source, but I can see how this would be more difficult for a company building a commercial product.</p><p>Since I was reading about licenses, it seemed as good a time as any to pick one for my project. After a lot of reading, I decided to go with the GNU GPL license. I want my project to be freely available, and I don’t want anyone to use it to make a profit without me being involved. If I were unsure this would be the safest choice anyway, because I always have the ability to switch to a less restrictive license later.</p><p>Now if I can just find an IoC container, a unit testing framework, and a mocking framework that are also compatible with GPL, Ill be ready to get started.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I’ve been spending time on the first part of the shared resource service project: “Receive event invitations by email and send an accepta</summary>
      
    
    
    
    <category term="projects" scheme="https://jessemcdowell.ca/categories/projects/"/>
    
    <category term="Themis" scheme="https://jessemcdowell.ca/categories/projects/Themis/"/>
    
    
    <category term="blogging" scheme="https://jessemcdowell.ca/tags/blogging/"/>
    
    <category term="open source" scheme="https://jessemcdowell.ca/tags/open-source/"/>
    
  </entry>
  
  <entry>
    <title>Changing From an iPod to a Creative Zen</title>
    <link href="https://jessemcdowell.ca/2010/08/changing-from-an-ipod-to-a-creative-zen/"/>
    <id>https://jessemcdowell.ca/2010/08/changing-from-an-ipod-to-a-creative-zen/</id>
    <published>2010-08-31T21:22:15.000Z</published>
    <updated>2024-11-01T20:30:30.000Z</updated>
    
    <content type="html"><![CDATA[<p>The time finally came for me to replace my iPod with something new. I was using an<a href="https://support.apple.com/kb/HT1353#clickwheel">iPod click wheel</a>that I got as a warranty replacement back in 2004.It was good to me, but I felt that it was time to look at other options.If iTunes ran a little better on my PC, I probably would have just bought another iPod without a second thought.</p><h2 id="Hardware"><a href="#Hardware" class="headerlink" title="Hardware"></a>Hardware</h2><p>It took me quite a while to choose a new device. I did look at the iPod Nano, but they couldn’t store quite as much as I wanted.The iPod classic on the other hand is way more storage thanI need, and I prefer the lighter weight, longer battery life, and the improved durability of flash storage.</p><p>The Microsoft Zune is another device I considered, but the windows software for that is even worse than iTunes. For example,songs can have one of three ratings: heart, broken heart, or no rating. I like that they’re trying to simplify the interface, but I really feel like I’m not their target audience.That said, if Zune Pass worked in Canada I might have gone with them anyway.</p><p>The Creative Zen X-Fi gave me everything I wanted. It’s small and light, has 32GB of flash storage and can be easily expanded, and it’s priced competitively. It has an FM radio, and buttons on the outside. I know touch screen is in these days, but I can change songs without even taking it out of my pocket.</p><h2 id="Software"><a href="#Software" class="headerlink" title="Software"></a>Software</h2><p>Replacing iTunes was another big job. I’ve built up a detailed system of smart play lists and ratings over the years and change scares me. I’m willing to try something new, but I need an app with enough features to do the job.</p><p>My search was quickly reduced to two main contenders, <a href="https://www.winamp.com/">Winamp</a> and <a href="https://www.mediamonkey.com/">Media Monkey</a>. Both had automatic play lists, song ratings, and file type conversion features.</p><p>I ended up downloading both and doing several tests before choosing Media Monkey. Both applications were comparable for features, but Media Monkey was a little better with syncing, and a little more straightforward for organising songs and play lists.</p><p>Converting to Media Monkey was easy, and all the data (like ratings, play counts, album art) came over fine. I did have some trouble with the comments getting scrambled in several tracks, but it seems like iTunes was causing that well before copying the data over. All told, I had all the data over and fixed up after a half day of elbow grease.</p><h2 id="Verdict"><a href="#Verdict" class="headerlink" title="Verdict"></a>Verdict</h2><p>My only real problem so far has been with the Zen. I’ve had to reboot the device a couple times when it’s hiccupped during a huge sync operation or failed to come back out of sleep.The UI can be a bit difficult sometimes as well. I won’t get into details, but let me just say that converting a WEP key to stars while typing with 9 buttons isn’t making my life any easier.</p><p>All in all, I’m pretty happy with the change so far. Just getting the weight of my old 20 GB monster out of my pocket has been nice.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;The time finally came for me to replace my iPod with something new. I was using an&lt;a href=&quot;https://support.apple.com/kb/HT1353#clickwheel</summary>
      
    
    
    
    
    <category term="gadgets" scheme="https://jessemcdowell.ca/tags/gadgets/"/>
    
  </entry>
  
  <entry>
    <title>SQL Management Studio Express with VS 2010 Express</title>
    <link href="https://jessemcdowell.ca/2010/08/ssmse-with-vs2010-express/"/>
    <id>https://jessemcdowell.ca/2010/08/ssmse-with-vs2010-express/</id>
    <published>2010-08-11T03:16:53.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>I had some fun trying to install SQL Management Studio on my home computer over the weekend. I was successful in the end, but it was a rough journey.</p><p>These seem to be the things that can lead you into my scenario:</p><ul><li>Windows 7</li><li>Visual Web Developer 2010 Express (which installs SQL Server 2008 Express)</li></ul><p>These are the things I ran into trying to install SQL Management Studio (both from an SQL 2008 standard disk, and a download of the express version from the Microsoft web site):</p><ul><li>A warning that the installer was not compatible with Windows 7. The warning leads you to the SQL 2008 SP1 installer, but that can’t install management studio.</li><li>The installer fails to load with this error: “Invoke or BeginInvoke cannot be called on a control until the window handle has been created.”(If you keep retrying you can eventually get past this error, but it seems to be possible at three different stage of the install process.)</li><li>A reboot is required before proceeding, even when running the installer immediately after a reboot.</li><li>When given a list of modules to add, SQL Management Studio isn’t an option on the list.</li></ul><p>What a frustrating evening!</p><p>The thing that finally worked turned out to be pretty simple though:</p><ol><li>Launch the Microsoft Web Platform Installer (which you need to install to have Visual Web Developer 2010 Express, as far as I know).</li><li>Go to the Web Platform tab, and click “Customize” on the Database option.</li><li>Check “SQL Server 2008 R2 Management Studio Express”, and click Install.</li><li>Stand back and let the web platform installer do it’s thing.</li></ol><p>I wish I’d tried that the first time.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I had some fun trying to install SQL Management Studio on my home computer over the weekend. I was successful in the end, but it was a ro</summary>
      
    
    
    
    
    <category term="sql" scheme="https://jessemcdowell.ca/tags/sql/"/>
    
    <category term="devops" scheme="https://jessemcdowell.ca/tags/devops/"/>
    
  </entry>
  
  <entry>
    <title>Shared Resource Service Requirements</title>
    <link href="https://jessemcdowell.ca/2010/08/shared-resource-service-requirements/"/>
    <id>https://jessemcdowell.ca/2010/08/shared-resource-service-requirements/</id>
    <published>2010-08-09T04:12:53.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>I want to start my project by dividing up the various things the system needs to do, then put them into the order I plan to approach them. I want to start with requirements that are the most risky and most important to success first, and work my way down the list until I finish with items that are neither risky nor important.</p><p>This approach gives me several advantages. It’s not an issue in this case since I don’t have any deadlines to meet, but if this were a professional project and a deadline had to be pushed up, I am more likely to have a product that does the most essential things. If I had a fixed deadline and the project starts running long, I’m more likely to have a working product that’s just missing a few less important features. In an ideal world those don’t happen, but at the end of the project the parts that were risky and&#x2F;or essential have had the most time to be tested and fine-tuned as the application matures.</p><p>I could just charge in and write some code, and sometimes the results can turn out okay, but in my experience it never saves time. If anything, it just increases the chance that the project will fail. On the other hand, I don’t want to do too much design up front. I will know more and more about the problem space the further I get into the project, so the further back I can push parts of the design, the more likely I am to get them right, or at least be close,the first time.</p><p>After a couple days of thinking and fooling around with the list, here is what I end up with in the general order I plan to approach them:</p><ol><li>Receive event invitations by email and send an acceptance email.</li><li>Record accepted invitations, and send a rejection email when a new invitation conflicts with an existing one.</li><li>If an event is cancelled, remove it from the record.</li><li>If an event is rescheduled, try to accept the new time and remove the previous time.</li><li>Receive invitations for multiple resources.</li><li>Users can view the schedule for resources.</li><li>Administrators can view, add, edit, and remove resources or change other options for the service.</li><li>Save free&#x2F;busy information for all resources in a format compatible with MS Outlook.</li><li>The schedule for a resource can be retrieved in the iCal format.</li></ol>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I want to start my project by dividing up the various things the system needs to do, then put them into the order I plan to approach them</summary>
      
    
    
    
    <category term="projects" scheme="https://jessemcdowell.ca/categories/projects/"/>
    
    <category term="Themis" scheme="https://jessemcdowell.ca/categories/projects/Themis/"/>
    
    
    <category term="design" scheme="https://jessemcdowell.ca/tags/design/"/>
    
  </entry>
  
  <entry>
    <title>Shared Resource Schedule Service</title>
    <link href="https://jessemcdowell.ca/2010/07/shared-resource-schedule-service/"/>
    <id>https://jessemcdowell.ca/2010/07/shared-resource-schedule-service/</id>
    <published>2010-07-29T04:53:02.000Z</published>
    <updated>2021-01-30T03:45:23.000Z</updated>
    
    <content type="html"><![CDATA[<p>I think I’ve got a new project to work on at home. We’ve got this problem at work coordinating a few shared resources in Outlook (like a couple of meeting rooms and a projector), but we couldn’t find any good solutions out there short of installing Exchange Server.</p><p>I’ve been wanting to play around with Entity Framework a bit, so this should be a good candidate. Perhaps I can slap a UI on it using Silverlight and Ria Services.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I think I’ve got a new project to work on at home. We’ve got this problem at work coordinating a few shared resources in Outlook (like a </summary>
      
    
    
    
    <category term="projects" scheme="https://jessemcdowell.ca/categories/projects/"/>
    
    <category term="Themis" scheme="https://jessemcdowell.ca/categories/projects/Themis/"/>
    
    
  </entry>
  
  <entry>
    <title>My First Post</title>
    <link href="https://jessemcdowell.ca/2010/07/my-first-post/"/>
    <id>https://jessemcdowell.ca/2010/07/my-first-post/</id>
    <published>2010-07-28T05:22:21.000Z</published>
    <updated>2024-08-07T05:04:17.757Z</updated>
    
    <content type="html"><![CDATA[<p>I’ve finally gone and done it.</p><p>To be honest, creating my own blog wasn’t foremost in my thoughts until fairly recently. Scott Hanselman had <a href="https://www.hanselman.com/blog/FoundVideoSocialNetworkingForDevelopersAndMakingYourBlogSuckLess.aspx">a couple of interesting videos</a> on his blog that finally convinced me to do it.</p><p>I hope you like it.</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;I’ve finally gone and done it.&lt;/p&gt;
&lt;p&gt;To be honest, creating my own blog wasn’t foremost in my thoughts until fairly recently. Scott Hans</summary>
      
    
    
    
    
    <category term="blogging" scheme="https://jessemcdowell.ca/tags/blogging/"/>
    
  </entry>
  
</feed>