Chip Wafer Analysis

This project uses AWS machine learning and IoT tools to develop a deep learning defect classification model and use it for real-time defect detection on a device.

Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
SPDX-License-Identifier: MIT-0

Chip wafer maps show a visual representation of a chip wafer produced in a semiconductory foundry (fab). The maps are generated by microscopic cameras or electronic line scanners that probe for faults.

The fabs look for common defect patterns in this map as a quality control measure. They use either manual inspection or appliances that scan for defect patterns using pattern recognition software. These defect detection methods have several problems:

• Human inspection is not real-time. Shipping wafers with defects is costly.

• The appliances are expensive.

• The appliances do not have redundancy.

• Fabs cannot easily improve the accuracy of the defect detection or account for new defect patterns.

This project uses AWS machine learning and IoT tools to develop a deep learning defect classification model and use it for real-time defect detection on a device.

License Summary

This sample code is made available under the MIT-0 license. See the LICENSE file.

Data set attribution

The data set we use is:

[Qingyi](https://www.kaggle.com/qingyi). (February 2018). WM-811K wafer map, Version 1. Retrieved January 2018 from https://www.kaggle.com/qingyi/wm811k-wafer-map/downloads/wm811k-wafer-map.zip/1.

Architecture

IoT

Each device (a Raspberry Pi) runs the GreenGrass Core software. Devices publish two kinds of messages:

• Raw images. These go to the topic fabwafer/<fabid>/<cameraid>/img/<imgid>.

• Classifications. These go to the topic fabwafer/<fabid>/<cameraid>/prediction/<imgid>

Here are some sample messages you can send to the topic fabwafer/faba/camera1/prediction/img1 to test the notifications. The first two should not cause an alert, but the last should. All should write into the DynamoDB table.

---

{ "imgid": "img1", "timestamp": 1554134552944, "fab": "faba", "camera": "camera1", "prediction": "none", "probability": 0.9 } { "imgid": "img2", "timestamp": 1554134552945, "fab": "faba", "camera": "camera1", "prediction": "loc", "probability": 0.4 } { "imgid": "img3", "timestamp": 1554134552946, "fab": "faba", "camera": "camera1", "prediction": "loc", "probability": 0.9 } ---

For the raw image topic, here’s a sample message. The bytes field is base64-encoded.

---

{ "imgid": "img3", "timestamp": 1554134552946, "fab": "faba", "camera": "camera1", "bytes": "iVBORw0KGgoAAAANSUhEUgAAAJYAAACWCAYAAAA8AXHiAAAACXBIWXMAABYlAAAWJQFJUiTwAAAgAElEQVR4nO1dCZhUxbU+3UzAFxhQI8MyA7Kvw74vSoRB1BhZTNSEzZCYKLgkz4hgjOHLFxU1efHlEf3ysijjGiOLCiq7UVE2Cfu+hX1xiTPgc1DS7zvVfW9X1fnv7Z6ll2FufV/P3D73VNU5VX/XcupUVWjYo+0iIQqRG0Kkf4uGUEjSomSb4kYWrxS/pAIShUAiBlsskkkDfBYR80sBQoARy6knGgK0JHUKAT7tnUwuhOURtBDMI84viSZ/CNDEI6yPMP+JUCROiZD+LUaLSFqUbFPcyOKV4pdUQIrLY4olIpk0wGcRMb8UIAIYsZx6ohFAS1KnCODT3snkIlgeQYvAPOL8kmjyRwBNPML6CAMRsDIBuDzklCUfgEsDFtkKIWUCcHnIKUu+poMrLOUPwAWSNxixnLLkazK4wkj5AFx+MgXgQmnY9REmD+UDcPnJFIALpaHHDIM4SASsTAAuDzllyaPyTahTNQaXNXiXGQXg8pMpABdKIxIdvIPiFvIH4ALJG4xYTlnyqHwT6lQNwRUbvIPiFvIH4ALJG4xYTlnyqHwT6lTNwKUN3kFxC/kDcIHkDUYspyx5VL4JdapG4LIG76C4hfwBuEDyBiOWU9QGLN+EOlUTcIHBOyhuIX8ALpC8wYjlFLUByzehTtUAXGHIGIALvwvAhevDeIh+CwO+AFwBuGAecX5JtMEVxgkH4ArAVTlwxQfvki8AVwAumEecXxIdijl4l3wBuAJwwTzi/JIYiQLLKkbJF4ArABfMI84viWGkAeALwBWAC+YR5zeJYS8NZMIBuAJwJQ+usMkYgAtgIAAXzhUl4tZHmIQwAbgABgJw4VxRIqo+wuI90EAmHIArAJc/uHJsZULuQ8TYgsgJ2/sbo2zmCxBVieBunWRl7E2YDC6wKVbmGY8cAXswGVz2JkxOo3VeB6pbJ5daNexI9erkUqP6+erDgektL2kvCiaZ8NnZ07Tv1A7FeeZsKe3/MPq8/8OddKaslPZ/tEP9tzUz9YpqInW1yzLGB8rXKGPtnRVV1gV51IcCl9wUK2XUUrNkCg15qI3AG9oN65Lk5tkYG9iDLDb1pnbHddeCvtQqrwM1rp9PrRp2UB8GTqbD1qPr6GTpUTpZcpS2HF3rAu983nEduvzB1pFkt1p7J5x+cDWqX0Bdm/elrgV9qHXDjtSqYcVanUyFU6XHVAu35eg62hoD2/kELgUsKsc+fu+EUwsu7sK6Ne9LA9oUUbeCvpRXv6nIvzoH7la3HFlLaw6soC1H1tGp00eFNtUJXC6wKAvBlVsnlwa2KaKBbYbRgDbDRF7lDdwVnSg5osZDztjodBk/b/evCO2bTstr0JTycvMVjYHOzxwK83tXWtYDH+2iFTtfoTX7V6hu1JCiGoDLABZlCbi6NetLVxaOoeGdR4m0E4UzZVHQ7D21Q4Fo38kdqmL4ObHcPhWhRQJDIwFCNSlo2IEa5TZVoOuS34fycptSw9zyt7Rbj36gQLZi56vxvLIdXDawKIPgurLLaJow8A5qVI5ujkG06dAa2ntqJ206vEYBCIiDdZIk/4rQIiUDLvDPBVxh094KbJ2bJt+6cXe5YNOztGDzc/RZWalACCrfhDqlCFyhy37Fg3fBhyvCeKg6cI0oHEMTBiUHKO7ONh5aQ+/tWUqbD69RXZmRuixDf50kyb8itEgVBZdJC6mus1/LoQpsLZIwfTgAW7j5OdWt6wmi8k2oUwrApYBlE+OMkliV4GrcIJ+mXj2TujbrK+Lp4UTJUQWkJVvn0d6TOxLkWb0Pf+Pusl+roTS0w3XU4mv+IOOZ5awVP6etx9YZCYKqSaxTFYPLBRZ5FTAgVgW4riocQ5OH3edrZ1qydT4t3jJXtVBO4iB5kOf5cbIgt+DXdh1HQzuMpK/WrifeO4Fbrqfee8xIEFRNYp2qEFwGsMirgAGxMuDirm/qNQ8LHooNvud+MJvmrium02dLpDI19NhKBteNvW+jhrlNBA+Ht3a+RrPe+rmRIKiaxDpVFbhsYJFXAQNiRcA1osv1nqAqfm8WzV03W4HLV5kafCYqA2zSoKmwBcsmcNVqPvSiGSKyVwEDYnnAxWOq//rOMyINHjdNf/kH9NaOhXT2XJlUCClTQ8HFFvpF216mggtbUf5FLQweHvifKj1KBz7aaSQIqiaxTpUElzzcNhbkaraHB4Hx4O8VMXHwnSL+4i3z6IdPj6S9J7Zb6QZeESB5Fc6UldDMN39Ms1Y8IPhu6H2bcG0AVZNYp0p6RYSTVSZOk8RkwTWicLTxjgflj7w+zStqAC5fmSK0fMer9JeVjxk8PAbrxLaxDINLc00GkYUyDk0Sy9NyOWH2u/8j0xDyB+ACybuMCzY9J3jYZGGWZfrBFdaZYGShjEOTxPKCq33jQvECRA3A5SNTHpglOkbTTIIrXBFl4jRJ9AOXHcYNnEyt8zqKxFHUAFxSJrYBTrv6cfFOn1VnClxhKqcykiaJXuDaeHCNwccF8/jY5xXAAnCJfxbNjDG4zQh6cORT0DrPjoUiiTSDS/q8+yhDXgUMiAhcp8tKBN9Xa9el7132Y/rTpAXUJb+nTEPIX7PBxYPz6Vc/Tndf+Sg1/1pbIQ+vI2I5RaXA8k2oU5LgytEdmSO6+SL2RfhIE/J99vA3d9JTDxHac2I7DWpbJBTh0LJhO3p87Au049hGennd07Ri+ukRTUMJ5n2oRe6/mba5Fw+er5mkT9XV69pnRX0YPUJq8z1cm5QJSfExy/eyxn+nzoc2wpsPJVA67jnx4WBWGHDk260f3f/C3dMuQeWrR5Li3aMpeOf3pEKF8TwFX3glzq33IoXddtXFJeD+SOr2KggXKmB1w5ccbUg+v4v0xnO7/AC7ATBt2uPuzdsHL3Ulq5e0l8Qfo8BdfgtldRj2YDqXN+b2rSoJlPCeGw37W6ZxZcOSZjasGFWqxHFk6j7s370Yguo8U7JzDIxvSaoD4c2LFvw6E1tPHgarUc5M6CqiG42NmPHf/6tbqC2jXq4tvN6YEH6C+sfZLuu/q/jXVD9lczdMoQuHKMjCi14Dr2qWyxuJubufBeevrd39HNl91Jg9sW+brScGD/LeXDNfB29Z1btBOfHqGNh1ar9ByXZHeykAXgKizoQ/Xq1KdWl7RX29Ly6udXaD/j6v0r6LWNz6qNF5y2vRh9svRIVuxbzMGMqQMXtzC8gdQJ3Zv3pQ0Ho4CYueBeyr2gPg1uV6QG+V4DfTtwi8afrs36iHfsunz681Lae2q78jYNKRm2u56nvO7mbKxwlUwSXNGNrzGv11B0X6MTuhT0cb1DKxt44Xn59ldo1f7lyn/fkamwqdTXabEyDa4cEsTUguv056bJod4F9Y3vpZ+X0Jub56oP/8IZZN2a90uqJUOBWwdSrZyshGwOvAVs9b7ltGr/shhYYmWo1SsqD31HTybBlYOIqQTX7hPb1f5AJ7Rhy7s9L4qlwV2ZA7JHiKhtXkfqdmk/FYfBVp5NF9kczvCewsNraR9vYD28ljbHujnymC06/yu9MCHO4WWmQJXDiJSCsFlG0lViwU0QGnsPrmd9pzc7r7gFq11ow4KcLw0xP5e/B/9krMlcLemxoCndqhn/n+St6bZ/lwJTBH8r25ts7XnNcLE9ZEecIHBO2KsOnBt+OdqosF3uDR3vAU0gGlQ/CASBikvE/EnRKbyvDeRZXJaRwYcA5FS2C06G2I5cOvDs1X+7gDps7LTUUZNTq/ZYlxZb3C1tI4V2HdqZ4wj8+DK0alpAZeKYIZGDfLVrA5pgNNIfMqNY/S7V4EPB+5xVQL4VaQXqQh8eSARtFkBJAGeVaEriwuXwmuPGvz65mzJeWoj9SCK8dWL9XgUi2WFRo3KIha16VWMA1KElyG4mRM+NzAkwkGH8CAyFNHpv0KzYCx3LgiKgIuG1g84Df5Mwcu7bLxCImnCAEa4NO+oBbJXhQ1NkvwAL5RRysPkbpII84WATSbz1REJFUNF67RoSiYXwpgOwtKmvbGAgMqX6ST4UGaLnDttvzbdZNDAC4zEi5fcg8g0QMbTTG/FCDV4LIOtzVjAFKVgMu2ZbVtZI5xAnCZkVD5trykg05V7jIguYyBCxxua8YApEqDa8+JbQatbp36kg9rZWdr8tcgcNkmFXf1QCaXEXD5mBsiKRvQH7O8HHhZBw5+jXwjaRvQc+ID2hbFjLekZqx8dsRpcbxj5rwiWlktFq8RaoUuZfKpj1QM6D3WCsUDIlUYXMjLwbMi0gwutnX9+qZidynICbeWTad7Xpqg1jqzAVx2ixX3asgOcFmDd9HapaRbtAfvHLpf2i+WjGxb09ktzhj1ewEqilXkYzcUqyMrZZ7p7xbtxe0TpXovEEH/LJoUoCq7xbCvMvIBkcoNLnvwziFXG2fBisBa2dma/OUEFx+S62eVZ3CNi7nqyDzTBy60ZMUtFtAWy5QGcJlb7JEy8gGRyg2uPVar1caeGQJtUg2uQW3lOae2za2bdpaXzDM94GoF/LicwTvQFsuUYnBpu3Rk7hJAADSQH/BZypy2KqwemhkCbdLRcunBbh24m3zsxmJ14C5IOi3gQrNo9QOQUQSqvOpD5A8YsZyiUtRf6xZ7mbsEEAAN5Ad82pd//HOVIWCbxp2w4ICYKnApz4kkAp8t/4uRs+iZW5bRlZ1HgzxTCy67xdqvu8vIKIKI6qOqwQVusZe5SwAB0EB+wCe+RAO7vHgKDoipABdv2LCPovQLvKxy91UPKYCNG3C7Gtgbqcui9NdJkmB95NU3re7cWqHyBdridykAl8ct9jJ3CSAAGsgP+LjFOmC2WA6wPAUHxFSA69E3pikfejt8ee4LQXMCA2zcgCk0+wfL6O6rHnbv6UkVuBpZwOJDfkUeMN/0gSvHYZL2Ch+7inxAJI0m7Vzsh24HXtpxTBFQJmQTqmI7F/vD/jpka7PPY/9eJPGoq3z6LPPS2m02i00UYy9KDYeG95plPq8v3cZzVtfrHYUVbWdCy1AIyOq88UqlbTYuUJ9H7jUxZ50E6EkT30L+bzTaSbfO/fvNfK669mx9A/LrQbKBIhmviGLX7DH2EKAZvOZijhGVAYYX3KQyD2aZ2sMMD6oFxSlv06SpORZcNcW9fzlv7+gnPBX6NDH+2jGq7dhT1TtC9AWv3NpUoAQYERy1sof0mCGAQzAlCpw3dhvEtXOqeN+ZxPEtiMbZP5IJkBMF7i+OFemWqJ5H8xWjn6t8zpZ46t4uKjuJWoGObrnBKpdq47qZp3dQkIeIKRN4lnpNV1uVM/hUC31v8F/XEQju49XaTKQvzh31iwP8gBQCsEVA5aZLNAvJeDq3/pyanxhgft+29ENaraYrjNR42yguBOAy/nGg30G2KbDa5XDoj3+cQL/gHgvJAOMPVbPfllGhz/ZL/igThqp4OKWVNRpFJ379zkKh8y5F285u7z9NbT/1A53tw7AQFrAVavp5Q1mhAAKgH5VDq4eLfobLjP87o1Nc5JUxqFJYrrBRbGF6sVb56suL+ruLJeFnNDs4pY0pP01ylTBiXBX5rQynjrFSANaF1GvSweryUStcI7g43HesI6jVHe9/uBKszy0L0Bb/K6C4HJbrEyAq23jTtQjtkZIsV3Rb2ycUw5lHJokZgJcHE6fLaX39yyj+euLFVi4FUMDfYqBoHeLwXRDn1tEKwZ1CpHi55bJAdVnZ8/QV2rVFrztG3dVG2hX7VtOZ8+d9QCLNxHzY5lsRqZpXWH6wdXkwgK6rP1wl8bd4lNv/66cyjg0ScwUuDgNBpUah60vVut43Ip5dZNktWK8uYRbsfg9OfEwdsAUI523d71Bz69+knpdehnVzjEBxrNHbt3e3v1G2sFlWt6BQQrbK6QhREYF9iuL/9i/pPuM66actF3FoUliKuxcQqYknAUXb51H9/x1Ak34YxEt2TZfrD/qgcEwqsd4evr7S2jW2Lk0qscEA0j2XkKePLy/dylNemq4ap3swJtaHx7zlFkepJevNxHzC22hnctosZyQrpartKyExg+61Uhz9d63VZdoR6quLZdOY0C9t2cZvb7pRTr0yX5lFOZZo1fgd71aDFZA47EVTwCKOo00uJdun69mgnzxwju73lSD9v6thxo8F331EiXH5tguHtDAVHnLFZb4S1/LhdxnuMUCP4DzouVyQunnpeoms8nPjKYpz4xO2IqRMjO0px8OuVc9n/v3l1T2ZfQGD/2CT85t6bb5NH3O90T8YZ1Gebo5C1oVtFxqVgh/bWlquYZ0uJIurtfQfXfwo32ukbRifbxDk8Rsabn0PD/57EM12H9p7R/pRMkxJTePt/wCmxlyYoN3Nj2wGYPtWwUXt6Kv5NRW8nA6usMiTxReWP2EoSwuX28i5peaheJHRQK/b8OMry3dQDN+xZd/Sq1Wq4m+Zqgrk/Ryg0MDSyWGKJVf/hH6e5VlkudzLdk6V7Vk9S7IpeGdR9PwTqOTup2fD2/jT9IhDcs/Yb29k41b6rtFezGaZ4peC9fnY7co84yodVSeTU5+djTd/Ofh9Ie/z3TPZaho4F3SEZAxLl9vIuYXSjizwsyBy16Mdi3xNRxcDuPxkiPKJjbl2dH01s6FIm4y4UzZafrft2eaumsZ4/L1JmJ+M0b8fCytvRNNeQq7xV3WHkOeKcXZpFeEKSpuhoVM1bBb1CM7Wfzrs4+NOK9vfokOf7xfjaV4MZz/68ZYniFuPryWnl/1RHSQb9eHljEu34p3izmm8ukHlz3GophFfvfxbQG4LHD94a2H1Ycoeiwlg0U/wc8JeCLiUR+UGnC5279Ex4Oa8hR0iwwgOzj+73E2IZ0lqhQAygSI1alb1F9tOrxGAQvqJEn+9aFFwuXrTcT8EfP2L1F9qEBSAK7j1mnKPbX1w5oALna50d1uZJ4muLi7q2vwiwggjQT1oUXC5etNRPw5Nn8mukVe2tFdk00mnf/87BaHF46mW78+Xbne8Poib+e3d1zzfseBrYtiNqv29MtX71Beqr46Qbl96kOLhMs3+W5RbKbIRMtlrxmyO403v5DOElUKAGUCxEy1XM4x3rz7h33nb73iPpHnT0c8TKN6jnftWl21/Y2+OkmSf31okXD5ehP1d2FUEekGlw2sXH1ZB/Cfb+DqZoFkk36tS4xx42HzSj4GIxAH6yRJvvWhR8Ll60103nleNp5OcNkDeHzCn/VYzcDFExLDLytWluiUZ+Pc1FjSmywat1zqHAlLlrrqbAkpJCB51ocdCZevNzGS6LLxdIELmRwc9xmQbLUEF/4mX/nWvrNTcVql0/0Zllyd1Q7gRejbWARuESUtC60S0Ff+tHXp9Pvx82jlyevjrZmQEhAgvVRFeAybqZAg19Kw4B+FzA5tGvckdYfWE1GahEw4KwmA3qnu3PvAYrdAWRvwlCgAnYuZbMqOWps/eIB/wPXmRe2U2ygz+YIqBOUu+oH9EldNp7qlgu5zzRpUEAgV/zLqgYtF9+kYQe2ltvdoNvlATuX3ZLB/YUxv3hfnSSpyluupC8bTzW47P2E8d07IlesfBaDi08GTPZ6lr36+REWuPiiqWSC7REBdZKkKgWXeeWJ9gV1ISnvFrUQ9XKQfSAgZX23uEftrh6lbsngy6aQqcAJo3tNNFsmrVtUu3o8Ao/N2A723t5lYqDvqROUu2q6xRxE9KuIVIFr/YFV1KNFvLto4no5SCQBUrUAF3/mrJutYg3iW82a9VODer0148E8A2+TBa7hhWPESYM85pq3frbi3etejZcdlxwYR0XiJg62mbKDqmS3WPr5p8Z3XoiWUUWuuNmuBmOulbuW0hPLH6QH5k0WcSZfcZ+RMpsqbrtiusHDLdRtxaNo7gfFGqjIqAggDtZJkirdLYLBO2KEOctqrgS47JmhM1uSUUWuWPksANfg2KEiXuDiPzymWrxlnvGuVV4HGtNzopsynxNhD/LZEdA5vE6KGK0ILsMBreUphVAnSaoUuEzLu0zHtyKqElxoK1jP2NKOjCpyxcpnEFxXdRlDvxzzBP32u88mBNcTyx8Smyn4kvV6tetTXoN8Gh8799QJbKZ45r1ZIk891KtTjx79drEyRxSBsRnUSZIqDC5peZfp+FZEVYELAQuIhR4QKaPgYlBNvSbqsclb7ROBi80txStNexS3UAyuybF1Qz08ufwhIHU8aW6pHr2h2J0d3n3lg2kHlzyDlChj4DpuXSygWiwpFnpApIyAi7007v3GTIMnGXDxoN4+8I27QNsyv+nQWuX9gOxcFDuN5onx88VA/9Yh0yA/1EmSyg0ufAYpUUbAZbdavFvFUxn5gEhpBxdfjvDIwmmCJxlwPfr6vSKeHZ5c8ZAhoJ4K58Fn0duGU+5mp7400ZQ1kU6SVC5wGTdTOJFD1hwzAqeVctpOlTRF7Dq+1TA5tGvc2X/qKx8QSaOlxxTx5uboiTmo5frjpFeEY2OyQc0EQdfoiNza8nknF1QTYjPH9N2gEd1MkSXgshej4wvRPsrIB0TKGnDxOQx+h4P4BQaN3wUHdjBBRUblpBpc8du/IjKy9oUEi08XUtFucdcx0+TQtnH87CzfZlg+IFLau0UG16LN8wRPugKbJPaIU6DTY+cKG8QMg8s2kpJuga+G4OLZ4YgucjaWrsDeqJk4h55sn/dMd4vrrV3RFAOWPqj3bYblAyKlpVsc0XUM3fuNR4Q+7Mv+7q4lsgBi+V/f+2YxTnICzxoXbZmLoqrAy0P2bJDPoafYUUp+W8tkfUidoO4e9ZFjJ5tpcNlBb7GEPBAs4gGRUgquq7qOoWkeoPrx8+NcNyE7DbbUe4GKYm420UsOtsODSOZ9UAyvxMsEuGIXCEQsRkFKW7dot1oMLAQ4v2YYPCBSSrpFtmN5g2qs4XtmpzGl6Gcinh2mDL1PyhQry9NlJfTTFyfEb1vVwvg031oWjjNmB7jsmSGbHJJVxniUD4hU5eDibnvmQtMepUD13Fg6/X8lolCdNCYOvkPMFotXzqKVu5cZNPZ84C5PyOQDLp4dznhlishTRBbSOfySCkhGfYRNxsyDy54ZukbSagQuPqDXAReD6q7nxsZ/MKBQ+Tavb/W+2aAxGOasfVp5QNhh8tCfabvF/cHF6fw0djOsv9xVC64cmzFk+xmlecxluylHF6K18VIGrmdJpBOSicHFJ+lsOLja1cnMN5767cN+JsZWv1/2oDpKk0HCLRevGzqBx1pjek901xfRplgHXHxQrvJKreBBJCZ/8mMusWE10y0XW9/tcMsVPzFbHZikFEK2TiJXD37Al0AnJNM7OxeLH4qZb0RtdbNNEty6vLl5rsvGLZe9jnh9r4nmxVYeLZfr6uyxtijlrpqWq1beoNwZNuLInbGYLwBJzE4ESwjwae9CFiN3Gdf3GW9chcKtVvsmnWnL4fXGeVpQbiBEyPsBvNNpgC+BTlAmQLys3XC6rsd36ebBd8RPio6FX736E2PZhw+uPVNW4o6tKHbTxaC2w9U58vz8yZkPFR+aLdoCChqU29sT1UsnnRTqMrVpBCecOXB9Z+D36T+vfkCwcli44WV68f2/uI6BUG4ghAQQAA3kB3wJdEIysUds9+b9FKDY990rsDnh/jmTYRq8iG3vmtYDuyhz/I2H1sZbqgyBywUWTjhz4PrF9b+ha7t/S7A6Yffx7bRgw9/o79sXK48CmaQUQgIIgAbyA74EOullya3KX6e8JXhQ+Mnz49W4zE6DLD+vRGH4Y5otKwPgqtVoUO4Mm4gY0w0uBgwL3atlf8HK4Wv1GtKANkPopgGT6Nru31b2rjo5ddS1Ia4BEgghAQRAA/kBXwKdnCg81rmm6/Wiy0OB7+TZoO161utjRJcx1KlpdxDLDDxGe23jC6I+hIQpBFetRoPqzRCgkXwZAdcH+1fR33csppYN20ALvBO4wgoLetDwwm/STf2jQOvZYgBdekkryr2ggRqDfHzmlC0CekCkKgEX33lt39TPpgAeG+mA4zScgbudxg+/fg9drF04wAN6TsPeTb1g44vuFrJMgSt2BqllYkBTyAyZItiu9aO/3Ei9W/anmwZ8X50Lnyg0vjBffS7vMNzg5M0HzgEkuoU/evE5Ki6jWK0HonaNOrmg4DEU29y429vwz9X00GtTjbJkGndl7+5aqro6/vD9jN/qczPdrlnco+MvWR9NLswXN4rNfvd3tGjLPLUhluPx4J7HYDzO8qoPsZQGtvMTxED5TBGhwqlNIvGCy86WS6fxEUfX9vi2+uhuNdkUuCu785nvgnKzyzKkWrE/TXrVoP98zmR6d/cSo8QYlLZv13WP91azRTvxZOpDiAPAheVOruWyNqzKX0q2tFwOjc0RL7z/Z/VhkA3pOEKNw3q1GKBaqWwI4nRCz7KMqFaLuzPdQMqzx6gHRLzEultnP/A4isduqjKtxJOpj1S3XNZpM9UDXA4ng2zBP/6mPiwjA61t484KaLzGyOOyTLRq0SPFpU64LCP0zq4lqkVyQnfnDFatULtbJgpn5hhnyy5w5ei81RFcujwMtPUH3jfGTyw3A0z/8IC+XeNObiL6ZZzlCWzEPB7zFSstK6U9sfFbdMyGdUJl6Yy/eGzEh6MwaOzln/vnTlZuNdxy8Xhq40HzEJVsA1eo8J4mpkHe6D+zf8wlWMEoG8oNhJBRRa4e/IAvgU5QJkA0802iPsALEFXUh0gKgAsk7TnmEmuF5jqRtWYI1oSyzZ/La2uZ4ANCyKgiVw9+wJdAJygTINpriwnTAC9AVFEfIqlKri2GkTIBuOADeBeAy47svAp7KROACz6AdwG47MgRY5eO5AzAhR/AuwBcdmTrsnHBGYALP4B3Abj0yGLwjpQJwAUfwLsAXE5kj8vGATUAF3pApBoPLorfTCEDUiYAF3wA7wJwhUEZaoyAGoALPSBSjQZXWI+FGQE1ABd6QKQaC66wHQszAmoALhFkYRkAAAEXSURBVPSASDUSXGFDVVmGWnxADcCFHmo8uEgO3gNwecoNhJBRRa4e/IAvgU5QJkDMFnCBwXsALk+5gRAyqsjVgx/wJdAJygSI2QAuj5spAnB5yg2EkFFFrh78gC+BTlAmQMw0uHxupgjA5Sk3EEJGFbl68AO+BDpBmQAxk+DyuFZOxhKRPZQJwAUfEOm8Bpe0vIvyCsDlKTcQQkYVuXrwA74EOkGZADET4DIPt7W+BOAyU4dyAyFkVJGrBz/gS6ATlAkQ0w2ucGJlZCwgB1QmABd8AO/OP3CFk1PGiiXT9lQmABd8AO/OL3CFk1dGIwiavzIBuOADeHf+gMscvAfgCsDllQZ4AaK6MsnBewCuAFyVBRdF6P8BlPe3s/zM1zAAAAAASUVORK5CYII=" } ---

DynamoDB

The classification table schema is:

• imgid (hash key)

• timestamp (range key)

• fab

• camera

• prediction

• probability

Data

The source data is from the Kaggle competition. Place this data into an S3 bucket organized into train and valid subdirectories. The notebook DataPrep.ipynb documents the data preparation steps.

Setup

First, create an S3 bucket to hold the CloudFormation templates.

aws s3 mb s3://<template bucket>

Now create the stack:

./scripts/create.sh <template bucket> <template prefix> <stack name> <region>

Note the CodeCommit repo output from the stack and check in the code from the pytorch_code, test_code, deploy_code, and trainer_code directories.

cd ..
git clone <clone URL>
cd ChipWaferMLRepo
cp -r ../ChipWaferAnalysis/pytorch_code/ .
git add .
git commit -m "First commit.  Trying out the build process."
git push -u origin master

cd ..
git clone <training repo clone URL>
cd ChipWaferTrainRepo
cp -r ../ChipWaferAnalysis/trainer_code/ .
git add .
git commit -m "First commit.  Trying out the build process."
git push -u origin master

cd ..
git clone <test repo clone URL>
cd ChipWaferTestRepo
cp -r ../ChipWaferAnalysis/test_code/ .
git add .
git commit -m "First commit.  Trying out the build process."
git push -u origin master

cd ..
git clone <deploy repo clone URL>
cd ChipWaferDeployRepo
cp -r ../ChipWaferAnalysis/deploy_code/ .
git add .
git commit -m "First commit.  Trying out the build process."
git push -u origin master

Now go into the GreenGrass console and deploy the group. You’ll need to deploy the group if the Lambda function changes.

Next go to the API Gateway console, select the proper API, go to the Resources section, and select Deploy API from the Actions menu. Set the Deployment stage to test.

Next, create a Cognito user for the review portal.

 ./scripts/set-user-password.sh <user email> <password> <user pool id> <client id> <group name>

You can obtain the user pool ID, client ID, and group name from the CFN output. The other parameters are at your discretion.

Finally, build and load the React app. Adjust any necessary values in frontend/src/config.js.

cd frontend
npm install # only needed once
npm run build
aws s3 sync build/ s3://<app bucket>

Updating stack

You can update the stack by passing the --update flag.

If you update the GreenGrass elements, reset the deployment on the group. Then update the stack and redeploy the group. If you update the Lambda function you must also update the subscription definition.

./scripts/create.sh <template bucket> <template prefix> <stack name> <region> --update

Setting up inference on Raspberry Pi

The automated demo right now runs a GreenGrass core device on an EC2 instance. It calls the SageMaker inference endpoint.

If you’d rather do inference on a real device, you can configure a Raspberry Pi.

• Build an MxNet model. (Eventually we can compile the PyTorch model using SageMaker Neo, but Neo does not yet support Pytorch 1.0.)

  • Run the notebook notebooks/Classify-MxNet-121.ipynb. This notebook builds a model using MxNet 1.2.1 and saves the artifacts. Grab the exported artifacts, zip them up, and save them in S3.

  • Alternatively, run the notebook notebooks/Classify-MxNet-SM.ipynb. This notebook trains the model in SageMaker, and the model artifact is automatically saved in S3.

• Follow the basic Raspberry Pi setup tutorial (parts 1 and 2).

• Follow the tutorial on deploying inference on the device using MxNet.

  • Copy the test image folder onto the device in the path /opt/images/test

  • Starting with the lambda zip package you got from the tutorial, replace the greengrassObjectClassification.py with the version in the folder lambda-rpi-inference and rebuild the zip file.

  • When you deploy the Lambda function, set environment variables for the fab, camera, and inference interval.

  • Add a file system resource that maps /opt/images to /volumes/images. Don’t bother with the camera resources.

  • Use the model artifact created from the MxNet notebook. The local path should be /greengrass-machine-learning/mxnet/wafers.

Also note that the Pi needs a 2.5 power source when you run inference. If you use a lesser power source, it’ll boot and seem to work, but it’ll crash when you invoke any neural network for inference.

Resources

• GreenGrass ML Inference Guide

Improvement list

• Run ML training jobs on multiple instances

• Use native PyTorch container rather than custom version (standardizes on fastai 1.0.39)

• Improve accuracy of MxNet model. It should probably not use CenterCrop as the cropping strategy; need to identify other deltas compared to the PyTorch model.

• Use incremental training rather than full training every time, pulling in manually reviewed data.

• Work on class imbalance problem. Consider oversampling, a different loss function, or an imbalanced sampler. The imbalanced sampler seems to work well but it’s very slow right now.

ML Metrics

Table 1. Metrics

Model	Resnet 34	Resnet 34 with imbalanced sampling
Accuracy	97.8	94.2
F1	97.8	94.8
Macro F1	88.5	82.4
Binary accuracy	98.4	94.9
Binary precision	97	74.7
Binary recall	92	98.7
Worst class accuracy	75	84