Project timeline
- First meeting with our mentor
- Discuss project's goals
- Research on the existing products
This week we did a brainstorming in order to find out what are the best components for our "intelligent t-shirt"(see materials list). For that, we had to established how far we can go, so the features that we will include for our t-shirt are:
- The t-shirt will interact with an intelligent house, by using simple gestures
- We will be able to measure vital functions (heart beat)
- T-shirt material is washable
- T-shirt contains electrostatic material (that will not affect electronic components)
As we already ordered most of the components and the materials for the t-shirt, we are ready to start the practice. Meanwhile, we prepared our working environment, installed Android IDE and the OpenHab runtime core.
During this project we will talk a lot about about sensors and other components, but now we will turn our attention to electrocardiogram sensors. We will often refer to them as ECG sensors. See here more details about how ECGs can determine the heart rate and the cardiac rhythm.
We got our first contact with an IMU, even though its model is not the one that will be used in the projects prototype. We connected a breakout board containing the MPU-6050 IMU to an Arduino Uno. Using its serial communication capabilities, we sent the measures of the sensor to a computer, where a program made in Processing received and processed them. Finally, the board's movement is reproduced through the display of a 3D model.
We continued our work with IMUs. In order to reach our objective, which is detecting gestures, we need to be able to mesure the angular position of an object. Both accelerometers and gyroscopes present in the IMU allow us to do this individually; however, each one present its problems when estimating an angle.
The idea is then combining both measures to obtain a good estimate. The Kalman filter is the ideal solution for this type of problem; nonetheless, it is difficult to understand and to code, and also very demanding in terms of processing power. The Complementary Filter is a simpler solution, which is very easy to implement, demands much less processing power and presents an angle's estimate which is sufficiently accurate for many applications. More details here.
During this week we also discovered Openhab (open Home Automation Bus), an open source software that supports a variety of features which allows you to interact and to control all types of smarthouse-gadgets from different suppliers. After its installation we observed that it has a very user-friendly (see image below) editors that allows you to configure your own runtime, to define your UIs and to implement your own rules. At the end of the development and of the implementation of the "motion recognition", our t-shirt should be able to "communicate" with an intelligent house, by the use of simple gestures.
We developed a diagram describing the placement of the desired components in the t-shirt, which can be found here. A more detailed schematic with the connections between the components used in the final prototype was later developed, and can be found here.
After receiving a good part of the components (IMUs, Bluetooth module, FTDI board) for our prototype, we went on to solder the required pins. We also got two JeeNode v6 boards since they run at the same internal voltage (3.3 V) of the IMUs, contrarily to the other boards we have. We we'll use them temporarily until the LilyPads arrive. They came disassembled, so we had to solder all the boards' components.
This week, we had a hard time figuring out how to program the JeeNode v6 board. To do it, we need to use an USB to TTL serial adapter (in our case, the BUB II board, which contains a FTDI's FT232R chip), since the JeeNode doesn't have the needed serial to USB capabilities. Since the BUB II is a very flexible and configurable board, we took some time to understand how it works and how should we configure its output and internal voltage to make with work with the JeeNode. We finally managed to make the two work together, and we are now ready to interface it with our IMU and to implement the code which will be present in our prototype.
We studied the different possibilities for the Bluetooth connection. We connected the BLE module to an Arduino and then controlled its outputs by pairing the module with our smartphone, which ran a demo application. We also analysed a library that implements the HCI protocol, which enabled us to easily send data from one module to another. Finally, we tried to connect the module directly to our computer's Bluetooth adapter by using the BlueZ, which is the Linux standard Bluetooth module. However, we didn't succeed due to compatibilities problems between the latest version of the library (which supports the BLE technology) and our operating system's version.
The Bluetooth connection was finished, using the HCI protocol. A BLE module connected to the microcontroller sends data to another module connected to the computer. The program that receives and interprets the data coming by Bluetooth was made in Python, using the HCI Library of the BLE Mini.
We developed the visualization program for the data received from the IMUs. We used VPython (also know as visual) to develop it. Our representation shows the received Euler angles and also displays them in an animated 3D box.
We sewed some pockets to the t-shirt, where we would put our IMUs and the Bluetooth module. We soldered pins and wires to the lastly received components (buzzer and LEDs).
We implemented a simple gesture detection algorithm, which detects when we turn our arms by approximately 180 degrees when holding it parallel to the ground.
We integrated a luminous and sonorous feedback to our system. When a gesture is detected, the LEDs of the corresponding arm will shortly turn on and a buzzer attached to the t-shirt will emit a beep. The signals required to do this are sent back by Bluetooth to the microcontroller by the gesture detection program.
We encountered many problems when integrating all the components to the t-shirt:
- The conductive threads didn't actually conduct current.
- Solution: using a lot of jumper wires, connected to each other with tape.
- We can't connect the jumper wires to the LilyPad SimpleSnap.
- Solution: we used the JeeNode v6 which was used earlier in our project, as it also works internally with 3.3V.
- The JeeNode doesn't have a battery, and neither do we.
- Solution: we made all our tests by feeding the JeeNode by USB, with the FTDI adapater.
- Many wires which we had soldered to our components broke.
- Solution: resolder them.
We developed a poster, a flyer, and a video for our final presentation. They can all be found here.