Researchers have been in constant pursuit for the perfect way to monitor human activity. In other words, they are looking to find the best solution for human activity recognition (HAR), which could help various technologies down the road. And, as smartphones and their sensors become more and more powerful, it seems as though these mobile devices are the newest answer to researchers’ plights.
HAR is now a prominent research area in the realm of human-computer interaction (HCI). Researchers are already using smartphones to identify the activity of a human, but they hope that this technology can eventually help contribute to even greater causes. In essence, we are using technology to identify how humans interact with their devices in an effort to improve that interaction in the long run.
But just how accurate and proficient is this technology when it comes to smartphone designs?
The Basic Sensors in a Typical Smartphone
Accelerometer and Gyroscope
The gyroscope is a common sensor that can help a smartphone identify axes in the physical plane. Basically, it helps the phone detect its orientation — whether it is facing up or down or even spinning.
Image courtesy of Wikimedia Commons.
The accelerometer, on the other hand, measures the phone’s linear motion in a particular direction. It gives the phone a sense of direction. It can also identify steps. So, the smartwatch that keeps track of consumers’ footsteps might be using an accelerometer.
The Digital Compass or the Magnetometer
The name of this sensor is pretty much self-explanatory — it acts as the compass. With it, a phone will never forget which way north is. When your navigation app changes its orientation as soon as you change your position, it is simply using the magnetometer.
Proximity Sensor
When you receive a phone call and hold the phone near your ear, the display shuts off. It’s because the proximity sensor can sense that the phone is too close to one’s ear. It can detect whether there is an object close to the sensor or not.
Barometer
A barometer can sense pressure. In a smartphone, it serves to calculate the altitude. The first generation smartphones didn’t have a barometer; those were introduced in Samsung Galaxy S4 and the Google Nexus 4. The health apps that tell you how many floors you have climbed might be using a barometer.
Biometrics
There are a few biometrics sensors — the fingerprint scanner, the IRIS scanner, and the heart rate scanner. Although, they can be found in the latest smartphones only. Even some smartphones have an SPO2 sensor that can measure arterial oxygen saturation level.
Image courtesy of Pixabay.
While these sensors can lend themselves to better HAR, they do have their faults, which could lead to weaker data for HCI research. So, what are these challenges?
The Challenges to Overcome
Not Accurate Enough
If we rely solely on accelerometer data for motion sensing, we cannot be accurate. There are two major reasons behind this — motion pattern varies from person to person, and a person exhibits different motion in different time. Thus, too much variation in the data causes results to fluctuate.
Is the Phone in the Right Place?
If the machine is dependent on the data taken from the accelerometer, the reading will vary with the phone’s position. For example, if you are walking while holding the phone on your hand, the reading will fluctuate from the reading taken while the phone is kept in a pocket. Similarly, the data will also differ if you hold the phone higher to your eye. The regularised Kernel algorithm might be useful, but, still, the problem persists.
Bizarre Reading During Multitasking
It’s true that our brain can only focus on a single task at a time. So, when we are multitasking, we are just switching our prominent activity too fast. Because of this, the sensors cannot give consistent, smooth data when the test subject is multitasking. One subject might be walking and talking on the phone at the same time, which could give off some strange step readings.
Image courtesy of Pixabay.
Limited Power and Energy
An optimal HAR process would need constant data collection and analysis. The device would also have to stay online all the time for a seamless and accurate experience. This causes some challenges — limited battery backup, high CPU power consumption, and spotty internet connection. Yes, phones do have better battery life after the massive use of Li-Poly battery, and there is Wi-Fi available almost everywhere, but still, it’s not enough.
User Variation
As we said earlier, not every single human being is the same. Each consumer will demonstrate different and unique activity patterns. Also, there is the case of age. Youth will definitely be faster and more agile with their devices than their seniors. In the end, reliable human activity recognition is not as easy as it sounds.
Concluding Thoughts
We certainly need to overcome a few critical hurdles before we can use the sensors in a smartphone to perfectly recognise human activity. The good thing is that smartphones are constantly updating, and new technology is always knocking on the door. Engineers just need to keep these goals in mind when working on the newest smartphone prototypes. If we can master this process, the field of machine learning will reach a new height of sophistication.
