Why do you want to try FitSleep?
I experience irregular sleep. Some weeks I sleep better. Other weeks it’s inconsistent. The most difficult aspect is quieting my mind to fall asleep. Sometimes it takes several hours to fall asleep. Stress plays a major role in my sleeping pattern. It’s not uncommon for me to have only a few hours of sleep than put in another 20-hour work day. No matter what time I go to bed, I consistently wake up between 5am to 6am.
Like any sleep aid technology, I am hopeful to try new devices that could stabilize and regulate my sleep. Hence, I was eager to try FitSleep.
What is your sleep goal?
My main sleep goal is to fall asleep quickly and stay asleep for the majority the night. If I could consistently sleep for 6 to 7 hours a night, that would be a tremendous improvement to my health and productivity.
What is your experience with FitSleep?
Setup
Getting started was probably the hardest. Mostly attributed to poor English translations, I found the instructions obtuse and difficult to understand. The paper guide instructed to send an email to a FitSleep staff member to obtain detailed instructions. Several days later, FitSleep staff replied instructing me to download the app. (On the same day of receiving the shipment, I had downloaded the app and self-navigated to setup the system.)
Device
The hardware device is slick and well-designed. There was initially some concern about the size and thickness as to whether it would be comfortable to sleep on. However, the size never posed any problems with the various pillows that I tried.
In the beginning, it was a bit unclear if the button on the device had a function. I would tried to push it to turn it on and off but realized quickly that it remains on standby mode and automatically starts tracking when your head is on the device.
The battery lasts about 3 – 4 days before needing recharge through the USB to computer. The blinking light indicator turns from green to red to signal battery recharge is needed.
Application
The general wireframe of the application is reasonable well designed and intuitive. I found the daily, weekly and monthly metrics most useful.
Probably the biggest user experience (UX) issue that I found unintuitive is the daily sync process to download the data from the device. It’s perfectly fine that the user needs to initiate the transfer but clicking on the tiny daily report button seems quite unintuitive. The entire page is useless since it displays non-useful information and the only feature that I use on the page is the small icon for daily sync and download.
Accuracy
For this product review, I intentionally did not concurrently test another sleep aid technology in the event of potential radio frequency interference.
Vital signs monitoring
The vital sign monitoring stat appears to be in line with my most recent physical examination report. Since I did not perform a side-by-side test comparison with a medical grade vital monitoring devices, I cannot conclusively affirm that it’s absolutely accurate but the general range seems to be in order.
Sleep tracking
I found the sleep tracking segmentation metrics most valuable in better understanding my sleep pattern.
It emphasized my deep sleep, light sleep, overall sleep duration and waking periods. Incredibly helpful.
Again, as a disclaimer, I did not conduct a side-by-side comparison with a hospital grade sleep monitoring system to measure the accuracy.
For the most part, it appeared to be reasonable in capturing my sleep pattern. However, on couple of nights towards the end of my third week, the tracking data was way off. For instance, I know that I slept for less than three hours yet the system reported 3 1/2 hours of deep sleep and almost 13 total sleep duration. Up to this point, I had reasonable confidence in the system but I then began to question the accuracy.
As a caveat, I recognize that algorithms are tricky to optimize out of the gate. It’s somewhat expected that the accuracy will continue to increase with more data and optimized algorithms but clearly there are some glaring issues to work out in subsequent releases.
Sleep health report
Perhaps the metric that I found most opaque was the overall sleep score. Personally, less than one hour of deep sleep and 4 – 5 hours of total sleep duration should not qualify for score in the 70’s – 80’s. The sleep score, at least according to their definition, indicated normal sleep pattern, which I would strongly disagree.
I find the weekly and monthly chart least valuable, mostly because there are not going to be too many deviation in sleep pattern from day to day, week to week. What I am most interested is the x-ray view in a night’s sleep and specific actionable ways to improve my key sleep metrics, such a longer deep sleep.
I found their overall advice section to be too generic and non-personalized to be of any use. Moreover, the English grammar issues made the content non-credible and therefore poorly cast light on the overall professionalism of the company and its product.
Natural wake-up alarm
Though I set the wake-up alarm several times, I woke up sometimes hours before my scheduled wake-up. As a result, I was usually already awake and never needed to utilize this function. (I, on average, sleep 3 – 4 hours at best. Waking up is not the problem, at least for me. It’s falling asleep and staying asleep.)
Alpha wave E-lullaby
This is the number one reason for trying out FitSleep. My hope for the Alpha wave E-lullaby was high.
Over the course of three weeks, per instructions, I kept the Alpha wave function off the first week and then enabled thereafter for the remainder of the testing period.
For me, the alpha wave did not work. Keep in mind that my stress and other factors are so strong that it’s possible that the alpha wave had no statistically significant impact to my sleep pattern.
By the third week, I had to start incorporating natural, organic Melatonin to increase my drowsiness to fall asleep.
I experienced no measurable impact from the Alpha wave E-lullaby.
Post this product review, I will have each of my family members try out the FitSleep for a 3-week duration to see if they see any material benefit from the Alpha wave function.
Note that as mentioned above, it’s not uncommon for early product development (Alpha and Beta) iterations to be not fully optimized. From my perspective, it’s somewhat expected from an early release product. The FitSleep team is doing the right thing by gathering real-life data from users to improve their data, algorithms, and the Alpha wave.
How well do you think you’ve reached your sleep goal after three weeks with FitSleep?
As you can see from my daily charts, I achieved no measurable improvement in the three weeks. If anything, my overall sleep number went down. I do believe that this is more attributed to my stress and lifestyle than the FitSleep system.
Do you have any suggestions for improvement?
Keep doing what they are doing. As someone who helps companies implement predictive data analytics, it’s expected that you’re not going to hit a homerun out of the gate. Optimization of models and algorithms, especially for machine learning requires continuous and large volume of data to teach the system. Hence the best way to improve the various dimensions of tracking and monitoring accuracy is to get the device into the hands of more users and have them keep using the system.
The two biggest immediate areas of improvement aside from accuracy are:
-
Daily sync. The system should auto-sync the next day after several hours of non-use. This should be fairly easy to do as the data trendline should be relatively flat and patterns are clearly different from sleep. Even if FitSleep chooses not to auto-sync, they need to revamp the manual daily sync process so that the page is more useful. In current form, it’s a waste of a page in the workflow. Also in order to view historical daily stats, it forces the user to sync the data. The workaround is to go to the monthly report and click on a specific date.
-
Another low-hanging fruit but yet critical for their company image is accurate language translation. Everything from the app content, field labels to paper instructions and guides, they need to have a native English speaking editor to iron out the translation.
As the system improves overtime, it would be helpful to generate personalized advice based on individual data patterns. For instance, “we noticed that when you go to sleep before 10pm, you tend to average 1/2 hour more deep sleep. We highly recommend that you go to sleep no later than 10pm. Similarly, we notice that the last two hours of your sleep is light and in awaken state. Based on the sun pattern in your geography, it could be that your blinds are allowing sunlight through which could be disturbing your sleep in the morning.
How likely are you going to recommend this to your significant other? And why?
As mentioned, I recognize that I clearly have sleeping issues which could be creating an anomaly in their data set so I am having each of my family members try the device for three weeks each to validate how effective FitSleep’s Alpha wave claims are when it comes to getting a better night sleep.
Until then, it’s hard to provide a conclusive recommendation.
I do believe that the FitSleep system provides transparency and insight into your sleeping pattern so even if it didn’t necessarily help you sleep better, there is value in self-quantification.
How likely are you going to keep the device? And why?
I will keep the device to continue the testing with my family.
Is there anything else that you’d like to tell us?
Overall, it’s clear that the FitSleep team has worked hard to develop a functioning hardware and app system.
There are some tactical issues that can be resolved quickly and easily and the rest is collecting more data to improve their algorithms for accuracy.
I am hopeful and optimistic for the FitSleep team.
Author Scott Amyx. Video by Christian Amyx.
Tune in for my next experiment!
Keep learning.
– Christian
YouTube: https://goo.gl/TLZ4SjWebsite: https://christianamyx.com/Medium: https://medium.com/@christianamyxTwitter: https://twitter.com/ChristianAmyx1
This blog is moderated by Christian’s parents: https://medium.com/@ScottAmyx/ All comments will be reviewed and approved before publishing.
