Evaluating mHealth Apps

BOSTON– February 10, 2016: Are mHealth app ratings actually useful? A study authored by Adam C. Powell, Ph.D., John B. Torous, MD, and colleagues examined the interrater reliability of 22 measures commonly used to evaluate the quality of mobile health apps for depression and smoking cessation. Six raters assigned ratings to a pool of 20 apps, and then the level of agreement between the ratings was calculated using Krippendorff’s alpha. Surprisingly, the six raters had rather poor agreement on 21 of the 22 measures which were examined. There was rather high disagreement over basic issues, like whether the apps were easy to use, stated their advertising policies, and had performance issues. However, the raters had strong agreement about the level of interactiveness and feedback within the apps. These findings suggest that mHealth app reviews should be used cautiously, especially if they rely upon measures which have not been validated.

To learn more about which measures of mHealth app quality have high interrater reliability, download the full text of the study at http://mhealth.jmir.org/2016/1/e15/