Mobile health and health applications

The rapid adoption of smart devices in the last decade has greatly contributed to the promise of using mobile technologies for health improvement. The m-health (i.e. mHealth) term also emerged, which was defined by the World Health Organization (WHO) as: “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants (PDAs), and other wireless devices” [11]. Mobile devices provide good platforms for developers to design third-party apps called mobile apps, software programs that are specifically designed to run on mobile devices to improve the functionality of mobile devices. Mobile applications installed on the mobile devices can utilize hardware and sensors (i.e. accelerometers, gyroscopes, magnetometers, sensors to measure heart rate, geo sensors GPS and cameras) to obtain the desired outputs. Consequently, mobile apps provide new methods for the continuous monitoring of biological, behavioral or environmental data, health indicators, and trends related to health behavior. Mobile apps can help change attitudes and behaviors by distributing, collecting, processing and interpreting health-related information and by enabling interventions [12,13]. Therefore, various objectives may be met through mobile applications targeting a wide range of user groups. It is possible to develop applications targeting healthcare professionals, healthcare recipients and the general public. Mobile devices and applications are being used for the rapid delivery of clinical information to healthcare workers to equip rural healthcare personnel with up-to-date information in both developed and underdeveloped countries. Yet, since the applications targeting health personnel are not within the scope of this study, only the applications targeting general public and healthcare recipients are discussed from this point on. Such mobile health interventions have achieved success in varying degrees in adherence to medication and treatment outcomes [14]. The effectiveness of various cell phone application and cell phone text message interventions have been tested in clinical trials such as diabetes control [15], hypertension control [16], and adherence to medication [17]. In cases where treatment is complex, such as with cancer patients, mobile applications are also used to improve health literacy in order to improve compliance [18]. Similarly, systematic reviews on the use of various smartphone apps and exercise platforms to improve diet, physical activity and sedentary behavior, and the effectiveness of related interventions have shown positive yet modest effects [12,19,20]. Mobile apps were also found to be influential in improving emotional and mental health. In a relevant study, it was observed that self-monitoring of personal mood with a mobile application reduces symptoms related to depression and anxiety and improves emotional well-being [21]. Consequently, mobile applications may be used as preventive medical tools in a multitude of ways [2,10,14].

Several studies have been conducted to analyze the features of mobile health software on app marketplaces, yet most of them have focused on a particular application area. Correspondingly, m-health app use literature is diversified and numerous subcategories exist. Depending on the use case mobile apps may be used as information sources, journals and personal digital assistants for quitting smoking, healthy eating, reducing calorie intake, increasing physical activity levels, communicating with the health system, improving adherence to treatments (e.g., on-time drug intake), medical monitoring and more [9,10,20,22–25]. Given the variety of apps available to the general public, only a limited number of studies analyzed all available categories [e.g., 4], creating a research gap. Such ambitious studies called for solid frameworks to categorize the available apps and utilized several popular health education, planning and promotion models. One such framework is the PRECEDE-PROCEED model (PPM) of health planning and health education, another is Health Education Curriculum Analysis Tool (HECAT) health education content classification [4,26]. Additionally, disease management models such as WHO Global Burden of Disease [27] have been utilized in relevant studies. The PPM is a widely applied ecological approach to the planning and promotion of health interventions [28]. The PPM has been applied to the m-health applications context by adopting the tripartite structure of predisposing, enabling, and reinforcing factors [4,26]. Within this framework, predisposing factors such as mobile applications try to influence attitudes by providing information and increasing awareness of conditions and or health outcomes, as well as changing beliefs and attitudes to establish confidence among users so that they can change their behavior to avoid adverse outcomes. Enabling factors/ mobile applications on the other hand, aim to change behavior and formed habits by providing an opportunity to learn a new skill and to follow up on the progress of a subject (e.g., applications that allow daily/monthly recordings and follow-ups of running/cycling times and duration while doing sports). Lastly, reinforcing factors/applications aim to encourage certain behaviors that will help in improving and sustaining health through different reward and feedback systems provided to users [4,29]. For instance, apps with auto-sharing to social media sites such as Facebook, or apps that provide ways to communicate and get feedback from an online coach are considered in this category.

Another framework used in the literature for categorizing m-health apps, which is also used in the present study, is HECAT [4] by the Centers for Disease Control and Prevention [30]. This framework focuses on health education curricula provided to students. The categories considered within the HECAT health education content classification are as follows: Alcohol and Other Drugs, Healthy Eating, Mental and Emotional Health, Personal Health and Wellness, Physical Activity, Safety, Sexual Health, Tobacco, Violence Prevention, Comprehensive Health Education.

Factors affecting m-health application use

Selection, data collection and screening

Mobile apps are selected by browsing the health and wellness category of Google Play app store and carrying out searches using ‘health’ and ‘wellness’ as keywords. The data on apps was collected in the second half of January 2020 via an application programming interface (API) in Python programming environment. Data on 520 free and 520 paid mobile applications classified under the health and wellness category were retrieved through the API. The app selection and exclusion process can be seen in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. App selection process.

https://doi.org/10.1371/journal.pone.0244302.g001

Data coding and classification and validation

In the data coding stage, 30 duplicates and 36 apps belonging to other categories or having different purposes (e.g., games, gym membership apps etc.) were left out of further analysis. App description data were cleaned with text mining tools. Firstly, all descriptions were translated into a single language i.e. Turkish as not all were in the same language. After that, texts have been tokenized and stemmed (i.e. converted to its base form). Following this process goes, going, went or gone were converted to their base form, ‘go’. Then, stop words (i.e. the most common words in a language) and punctuation filtering were applied to the data. In this stage words such as ‘a’, ‘an’, ‘the’, ‘but’, ‘only’ was filtered. Additionally, punctuation marks such as ‘?’, ‘;’ and bullet points were also removed from the dataset. After this process, all text data have been vectorized, in other words converted to binary dummy variables. Since this creates a large dataset that is hard to administer, frequency elimination was applied. Classical ways of calculating the weighting of words are term-frequency and Document Frequency (DF). DF is the number of documents in which the term appears. One may think of the definite article ‘the’, which occurs more than any other lexical words in an English text. However, it does not mean that ‘the’ is more informative than other words are. For this purpose, the Inverse Document Frequency (IDF) method was used. This method applies a technique considering all reviews and the frequency of each word in each document inversely. In this way, if a word occurs too many times in all of the documents, its frequency value drops. The description data which was vectorized were merged with other variables. In the Eq (1), fr_t,d is the frequency of each term in the documents, while ‘d’ represents the number of documents in which the term occurs. In our case d is the number of reviews.

(1)

Inverse document frequency (IDF) is a measure which aims to deliver how much information the term gives. IDF measure does not count on whether the term occurs frequently or rarely. It takes one or more documents in the corpus. N is the total number of documents in the corpus and nt is the number of documents in which the term appears. If the word does not occur in the document nt = 0, this leads to a division-by-zero case. In order to avoid this problem, it is smoothed with 1 + nt.

(2)

Regarding the manual data classification, two distinct frameworks were used to categorize the mobile applications. First, the PPM [28] was utilized to categorize mobile applications with regard to how they promote health (i.e. their major aims and how they provide value to users). Within this framework, mobile applications were classified as predisposing, reinforcing, or enabling. This model has been widely used in health education planning, promotion, diagnosis and evaluation activities [49]. Similar to the methodology followed by West et al. [4] in a related study, the applications have been grouped with regard to the PPM framework in this study as well. Consequently, the coders coded the applications as predisposing, enabling or reinforcing. Within this framework m-health apps were coded as predisposing if they provided health information and statistics related to influencing attitudes, knowledge, awareness, beliefs and values or if they aimed to increase the confidence, motivation or self- efficacy of users (e.g., an application that provides smoking and cancer related statistics; ways to avoid adverse health outcomes). If the apps were used for tracking or recording status or progress (e.g., weight or calorie counting, geo-locating running/biking activities), or they facilitated behavior by teaching a skill (e.g., an app showing images or videos on proper posture) they were coded as enabling. Enabling apps are commonly used at the same time as the desired behavior. The apps were coded as reinforcing if they interfaced with a social networking site (e.g., apps with an automatic upload to Facebook), provided encouragement from trainers/coaches (e.g., an app that featured easy communication with a trainer), and included an evaluation based upon the user’s self-monitoring (e.g., an app that provided automated notifications). In the event that an app was both considered enabling or reinforcing, the reinforcing category was coded. If an app can both be considered predisposing or enabling the enabling category was coded. Examples of applications that were coded as predisposing are Health Articles, Info & Motivation–Lets Healthify, Ketogenic Diet and EO Guide. Applications that were coded as reinforcing are Drink water reminder, Soccer Training and Quit Smoking Tracker GOLD; coded as enabling are Period Tracker Mia, YAZIO Calorie Counter, Nutrition Diary & Diet Plan, and Calorie Counter–MyFitnessPal.

The second classification method used is HECAT health education content classification by the Centers for Disease Control and Prevention [30]. The categories of coding within the HECAT health education content classification are: AOD: Alcohol and Other Drugs, HE: Healthy Eating, MEH: Mental and Emotional Health, PHW: Personal Health and Wellness, PA: Physical Activity, S: Safety, SH: Sexual Health, T: Tobacco, V: Violence Prevention, CHE: Comprehensive Health Education.

Two research assistants working at a management information systems department with a focus on healthcare, who provided informed consent to participate in this study, were trained by the authors to collect and code application data into the aforementioned categories based on the two frameworks discussed. Two training sessions were held over the course of data coding by two authors and two research assistants, one before coding and another after the coding of 60 apps in a pilot study. The manual coding detailed in this section helped in obtaining further variables (i.e. the PPM and HECAT categories, app sponsor type, privacy policy availability) in addition to the information retrieved from Google Play Store. Table 1 presents all of the metrics and qualitative parameters that were extracted in this manner and fed into the machine learning models. These predictor variables were used in the analysis to predict the total number of downloads. Supervised learning algorithms need a class variable for training using the available data. In the current study, ‘the total number of downloads’ was chosen as the class variable parallel to the main aim of this study, which is to discover the ways to improve the use of m-health apps.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Code sheet: Metrics and parameters.

https://doi.org/10.1371/journal.pone.0244302.t001

Inter-coder reliability was computed on 120 apps (approximately 12% of the total dataset) after the apps were coded. The degree of agreement between the two coders was calculated and the overall 87% agreement figure led to the conclusion that there was no significant inter-coder issue.

Data analysis and results

Following the re-coding process, the frequencies of each variable and the results of the classifications are provided in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Frequencies of apps and app classification(s).

https://doi.org/10.1371/journal.pone.0244302.t002

As a next step, the means of different categories were compared via ANOVA analysis on SPSS software package to reveal the factors that are influential in review scores. User scores variable was set as the dependent variable whereas the app sponsor, PPM categories, HECAT categories, content ratings, price groups, being an editors’ choice, having a privacy policy, having a video in the description, in-app purchases, the number of days since last update and interactive elements were set as independent variables. 75 apps out of 974 did not have any published review scores, thus were left out of this analysis. The results of the analysis are detailed in Table 3. According to the results, the PPM category, content rating, price group, the days since last update and the interactive elements were not significantly different in terms of user scores and F values are not provided for these variables in an effort to save space.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. F-test compare means results.

https://doi.org/10.1371/journal.pone.0244302.t003

As a further step of the analysis, three machine learning algorithms were used to predict the number of downloads. The overall workflow of the machine learning analysis is depicted in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overall workflow.

https://doi.org/10.1371/journal.pone.0244302.g002

A decision tree model was used to extract hidden patterns behind the number of downloads. Decision trees create IF-ELSE-RULES which can be interpreted by humans and can also be implemented in third -party applications. The most frequently used decision tree algorithms are Gini and Entropy (gain ratio) based algorithms. However, it is well known that they are weak learners. They are easily affected by dataset variations and outliers. In order to circumvent this overlearning or overfitting problem, ensemble and random forest decision tree (RFDT) models have been developed. They yield successful results on many data bases. In this study, the decision tree model we have employed is the random forest model. As the name suggests, the random forest model creates more than one tree. Trees are created with randomly selected variables from the dataset by using different algorithms [50]. The total number of the trees in the forest is determined by the data scientist. The root of the tree is essential for a decision tree model. It is considered to be the most important parameter to explain the target class variable. Thus, in a forest it is critical to know how many times each variable was selected as the algorithm to be included in the root of the trees. Traditionally, random forests may use GINI or Entropy based decision tree algorithms. In this study, GINI based random forest algorithm, which employs probability concept as indicated in Eq (3), was used. In the equation, p is the probability of a variable being in a field of the data set.

(3)

The second machine learning model we used is Artificial Neural Networks (ANN). ANN are composed of layers, which are interconnected via weights as visualized in Fig 3. The simplest ANN has one input layer (variables used for learning), a hidden layer and an output layer (class variables). Hidden layers contain neurons that hold activation functions. Each neuron makes a decision about how strongly it should activate. Each neuron is also fed with weights coming from previous layers. A backpropagation method is applied to adjust weights with minimum prediction error. ANN generates weight matrixes as learning outputs [51]. Neural networks need to be fined-tuned with certain hyperparameters.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Artificial neural networks.

https://doi.org/10.1371/journal.pone.0244302.g003

Thirdly, Naive Bayes (NB) machine learning models, which are one of the most popular probabilistic classifiers in data science, were used. NB’s are based on the Bayes Theorem of conditional probabilities. They tolerate noise and outliers well; their training time is relatively short, and they need few hyperparameters for fine-tuning. In the Eq (4) P(x) is probability of x in the document (data set), P(c) is the probability of the labeled class in the data set and finally P(x/c) is the probability of variable x in a given class.

(4)

The KNIME analytics platform was used to generate all machine learning models and to apply algorithms. For all the algorithms applied, 30% of the data was used for validation, and 70% was used for training. Stratified sampling method was used for partitioning dataset as learning and validation parts. To overcome minority class problem, Synthetic Minority Over-sampling Technique (SMOTE) was employed. Hyperparameter selection for the algorithms utilized is as follows:

• The quality measure was chosen as Gini index for random forest decision tree model. The number of levels was set at 10 and the minimum node size was 3. The n-estimator was chosen as 100. A 5-fold sampling (without replacement) was done along with stratified sampling.
• In ANN, sigmoid activation function was preferred, and z-score normalization was applied to the dataset to speed up the training. Stochastic depth and early stopping were to prevent any possible overfitting. The best performing model for ANN was achieved with two layers and 10 nodes in each layer after several trials.

We calculated the accuracy, precision, sensitivity, specifity, Cohen’s Kappa, and F- values using true positive, true negative, false positive, and false negative cases to evaluate the learning quality and performance of the algorithms as provided in Table 4 and Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. HECAT application categories.

https://doi.org/10.1371/journal.pone.0244302.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Model prediction performance.

https://doi.org/10.1371/journal.pone.0244302.t004

The influences of different factors on download behavior were assessed using machine learning algorithms. When the results of these analyses are assessed, the best learner emerges as the ANN, which can be seen in Table 4. The output of the ANN analysis is not directly discussable, yet the sensitivity of each input variable can be calculated and interpreted. Thus, we calculated the sensitivity of input variables using the weights between inputs and the first layer of the ANN. The weights were then normalized by dividing the weight of each variable by the grand total of the weights. The sensitivity analysis of the Top-100 parameters (consolidated and filtered) led to the following influential keywords:

• Keywords: part, young, band, weight, period, breath, pedometer, guts, treatment, calculate, education, smoking, use, outside, activity, timer, running, water, educate, Bluetooth, development, sutra, chest, glycemic, step, cycle, run, dance, points, plans, use, photo, music, minute, notification, progress

Moreover, a second run of the ANN by excluding the description text data has led to the findings provided in Table 5. Please note that random forest and naive Bayes algorithms have not been used further since they yielded relatively weaker accuracy and precision.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. ANN sensitivity (excluding description text data).

https://doi.org/10.1371/journal.pone.0244302.t005

The top five factors that are related to the total number of downloads is the ‘Required Android Version’, ‘Interactive elements’, ‘HECAT category’, ‘Content Rating’ and whether the app is ‘free’ or not. Lastly, the second-best performing algorithm RFDT’s tree attribute selection results highlight the role of the following parameters on download behavior:

• Variables: Sponsor, HECAT, Editor ‘s Choice, Android version, Free.
• Keywords: Light, Routine, Follow, Information, Morning, Food, Exercise, Step recorder, Ovulation period, Develop, Relax, Friends, Heart.

These findings are discussed in the following section in detail.