Optimizing Aging Male Symptom Questionnaire Through Genetic Algorithms Based Machine Learning Techniques

The Aging Male Symptom (AMS) questionnaire, though widely used in estimating symptoms for late onset hypogonadism (LOH), has been notoriously inconsistent in its assessment of initial screening or follow-up [1]. Compared to the simpler Androgen Deficiency in the Aging Male (ADAM) questionnaire, it provides a detailed query of clinically relevant points, while also providing a scaled response. Hence, despite the lack of physiological relevance to these questions, AMS continues to enjoy a wide measure of use, despite most studies decrying its lack of specificity [2].

Conventional statistical methods employing various multivariate regression analyses to identify relevant questionnaires often fall short [3, 4, 5]. Regression methods require each component of the multivariate equation to be independent of each other. Even if a cross correlation presents itself through a validation study in an epidemiologic setting to state that all items showed little inter-dependence, one could simply read through the confusing list of all seventeen items to understand the overlapping spectrum of symptoms. Ironically, AMS propounds the complex overlap of its questionnaires by clustering items into symptom groups, further emphasizing that each item was never designed to be independent, and thus, the statistically predictive power of each item on the overall symptom is spread out and weakened.

Furthermore, conventional statistical methods to remove each component from the regression equation is difficult. Removing questions based on statistical relevance in a univariate setting cannot avoid the aforementioned problem of interdependence between each questionnaire. Conversely, including individual questionnaires and comparing various iterations confronts the problem of investigating 2ⁿ possible sets of different strategies for relevance, where n is the number of items in the questionnaire.

Genetic algorithm is a machine learning method best suited to evolve an optimal outcome from near infinite iterations [6, 7]. Originally created as an optimization method for complex problems, genetic algorithms create generations of sample strategies that then compete against each other for optimum fitness. Once each generation is complete, all sample strategies are then changed through various methods to create a new generation, where the most fit of the previous generation produce similar but slightly altered offspring strategies (crossover), while poor fit samples are discarded and new random samples are introduced (mutation).

This study applied genetic algorithm on a training set of a previous epidemiological study. The resultant best fit strategy of AMS questionnaires was then applied to a validation set of newly recruited patients.

The initial machine learning process to train the algorithm best suited to correlate with serum testosterone was performed against the original epidemiologic data. The data from 1,895 participants were cleaned for missing values, leaving 1,335 participants. Initially we attempted to take advantage of the benefit of using an AMS score over simpler identifiers, such as the ADAM score, i.e., a gradually scaled response providing, hopefully, a full spectrum of responses. However, the correlation function showed weak reproducibility primarily through oversimplifying the outcome.

Hence, sensitivity was chosen as the fitness function, while expanding the complexity of the genetic algorithm to inspect the entire spectrum of serum testosterone thresholds. Interestingly, the initial internal training provided a wide array of questionnaires fit for validation, but was most successful to show that a significant portion of the items, such as 2, 3, 6, 7, 11 had no function in relevance to serum testosterone.

Additionally, through rigorous internal validation, we were also able to discern ‘false positive’ questionnaires, such as items, 1, 9, 10, 13, 15, and 16, which showed high fitness during training but sharply lost relevance when applied to internal validation. Through this process, only a select number of items showed sharp distinction and were successful in identifying low serum testosterone in the independent validation set.

It is noteworthy to see that these select items do not overlap with items selected through conventional methods, such as linear regression (Table 2). Despite the robustness for selecting matrices best fitting the outcome, frequentist methods could not thoroughly predict all possible combinations that could be mired from deviations from the prerequisites of variable independence and normality [9]. For example, items such as 4 and 5, both describing some symptom of sleep cannot possibly be independent of each other, yet a multitude of investigations concerning the AMS often present outcomes where investigators treat these as independent variables in analysis. Genetic algorithms can overcome these pitfalls by dissociating the process of selection from the assumption of normality and independence [6, 7, 10]. The algorithm itself only compares and develops the outcomes based on fitness alone; if codependent factors are included, it does not claim to make judgement that these factors independently affect the overall outcome. Indeed, if such codependence would hamper evolving the fitness function, the genetic algorithm would most likely either eliminate it entirely from the process until such time that randomness reintroduces such factors into the equation again, or stall into a local minima, again, in which randomization would rescue it from over fitting.

Ultimately, the overall outcome shows only relatively distinct items included in the final questionnaire. One single item for sleep (item 4), one single item for mood (item 8), one single item for vitality (item 12), one single item that had been generally ignored (item 14), and one single item most central to the AMS (item 17). Also, it is interesting to note that, during the entire process, the threshold for serum testosterone had been left randomized, only limiting the focus when predictability went out of hand, yet ultimately the threshold had settled down to 3.16 ng/mL as the most deterministic level, which can be converted to 11 nmol/L, a widely accepted level for LOH [4, 11, 12].

Overall, one cannot say that a validation sensitivity of 0.77 is very high, especially considering a poor specificity of only 0.19. Recently Lu et al [13], approached LOH diagnosis through machine learning techniques employing a decision tree method to incorporate various serologic criteria and compared these to questionnaires. The results showed higher predictive value in serologic criteria alone.

However, dismissing questionnaires in diagnosing LOH entirely runs the fundamental problem of approaching a disease without clarifying its symptoms. How can a disease be independent of presenting symptoms? Without clarification of what centrally constitutes the constellation of presentations of a symptom leaves us with no symptoms and only a series of equations describing serology.

Hence, any attempt to salvage these questionnaires through any means is worthwhile. While AMS suffers from any credible criteria, it does not suffer from any lack of symptoms queried. AMS provides a wide range of symptoms, whether they are specifically related to the diagnosis of LOH

Finally, in clarification, the questionnaire used in this study was the Korean translation of the AMS questionnaire, previously used in several nationwide studies [8, 14, 15]. There is poor validation information regarding the AMS, as questions of its utility had already arose by the time several nationwide studies had been conducted, and no future prospects of formal validation seems likely. Perhaps a refinement of the questionnaire may help improve these matters.

Genetic algorithm method of selecting questionnaires improved AMS questionnaire significantly. This method can be easily applied to other questionnaires that do not correlate with physiological markers.