INTRODUCTION
Various tools are used to manage patients with voiding dysfunction, such as voiding diary which is selected as one of the most important diagnostic modality for voiding dysfunction [1-3]. The voiding diary is a method that doctors use to objectively monitor the subjective symptoms of patients with voiding dysfunction. Since the diagnosis or treatment proceeds after a doctor objectively assesses the patient’s symptoms, the voiding diary is the starting point of studies on voiding dysfunction, making it one of the most important diagnostic methods. In particular, patients subjectively express their symptoms in different ways according to their characteristics and the surrounding environment even if the underlying symptoms are the same; even, there is a famous saying that the patient’s symptoms are an unreliable witness. Therefore, making an objective diagnosis is very important, and the voiding diary is an important diagnostic tool. Nevertheless, it is very difficult to get exact data from voiding diary because recording every events and information exactly is very hard, even to the patients who are very careful and well educated about voiding diary [4-6]. In particular, it is almost impossible for children and elderly people to record all the variables in the diary, so they are only required to write their urination time. As such, although there are many variables that can be monitored in voiding diaries, it is clinically impossible to apply this method to all patients, because patients have various levels of ability to comply with the requirements of using a voiding diary. Accurate sensing technology with the ability to monitor urination would be able to replicate the function of voiding diaries, thereby enabling the systematic and efficient management of patients’ urination. If patients can carry their voiding diaries in the form of a device such as a watch or beeper and it can automatically record the urination related data, it will be helpful for studying symptoms and mechanisms that have not been characterized in many patients. To solve these problems, this study aimed to develop a technology for recognizing urination by collecting and analyzing sensed movement (acceleration and tilt angle) information in smart bands worn by patients. This development is expected to lead to the implementation of a urination management monitoring system for the end-user.
The technology for recognizing urination presented in this study is an extension of existing pattern recognition technology based on signal processing. This technology is similar to that involved in recognizing specific motions in smart home care services. The proposed technology recognizes the signal pattern of urination and measures the frequency and time of urination to automatically record patients’ urination information.
Various methods and learning algorithms for motion recognition have been suggested. Most studies have used static algorithms such as artificial neural network [7] and K-means [8], or have used dynamic time warping [9] in combination with algorithms. However, it is appropriate to use a time series algorithm to predict or classify dynamically changing time series data. Typical time series algorithms include dynamic Bayesian networks [10], hidden Markov models [11], and recurrent neural networks (RNNs) [12]. The HMM and RNN methods have been widely used for time series data. However, HMMs are not appropriate for learning sequential data because each step is only influenced by the previous one.
In this study, we propose an algorithm for recognizing the movements involved in urination by analyzing data from acceleration signals and gyro signals collected from smart bands. To do so, we developed an analytical technique that enables high accuracy, while solving the limitations of existing studies. Thus, we applied an RNN-based long short-term memory (LSTM) [13] method for urination recognition. Using the proposed technology, we aim to improve the recognition rate of the user’s urination and to efficiently solve existing issues, such as the problem of not applying previous data. The conceptual diagram used for urination recognition was processed according to the following stages, shown in Fig. 1 [14].
MATERIALS AND METHODS
In this study, we developed an algorithm that recognizes urination activities (frequency and time) based on the patient’s posture and changes in posture. Movement information was obtained by a smart band worn on the wrist. Three-axis accelerometer information and tilt angle (gyro sensor) information were collected, and an algorithm for recognizing the 3-step motion corresponding to urination (forward movement, urination, and backward movement) was developed. The acceleration information accounted for a larger proportion of the information used for recognition than the tilt angle information, so it was used in the final recognition algorithm to automatically calculate the frequency and time of urination. In other words, this study presents an extended algorithm that also measures time, in comparison to our previous study [14], which presented an algorithm for automatically measuring urination frequency. The RNN method with an LSTM structure was applied to develop the extended algorithm used for urination recognition. Fig. 2 shows the flow diagram of the proposed algorithm.
RNN Structure
RNN is a deep-learning technology that is effective for learning sequences based on a structure with certain repeated parts. Fig. 3 shows a basic structure with the recurrent weight W in the hidden layer, which recognizes the regular patterns of the data in the same way as a convolutional neural network (CNN) [12,15]. This structure can receive help to grasp current information based on past information through W.
There are 2 main processing stages for pattern recognition using machine learning. First, in the learning stage, pattern characteristics are analyzed using data obtained from instances of the behavior to be modeled (for instance, falling), and key information for classifying different patterns is extracted. After completing the learning stage, newly obtained data are classified and recognized. Usually, the learning stage is carried out a single time during the production of the recognition technology, and the recognition stage is carried out every time new data are given. The proposed method used for urination recognition included the following stages, shown in Fig. 3.
xt is the input value in the current time step t. st is the hidden layer that functions as memory and is calculated by st-1 (previous time step t) and xt as shown in equation (1). In this case, the activation function f is a nonlinear function, for which tanh or ReLU is used. st-1 is initialized to 0 to calculate the first hidden state. Since the past output is used again for weight calculation, the RNN can remember past information. In addition, for RNNs, unlike the back-propagation algorithm of the general artificial neural network, the shared weight is updated at each time step through a back-propagation algorithm.
The hidden layer of the RNN has a function to store the calculated result based on the input information, so it can effectively process a short sequence. zt is the output value at time step t and is calculated by equation (2).
The RNN learns via back-propagation through time, which is a variant of the back-propagation algorithm, since the weights U, V, and W are shared at each time step.
Application of LSTM
The LSTM algorithm is used to solve the problem of the zero gradient, which is also known as the vanishing gradient problem. The LSTM is a weight structure designed to transmit the gradient throughout the back-propagation process. The LSTM unit consists of cells with multiple gates attached, and it has the function of storing, retrieving, and maintaining cell information. This function solves the problem of the vanishing gradient observed in the RNN and makes it easier to remember information from a long time ago. The structure of the LSTM is shown in Fig. 4 [13].
Fig. 3 shows 1 memory block, and each gate controls the block to read, use, and output the data, entering the input node in the time unit. The operation of each gate can be expressed by multiplying each weight and the input value, as shown in the basic neural network algorithm. The operation reads the information from the output value of the cell, the input node, and the other memory block, and activates the gate by the sigmoid function. The input gate confirms whether the cell contains data and then determines whether data should be read from the input node. The output gate determines whether to export the data to the output node and then determines the status value based on the information from the forget gate and the previous cell. The fixed weight value of 1.0 makes it possible to keep the previous information in the hidden node without loss. The forget gate determines the recursive iteration of the cell. The hyperbolic tangent function is used to activate the cell, and is expressed by the multiplication of the output gate status values. The LSTM solves the vanishing gradient problem of RNNs by applying fixed weight values of 3 gates and cells.
Application of Clinical Biomarkers for Urination
Under the guidance of a clinician, we applied guidelines for urination management analysis. The variables necessary for the analysis were defined based on the records of the electronic voiding diaries and the urination pattern data collected from the patients. The defined variables were the frequency of urination per day, the maximum urine volume during the day and in a day, and the average total urine volume. Based on these variables, the above-described algorithm for urination recognition was applied for the integrated management of patients’ urination. This design enabled the implementation of quantitative and behavioral biomarkers. The proposed technology of urination analysis was verified by the urologist, thereby confirming its validity and reproducibility, as shown for paper voiding diaries [16-19].
RESULTS
Experimental Environment
An experiment was carried out to assess the performance of the recognition technology proposed in this study. A total of 30 patients were selected, and the performance of the algorithm was assessed based on the data (acceleration, tilt angle) collected from smart bands for 60 days. The final accuracy of the algorithm was calculated based on clinical guidelines used by urologists. The proposed experimental method was divided into 2 major steps for the recognition of urinary activity, and the urination time was calculated in the recognition group. This method was compared to existing methods to demonstrate its robustness. The corresponding scenario and data sample are shown in Fig. 5.
Information on acceleration and tilt angle was collected at a frequency of 12–13 Hz, and the coordinates of the x-, y-, and z- axes were stored in the smartphone as time series data. The data were formatted again in XML (extensible markup language), and divided into learning data and test data at a ratio of 3:7.
Features of the Urination Data
The feature information we aimed to extract included 3 steps: forward movement to prepare for urination, urination, and backward movement for cleaning up after urination. The final urination recognition result was provided through a consideration of the acceleration and tilt angle information in these 3 steps [14]. Fig. 6 shows the acceleration and tilt angle information during urination in comparison with it during other daily activities.
As shown in Fig. 6, the movements involved in urination and other daily activities showed different characteristics. The start and end of urination were classified based on the recognized urination time to calculate the final urination time.
Results of Urination Recognition
The highest probability distribution value of the sequence data was selected as the output of the network. Fig. 7 shows the overall network structure over time. The output nodes were labeled for each action and learned using a back-propagation algorithm. Here, x, y, and z represent the structures to which the accelerometer and gyro sensor signal data are input. This is the process of calculating the node continuity and the status of the current time point (t) and the next time point (t+1). The following Fig. 7 shows the entire network structure over time. Processing the target network involved 2 major steps. The urination stage recognized in the network through the input of accelerometer data from step 1 and the urination step recognized from the gyro data in step 2 were compared and a weight of 0.5 was assigned to determine the final urination step. As shown in Fig. 7, the presence of urinary activity and recognized urinary activity in relation to time are processed by the proposed network structure.
Step 1: recognition of urinary activity
In this step, the 24-hour cumulative signal data of patients with urinary dysfunction were analyzed to recognize forward movement, urination, and rear movement as a single group to determine the daily frequency for that group. In other words, this was the step in which the presence of urinary activity was recognized within daily activity data. To assess the accuracy of urinary activity recognition, a comparative evaluation was performed against existing recognition algorithms. Comparisons were made against existing HMM-based and support vector machine (SVM)-based recognition algorithms, which are typical classification algorithms. The results showed accuracy of 94.2%, which sufficiently demonstrated the robustness of the proposed algorithm. The results of the confusion matrix are shown in Table 1.
The comparative evaluation of the proposed method assessed the RNN-based method presented in this study and the commonly used HMM and SVM methods. This comparative evaluation was performed in order to ensure that the classification was as usable as possible. We used a confusion matrix for this experiment, in which the accuracy, sensitivity, and specificity of the suggested algorithm and the typical algorithm were compared. True positive (TP), false positive (FP), false negative (FN), and true negative (TN) results were defined as follows:
TP: a match for urination is determined to correctly reflect a real urination event.
FP: a match for urination is determined not to reflect a real urination event.
TN: a nonmatch for urination is determined to correctly reflect the absence of urination.
FN: a nonmatch for urination is determined to correspond to a real urination event.
The confusion matrix was calculated, and the average results are shown above (Table 1). Based on the confusion matrix, accuracy, sensitivity, and specificity can be calculated as shown in equation (3) and Fig. 8.
Some inaccurate results for the recognition of urinary activity were obtained due to some errors in the calculation of weighted values for the accelerometer and slope data, indicating areas for improvement. The calculated confusion matrix and the accuracy, sensitivity, and specificity results confirmed that the proposed method was more accurate than the existing algorithms. The proposed method was further confirmed to be more effective through receiver operating characteristic (ROC) curve analysis. A ROC curve is drawn based on the average value of the sensitivity and specificity of the measured values. As shown in Fig. 9, an area under the curve (AUC) of 1 would represent perfect results. If the AUC is 0.5 or less, the method is not useful. AUC values of 0.9-1 represent very accurate results, and values of 0.7–0.89 represent comparatively accurate results. The ROC curve analysis further confirmed that our algorithm led to more accurate results than the existing comparison methods.
Step 2: calculation of urinary activity time
In this step, urination time after the recognition of urinary activity was calculated. The time was calculated by recognizing forward movement, urination, and rear movement stages in the 24-hour cumulative data from all patients. Time was calculated from the point when forward movement began to the point when the rear movement ended. The time was derived by each method included in our comparative analysis and compared to the reference values to determine the accuracy of recognition. As discussed earlier, data collected for a total of 60 days from 30 patients were used for the accuracy evaluation, and the mean error relative to the reference value was calculated. The following Table 2 shows some examples of the results. That values showed the duration of urinary activity for each patient, and the values for each method are presented as the absolute average value of the differences in duration, with smaller values indicating higher accuracy in time calculation. Urinary frequency ranged between 4 and 6 times per day. The absolute average difference value was derived by the average value of the entire difference in the corresponding frequency. Matching results were defined as those equal to what was obtained using the method with the lowest absolute average value. Overall, the proposed, HMM-based, and SVM-based methods showed 22, 5, and 3 matching results, respectively, which demonstrated the superiority of the proposed method. Although relatively inaccurate results, as compared to the HMM-based method, were obtained from some specific data points, this is believed to be attributable to the use of a single unidirectional hidden node in the RNN input/output nodes. As a result, errors occurred in recognizing the forward and rear movements. The recognition rate of LSTM was lower than that of the compared method. Because neural networks are composed of random initial weights, however, the average recognition accuracy was still high. This demonstrates the need to improve the neural network nodes by incorporating 2 hidden nodes in the forward and reverse directions to address this problem.
DISCUSSION
In this paper, we developed an urination recognition algorithm based on an RNN, which is specialized for time series data, to examine the feasibility of constructing an automated urination management system for patients with voiding dysfunction. Previous study [14] have suggested recognition of the number of urination events through a K-nearest neighbors (k-NN) method. The k-NN algorithm is a type of supervised learning that uses labeled data to perform classification. As can be seen from the name of the algorithm, it is an algorithm that refers to and labels K different data points that are close to the data. When measuring the distance in a k-NN, the calculation becomes complicated as the size of the vector increases. In addition, although k-NN performs well for numerical data classification, the classification speed becomes slow if the amount of learning data is large.
Although the use of k-NN would be possible if the goal was only to recognize the number of urination events, it is difficult to use this method to analyze urination time due to the loss of the position information at the time of spatial conversion. If the urine collection time is analyzed and the data are continuously collected, the RNN method using an LSTM structure becomes appropriate. The urination time calculation is handled through the identification of positional information over time in the forward and backward movements of urination. The processing can be calculated through the recognition method of the RNN structure. The proposed urination recognition technology collected acceleration and tilt angle information through a smart band and applied an RNN based on an LSTM structure to derive the final urination frequency and time.
RNNs are an extension of existing multilayer perceptron neural networks and can be effectively used for shape extraction and recognition, as they can maintain previous continuous data for a long time through the circulation of the hidden layer. However, RNNs have a limited ability to maintain data continuously in the hidden layer. In other words, the data in the hidden layer disappears over time, which is referred as the vanishing gradient problem. To solve the vanishing gradient problem, we utilized an LSTM structure with 3 gates as part of an RNN-based classification algorithm to recognize the user’s urination activity. In addition, results for feature similarity between the accelerometer information and the tilt angle information were derived, and the final recognition result was optimized by assigning different weights.
The performance of this algorithm was evaluated using a study population consisting of 30 actual patients with urinary dysfunction from whom data were collected for 60 days. The results showed an average accuracy of 94.2% in recognition of urinary activity, confirming the feasibility of urinary activity recognition. Moreover, urination time was calculated after the recognition of urinary activity, which ultimately allowed a comprehensive assessment of urinary activity in patients with urinary dysfunction. Based on this, the eventual goal is to take into account the personal characteristics of each patient for use in calculating urinary volume. Some inaccurate results in the recognition of urinary activity were derived due to some errors in the calculation of weighted values for the accelerometer and slope data. For urinary activity time measurement, the recognition rate of LSTM was found to be lower than that of the methods it was compared to. Because neural networks are composed of random initial weights, however, the average recognition accuracy was still high. To improve recognition accuracy, we plan to utilize Dempster-Shafer theory [20], a method for finding the optimal weight, or bidirectional LSTM [21], which places 2 hidden nodes in the forward and reverse direction in the RNN input/output nodes. In addition, if the quantity and quality of future data are guaranteed, we can consider data analysis based on big data and develop a CNN and RNN convergent deep learning model that would apply CNN to a time series.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Print
