Development of Early-Stage Stroke Diagnosis System for the Elderly Neurogenic Bladder Prevention

Article information

Int Neurourol J. 2022;26(Suppl 1):S76-82
Publication date (electronic) : 2022 February 28
doi : https://doi.org/10.5213/inj.2244030.015
1Department of Media, Soongsil University, Seoul, Korea
2Blaubit Co. Ltd., Seongnam, Korea
3Digital Health Industry Team, National IT Industry Promotion Agency, Jincheon, Korea
4Department of Medical Device Engineering and Management, Yonsei University College of Medicine, Seoul, Korea
Corresponding author: Sung-Jong Eun Digital Health Industry Team, National IT Industry Promotion Agency, 10 Jeongtong-ro, Deoksan-eup, Jincheon-gun 27872, Korea Email: sjeun@nipa.kr
Co-corresponding author: Jun-Young Lee Department of Medical Device Engineering and Management, Yonsei University, 20 Eonju-ro 63-gil, Gangnam-gu, Seoul 06229, Korea Email: jjunyoung76@gmail.com
Received 2022 January 10; Accepted 2022 February 20.

Abstract

Purpose

There are various neurogenic bladder patterns that occur in patients during stroke. Among these patterns, the focus was mainly on the patient’s facial parsy diagnosis. Stroke requires early response, and it is most important to identify initial symptoms such as facial parsy. There is an urgent need for a diagnostic technology that notifies patients and caregivers of the onset of disease in the early stages of stroke. We developed an artificial intelligence (AI) stroke early-stage analysis software that can alert the early stage of stroke through analysis of facial muscle abnormalities for the elderly neurogenic bladder prevention.

Methods

The method proposed in this paper developed a learning-based deep learning analysis technology that outputs the initial stage of stroke after acquiring a high-definition digital image and then deep learning face analysis. The applied AI model was applied as a multimodal deep learning concept. The system is linked and integrated with the existing urine management integrated system to support patient management with a total-care concept.

Results

We developed an AI stroke early-stage analysis software that can alert the early stage of stroke with 86% hit performance through analysis of facial muscle abnormalities in the elderly. This result shows the validation result of the landmark image learning model based on the distance learning model.

Conclusions

We developed an AI stroke early-stage diagnostic system as a wellness personal medical service plan and prevent cases of missing golden time when existing stroke occurs. In order to secure and facilitate distribution of this, it was developed in the form of AI analysis software so that it can be mounted on various hardware products. In the end, it was found that using AI for these stroke diagnoses and making them quickly and accurately had a positive effect indirectly, if not directly, on the neurogenic bladder.

INTRODUCTION

Neurogenic bladder refers to a stroke patient among patients diagnosed with abnormal bladder storage function through detrusor hyperreflexia in urodynamic test in which bladder hyperreflexia occurs due to nerve damage. Exist studies [1,2] have reported that stroke is affecting symptoms or disease expression of neurogenic bladder. There are some of these patterns, of which the pattern for facial parsy can be easily identified. Asymmetry criteria for the face are mainly important, and fine detection of this is essential to prevent stroke. This paper developed an early stroke prediction system equipped with a highdefinition 4K camera for fine detection of facial parsy symptoms in stroke patients. The use of the camera is for the purpose of photographing a patient’s face image, and the initial symptoms of stroke are analyzed through machine learning-based recognition [3-9] of the collected face image.

Wearable devices are developing into a companion product that can be compared to smartphones at a time when smartphones are recently activated and the Internet of things is connected and information is exchanged from the Internet of Things. Various wearable medical devices are also developing, and optical diagnostic devices that provide high resolution can be miniaturized and have the advantage of high portability and immediate diagnosis, so their market share is gradually increasing. In particular, since an optical diagnostic device using a camera can perform artificial intelligence (AI) analysis using only photographic images, remote treatment and analysis are possible. Optical diagnostics have a disadvantage in that they have lower detection accuracy than blood and laser, but with the recent development of high-speed cameras and resolutions, they can have a level of examination accuracy similar to that of actual examinations with the naked eye. In addition, with the development of the network, large-capacity image processing of tens of thousands of images can be sent to the central server to receive the examination results within a short period of time. With such technological support, the need for developing an AI image analysis system that can be applied to the treatment of neurological diseases in the elderly is increasing.

From the heavy system of the existing inspection camera, we developed a 4K inspection camera that is a portable, high-performance dedicated camera and provides ultra-high-resolution images that can ensure the stability and accuracy of inspection. The existing stroke recognition studies [10-12], there were issues such as restrictions on image information, so in the case of this proposal, the focus was on supplementing data. While providing a variety of light sources, it also provides various image format conversion technologies required for diagnosis. In addition, since it is intended for the elderly, it is possible to install AI analysis software that can accurately diagnose the early stage of stroke in daily life.

MATERIALS AND METHODS

The method proposed in this paper developed a learning-based deep learning analysis technology that outputs the initial stage of stroke after acquiring a high-definition digital image and then deep learning face analysis. An AI auxiliary diagnostic software was developed to predict and diagnose the early stages of stroke by analyzing eye shape, facial wrinkles, and muscle abnormalities in the area of the mouth in facial images when stroke symptoms are expressed. The proposed method was to add and apply an AI model to the previously developed elderly urination management system [13] for neurogenic bladder prevention. Through this, an integrated system capable of total-care of patients was established. (Fig. 1) represent to the concept of system integration flow.

Fig. 1.

The concept of personalized total-care system.

AI (CNN+GAN) Diagnostic System

The AI diagnosis system [14-21] primarily performs the lesion classification step through the convolution neural network (CNN) algorithm based on the input data. In combination, upgrading is performed through the generative adversarial network (GAN) algorithm, and personalized lesions can be tracked in conjunction with the personal lesion tracking system. The learned data was implemented as a DB system by constructing patient photos and facial photos of normal people. Using this, it was reflected in the expansion of learning data and algorithm improvement. Fig. 2 shows the processing configuration of the AI diagnostic system.

Fig. 2.

The flow of AI diagnosis method.

In evaluating stroke severity, analysis guidelines were developed by setting weights for various factors such as consciousness, language, pupil abnormalities, eye paralysis, arm weakness, facial paralysis, foot circulation, and sensory damage. Using existing studies on weight factors for determining stroke, the guidelines were modified to suit this task and set as criteria for setting weights for facial paralysis, eye paralysis, and arm weakness. The initial symptoms of stroke are determined by comprehensively determining the diagnostic analysis of facial paralysis, eye paralysis, and arm weakness. Previous studies mainly deal with only the degree of symmetry of left and right lips among facial paralysis, but to determine accurate stroke symptoms, we deal with comprehensive information on (1) forehead wrinkles, (2) both cheeks, (3) lip tilt, and (4) eye movement, and (5) arm movement to increase accuracy.

Database and Image Processing System

In conjunction with the AI diagnostic system, feedback functions were established to reflect results, classify and store images, and improve algorithms. It was developed as an Amazon Web Services platform system capable of inquiring diagnostic results and manipulating and reporting in a desired form.

Data augmentation of insufficient learning data was performed by additionally learning stroke patient data collected by Gil Hospital, data processed with image processing technology, and open face image sets for wrinkle detection as follows. Among the 2,000 faces data, 1,000 datasets suitable for wrinkle detection for wrinkle stroke analysis were selected and used for learning and evaluation, and the learning image was labeled through 200 patient video data and used as abnormal data. Fig. 3 shows the data augmentation method of the learning system.

Fig. 3.

Data composition by augmentation method.

Integrated Patient Management System for Clinical Diagnosis Support

The system is linked and integrated with the existing urine management integrated system to support patient urine management with a total-care concept. The proposed image analysis-based stroke diagnosis function is for neurogenic bladder prevention of patients. By adding the stroke diagnosis function, it was possible to upgrade the function of monitoring existing urination activities by supporting the prediction/prevention field of urination management.

An important part of the patient’s urinary activity management is the patient’s feedback. To this end, a web-based (HTML5) urinary activity management system has been developed, and urinary activity of patients such as neurogenic bladder prevention can be systematically and cumulatively managed. The system construction was largely carried out in 3 stages: DB development, backend development, and client development. The client optimized the switching speed between utilization programs through the AngularJS framework, the backend was configured using Firebase, and DB development was stored in a NoSQL-based document method. The data collected and managed by the integrated patient management system consists of urinary tract data for patient urination activity management and facial image data for stroke diagnosis. A history viewer function is provided for the corresponding collected data, and an AI-based multimodal deep learning applied to the collected data. Through the application of the deep learning model, integrated urination activity management of patients can be performed.

The proposed form is a total-careable patient management system, implemented as a system supported up to neurogenic bladder prevention by adding a stroke diagnosis function, which is a proposed function, to the previously developed urination activity management system.

RESULTS

The experimental test environment proposed in this paper is as follows. It is software that analyzes images obtained from facial and arm motion images and displays suspicion of stroke based on learning (Object) with AI. It can be used as a reference index for stroke diagnosis by supporting the function of detecting and capturing objects in the acquired facial and arm motion images among the analyzed result values. Take a still cut by selecting front photos, face landmarks, eyebrows and eyes, mouth corners, and lip frames among the video frames.

In the experimental process, one of the precursor symptoms of stroke patients is that the left and right symmetry of the face is not matched due to paralysis of one face. In order to distinguish between patients and the general public from the input image using these features, the difference must be learned by extracting features from the facial databases of the 2 groups. To this end, distance learning and feature point learning were used in 2 methods. Fig. 4 shows the detected facial main feature points [22,23].

Fig. 4.

The primary feature point of face.

In addition, first of all, the facial area and 69 landmarks are detected in stroke patients and ordinary daily faces. Angle and slope must be detected to calculate the degree of asymmetry by calculating the distance between features in the face area and comparing left and right differences. Since the existing database has a different face area size and its slope, it must undergo a preprocessing process to normalize the size and angle of the detected face. Then, in order to obtain the rotation angle, the left-hand effect values of the left-end and right-end landmarks are stored as rotation angles. In order to align the images using this rotation angle, the affine transformation matrix is stored. After extracting the landmark from the rotated image by applying the angle, distance values that can find features for left and right symmetry analysis are calculated.

In order to extract the calculated distance values as meaningful data, the left and right distances are divided and compared to store larger values, and the values calculated such as slope, angle, and ratio of eyebrows and lips are also extracted. The extracted 27 data are normalized, labeled, and then learned with the CNN model. Through the convolution 2-dimensional layer of the landmark image learning model Keras, the model directly finds the feature points of the image. For image preprocessing, the face area and landmarks are extracted and a binary image of size 500×500 is generated by dotting the coordinates. This image is converted into a gray scale image of size 100×100 to learn from the model. Thereafter, the binary image is arranged so that the convolution layer can learn, and a label suitable for each image is assigned. And we learned using 605 stroke patient data and 725 general public data. Fig. 5 shows the learning data results of the distance learning model.

Fig. 5.

Result analysis of training data.

As a result of learning data analysis, train accuracy averaged 86% and validation accuracy averaged 83%. In addition, the accuracy of the control group and the patient group was also close to 80%. Accuracy and loss were recorded according to the number of epochs, and similar performance was derived overall for learning data. The results are shown in Tables 1, 2.

Training result

Test result

And we learned using 496 stroke patient data and 678 general public data, a total of 1,174 data. The results can be confirmed in Fig. 6, and similarly, the loss value and account value according to epoch were recorded. As the epoch increased, the loss decreased, and the accuracy increased, resulting in an average of 97% for train accuracy and an average of 86% for validation accuracy. Specifically, the patient data classification accuracy was 88%, and the control group was 87%, which can be found in Tables 3, 4. In the end, it was found that using AI for these stroke diagnoses and making them quickly and accurately had a positive effect indirectly, if not directly, on the neurogenic bladder. And Table 5 represents the configuration items of the system.

Fig. 6.

Result analysis of test data.

Training result

Test result

System structure

DISCUSSION

In this paper, we developed an AI stroke early-stage analysis software that can alert the early stage of stroke through analysis of facial muscle abnormalities for the elderly neurogenic bladder prevention. This is to support neurogenic bladder prevention by adding stroke diagnostic technology, a proposed method, to the previously developed urination patient management system. The system is linked and integrated with the existing urine management integrated system to support patient urine management with a total-care concept. By adding the stroke diagnosis function, it was possible to upgrade the function of monitoring existing urination activities by supporting the prediction/prevention field of urination management.

AI stroke early-stage diagnostic system as a wellness personal medical service plan and prevent cases of missing golden time when existing stroke occurs. In order to secure and facilitate distribution of this, it was developed in the form of AI analysis software so that it can be mounted on various hardware products. It also aims to provide a service that allows users to receive images taken from personal mobile phones in a cloud environment in a 5G mobile communication environment and receive analysis results in seconds through a large-capacity analysis server. In addition, in an environment where a network environment is not provided, we intend to provide services through deep learning analysis through a portable 4K camera.

The method proposed in this paper developed a learning-based deep learning analysis technology that outputs the initial stage of stroke after acquiring a high-definition digital image and then deep learning face analysis. The applied AI model was applied as a multimodal deep learning concept. We developed an AI stroke early-stage analysis software that can alert the early stage of stroke with 86% hit performance through analysis of facial muscle abnormalities in the elderly. This result shows the validation result of the landmark image learning model based on the distance learning model.

Finally, it was developed in the form of AI analysis software so that it can be mounted on various hardware products. It was found that using AI to make stroke diagnosis quickly and accurately affects indirectly, if not directly, neurogenic bladder. It was confirmed that such a total-care system could also enable early prediction of neurogenic bladder in the future.

Notes

Funding Support

This study was conducted with the support of the government (Ministry of Science and ICT and the pan-ministerial medical device project group) in 2021 (No. 2017M3A9E2072931, ICT Medical Device Development Task/No. KMDF_PR_20200901_0188-01, Development of technology for monitoring neurological and mental disorders in the elderly).

Research Ethics

This research was approved by the Institutional Review Board of Gachon University Gil Medical Center (approval number: GBIRB2021-068).

Conflict of Interest

No potential conflict of interest relevant to this article was reported.

AUTHOR CONTRIBUTION STATEMENT

·Conceptualization: SJE

·Data curation: JMH

·Formal analysis: ESK

·Funding acquisition: ESK

·Methodology: SJE

·Project administration: JYL

·Visualization: JMH

·Writing-original draft: ESK

·Writing-review & editing: SJE, JYL

References

1. Özcan F, Özişler Z. The relationship between urinary symptom severity and functional status in patients with stroke. Scott Med J 2022;:369330211072247.
2. Musco S, Giraudo D, Antoniono E, Lombardi G, Del Popolo G, Li Marzi V, et al. Prevalence of nocturia after brain injury: a cross-sectional study in a single rehabilitation center. Brain Inj 2021;35:90–5.
3. Zhao W, Chellppa R, Phillips PJ, Rosenfeld A. Face recognition: a literature survey. ACM Comput Surv 2003;35:399–458.
4. Agrawal B, Gupta C, Mandloi M, Dwivedi D, Surana J. GPU based face recognition system for authentication. Int J Eng Dev Res 2017;5:931–5.
5. Xiang C, Tang C, Cai Y, Xu Q. Privacy-preserving face recognition with outsourced computation. Soft Comput 2016;20:3735–44.
6. Introna L, Wood D. Picturing algorithmic surveillance: the politics of facial recognition systems. Surveill Soc 2004;2:177–98.
7. Pagano C, Granger E, Sabourin R, Marcialis GL, Roli F. Adaptive ensembles for face recognition in changing video surveillance environments. Information Sciences 2014;286:75–101.
8. Kim ES, Bang G, Chung D, Ko IJ. Environment independent hybrid face recognition system using a fixed camera and a PTZ Camera. Adv Sci Technol Lett 2016;129:196–202.
9. Shen Y, Hu W, Yang M, Wei B, Lucey S, Chou TC. Face recognition on smartphones via optimised sparse representation classification. IEEE 2014;:237–48.
10. Gupta A. StrokeSave: a novel, high-performance mobile application for stroke diagnosis using deep learning and computer vision [preprint]. 2019. [cited 2021 Dec 5]. arXiv:1907.05358. Available from: https://arxiv.org/abs/1907.05358.
11. Foong OM, Hong KW, Yong SP. Droopy mouth detection model in stroke warning. IEEE 2016;:616–21.
12. Chang CY, Cheng MJ, Matthew HM. Application of machine learning for facial stroke detection. IEEE 2018;:1–5.
13. Eun SJ, Lee JY, Jung H, Kim KH. Personalized urination activity management based on an intelligent system using a wearable device. Int Neurourol J 2021;25:229–35.
14. Haykin S. Neural networks: a comprehensive foundation New York: MacMillan; 1994.
15. Karmonik C, Boone T, Khavari R. Data-driven machine-learning quantifies differences in the voiding initiation network in neurogenic voiding dysfunction in women with multiple sclerosis. Int Neurourol J 2019;23:195–204.
16. Kim JW, Kim SJ, Park JM, Na YG, Kim KH. Past, present, and future in the study of neural control of the lower urinary tract. Int Neurourol J 2020;24:191–9.
17. Nikkola J, Holm A, Seppänen M, Joutsi T, Rauhala E, Kaipia A. Repetitive transcranial magnetic stimulation for chronic prostatitis/ chronic pelvic pain syndrome: a prospective pilot study. Int Neurourol J 2020;24:144–9.
18. Prabhakar AT, Iqbal Ahmed AS, Vijayakrishnan Nair A, Mathew V, Aaron S, Sivadasan A, et al. Neural correlates of urinary retention in lateral medullary infarction. Int Neurourol J 2019;23:205–10.
19. Murphy KP. Dynamic bayesian networks: representation, inference and learning thesis Berkeley (CA): University of California, Berkeley; 2002.
20. Sonnhammer EL, von Heijne G, Krogh A. A hidden Markov model for predicting transmembrane helices in protein sequences. Proc Int Conf Intell Syst Mol Biol 1998;6:175–82.
21. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436–44.
22. Gemma S, Raul E, Sanchez Y, Carlos H, Garcia C. Facial paralysis detection on images using key point analysis. Appl Sci 2021;11:2435.
23. Park E, Han T, Nam HS. mFAST: automatic stoke evaluation system for time-critical treatment with multimodal feature collection and machine learning classification. In : Proceedings of the 2020 12th International Conference on Computer and Automation Engineering; 2020 May 16 (online); 2020. 38–41.

Article information Continued

Fig. 1.

The concept of personalized total-care system.

Fig. 2.

The flow of AI diagnosis method.

Fig. 3.

Data composition by augmentation method.

Fig. 4.

The primary feature point of face.

Fig. 5.

Result analysis of training data.

Fig. 6.

Result analysis of test data.

Table 1.

Training result

Accuracy Percentage
Validation 83
Train 86

Table 2.

Test result

Category Normal Abnormal
Dataset 205 201
Accuracy 76% 82%

Table 3.

Training result

Accuracy Percentage
Validation 86
Train 97

Table 4.

Test result

Category Normal Abnormal
Dataset 203 114
Accuracy 87% 88%

Table 5.

System structure

No. Category Description
1 Operating system Upper OS: Microsoft Windows 10
2 Hardware Upper CPU 8 core/Memory 32 GB
Upper cudacore 3500
Upper CUDA 9.0
Upper 1TB SSD
3 Display Upper 1920 × 1080 resolution