Bionic blink improves real-time eye closure in unilateral facial paralysis

Facial paralysis (FP) results from impairment of cranial nerve (CN) VII leading to loss of facial movement and expression. FP can be peripheral (infranuclear; damage to ipsilateral CN) or central (supranuclear; contralateral lesion in the brain stem) [1], as well as being unilateral or bilateral. The most common type (20–30 per 100 000 people) is unilateral idiopathic palsy also known as Bell's palsy. However, FP can also occur as a result of stroke, tumor, infection, surgery or trauma [2].

The loss of facial muscle activity has both physiological complications and psychological consequences. The most important physiological consequence of FP for vision health is the impairment of the blink function [3]. Blinking is fundamental for eye health through maintaining the tear film and removing debris, facilitating vision and preventing chronic irritation and pain [4–6]. Additionally, optimal visual performance was shown to rapidly decrease after 4–5 s from a blink [7]. In general, the inability to move facial muscles (i.e. forehead, eyebrow, cheeks or the lips) also results in difficulties in eating, smiling, speaking and expressing emotions [8]. This can psychologically impact FP patients through social alienation, isolation, and emotional disruption with 42% of this population eventually developing depression [9–11].

FP is highly variable, describing mild weakness to complete paralysis [12], and has been shown to resolve completely on its own in 70%–85% percent of patients in a few weeks or months [2, 12]. However, FP patients who do not recover completely, or in the time before complete recovery, are still susceptible to health and social consequences.

The most common strategy to treat eye dryness caused by tear film degradation related to impaired blink function is the application of lubricating drops or ointment [13]. Other strategies such as the use of a moisture chamber, mechanical eyelid taping, and canalicular or punctal (tear drainage duct) occlusion are other popular non-invasive methods [12, 13]. Facial exercises and massages have a positive psychological impact on FP patients and are speculated to support facial muscle tone [2]. However, there is little evidence to support their effect on recovery. Various pharmaceutical approaches to treat facial muscle tone problems exist, but they are not widely used and have mixed success rates [2, 12, 14, 15]. Surgical procedures to reduce the palpebral fissure and corneal exposure include: the implantation of gold eyelid weights [16], lower eyelid ectropion surgery and tarsorrhaphy.

Neuroprosthetics have been used to deliver targeted electrical stimulation [17–20], to nerves and muscles restoring sensory-motor functions in individuals with different neurological disorders [17, 21, 22]. Electrical stimulation has also been used to improve facial function through non-invasive activation of muscles [23]. Although electrical stimulation has not demonstrated significant therapeutic effects in improving facial muscle function [24, 25], it shows promise as an assistive tool to restore both blink function [26–28] and facial muscle tone [15] without surgery in acute cases (2–3 months) [26, 28]. Moreover, restoration of the blink function was shown to improve visual acuity as well as ocular symptoms such as foreign body sensation and dryness after a few hours [26, 27].

Real-time, myoelectric control of neuroprosthetics make them more intuitive to the user and allow for precise control [29, 30]. Blink detection methods on the healthy side of the face have been explored in an effort to close the control loop and take steps towards usable devices.

Vision-based methods such as an infrared beam have been used to detect blinks [31] but is highly susceptible to false positives caused by looking down [32]. Camera-based methods, such as image classifying convolutional or recurrent neural networks are robust, but inherently computation heavy [33, 34] challenging real-time applicability, miniaturization and portability.

Electromyography (EMG) has been shown to robustly detect blinks from the healthy orbicularis oculi muscle (OOM) [5, 35, 36], but is susceptible to crosstalk with other facial muscles and movements [35]. EMG activity is particularly useful for a myoelectric control system development due to its anticipatory nature of muscle movement, typically preceding facial muscle actuation by 30–40 ms [37, 38] and OOM activity by 10–12 ms [39]. This is something vision-based methods fail to take advantage of. Using EMG, Rantanen et al developed and tested a myoelectric control pacing device that could detect a blink and deliver stimulation in real-time (<33 ms) in healthy individuals, but did not validate this myoelectric control system in patients [40]. McDonnell et al tested a myoelectric control system in 6 FP patients and demonstrated that both percutaneous and transcutaneous electrical stimulation could restore blink function detected through EMG [41]. Notably, both systems used large table-top stimulators completely lacking portability necessary for an everyday life adoption and do not take clinical outcomes such as tear film quality and visual acuity into consideration.

Here we present a real-time blink restoration system (NEURO-BLINK) for unilateral FP patients that detects the healthy blink with surface EMG [42] and supports the controlateral blink impaired by paralysis with surface electrical stimulation. We evaluated the success of real-time blink restoration using NEURO-BLINK as well as ocular symptoms (e.g. eye dryness) linked to the loss of blink function before and after use.

The experiments were conducted in accordance with the Declaration of Helsinki and informed consent was obtained from all subjects. Subjects shown in video consented to have identifiable images and videos used for scientific publication. The experiments were approved by Swiss Ethics (BASEC ID: 202-D0065), Swiss Medics (Ref. Number 10000966), and were registered with ClinicalTrials.gov (NCT05504473).

2.1. Patient's recruitment

2.2. Experimental setup

A GoPro 10 was mounted on a tripod and set to a framerate of 120 fps. Videos were acquired in landscape orientation at 1080p full HD resolution. The GoPro was placed at a distance that allowed the entire head and the tops of the shoulders to be in the frame (typically ∼30 cm). Patient position relative to the camera was kept consistent in all the recordings. Experiments were conducted in the same room with the same artificial lighting conditions for all the participants.

Electrodes 1.5 cm in diameter were used to elicit a blink in subjects. The active electrode was placed on the paretic side at the innervation point of zygomatic branch to the OOM just lateral to the orbital rim [26].

Active EMG electrodes 1 cm in diameter were used to detect the healthy blink from the healthy contralateral side. One EMG electrode was placed directly under the eye and the other was placed towards the outside of the eye along the length of the OOM in the direction of its fibers. The EMG electrodes were connected directly to a MyoWare Muscle Sensor (Advancer Technologies, USA). The MyoWare signal was transmitted to a Raspberry Pi through an analog-to-digital converter (figure S2). Subsequently, the microcontroller unit was wire-connected to the portable stimulator, RehaMove 3 (HASOMED GmbH, Germany). To complete the setup, the stimulator was interfaced with the patient via the electrodes positioned on the paretic side of the face. The wiring of the electrodes was integrated into a glasses frame, and all electronic components were housed within a custom designed 3D-printed case.

The described NEURO-BLINK system is depicted in figure 1(A) and video S1.

Zoom In Zoom Out Reset image size

2.3. Protocol description

2.4. Electrical stimulation and characterization

2.5. EMG blink detection and myoelectric controlled system development

2.6. Data acquisition and analysis

Frames at maximum (open) and minimum (closed) eye closure were manually extracted from the videos. IntelliJ was used to measure the distance in pixels between the upper and lower eyelids. Given camera placement and resolution, our system has a sensitivity of approximately 7% of overall eye closure. This was mitigated through repetitions allowing us to achieve statistical power. In addition, a second researcher analyzed some random data points that had been previously measured obtaining the same results, mitigating the impact of human error and strengthening the reliability of the measurements.

Percentage of eye closure was calculated using the following formula:

$$\begin{equation*}{\text{EC}} = {\text{ }}\left( {1 - {\text{ }}\frac{{{\text{aperture}}_{{\text{blink}}}}}{{{\text{aperture}}_{{\text{open}}\_{\text{eye}}}}}} \right)\end{equation*}$$

where EC represents the eye closure, aperture_blink is the distance between eyelids during a blink, and aperture_{open_eye} is the distance between eyelids between blinks. This is illustrated in figure S4(A).

Percentage of cornea coverage (CC) was calculated in three cases (figure S4(B)). In case 1, the cornea was covered both above and below by the eyelid and CC was calculated using the following formula:

$$\begin{equation*}CC = 2*\frac{{{r^2}}}{2}\left( {\theta - {\text{ }}\sin \theta } \right).\end{equation*}$$

In case 2, the cornea was <50% covered by one eyelid and CC was calculated using the following formula:

$$\begin{equation*}CC = \frac{{{r^2}}}{2}\left( {\theta - {\text{ }}\sin \theta } \right).\end{equation*}$$

In case 3, the cornea was >50% covered by the eyelid and CC was calculated using the following formula:

$$\begin{equation*}CC = \pi *{r^2} - \frac{{{r^2}}}{2}\left( {\theta - {\text{ }}\sin \theta } \right).\end{equation*}$$

In all previous formulas r is the radius of the cornea, and θ is the central angle created by the intersection of the eyelids and the cornea.

The overall testing time, including calibration was approximately 2 h for each subject.

All authors had complete access to anonymized data. The Wilcoxon signed-rank test was used to compare the change in percentage closure. An ANOVA for repeated measures was used to analyze the effect of NEURO-BLINK on all patient blinks together.

Of the 10 enrolled patients, all participated in the initial characterization phase. Following this, 2 patients with no response were excluded. A total of 8 patients underwent paced control testing and 6 underwent myoelectric control testing.

3.1. Eye closure increases with increasing charge

Electrode placement for optimal eye closure and minimal discomfort for patients generally fell into two categories: bottom placement and lateral placement (figure 2(A)) with all locations on the OOM.

Zoom In Zoom Out Reset image size

Patients who had a motor response to stimulation all demonstrated a monotonically increasing relationship between eye closure percentage and charge injected, with many subjects falling inside a healthy range (figure 2(B)). Two subjects demonstrated no motor response and were excluded from further testing.

The motor threshold of paretic patients (0.91 ± 0.45 µC) was found to be significantly higher (p < 0.01) than that eight of healthy participants (0.45 ± 0.21 µC) when compared with a Wilcoxon rank sum test (figure 2(C)).

The types of sensation reported by patients were 38.4% electricity, 23.1% pulsation/pressure, 15.4% stinging, 7.7% tingling, 15.4% pinching (figure 2(D)). With words chosen from a list [44]. No patients reported pain.

3.2. Blink restoration with optimal charge

3.3. Electrical stimulation increases eye closure and cornea coverage with paced control

Overall eye closure and percentage of cornea coverage was evaluated with a metronome set to 60 BPM with a high tone every third beat to signal a blink every 3 s.

Eye closure over all subjects together was found to significantly (p < 0.01, n = 562, statistical power >99%) increase with NEURO-BLINK. The median change over 8 subjects is 14.51%. There was also a significant (p < 0.001) increase in eye closure with NEURO-BLINK in all eight patients individually (figures 3(A) and S5).

Zoom In Zoom Out Reset image size

Cornea coverage over all subjects together significantly (p < 0.001, n = 562, statistical power >99%) increased with NEURO-BLINK. The median change over 8 subjects is 7.42% There was also a significant (p < 0.001) increase in cornea coverage in P4, P5, P6, P7, and P8 (figures 3(B) and S5).

3.4. Lagophthalmos reduced with electrical stimulation

3.5. Myoelectrically controlled stimulation increases eye closure and cornea coverage

Eye closure and cornea coverage were also evaluated in the six of the patients who opted to continue testing the myoelectric control system that delivered electrical stimulation to aid the paretic blink when a blink was detected by EMG on the healthy side (figure 4(A)). The EMG detection algorithm was tested offline with 95% accuracy (4% false positives, 1% false negatives) when compared to blinks observed in video footage (as in experimental set up) where a blink was defined as having at least one frame with full eye closure. Electrical stimulation was delivered 46 ± 13 ms after the blink being detected. This delay was measured by timing the difference between blinks in two healthy people, where one is controlling the electrical stimulation being applied to the other.

Zoom In Zoom Out Reset image size

Eye closure over all subjects significantly (p < 0.05, n = 528, statistical power >99%) increased with the myoelectric control NEURO-BLINK system. The median change over 6 subjects is 24.64%. There was also a statistically significant (p < 0.05 in P8 and p < 0.001 in all others) increase in eye closure with NEURO-BLINK individually in all six patients who tried the myoelectric control system (figure 4(C)).

Cornea coverage over all subjects together significantly (p < 0.05, n = 528, statistical power >99%) increased with the myoelectric control NEURO-BLINK system. The median change over 6 subjects is 24.14%. There was a statistically significant (p < 0.01) increase in cornea coverage with NEURO-BLINK individually in all patients except P8 (figures 4(C) and S5).

3.6. Clinical outcomes

This study presents NEURO-BLINK, a fully portable myoelectric control blink restoration system, that can successfully detect healthy blinks using EMG and trigger electrically assisted blinks in the impaired eye. NEURO-BLINK increased eye closure and cornea coverage in eight patients who had a functional response to electrical stimulation eliciting unnatural, but not uncomfortable sensations. The most common electrode placement to best induce a blink was lateral to the eye, consistent to what was found by previous studies [26, 28]. However, we also present an alternative optimal bottom location that proved especially assistive in patients who had residual muscle activity in response to an intentional blink movement mainly in the upper lid. Unlike any previous attempt with assistive devices, seven out of eight patients (87.5%) had an elicitable blink movement that covered their entire pupil corresponding to a full blink [46]. This contrasts with the 53% of patients in whom Mäkelä et al were able to elicit a blink [26], 55% by Frigerio et al [28] and 66% by McDonnall et al [41]. This could be attributed to the combination of a methodological calibration approach and synchronized nature of the myoelectric control system. NEURO-BLINK's comprehensive calibration methodology allowed us to consider positions under the eye as opposed to only lateral positions [26, 28]. Notably, our methodological calibration additionally allowed NEURO-BLINK to achieve significant motor improvement while completely avoiding pain, something previous work [26, 28, 41] has failed to do. Our larger active area likely helped us avoid the unnecessary pain that accompanied the use of small electrodes [47] by McDonnall et al on the eyelid [41], but limited us in our testable locations.

Additionally, the myoelectric control NEURO-BLINK system takes advantage of the patient's natural bilateral activation of a blink, therefore delivering stimulation near to the time of natural muscle activation. This synchronizes electrical stimulation with any residual muscle activity on the impaired side allowing the stimulation to intuitively support the activity without requiring the subject to think about correctly timing a blink as in the paced control condition. Synchronized blinking also improves overall facial symmetry [11, 48] and takes advantage of a relaxed levator palpebrae muscle that would be active if the subject was trying to keep their eye open reflecting important psychological benefits of using a myoelectric control device. These results and implications underline the importance of moving forward with a myoelectric control system that can synchronously pair stimulation with the intention to blink. It also highlights the usefulness of NEURO-BLINK as an assistive device in addition to the known therapeutic benefits of OOM stimulation [26, 27].

Two patients with no response to electrical stimulation were excluded from the remainder of the study. Among these, one had impaired sensation in the entire check and side of the chin and the other had loss near the eyebrow and in the cheek (figure S1(A)). Both demonstrated little to no movement in the OOM even when electrodes were placed in locations they could feel. Although the sensory component of the facial nerve (CN VII) innervates the external ear and the tongue [49, 50] not the skin on the face, we considered sensory feedback to be an intuitive measure of facial nerve health. Indeed recent research suggests that sensory axons in the face play an important role in maintaining facial muscle tone [51]. In patients with intact cutaneous sensory feedback, eye closure was consistently shown to increase with injected charge at various electrode placements around the eye. Impaired cutaneous feedback in the face (innervated by CN V) and lack of response to OOM (innervated by CN VII) stimulation in the two excluded patients may indicate that the pathology of their palsy affected both CN V and VII or that these patients were both more impaired in CN VII causing a high sensory-motor mismatch which can lead to perceived numbness in the cheek [52]. Although impaired sensory feedback coincided with poor response to stimulation in these two patients, sensory feedback testing cannot be a robust indicator of stimulation response in the OOM. While NEURO-BLINK primarily targets the innervation point of the OOM, non-invasive stimulation has the potential to affect both the nerve and the underlying muscle tissue. This could theoretically lead to the direct activation of a deinnervated muscle. However, in individuals with chronic conditions, factors such as muscle atrophy may contribute to their limited response. As a result, more research is required to pinpoint the specific criteria that indicate a patient's potential response to NEURO-BLINK therapy. Regarding the clinical value of this intervention, eye closure and cornea coverage reflect the ability to remove debris from the eye and reduce lagophthalmos, and keep the cornea moist to improve vision respectively [32, 53]. Patients demonstrated a comfortable charge range similar to that of healthy subjects with all responding to charge between 0 and 3 µC except for P1. This suggests that the distal peripheral nerve is functional or that the OOM was not atrophied allowing it to respond to stimulation, and that damage is typically more proximal than the innervation point of the OOM where stimulation is given. In this charge range, lagophthalmos reduction was consistently demonstrated in all patients. NEURO-BLINK consistently elicited a complete blink suggesting its potential to longitudinally improve ocular health. In the most acute patients with idiopathic palsy (P4, P5, P6), cornea coverage was improved both in paced and myoelectric control cases from below 75% (incomplete blink) [46, 54] to a complete blink. It is worth noting that NEURO-BLINK yielded positive results in patients diagnosed up to 4 years ago. This contrasts with previous studies that observed effects in individuals diagnosed only 2–3 months earlier [26, 28]. Additionally, the consistent complete blink restoration, supported by NEURO-BLINK, has the potential to reduce corneal dryness and its symptoms such as visual acuity [55, 56]. This is an improvement with respect to previous devices that cannot consistently elicit complete blinks [28, 41] or have only been tested on healthy participants [5, 40]. In our study, we did not observe any of these clinical changes likely due to the short time of application of our assistive device. However, reducing these dry eye and corneal symptoms in the long term could lead to a difference in visual acuity of 0.05 on the LogMar scale [57], which is equivalent to about 5 years of age-related visual acuity decay [58, 59]. On the other hand, NEURO-BLINK is less effective in patients with a Bell's reflex whose cornea rolls up to be covered by the eyelid even in an incomplete blink. This was especially true for P1 whose blink was assisted from the bottom, reducing lagophthalmos from the bottom but not improving coverage (videos S1 and S2). This may be further exacerbated by the electrical stimulation itself, which has been linked to a missing Bell's reflex [60]. While NEURO-BLINK has shown proof-of-concept promising results, in this short-term study, it opens for a next long-term study, which would help us to better understand its long-term clinical effects.

The myoelectric control (with synchronized blink intention and stimulation) system was tested in the six available patients. This allowed us to assess both the effectiveness and the enhanced portability of the system (5 × 7 × 3 cm wearable stimulator and 15 × 10 × 5 cm processor with a 10 mAh power bank) compared with existing systems (25 × 20 × 10 cm or 8 × 11 × 19 cm tabletop EMG recorder, stimulator and PC computer) [40, 41]. Furthermore, NEURO-BLINK, weighing approximately 500 g, stands in contrast to the several-kilogram counterparts of other systems, making them unsuitable for portability. The EMG detection proved robust to small facial movements present during testing and demonstrated high accuracy in blink detection. The myoelectric control system was able to improve both overall eye closure and cornea coverage in all patients except the one with the most residual muscle activity (least disability). Out of these five with significant changes, two had improvement in cornea coverage with respect to the paced control condition. Although results seem slightly better in the paced control scenario, focusing on the timing of the blink required significant effort in some subjects causing highly unnatural blinks that involved other facial muscles. Consequently, eye closure in the paced control condition was sometimes higher than in the more myoelectric control condition. Despite this, the median change in percentage of eye closure and cornea coverage are both higher with the myoelectric control system than the paced control system (figure S5), suggesting a greater effect on more natural blinks. Due to these differences in blinking effort introduced by using a timer, intra-scenario comparison is more meaningful for understanding the effect of NEURO-BLINK.

Invasive stimulation has been shown to be more precise that non-invasive stimulation likely due to the proximity of active sites to the nerve [18]. While this may be a solution that could further reduce delay time and also more selectively activate OOM muscle fibers only, the requirement of a surgery make it less appropriate than non-invasive stimulation in these patients whom often heal spontaneously [2, 12] and only require assistance before that.

The NEURO-BLINK system has the potential to match the wearability of other glasses mounted blink detection systems [31, 33] and is significantly less complex than visual blink detection systems [33, 34] through its use of EMG. NEURO-BLINK is also significantly more portable, taking advantage of a wearable stimulator that can be clipped on the belt, as opposed to table top stimulators used in other studies [26, 28, 40]. It was demonstrated to be useful at improving eye closure and cornea coverage in facial palsy patients diagnosed up to 4 years ago who have residual motor and sensory facial function.

Further development of the EMG detection system can improve its robustness to noise and other movements such as facial expressions in a way that infrared systems cannot be improved [32]. Further work can also aim at reducing delay time so that blinks are perceived more naturally [28] and increasing robustness to crosstalk from other facial movements with more complex EMG decoding algorithms [35]. This could be achieved through integration of a more sensitive EMG hardware, faster processor, improved software, and more selective stimulation strategies to precisely activate OOM fibers and therefore actuate them faster. Electrical stimulation policies like the one used in NEURO-BLINK can target both muscles and nerves restoring both sensory and motor control [22]. This suggests a promising future for devices exploiting electrical stimulation to improve both facial muscle tone and restore blink function. Therefore, further miniaturization and integration into a single wearable (i.e. a pair of glasses) would allow for more long-term investigations that could further examine the effect of electrical stimulation and blink restoration on lagophthalmos overtime, overall recovery time, and improvements in ocular health.

This work takes exciting steps towards the development of an accessible myoelectric control device that could be used for real-time assistance in FP patients at home and in their daily life.

The authors are immensely grateful to the volunteers who freely donated their time to the advancement of knowledge and the betterment of the lives of those with unilateral facial paralysis.

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to ethical reason.

This project has received funding from the Research Council of the Cantonal Hospital Aarau (Funding Number. 1410.000.164). This project has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (FeelAgain Grant Agreement No. 759998), Swiss National Science Foundation (SNSF) (MOVEIT No. 197271). The project received the support of a fellowship from 'la Caixa' Foundation (ID 100010434). The fellowship code is LCF/BQ/EU20/11810072.

M C developed the NEURO-BLINK system, performed the experiments and the analysis, made the figures, and wrote the manuscript; L C designed and performed the experiments and the analyses, made the figures, and wrote the manuscript; A P designed the experiments, prepared the clinical documentation, performed recruitment and experiments, and reviewed the manuscript; A C designed and performed the experiments, supervised the analyses, made the figures, discussed the results and reviewed the manuscript; G V prepared the clinical documentation, discussed the results, and reviewed the manuscript; M T performed the experiments and reviewed the manuscript; M M designed and supervised the experiments, discussed the results and reviewed the manuscript; S R designed and supervised the experiments, supervised the analyses, discussed the results and reviewed the manuscript.

All authors had complete access to anonymized data. All the authors authorized submission of the manuscript, while the final submission decision was taken by the corresponding author.

Nothing to disclose.