Lower Body Kinematics Monitoring in Running Using Fabric-Based Wearable Sensors and Deep Convolutional Neural Networks

Continuous kinematic monitoring of runners is crucial to inform runners of inappropriate running habits. Motion capture systems are the gold standard for gait analysis, but they are spatially limited to laboratories. Recently, wearable sensors have gained attention as an unobtrusive method to analyze performance metrics and the health conditions of runners. In this study, we developed a system capable of estimating joint angles in sagittal, frontal, and transverse planes during running. A prototype with ber strain sensors was fabricated. The positions of the sensors on the pelvis were optimized using a genetic algorithm. A cohort of ten people completed 15 min of running at ve di erent speeds for gait analysis by our prototype device. The joint angles were estimated by a deep convolutional neural network in inter- and intra-participant scenarios. In intra-participant tests, root mean square error (RMSE) and normalized root mean square error (NRMSE) of less than 2.2 and 5.3%, respectively, were obtained for hip, knee, and ankle joints in sagittal, frontal, and transverse planes. The RMSE and NRMSE in inter-participant tests were less than 6.4 and 10%, respectively, in the sagittal plane. The accuracy of this device and methodology could yield potential applications as a soft wearable device for gait monitoring of runners. Keywords:strain sensors; running; convolutional neural networks; gait analysis; kinematics 1. Introduction Running kinematics are important biomechanical parameters that are associated with injury risk and running economy [1,2]. Previous studies have been conducted on runners on the relationship between di erent kinematic parameters [3]. The range of motion of the hip joint in the frontal plane during running has been correlated with a risk of injuries of the pelvis, hip, and knee

Running kinematics are important biomechanical parameters that are associated with injury risk and running economy [1,2]. Previous studies have been conducted on runners on the relationship between di erent kinematic parameters [3]. The range of motion of the hip joint in the frontal plane during running has been correlated with a risk of injuries of the pelvis, hip, and knee [2]. Foot and shank angles at the initial ground contact, along with knee and hip range of motion during the stance phase are related to running performance [1]. The ankle joint angle at initial contact has also been related to several kinetic risk factors for running-related injuries [4]. Therefore, continuous multi-axis kinematic monitoring of lower extremities is an important consideration for the prevention of running injuries and performance improvement. Sophisticated gait analysis, conducted in clinics, provides the most accurate results, but this solution is not practical for long-term monitoring of runners during daily training. Gait laboratories are also not accessible to the majority of recreational runners. An alternative solution to this problem has been the development of wearable sensors to measure running kinematics. Inertial measurement units (IMUs) are the most common wearable sensor systems. IMUs have been used to measure kinematics and kinetics of the lower body including two- and three-dimensional joint angles [5–8], changes in the kinematics [9], ground reaction forces [10], and lower body joint Sensors2019,19, 5325; doi:10.3390 /s19235325 /journal/sensors

Sensors2019,19, 5325 2 of 18 power [11]. IMU-based wearable sensors have limitations when measuring the multi-axis joint angle. Magnetometers are susceptible to ferromagnetic disturbances [6], and therefore heading drift in the IMUs is a challenge. The physiological bone coordinate system is also di erent from the IMU's coordinate system attached to each segment which can lead to a measurement error [8]. Thus, the correlation between angles measured by IMUs and a motion capture system have been reported to be poor for the hip frontal plane [12]. Considering the accuracy of IMUs for joint angle measurement, it should be highlighted that a large number of the previous works have attached the motion capture markers to the IMU units for measuring reference joint angles [5], however, attaching markers to anatomical bone marks, which is the common method in gait analysis, can signi cantly increase the error [13]. Soft strain sensors are an alternative solution for human motion monitoring that have been employed for lower body monitoring [14–17], trunk angle measurement [18], gait phase detection [19], and posture classi cation [20]. The primary advantage of fabric-based strain sensors as compared with rigid IMUs is exibility and comfort during their use. Strain sensors have been shown to measure hip, knee, and ankle joint angles in the sagittal plane with a root mean square error (RMSE) of 6 [16], 2 to 15 [15–17], and 3 to 10 [16,17], respectively. The peak knee exion angle during gait was also estimated with an RMSE less than 2 [21]. Totaro et al. [17] extended the use of strain sensors for lower body monitoring from the sagittal plane to the frontal and transverse planes for only the ankle joint, however, the performance of the sensors for non-sagittal plane angle measurements during walking and running was not investigated. Mengüç et al. [16] introduced the use of stain sensors to monitor running in the sagittal plane. While an RMSE of 2 to 10 was reported for a lower body joint measurement during walking and simple joint exion tasks [15–17], the error was shown to be less than 15 for running [16].

measurements during walking and running was not investigated. Mengüç et al. [16] introduced the use of stain sensors to monitor running in the sagittal plane. While an RMSE of 2 to 10 was reported for a lower body joint measurement during walking and simple joint exion tasks [15–17], the error was shown to be less than 15 for running [16]. A comparison of the performance of strain sensors for running [16] with walking [16] and joint bending exercises [15,17] highlights the challenges of this technology in running applications. A primary problem of soft strain sensors is the calibration of sensors for each individual person. Methods such as linear tting [16,17], Fourier series [15], machine learning (random forest and neural network) [14], and deep learning algorithms (LSTM) [22] have been previously reported addressing calibration issues. In a previous study, we showed that a machine learning model decreased the error of knee angle measurements approximately 3 in contrast to a linear regressor [14]. Incorporating a deep learning model has also been shown to lower the error of full-body motion tracking compared with linear models [22,23]. Although calibrating a sensor to a speci c person leads to high accuracy, it requires a gold standard motion capture system. An alternative approach is to introduce a model that is calibrated on several people and can be used by others. In previous works, a random forest regressor obtained an RMSE of 7 for the knee angle measurements in an inter-participant test [14]. Upper body postures were also classi ed with an accuracy of 65% in an inter-participant scenario [20]. Changes in the position of sensors on the body and di erences in body shapes are some challenges in inter-participant tests. Considering the need for the use of wearable sensors (IMUs or exible strain sensors) for running kinematic monitoring, reliability and opportunities of IMUs have been investigated for indoor and in- eld applications [24,25]. Previous works related to soft strain sensors have focused on sensor development [26] and their application for tracking simple activities such as joint bending [26] and two-dimensional kinematics of the gait during running

of wearable sensors (IMUs or exible strain sensors) for running kinematic monitoring, reliability and opportunities of IMUs have been investigated for indoor and in- eld applications [24,25]. Previous works related to soft strain sensors have focused on sensor development [26] and their application for tracking simple activities such as joint bending [26] and two-dimensional kinematics of the gait during running [16]. Herein, this work addresses the validation of a ber-based strain sensor for three-dimensional kinematic monitoring during running. Improvements in measurement accuracy were addressed by optimization of sensor placement and advanced signal processing.

Sensors2019,19, 5325 3 of 18 2. Materials and Methods 2.1. Strain Sensor The ber sensors that were incorporated into the prototype were produced as previously reported [27]. A multi lament spandex yarn (polyether, urethane, and urea) was dip coated with a carbon black thermoplastic elastomer composite. A solution consisting of Hytrel 3078 (H3078, DuPont Kingston, ON) in dichloromethane (5 wt % H3078) and carbon black (50 wt % with respect to H3078) was used to coat the spandex yarn using a continuous roll-to-roll method at a rate of 3.81 cm/second and wound onto a bobbin 1.83 meters from the exit of the solution (to allow su cient drying prior to collection to reduce bers sticking together). Any remaining solvent was removed in vacuo at 60 C for 30 min. Prior to use, all sensors were conditioned by straining to 40% 100x at 10% per second with a sinusoidal wave pattern. To protect the sensor from shorting when exposed to conductive liquids (such as sweat), an additional insulating sheath was applied by dip coating in a 5 wt % poly(styrene-b-ethylene-co-butadiene-b-styrene) cyclohexane solution once connections were made to the connecting wires. The analysis of the sensors range, gauge factor, hysteresis, long-term sinusoidal testing, strain rate, and long-term random testing were reported in a previous study [27]. The sensor had no hysteresis below 30% strain, consistent signal over 4 hours, and a gauge factor of 5. Strain response of the sensor at 10%, 20%, and 30% strain were linear and consistent. The piezoresistive sensing was limited to 90% strain, while the working range was limited to 30% to ensure signal linearity and limit plastic deformation. The performance of the sensor strained at di erent frequencies has been included in this study (Appendix). The sensors were able to track up to 10 Hz (200% per second; 100 mm/s for a 50 mm sample). 2.2. Optimization of Sensor Placement Optimization of sensor placement is an important step to determine the minimum number of sensor positions that lead to the highest accuracy. Optimization was only studied for the hip joint since it requires monitoring three degrees

sensors were able to track up to 10 Hz (200% per second; 100 mm/s for a 50 mm sample). 2.2. Optimization of Sensor Placement Optimization of sensor placement is an important step to determine the minimum number of sensor positions that lead to the highest accuracy. Optimization was only studied for the hip joint since it requires monitoring three degrees of freedom with many possible positions and orientations that could a ect the sensor performance. The knee and ankle have limited degrees of freedom and potential positions. Therefore, sensors were placed on the ankle and knee empirically based on the primary axis of movement and re ned with trial and error. Sensor placement for the hip joint angle was completed with the goal of nding the combination of positions that led to the highest accuracy in the joint angle estimation. The positioning was treated as a feature selection problem, and positions were considered as features according to the method introduced in a previous work [28]. The strain at all possible positions was measured using optical cameras and re ective markers. The objective function was a linear regressor's accuracy (R 2 error) that estimated joint angles in the sagittal, frontal, and transverse planes in a leave-one-person-out cross-validation. Two methods were employed to solve the feature selection problem: a forward sequential (FS) [29] method and a genetic algorithm (GA) [30]. There are two main categories of feature selection methods: lter and wrapper. Filter methods use statistical tests to nd the best feature subset without training a machine learning model. Wrapper methods evaluate the usefulness of a feature subset by training a machine learning model and computing the corresponding accuracies [31]. The wrapper method has been reported to nd better feature subsets as compared with lter methods because it considers the accuracy of any feature subset for the prediction problem. Therefore, wrapper methods were employed in this study. Two main search strategies used for wrapper methods are sequential algorithms and randomized algorithms [31]. In this study, a forward sequential algorithm from the sequential algorithms and a genetic algorithm from randomized algorithms were implemented.

compared with lter methods because it considers the accuracy of any feature subset for the prediction problem. Therefore, wrapper methods were employed in this study. Two main search strategies used for wrapper methods are sequential algorithms and randomized algorithms [31]. In this study, a forward sequential algorithm from the sequential algorithms and a genetic algorithm from randomized algorithms were implemented. A linear regression model was used as the machine learning algorithm which previously has achieved satisfactory accuracies in joint angle estimation using strain data while being less computationally demanding as compared with more complex machine learning algorithms [14].

Sensors2019,19, 5325 4 of 18 2.2.1. Data for Sensor Placement To nd the best positions and number of sensors to monitor the hip joint angle, strains on the garment during running were recorded using six motion capture cameras (Vicon, Oxford, UK). A total of 50 motion capture markers (diameter of 6 mm) were placed on tights 4 cm apart in a grid pattern (Figure). Five participants (all male, age 25 2 years, weight 75 7 kg, and height 180 2 cm) participated in the data collection. The experimental protocol was approved by the O ce of Research Ethics at Simon Fraser University. Prior to any data collection, written informed consent was obtained from all participants. Data recording was completed at the following three speeds: 8 km/h, 10 km/h, and 12 km/h, each for two minutes. The running test was conducted on a split-belt instrumented treadmill (Bertec Corporation, Columbus, OH, USA).Sensors 2019, 19, x FOR PEER REVIEW 4 of 17 pattern (Figure 1). Five participants (all male, age 25 ± 2 years, weight 75 ± 7 kg, and height 180 ± 2 cm) participated in the data collection. The experimental protocol was approved by the Office of Research Ethics at Simon Fraser University. Prior to any data collection, written informed consent was obtained from all participants. Data recording was completed at the following three speeds: 8 km/h, 10 km/h, and 12 km/h, each for two minutes. The running test was conducted on a split-belt instrumented treadmill (Bertec Corporation, Columbus, OH, USA). Figure 1. (a) Grid of markers on garment for optimization of sensor position on pelvis. Initial vertical and horizontal distances of markers were 4 cm before wearing the tight and (b) all potential sensor orientation including horizontal, vertical, and two diagonals were considered. The distances of neighboring markers in vertical, horizontal, and both diagonal directions were computed using the three-dimensional marker positions (Figure 1b). Strains on the garment were computed using the following formula: í µí± í µí±¡í µí±Ÿí µí±Ží µí±–í µí±› = í µí°¿âˆ’í µí°¿ 4 í µí°¿ 4 , (1) where L was the distance of markers and

diagonals were considered. The distances of neighboring markers in vertical, horizontal, and both diagonal directions were computed using the three-dimensional marker positions (Figure 1b). Strains on the garment were computed using the following formula: í µí± í µí±¡í µí±Ÿí µí±Ží µí±–í µí±› = í µí°¿âˆ’í µí°¿ 4 í µí°¿ 4 , (1) where L was the distance of markers and í µí°¿ 4 was the initial distance of markers in a neutral standing posture (í µí°¿ 4 was greater than or equal to 4 cm). When the garment was worn, the fabric had a natural stretch, and therefore was able to increase or decrease in length. If the garment stretch decreased such that wrinkles were formed, and effectively resulting in data that was not ”real”, a limit was created so that this data would be excluded. Therefore, during the kinematic tracking, the value of L was limited to a value greater than or equal to 4 cm, which was determined to be a greater length than when wrinkles would form. A condition was applied such that each sensor would strain a minimum of 10% (relative to its starting position) during the intended running motion to maximize the obtained signal and accuracy. This excluded any sensor positions that would not strain during the running motion. Twelve reflective markers were used to define the pelvis and thigh segments to measure the hip joint angle during the test. The marker set and segments are discussed in the next sections. 2.2.2. Sequential Forward Method In this method, we started from an empty subset of positions (features) and added the next position to the already selected positions, which led to the highest value of the objective function [29]. The method is described in Algorithm 1 (Appendix B) and was implemented in Python. In the pseudocode, í µí±Œ is the feature set that contains all features, í µí±‹ Þ is the feature subset selected by the algorithm after í µí±˜ iterations and contains í µí±˜ features, í µí°½ is the objective function that we aim to maximize, and í µí±¥ > is a new feature that

(Appendix B) and was implemented in Python. In the pseudocode, í µí±Œ is the feature set that contains all features, í µí±‹ Þ is the feature subset selected by the algorithm after í µí±˜ iterations and contains í µí±˜ features, í µí°½ is the objective function that we aim to maximize, and í µí±¥ > is a new feature that is selected to be added to the current feature subset (í µí±‹ Þ) in each iteration. 2.3.3. Genetic Algorithm Method The genetic algorithm has been frequently used as a feature selection for machine learning problems [32]. A custom genetic algorithm model was implemented in Python for this study [30]. The pseudocode of the genetic-based feature selection algorithm is described in Algorithm 2 (Appendix B). (a) (b) Positions selected by the genetic algorithm Figure 1. (a) Grid of markers on garment for optimization of sensor position on pelvis. Initial vertical and horizontal distances of markers were 4 cm before wearing the tight and (b) all potential sensor orientation including horizontal, vertical, and two diagonals were considered. The distances of neighboring markers in vertical, horizontal, and both diagonal directions were computed using the three-dimensional marker positions (Figureb). Strains on the garment were computed using the following formula: strain= L L0 L0 , (1) whereLwas the distance of markers andL0was the initial distance of markers in a neutral standing posture(L0was greater than or equal to 4 cm). When the garment was worn, the fabric had a natural stretch, and therefore was able to increase or decrease in length. If the garment stretch decreased such that wrinkles were formed, and e ectively resulting in data that was not ”real”, a limit was created so that this data would be excluded. Therefore, during the kinematic tracking, the value ofLwas limited to a value greater than or equal to 4 cm, which was determined to be a greater length than when wrinkles would form. A condition was applied such that each sensor would strain a minimum of 10% (relative to its starting position) during the intended running motion to maximize the obtained signal and accuracy. This

tracking, the value ofLwas limited to a value greater than or equal to 4 cm, which was determined to be a greater length than when wrinkles would form. A condition was applied such that each sensor would strain a minimum of 10% (relative to its starting position) during the intended running motion to maximize the obtained signal and accuracy. This excluded any sensor positions that would not strain during the running motion. Twelve re ective markers were used to de ne the pelvis and thigh segments to measure the hip joint angle during the test. The marker set and segments are discussed in the next sections. 2.2.2. Sequential Forward Method In this method, we started from an empty subset of positions (features) and added the next position to the already selected positions, which led to the highest value of the objective function [29]. The method is described in Algorithm 1 (Appendix) and was implemented in Python. In the pseudocode,Yis the feature set that contains all features,X kis the feature subset selected by the algorithm afterkiterations and containskfeatures,Jis the objective function that we aim to maximize, andx +is a new feature that is selected to be added to the current feature subset (X k) in each iteration.

This study develops a system for estimating joint angles during running using wearable sensors.