1. Introduction
The number of stroke patients is increasing every year, and 65% of them suffer from hand disability as an aftereffect (Kwakkel et al., Reference Kwakkel, Kollen, van der Grond and Prevo2003; Feigin et al., Reference Feigin, Brainin, Norrving, Martins, Sacco, Hacke, Fisher, Pandian and Lindsay2022). Hand dysfunction is a significant burden for patients since the hands play an essential role in daily life and labor (Mouri et al., Reference Mouri, Kawasaki, Aoki, Nishimoto, Ito and Ueki2009; Proietti et al., Reference Proietti, Nuckols, Grupper, Schwerz de Lucena, Inirio, Porazinski, Wagner, Cole, Glover, Mendelowitz, Herman, Breen, Lin and Walsh2024). Finger flexor spasticity is one of the most common motor impairments after stroke. Spasticity affects the recovery of hand motor function by increasing the stiffness of the finger joints and decreasing the range of motion (Sadarangani et al., Reference Sadarangani, Jiang, Simpson, Eng and Menon2017). Therefore, in the hand rehabilitation of stroke patients, determining the optimal therapy based on the spasticity condition is important to enhance the effectiveness of therapy (Heung et al., Reference Heung, Tang, Shi, Tong and Li2020).
In clinical practice, healthcare providers assess spasticity by palpation based on an ordinal scale (e.g., Modified Ashworth Scale [MAS]) (Bohannon and Smith, Reference Bohannon and Smith1987; Harb and Kishner, Reference Harb and Kishner2023). However, this method lacks objectivity because it relies on the evaluator’s experience. In addition, in an environment where palpation is not available, such as in telerehabilitation, it is not easy to ascertain the daily condition of spasticity (Auger et al., Reference Auger, Moreau, Côté, Guerrera, Rochette and Kairy2023). As spasticity conditions fluctuate daily, depending on physical and emotional stimuli (TSUJI et al., Reference TSUJI, OTA, KIMURA, CHINO and ISHIGAMI2002), it is difficult to provide optimal therapy to patients if spasticity cannot be assessed daily. In order to provide effective therapy to patients in any environment, a simple and objective method to quantify spasticity is needed.
It has been suggested that spasticity conditions can be quantified by measuring finger joint stiffness values (Shi et al., Reference Shi, Heung, Tang, Tong and Li2020). Various devices have been developed to measure finger joint stiffness after stroke (Kuo and Deshpande, Reference Kuo and Deshpande2012; Peperoni et al., Reference Peperoni, Capitani, Fiumalbi, Capotorti, Baldoni, Dell’Agnello, Creatini, Taglione, Vitiello, Trigili and Crea2023; Ranzani et al., Reference Ranzani, Chiriatti, Schwarz, Devittori, Gassert and Lambercy2023). However, most of them were developed to measure the stiffness of specific finger joints. Therefore, it is challenging to measure joint stiffness of multiple joints (e.g., the metacarpophalangeal [MCP] and proximal interphalangeal [PIP] joints) or different fingers (e.g., index and middle fingers) simultaneously or separately. Changing the devices’ settings is necessary when measuring different joints or fingers. Thus, it increases the measurement time and makes it difficult to use the device conveniently.
Recently, an analytical model-based stiffness estimation method was proposed using flexible and lightweight soft elastic composite actuators (SECAs) (Heung et al., Reference Heung, Tong, Lau and Li2019, Reference Heung, Tang, Shi, Tong and Li2020). Unlike conventional stiffness measurement devices, the stiffness of all fingers and joints can be measured simultaneously if the bending angle of each joint can be measured. The joint stiffness values estimated by this method have been compared to the MAS scores, which have shown potential as an indicator to quantify spasticity (Shi et al., Reference Shi, Heung, Tang, Tong and Li2020). However, whole-finger soft actuators, such as the SECA, which support the entire finger with a single actuator, have several problems, including high drive cost, limitations in supporting individual joints, and difficulty in customization (Kokubu et al., Reference Kokubu, Wang, Tortós Vinocour, Lu, Huang, Nishimura, Hsueh and Yu2022; Kokubu et al., Reference Kokubu, Nishimura and Yu2024a, Reference Kokubu, Tortós Vinocour and Yu2024b; Matsunaga et al., Reference Matsunaga, Ke, Hsueh, Huang, Gomez-Tames and Yu2024).
Joint modular soft actuators were developed to address these issues (Yun et al., Reference Yun, Kang and Cho2017; Kokubu et al., Reference Kokubu, Tortós Vinocour and Yu2024b; Tortós-Vinocour et al., Reference Tortós-Vinocour, Kokubu, Matsunaga, Lu, Zhou, Gomez-Tames and Yu2024). This actuator is more energy efficient than a whole-finger type because each joint is divided into individual actuators, and each joint can be controlled independently by air pressure (Kokubu et al., Reference Kokubu, Wang, Tortós Vinocour, Lu, Huang, Nishimura, Hsueh and Yu2022; Matsunaga et al., Reference Matsunaga, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). In addition, the actuators can be easily customized to the individual by simply changing the size of the connectors that connect the actuators. This soft actuator will enable us to provide an inexpensive and rapid rehabilitation system that can be adapted to many patients.
In our previous study, we newly designed a joint modular version of the SECA (Modular-SECA) and proposed a stiffness estimation method for joints using it (Matsunaga et al., Reference Matsunaga, Kokubu, Tortos Vinocour, Ke, Hsueh, Huang, Gomez-Tames and Yu2023). By adding correction parameters to the analytical model of the SECA, we constructed an analytical model for stiffness estimation according to the behavior of the Modular-SECAs. This analytical model is defined as the modular-analytical model. However, when estimating the stiffness values of multiple joints simultaneously, it was found that the modular-analytical model’s accuracy became unstable because the influence of the interactions between the finger and the Modular-SECA, as well as between the Modular-SECAs, changed for each joint and measurement (Matsunaga et al., Reference Matsunaga, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). It was also suggested that these influencing factors are complex and mixed up, making it difficult to incorporate them individually as correction terms in the modular-analytical model. Therefore, we proposed a method to directly estimate joint stiffness from the trend of joint angle changes using an artificial neural network (ANN) (Matsunaga et al., Reference Matsunaga, Oba, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). By using ANNs, we were able to improve the estimation accuracy by learning the effects of nonlinear interactions and incorporating elements that the modular-analytical model cannot capture. The ANN model also showed good estimation performance for different finger sizes and high stiffness values, which the modular-analytical model could not estimate.
Although the ANN-based model consistently outperformed the modular-analytical and other machine-learning models, its generalization performance remained limited. Machine-learning models generally require the test data to follow the same distribution as the training data to maintain prediction accuracy (Krueger et al., Reference Krueger, Caballero, Jacobsen, Zhang, Binas, Zhang, Le Priol and Courville2021; Yuan et al., Reference Yuan, Park and Lejeune2022; Zhou et al., Reference Zhou, Liu, Qiao, Xiang and Loy2022). However, in our previous study, when estimating stiffness for fingers with sizes different from those included in the training data, the ANN model’s performance declined (Matsunaga et al., Reference Matsunaga, Oba, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). This result was due to shifts in data distribution caused by differences in finger size, which the ANN could not fully adapt to. In human finger joint stiffness estimation, the data distribution of the patient’s finger joints is unknown. Therefore, a large amount of training data covering the range of possible situations in finger joint stiffness estimation is required to make a model that can accurately estimate joint stiffness values for various patients. However, it is difficult to reproduce various situations that may occur in stiffness estimation using dummy fingers, and much effort is required for data measurement.
One way to improve the generalization of machine-learning models is to use physical information (Yuan et al., Reference Yuan, Park and Lejeune2022). Physical information is not affected by changes in the data distribution. Therefore, it may construct models with high generalization and robustness, even with small training data, by using it as an input or output variable for models or as a constraint during training (Kalina et al., Reference Kalina, Linden, Brummund and Kästner2023). In our previous study, the ANN models incorporated information from the modular-analytical model as an output variable (Matsunaga et al., Reference Matsunaga, Oba, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). However, because the prediction accuracy of the ANN depends on the accuracy of the modular-analytical model, the model with the modular-analytical model’s information resulted in lower accuracy. On the other hand, this result could be taken to mean that the ANN with the modular-analytical model information could learn invariant information independent of the data distribution. Therefore, the modular-analytical model information may lead to improved generalization in estimating joint stiffness, which makes it difficult to identify data distribution and collect data.
It is essential to improve the accuracy of the modular-analytical model to take advantage of the stiffness estimation’s physical information. Doing so should enable more accurate and stable stiffness estimation, either by the analytical model alone or combined with machine learning. Our previous study has identified factors that affect the stiffness estimation accuracy with the Modular-SECAs (Matsunaga et al., Reference Matsunaga, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). However, we have yet to comprehensively analyze the specific effects of these factors on each term of the modular-analytical model. The modular-analytical model for stiffness estimation is derived from the conservation of energy in the Modular-SECA and finger complex. Consequently, their energy conservation relationship does not hold if the stiffness estimation accuracy is low. Therefore, the stiffness estimation accuracy may be improved by correcting the energy balance of the modular-analytical model.
In this study, we aimed to improve the stiffness estimation accuracy by applying data-driven tuning to the modular-analytical model. By analyzing the effect of each term of the modular-analytical model due to the variation of the stiffness estimation results, the cause of the failure of energy conservation in the Modular-SECA and finger complex was clarified. Then, correction parameters were added to the modular-analytical model to adjust the modular-analytical model’s energy balance defects so that the complex’s energy conservation is established. After that, the modular-analytical model was corrected to fit the measurement data by predicting the correction parameters using an ANN. The proposed method was compared with the original modular-analytical model and our previously proposed ANN-based model. In addition, subject experiments were also conducted on healthy subjects to verify the performance of the proposed method in the stiffness estimation of human finger joints.
2. Methods
This section first explains the modular-analytical model-based joint stiffness estimation. Next, each term of the modular-analytical model is analyzed in detail based on the stiffness estimation results of previous studies to identify the cause of the error in stiffness estimates. After that, the method to identify correction parameters to improve the accuracy of the modular-analytical model and to estimate these parameters using an ANN based on the measurement data is explained. Finally, an experiment to estimate human joint stiffness to verify the performance of the proposed method is described.
2.1. Joint stiffness estimation based on the modular-analytical model
The process of deriving the modular-analytical model and the flow of stiffness estimation will be briefly presented.
2.1.1. Analytical model for stiffness estimation with the Modular-SECA
More details regarding the analytical model can be seen in Heung et al. (Reference Heung, Tang, Shi, Tong and Li2020) and Matsunaga et al. (Reference Matsunaga, Kokubu, Tortos Vinocour, Ke, Hsueh, Huang, Gomez-Tames and Yu2023).
First, the analytical model for free bending of the SECA (Heung et al., Reference Heung, Tang, Shi, Tong and Li2020) is shown in Equation (2.1).
$$ P=\frac{W_A+{W}_L}{\Delta V} $$
where
$ P $
,
$ {W}_A $
,
$ {W}_L $
, and
$ \Delta V $
are the input air pressure, the bending strain energy stored in the soft actuator body, the bending strain energy stored in the torque-compensating layer, and the increase in the chamber’s volume, respectively. Free bending is the bending behavior of an actuator in a measurement environment where nothing but gravity is applied to the actuator. The SECA’s bending angle is the angle at which the work done by the air pressure input to the soft actuator (
$ P\Delta V $
), is balanced by the energy required to bend the actuator and torque-compensating layer (
$ {W}_A $
and
$ {W}_L $
).
However, Equation (2.1) cannot be used for the Modular-SECA’s bending analysis because the SECA and the Modular-SECA have different bending performance (Matsunaga et al., Reference Matsunaga, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). Therefore, we identified correction parameters to add to Equation (2.1) to match the free bending angles of the Modular-SECA to obtain the modular-analytical model (Matsunaga et al., Reference Matsunaga, Kokubu, Tortos Vinocour, Ke, Hsueh, Huang, Gomez-Tames and Yu2023) shown in Equation (2.2). Specifically, the free bending angles of the Modular-SECA were measured, and correction parameters were identified by the least-squares method to minimize the difference from the theoretical bending angle derived from the SECA model.
$$ P=\frac{W_A+0.5{W}_L}{1.15\Delta V}. $$
Next, the energy stored in the joints at different positions,
$ {W}_{joint} $
, is as follows:
$$ {W}_{joint}=\frac{1}{2}k{\left(\theta -{\theta}_0\right)}^2 $$
where
$ k $
,
$ \theta $
, and
$ {\theta}_0 $
are the joint stiffness value, the finger joint angle, and the resting angle, respectively. The stiffness value is the stiffness value in the flexion direction when
$ \theta >{\theta}_0 $
and in the extension direction when
$ \theta <{\theta}_0 $
. In this study, we set the range of change in joint angle as
$ \theta <{\theta}_0 $
because we want to measure the stiffness value in the extension direction in a stroke patient. It is noteworthy that
$ {W}_{joint} $
is the energy that does not vary with the type of actuator.
The modular-analytical model for stiffness estimation is derived from energy conservation in the finger and the Modular-SECA complex and is obtained by incorporating Equation (2.3) into Equation (2.2). When the joint is in the extension direction from the resting angle (
$ \theta <{\theta}_0 $
),
$ {W}_{joint} $
is treated similarly to the energy stored by bending in the Modular-SECA. Thus, from Equations (2.2) and (2.3), the modular-analytical model for stiffness estimation can be expressed by the displacement direction of the joint as follows:
$$ 1.15P\Delta V={W}_A+0.5{W}_L-{W}_{joint},\hskip1em 0\le \theta <{\theta}_0. $$
In
$ \theta <{\theta}_0 $
, the finger joint rests in the position where the energy generated by the air pressure input to the Modular-SECA (
$ P\Delta V $
) and the energy stored in the joint (
$ {W}_{joint} $
) are balanced by the energy required to bend the Modular-SECA (
$ {W}_A $
and
$ {W}_L $
). Transforming Equation (2.4), the joint stiffness value can be calculated using Equation (2.5):
$$ k=\frac{2\left({W}_A+0.5{W}_L-1.15P\Delta V\right)}{{\left(\theta -{\theta}_0\right)}^2},\hskip1em 0\le \theta \le 0.7{\theta}_0. $$
It is noteworthy that the stiffness estimation range is changed from
$ 0\le \theta <{\theta}_0 $
to
$ 0\le \theta \le 0.7{\theta}_0 $
to avoid singularity (
$ \theta ={\theta}_0 $
) effects (Heung et al., Reference Heung, Tang, Shi, Tong and Li2020).
2.1.2. Stiffness estimation of three joints
The procedure for estimating the joint stiffness using the Modular-SECAs follows the same protocol as proposed in our previous work (Matsunaga et al., Reference Matsunaga, Oba, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). The flow of stiffness measurement is shown in Figure 1(a). First, the resting joint angles are measured before attaching the Modular-SECAs to the joints. Then, the actuators are pressurized from 0 to 80 kPa and depressurized to 0 kPa, with joint angles recorded every 10 kPa during one pressurization–depressurization cycle (denoted as REP). Following the recommendation by Shi et al. (Reference Shi, Heung, Tang, Tong and Li2020), three REPs are conducted per measurement to account for short-term effects of repeated joint motion.
Figure 1. The flow of the modular-analytical model-based stiffness estimation with the Modular-SECAs. (a) Measurement flow and (b) estimation flow.
The flow of the stiffness estimation is shown in Figure 1(b). The stiffness value is estimated for each REP. That is, up to three stiffness values are estimated in one measurement. The joint stiffness values are calculated by substituting the joint angle values (among the 17 included in one REP), that fall within the stiffness estimation range
$ 0\le \theta \le 0.7{\theta}_0 $
, into Equation (2.5). No stiffness value is calculated for values where the joint angle is not inside the estimation range. Finally, the stiffness values that could be calculated for each air pressure value for one REP are averaged by Equation (2.6), in order to obtain the final stiffness value (
$ {k}_{analytic} $
).
$$ {k}_{analytic}=\frac{1}{n}\sum \limits_{i=1}^n{k}_i,\hskip1em 1\le n\le 17 $$
where
$ n $
is the number of joint angles that are included in the stiffness estimation range among the 17 joint angle values measured from the REP. The accuracy of the final stiffness value is affected by every stiffness value obtained in the REP. In other words, if the estimation accuracy of one part of the REP is high, but the accuracy of the other parts is low, the final stiffness value may be low. Also, if all 17 joint angle values included in one REP are not included in the stiffness estimation range, no stiffness value can be estimated by Equation (2.5).
Previously, we simultaneously estimated the stiffness values of three joints (the distal interphalangeal [DIP], PIP, and MCP joints). The stiffness values of the finger joints were varied by placing torsion springs at each joint of dummy fingers made with three different sizes (small, medium, and large). These dummy fingers were designed based on the sizes of the Japanese index fingers (Kokubu et al., Reference Kokubu, Wang, Tortós Vinocour, Lu, Huang, Nishimura, Hsueh and Yu2022). We set the stiffness values of the finger joints based on the stiffness value of MAS score
$ \le $
1+. The connectors’ lengths between the Modular-SECAs were adjusted according to the size of each dummy finger. We estimated the stiffness after measuring the angles at each joint according to the flow shown in Figure 1. It is noteworthy that all Modular-SECAs attached to each of the three joints had the same pneumatic control. The data from the medium-sized dummy finger were used to train the ANN model that estimates stiffness values directly, and the data from the small- and large-sized dummy fingers were used to verify the generalization performance of the ANN model. In addition, for the medium-sized dummy finger, we also estimated stiffness values equivalent to the stiffness of MAS score
$ \ge $
2. It is noteworthy that these high stiffness values could not be estimated by Equation (2.5) because all 17 joint angles obtained by 1 REP were not included in the estimation range
$ 0\le \theta \le 0.7{\theta}_0 $
(Matsunaga et al., Reference Matsunaga, Kokubu, Tortos Vinocour, Ke, Hsueh, Huang, Gomez-Tames and Yu2023). More details regarding this experimental data can be seen in Matsunaga et al. (Reference Matsunaga, Oba, Ke, Hsueh, Huang, Gomez-Tames and Yu2024).
This study used the data measured in our previous study (Matsunaga et al., Reference Matsunaga, Oba, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). The data were also divided into the same five datasets as before, as follows:
-
• The medium-sized dummy finger data for a model’s training (M-Training data).
-
• The medium-sized dummy finger data for a model’s test (M-Test data).
-
• The small-sized dummy finger data (S-Prediction data).
-
• The large-sized dummy finger data (L-Prediction data).
-
• The medium-sized dummy finger data with high stiffness values (H-Prediction data).
2.2. Correction of the modular-analytical model
2.2.1. Reasons behind the failure of energy conservation in the modular-analytical model
First, each term of the modular-analytical model is analyzed in detail based on previous stiffness estimation results to identify the causes of errors in stiffness estimates.
From our previous study (Matsunaga et al., Reference Matsunaga, Ke, Hsueh, Huang, Gomez-Tames and Yu2024), it is clear that the accuracy of the modular-analytical model is mainly influenced by the following two factors:
-
• Interaction between the Modular-SECA and the finger: This interaction is affected by differences in the force used to attach the soft actuator to the finger and the type of fixture. The assumption in Equation (2.5) is that the bending angles of the Modular-SECA and the finger joint are the same (Heung et al., Reference Heung, Tong, Lau and Li2019, (Reference Heung, Tang, Shi, Tong and Li2020); thus, if those differences cause non-negligible differences in the bending angles of the soft actuator and the finger joint, the estimation accuracy will be low.
-
• Interaction between the Modular-SECAs: The Modular-SECA is affected by the deformation due to the expansion of actuators in the other joints. Therefore, the Modular-SECA’s bending performance during stiffness estimation differs from that of the free bending. As shown in Equation (2.5), the model is the energy conservation equation for the free bending of the Modular-SECA plus the energy stored in the joints; thus, this assumes that the Modular-SECA’s bending performance is the same during stiffness estimation and free bending. However, when the bending performance changes, the behavior of the Modular-SECA during stiffness estimation is no longer accurately represented by Equation (2.5), resulting in low estimation accuracy.
We know that the stiffness estimation accuracy is reduced mainly due to the influence of these two factors, which results in the energy conservation in Equation (2.5) no longer being valid. However, since the two influences are mixed, it is difficult to partition and express the influence of each factor in terms of the extent to which each factor affects each measurement.
In
$ \theta <{\theta}_0 $
, each term in Equation (2.4) is divided into two types of energy:
$ {W}_A $
and
$ {W}_L $
, which are the energies required for the Modular-SECA and finger to bend to
$ \theta $
(bending energy:
$ {\mathrm{E}}_{\mathrm{bend}} $
).
$ P\Delta V $
and
$ {W}_{joint} $
are the energies stored by the Modular-SECA and the finger to reach
$ \theta $
(stored energy:
$ {\mathrm{E}}_{\mathrm{stored}} $
). The finger is held at the angle where these two types of energy are balanced. If the stiffness estimation results are low, there is a possibility that
$ {\mathrm{E}}_{\mathrm{bend}} $
and
$ {\mathrm{E}}_{\mathrm{stored}} $
are not balanced due to the inflow and outflow of energy. Therefore, comparing the stiffness estimation results with the values of these energies may reveal the cause of the errors in the stiffness estimation results. Therefore,
$ k $
,
$ {W}_A $
,
$ {W}_L $
,
$ P\Delta V $
, and
$ {W}_{joint} $
were calculated from the set of joint angles (
$ {\theta}_{exp} $
) and input air pressure values (
$ {P}_{exp} $
) measured in one REP of the stiffness measurement, respectively. Figure 2 illustrates
$ \Delta k $
,
$ {W}_A+0.5{W}_L $
, and
$ 1.15P\Delta V+{W}_{joint} $
, separately.
$ \Delta k $
is the difference between joint stiffness estimates (
$ {k}_{analytic} $
) and actual target values (
$ {k}_{target} $
), calculated by the following:
$$ \Delta k={k}_{analytic}-{k}_{target}. $$
From Figure 2, in the cases where the stiffness estimation results are not accurate (i.e.,
$ \Delta k\ne 0 $
), the bending energy (
$ {\mathrm{E}}_{\mathrm{bend}} $
) and the stored energy (
$ {\mathrm{E}}_{\mathrm{stored}} $
) calculated from the experimental joint angle (
$ {\theta}_{exp} $
) are not balanced. This imbalance indicates that the energy conservation in Equation (2.4) is broken. In the range
$ \theta >0.7{\theta}_0 $
, even a small difference between
$ {\mathrm{E}}_{\mathrm{bend}} $
and
$ {\mathrm{E}}_{\mathrm{stored}} $
results in a large error in the stiffness estimation (Figure 2c). The relationship between
$ \Delta k $
and two energies (
$ {\mathrm{E}}_{\mathrm{bend}} $
and
$ {\mathrm{E}}_{\mathrm{stored}} $
) can be divided into two types:
-
(i)
$ {W}_A+0.5{W}_L>1.15P\Delta V+{W}_{joint}:{k}_{analytic}>{k}_{target} $
-
(ii)
$ {W}_A+0.5{W}_L<1.15P\Delta V+{W}_{joint}:{k}_{analytic}<{k}_{target} $
Figure 2. Relationship between stiffness estimation error and energy. (a) Data for the MCP joint with good stiffness estimation result. (b) Data for the PIP joint estimated to be lower than the actual stiffness value. (c) Data for the DIP joint estimated to be higher than the actual stiffness value.
In the case of type (i), the finger is at rest even though the
$ {\mathrm{E}}_{\mathrm{bend}} $
is larger in the analysis results. This discrepancy could be because the actual
$ {\mathrm{E}}_{\mathrm{bend}} $
was less than the analysis values (i.e., the values in Figure 2), or there could have been additional energy in the
$ {\mathrm{E}}_{\mathrm{stored}} $
due to the fixation conditions of the actuator and the Modular-SECA of other joints. On the other hand, in the case of type (ii), the finger is at rest even though
$ {\mathrm{E}}_{\mathrm{bend}} $
is smaller in the analysis results. This discrepancy could mean that the actual
$ {\mathrm{E}}_{\mathrm{bend}} $
required more than the analysis values. Alternatively, the energy from the input air pressure
$ P\Delta V $
could have been used in others due to several possibilities. For example, the PIP joint is affected by the two Modular-SECAs because the Modular-SECAs of the DIP and MCP joints are adjacent on both sides. The energy from the input air pressure of the PIP’s Modular-SECA may be used to counteract the influence of the adjacent Modular-SECAs and the bending of the PIP joint. Thus, in the stiffness estimation with the Modular-SECAs, although the
$ {\mathrm{E}}_{\mathrm{bend}} $
and
$ {\mathrm{E}}_{\mathrm{stored}} $
in the analysis were not balanced, the Modular-SECA and finger are at rest. This is because the energy inflow and/or outflow changes the energy relationship in Equation (2.4), and the energy conservation of the Modular-SECA and finger complex actually is maintained.
Therefore, if the energy balance is corrected so that the analytical values of
$ {\mathrm{E}}_{\mathrm{bend}} $
and
$ {\mathrm{E}}_{\mathrm{stored}} $
are balanced, the stiffness estimation results can be improved. The energy stored in the joint
$ {W}_{joint} $
is invariant regardless of the actuator type and the interaction effect; thus, adjusting the balance between
$ {\mathrm{E}}_{\mathrm{bend}} $
and
$ {\mathrm{E}}_{\mathrm{stored}} $
is equivalent to correcting the energy balance in the Modular-SECA. Therefore, the energy balance is adjusted by adding the correction parameters
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
to Equation (2.4) as follows:
$$ {\beta}_{stored}\left(1.15P\Delta V\right)={\alpha}_{bend}\left({W}_A+0.5{W}_L\right)-{W}_{joint},\hskip1em 0\le \theta <{\theta}_0. $$
2.2.2. Identification of correction parameters to satisfy the energy conservation
The flow for identifying the correction parameters
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
in Equation (2.8) from the measurement data of the stiffness experiment is shown in Figure 3(a). The modular-analytical model is corrected for each REP. In addition, only the depressurization data, carefully selected for their precision, are used for the correction. Figure 3(b) shows the angle change of three REPs in one measurement. In the first REP, the angle change trend during pressurization often differed from that during the second and third REPs’ pressurization. This difference is thought to be due to the fact that the effect of the force applied when the soft actuator is attached is largest at the first 0 kPa, after which the effect is absorbed, and the change is similar during the remaining process. Consequently, when the data during pressurization was used to identify the correction parameters, the correction parameters for the first REP differed from those of the other REPs. Therefore, only depressurization data were used for parameter identification because the subsequent estimation of correction parameters by an ANN did not work.
Figure 3. (a) The flow of identifying correction parameters for the modular-analytical model to satisfy the energy conservation. (b) Differences in angle values for the three REPs included in one measurement. Especially in the PIP joint, differences in the trend during pressurization between the first REP and the second/third REP are often observed. (c) Comparison of the relationship between the energy in the modular-analytical model and between the air pressure experimental values (
$ {P}_{exp} $
) and the air pressure analytical values (
$ {P}_{analytic} $
) in the stiffness estimation results (good, low, and high estimates). It is noteworthy that only depressurization data is shown.
The depressurization data included nine joint angles measured at air pressure values ranging from 80 to 0 kPa. Next, among this depressurization data, those with
$ \theta \ge {\theta}_0 $
are removed; when
$ \theta \ge {\theta}_0 $
, the modular-analytical model of stiffness estimation is not Equation (2.5). Therefore, the energy balance differs from
$ \theta <{\theta}_0 $
, and the correction parameters may differ. In this study, we wanted to improve the stiffness estimation accuracy at
$ \theta <{\theta}_0 $
, so we did not use data for
$ \theta \ge {\theta}_0 $
, which includes the possibility that the parameter identification results may change. After filtering the angles of the depressurization data by
$ {\theta}_0 $
, the remaining angle data are used to calculate the analytical value of air pressure,
$ {P}_{analytic} $
, from Equation (2.9).
$$ {P}_{analytic}=\frac{2{\alpha}_{bend}\left({W}_A+0.5{W}_L\right)-k{\left(\theta -{\theta}_0\right)}^2}{2{\beta}_{stored}\cdot 1.15\Delta V},\hskip1em 0\le \theta <{\theta}_0. $$
Figure 3(c) compares the air pressure values used in the experiments (
$ {P}_{exp} $
) with the analytical values (
$ {P}_{analytic} $
at
$ {\alpha}_{bend}=1 $
and
$ {\beta}_{stored}=1 $
) at the same measured joint angle values (
$ {\theta}_{exp} $
). The relationship between
$ {P}_{exp} $
and
$ {P}_{analytic} $
is similar to that of
$ {\mathrm{E}}_{\mathrm{bend}} $
and
$ {\mathrm{E}}_{\mathrm{stored}} $
, which may represent the energy imbalance relationship in Equation (2.4). Therefore,
$ {P}_{exp} $
and
$ {P}_{analytic} $
were compared, and the correction parameters (
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
) were identified by the least-squares method.
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
were searched in the range of 0–10 real numbers. This range was empirically determined to limit the parameter values and improve the accuracy of subsequent parameter estimation. It was set in consideration of the balance between the ease of parameter estimation and the fitting accuracy of the identified parameters.
This method was used for all REPs in the stiffness experiment data to identify
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
such that the modular-analytical model accurately represented the measured data for each REP. The effect of adding correction parameters is evaluated by comparing the stiffness values estimated from the modular-analytical model before and after correction. The mean absolute percentage error (MAPE) was used as the evaluation indicator, and the acceptable error was set at 20%. The identified parameters are then substituted into Equation (2.10) to estimate the joint stiffness values from the measured joint angles.
$$ k=\frac{2\left({\alpha}_{bend}\left({W}_A+0.5{W}_L\right)-{\beta}_{stored}\cdot 1.15P\Delta V\right)}{{\left(\theta -{\theta}_0\right)}^2},\hskip1em 0\le \theta <{\theta}_0. $$
Finally, the corrected stiffness values are averaged in Equation (2.6) to obtain the final stiffness value
$ {k}_{corr} $
. The corrected modular-analytical model is denoted as the corrected modular-analytical model. It is noteworthy that unlike Equation (2.5), the stiffness estimation range is
$ 0\le \theta <{\theta}_0 $
. This is because we expected that the correction parameters would correct the modular-analytical model for
$ \theta <{\theta}_0 $
, so that stiffness estimation would be possible even for the range
$ 0.7{\theta}_0<\theta <{\theta}_0 $
.
2.2.3. Prediction of correction parameters using ANN
Suppose the correction parameters
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
identified in Subsection 2.2.2 can be estimated from the measurement data. In that case, accurate stiffness estimation can be performed by online tuning the modular-analytical model based on the measurement data.
Our previous study estimated stiffness values from 17 joint angle values obtained in 1 REP using the ANN (Matsunaga et al., Reference Matsunaga, Oba, Ke, Hsueh, Huang, Gomez-Tames and Yu2024). The relationship between joint angles and stiffness value is nonlinear and complex. The ANN was suitable as a machine learning algorithm for stiffness estimation because it is suitable for learning such a relationship. In this study, the values to be estimated are not stiffness values but correction parameters of the modular-analytical model. As with estimating the stiffness values, the relationship between the measurement data and the correction parameters is considered complex because it is affected by multiple interactions. Therefore, we also employed an ANN to predict the correction parameters.
However, just giving multiple joint angle values as input, as in stiffness value estimation, may not be enough information to predict correction parameters. For example, in the case of Figure 3(c), we can know the difference between the analytical values relative to the measured values and how to correct the modular-analytical model. On the other hand, we do not know how to correct the model without analytical values. The measured joint angles may be consistent with the analytical values or completely different. We cannot know these from the measured joint angle values alone.
This study estimates correction parameters using the stiffness values derived by substituting the measured angles into Equation (2.5). However, the stiffness estimation range is not
$ 0\le \theta \le 0.7{\theta}_0 $
but
$ 0\le \theta <{\theta}_0 $
. These stiffness values contain information about the modular-analytical model and the positive and negative difference between the measured and analytical values. Figure 4(a) shows that the sign of the difference in the stiffness values from 0 roughly reflects the difference between the two values of
$ {P}_{exp} $
and
$ {P}_{analytic} $
, even if the sign may be reversed when the stiffness value is close to 0. Therefore, we expected that the values of
$ {k}_{analytic} $
have more suitable information for estimating correction parameters than joint angles. However, as shown in Figure 2(c), the stiffness error becomes very large if the measured joint angle value differs from the analytical value even slightly in
$ 0.7{\theta}_0<\theta <{\theta}_0 $
. As a result, when the stiffness values calculated in Equation (2.5) were used as input values as they were, the range of stiffness values as input variables was too wide, and learning did not work well even if the input data were standardized or normalized. Therefore, we restrict the stiffness values as in the following:
$$ {k}^{\prime }=\mathit{\operatorname{sgn}}(k)\cdot \log \left(|k|+1\right). $$
Figure 4. (a) Relationship between the sign of the stiffness values calculated from the modular-analytical model and the difference between the measured and analytical values of air pressure. (b) The change in value by transforming the stiffness values obtained from the modular-analytical model; three stiffness results are shown as examples. (c) The flow of parameters estimation to correct the modular-analytical model using the ANN.
Taking the logarithm of the stiffness value suppresses the effect of excessive errors in
$ 0.7{\theta}_0<\theta <{\theta}_0 $
. The sign of the stiffness value is also essential information because it indicates the sign of the difference between the measured and analytical values. Therefore, by adding a signal function, the sign of the original stiffness value is left intact.
$ k $
and
$ {k}^{\prime } $
are shown in Figure 4(b). By transforming to
$ {k}^{\prime } $
, the effect of estimates that are too large is suppressed.
The flow of correction parameters estimation with the ANN is shown in Figure 4(c). As in Figure 3(a); first, the depressurization data are filtered by
$ {\theta}_0 $
. Although the identification of correction parameters can be performed even if the number of input values varies, the ANN needs to unify the number of input variables. Therefore, the angles removed by filtering were supplemented with the average of the data for
$ \theta <{\theta}_0 $
so that the number of input variables for the ANN was equalized to 9. Next, after substituting the measured joint angles and air pressure values into Equation (2.5) to calculate the stiffness values, they are transformed into
$ {k}^{\prime } $
by Equation (2.11). The nine stiffness values are input to the ANN, and the correction parameters
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
are estimated as output. The correction parameters are then substituted into Equation (2.10) to estimate the joint stiffness values from the measured joint angles. Finally, the corrected stiffness values are averaged in Equation (2.6) to obtain the final stiffness value
$ {k}_{corr} $
.
The ANN used was the multilayer perceptron (MLP), which contains three hidden layers, each with nine neurons in the node. After each hidden layer, there is a Rectified Linear Unit (ReLU) as an activation function and a dropout layer with a dropout rate of 0.1. Moreover, since the identification range of the correction parameters is 0–10, the two output values of the MLP were also clamped to the same range to limit the estimation range. The mean squared error (MSE) was used as the loss function, and the average of the two MSEs of
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
was trained as the loss. Adam is used as the optimization function, and the ANN was trained 2,000 epochs with a learning rate of 1e−4 and a weight decay of 1e−4. These parameters were determined by using a threefold cross-validation to balance overfitting and underfitting. All models were developed with Python 3.9.12 64 bits. PyTorch 2.1.2 + cu118 was used to build the neural networks.
In our previous study, stiffness values were estimated directly from joint angle sequences, making the trend of angle change an important feature. Since the interaction effects differ by joint (DIP, PIP, and MCP), separate models were constructed for each joint. In contrast, this study aims to estimate correction parameters that align analytical values with measured ones. Here, the key feature is the difference between the measured and analytical values, which is similar across joints. Therefore, a single model was constructed for all three joints.
We used the M-Training data to train the ANN, with standardized input features. The test data included M-Test, S-Prediction, L-Prediction, and H-Prediction datasets. Only depressurization data were used for both training and evaluation, ensuring consistency and reliability. The same condition was applied to the analytical model used for comparison. We also compared our method with the ANN-only model proposed in our previous work (Matsunaga et al., Reference Matsunaga, Oba, Ke, Hsueh, Huang, Gomez-Tames and Yu2024), which directly estimates stiffness values from joint angles. It is noteworthy that the ANN-only model was trained using both pressurization and depressurization data.
Evaluation metrics included the root mean squared percentage error (RMSPE) and mean absolute percentage deviation (MAPD), as defined in our previous study. The RMSPE was also used to evaluate correction parameter accuracy.
2.3. Subject stiffness estimation experiment
To verify the performance of the proposed method in the stiffness estimation of human finger joints, we conducted stiffness estimation experiments on healthy subjects.
2.3.1. Subjects
Four subjects participated in this experiment (Table 1).
Table 1. Subjects information
2.3.2. Experimental protocol
In this study, we measured the reference stiffness values of the MCP joint using the baseline method (Figure 5(a)) and estimated the stiffness using the proposed method with the Modular-SECA (Figure 5(b)).
Figure 5. (a) MCP joint stiffness measurement device for index finger. (b) Stiffness estimation with the Modular-SECA.
2.3.2.1. Subject’s posture
For all measurements, the subject’s elbow was bent at 90°, the wrist was fixed in the neutral position (no flexion or extension), and the forearm was held in the neutral position (no supination or pronation) by the arm holder on the armrest. Furthermore, the subject sat parallel to the desk with the armrest, and the chair’s height was adjusted so that the subject could relax during the measurement.
2.3.2.2. Reference value measurement by a baseline method
A standard MCP joint stiffness measurement device was reproduced for a baseline method, and the stiffness values measured with this device were used as the reference values of the subject experiment (Figure 5(a)). This standard stiffness measurement device measures the joint angle and torque of the index finger MCP joint and has proven effective in quantifying the stiffness of the MCP joint. A brief description of the baseline method is as follows. An overview of this device and details on the stiffness measurement theory can be found in Kuo and Deshpande (Reference Kuo and Deshpande2012) and Shi et al. (Reference Shi, Heung, Tang, Tong and Li2020).
The index finger of the subject’s hand was fixed to the reference value measurement device so that it could freely rotate in the horizontal plane. The other fingers were kept in a relaxed position while grasping the hand holder. Using a servo motor (RDS5160, Torque 65 kg.cm, DSservo Inc., China), the MCP joint of the index finger was extended at a rotational speed of 2°/s from 90° to 0° and held every 10° for 30 s. The MCP joint angle was then measured by a camera (C930eR, Logitech, Lausanne, Switzerland)-based two-dimensional marker detection system for 5 s. Simultaneously, a load cell (USL06-H5 Load cell, max: 100 N, Tec Gihan, Kyoto, Japan) was used to measure the passive force at the fingertip end. After reaching 0°, the joint was flexed at 2°/s to 90° and repeated three times.
By the following procedure, the measured data were analyzed. First, the torque was calculated by multiplying the measured force by the distance from the load cell to the center of the MCP joint. Next, using all the MCP joint angle and torque pair data measured during the three repetitions, the following classic double exponential function-based model parameters were identified by a nonlinear least-squares method.
$$ \tau =A\left({e}^{-B\left(\theta -E\right)}-1\right)-C\left({e}^{D\left(\theta -F\right)}-1\right) $$
where
$ \tau $
and
$ \theta $
are the torque and MCP joint angle, respectively, and
$ A $
to
$ F $
are the parameters of the double exponential function-based model describing the relationship between the passive elastic moment and the MCP joint. Furthermore, the MCP joint stiffness
$ {k}_{MCP} $
was calculated by substituting the identified parameters into the derivative of Equation (2.12) expressed as follows:
$$ {k}_{MCP}=\left|-{ABe}^{-B\left(\theta -E\right)}-{CDe}^{D\left(\theta -F\right)}\right| $$
For the joint angle
$ \theta $
of stiffness calculation, six angles were selected at regular intervals in the range from the MCP joint angle when the Modular-SECA was attached to the finger in an unpressurized state (
$ {\theta}_{initial} $
) to the resting angle (
$ {\theta}_0 $
). Then, the stiffness at each angle was calculated. Finally, the average of these six stiffness values was taken as the reference value of MCP joint stiffness.
2.3.2.3. Stiffness estimation with the Modular-SECA
After measuring the reference values using the baseline method, the Modular-SECAs were attached to three joints (DIP, PIP, and MCP) of the subject’s index finger to estimate the stiffness using the proposed method (Figure 5(b)). In this study, we verified the accuracy of the proposed method in estimating stiffness at the MCP joint, considering that the baseline method device is dedicated to measuring the MCP joint stiffness of the index finger.
In the proposed method, we measured according to the flow shown in Figure 1(a). However, unlike the prototype experiment using dummy fingers, the air pressure steps were set at 20 kPa intervals in the subject experiment. This step change shortened the measurement time and reduced the variation in stiffness values due to measurement fatigue. The model was restructured because the pressure step change varied the input variables to nine for the ANN-only model and to five for the correction parameters predicting model. The reduction in input variables slightly decreased the models’ prediction accuracy for the prototype experiment data. In addition, the air pressure was gradually changed at 2 kPa/s. After the measurements, the MCP joint stiffness
$ {k}_{uncorr}\left(0.7{\theta}_0\right) $
,
$ {k}_{ANN} $
, and
$ {k}_{corr}\left(0.7{\theta}_0\right) $
were estimated according to Figure 1(b), Matsunaga et al. (Reference Matsunaga, Oba, Ke, Hsueh, Huang, Gomez-Tames and Yu2024), and Figure 4(c).
3. Results
3.1. Identified correction parameters of the modular-analytical model
Figure 6 shows the results of identifying the correction parameters using the least-squares method based on the measured and analytical air pressure values. The MAPE of the stiffness values estimated by the modular-analytical model and the corrected-modular-analytical model are also shown in Table 2. In addition, Figure 7 illustrates the relationship between each energy in the corrected-modular-analytical model based on the three stiffness estimation data in Figure 2. From Figure 6, when the stiffness estimation range in Equation (2.10) was set to
$ 0\le \theta \le 0.7{\theta}_0 $
, the stiffness estimates were almost consistent with the target stiffness values. The MAPEs were also within acceptable limits, <20% for all datasets (Table 2). Figure 7 shows that adding the correction parameters into the modular-analytical model improved the stiffness estimation results by bringing
$ {\mathrm{E}}_{\mathrm{bend}} $
(bending energy) and
$ {\mathrm{E}}_{\mathrm{stored}} $
(stored energy) into balance. On the other hand, for the stiffness estimation range that remains in Equation (2.10), some stiffness values calculated from the corrected modular-analytical model remained in error with the target stiffness values (e.g., the DIP for M-Training and the MCP joints for L-Prediction). This was especially true for REPs of depressurization data that included joint angles close to
$ {\theta}_0 $
, as shown in Figure 7(c).
Figure 6. The identification results of the parameters that correct the modular-analytical model, and comparison of stiffness values calculated from the modular-analytical models before and after adding correction parameters. All data are plotted.
$ {k}_{uncorr}\left(0.7{\theta}_0\right) $
,
$ {k}_{corr}\left(0.7{\theta}_0\right) $
, and
$ {k}_{corr}\left({\theta}_0\right) $
are the stiffness values calculated from the modular-analytical model in
$ 0\le \theta \le 0.7{\theta}_0 $
, the corrected modular-analytical model in
$ 0\le \theta \le 0.7{\theta}_0 $
, and the corrected modular-analytical model in
$ 0\le \theta <{\theta}_0 $
, respectively. Only the H-Prediction figure has the vertical axis of the stiffness value plot on a logarithmic scale. In the figures on the right, the points on the black line indicate that the target stiffness values and the estimated stiffness values coincide, and the stiffness estimation error rate is 0%. The thin black areas on either side of the line indicate the range within 20% of the acceptable error rate.
Table 2. Accuracy of stiffness estimates calculated from the modular-analytical models before and after adding correction parameters
Note. Bold type indicates the estimation type with the highest estimation accuracy.
a The stiffness values calculated from the modular-analytical model in
$ 0\le \theta \le 0.7{\theta}_0 $
.
b The stiffness values calculated from the corrected modular-analytical model in
$ 0\le \theta \le 0.7{\theta}_0 $
.
c The stiffness values calculated from the corrected modular-analytical model in
$ 0\le \theta \le {\theta}_0 $
.
Figure 7. Relationship between stiffness estimation error and energy after the modular-analytical model correction.
$ {k}_{target} $
,
$ {k}_{uncorr} $
, and
$ {k}_{corr} $
are the target stiffness, the stiffness values calculated from the modular-analytical model, and the corrected-modular-analytical model, respectively. (a) Data for the MCP joint with good stiffness estimation result before correction. (b) Data for the PIP joint estimated to be lower than the actual stiffness value before correction. (c) Data for the DIP joint estimated to be higher than the actual stiffness value before correction.
Although the two correction parameters differ slightly from joint to joint, the relationship is generally linear for all joints. Figure 6 shows that the correction parameter values are higher in the PIP joint, where the stiffness values were estimated to be smaller overall than in the other joints. In addition, the values of
$ {\alpha}_{bend} $
are higher than those of
$ {\beta}_{stored} $
. In the data where the stiffness value was estimated higher than the target value, the correction parameter tends to be <1. In contrast, in the data where the stiffness value was estimated as lower than the target value, the correction parameter tends to be higher than 1. Most of the data for both
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
is concentrated in the range of 0–2.
3.2. Estimation of correction parameters with the ANN
Figure 8 shows the results of estimating the correction parameters of the modular-analytical model from depressurization data using the ANN and the stiffness estimation results using the estimated parameters. Tables 3 and 4 show the correction parameters estimation accuracy and the stiffness estimation accuracy. Also, the results of the previously proposed ANN-only model are included for comparison. From Table 3, the estimation accuracies of
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
are almost the same, except for the MCP joints in the L-Prediction and all joints in the H-Prediction. Since there is a strong positive correlation between the actual
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
values (Figure 6), the predicted
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
change in tandem and monotonically increase with actual
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
in such cases. On the other hand, the RMSPEs of the two parameters are different for the MCP joints in the L-Prediction and all joints in the H-Prediction. In such cases, the predicted changes in the values of
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
are not linked, or the predicted values do not monotonically increase with the actual values. These data did not have good stiffness estimation results either (
$ {k}_{corr}\left(0.7{\theta}_0\right) $
and
$ {k}_{corr}\left({\theta}_0\right) $
in Table 4). It can also be seen that values above
$ {\alpha}_{bend}=5 $
were not output.
Figure 8. The correction parameters estimation results using the ANN, and comparison of stiffness values estimated by the modular-analytical models before and after adding correction parameters and the ANN-only model.
$ {k}_{uncorr}\left(0.7{\theta}_0\right) $
,
$ {k}_{corr}\left(0.7{\theta}_0\right) $
,
$ {k}_{corr}\left({\theta}_0\right) $
, and
$ {k}_{ANN} $
are the stiffness values calculated from the modular-analytical model in
$ 0\le \theta \le 0.7{\theta}_0 $
, the corrected modular-analytical model in
$ 0\le \theta \le 0.7{\theta}_0 $
, the corrected modular-analytical model in
$ 0\le \theta <{\theta}_0 $
, and the ANN-only model, respectively. Only the H-Prediction figure has the vertical axis of the stiffness value plot on a logarithmic scale. In the figures on the right, the points on the black line indicate that the target stiffness values and the estimated stiffness values coincide, and the stiffness estimation error rate is 0%. The thin black areas on either side of the line indicate the range within 20% of the acceptable error rate. (a) The M-Test has one estimate for each target stiffness value. (b) The S-Prediction and (c) the L-Prediction have nine estimates for each target stiffness value, and their mean and standard deviation are shown. (d) The H-Prediction has nine estimates for a target stiffness value and shows all nine estimates.
Table 3. Correction parameters estimation accuracy using the ANN
Table 4. Stiffness estimation accuracy
Note. Bold type indicates the estimation type with the highest estimation accuracy.
a The stiffness values estimated from the ANN-only model in
$ 0\le \theta <{\theta}_0 $
.
b The stiffness values calculated from the modular-analytical model in
$ 0\le \theta <0.7{\theta}_0 $
.
c The stiffness values calculated from the corrected modular-analytical model in
$ 0\le \theta <0.7{\theta}_0 $
.
d The stiffness values calculated from the corrected modular-analytical model in
$ 0\le \theta <{\theta}_0 $
.
In the stiffness estimation results, many data show that the corrected modular-analytical model’s estimates are improved compared to the original modular-analytical model (Table 4). However, when the stiffness estimation range was set to
$ 0\le \theta <{\theta}_0 $
, there were cases where the results were lower than the modular-analytical model’s results. Compared to the ANN-only model, the results did not improve as much as the ANN-only model, with many cases not reaching the accuracy of the ANN-only model in both the RMSPE and MAPD.
3.3. Human joint stiffness estimation performance
The results of estimating the subject’s MCP joint stiffness are shown in Table 5 and Figure 9. Since the reference stiffness values are generally consistent with the stiffness ranges of healthy subjects in a previous study (Shi et al., Reference Shi, Heung, Tang, Tong and Li2020), it is reasonable to state that the baseline method device could accurately measure the actual stiffness.
Table 5. Subjects’ joint stiffness estimation results
a Mean (± mean absolute deviation) of six stiffness values (
$ {k}_{MCP} $
).
b Mean (± mean absolute deviation) of the three stiffness values estimated from each of the three REPs.
c The stiffness values estimated from the ANN-only model in
$ 0\le \theta <{\theta}_0 $
.
d The stiffness values calculated from the modular-analytical model in
$ 0\le \theta <0.7{\theta}_0 $
.
e The stiffness values calculated from the corrected modular-analytical model in
$ 0\le \theta <0.7{\theta}_0 $
.
Figure 9. (a) Subject’s joint stiffness reference values and results estimated by each model. (b) The reference values and predicted results of the modular-analytical model’s correction parameters. REP is simplified as R.
Table 5 and Figure 9(a) show that the corrected modular-analytical model has a smaller estimation error than the modular-analytical model. In particular, from Figure 9(b), the accuracy of the modular-analytical model was greatly improved by adding the correction parameters in H2, where the correction parameter
$ {\alpha}_{bend} $
was around 1. On the other hand, H1, H3, and H4, where the actual value of the correction parameter
$ {\alpha}_{bend} $
was near 0, resulted in more significant estimation errors compared to the ANN-only model.
4. Discussion
4.1. Change in the modular-analytical model performance with additional correction parameters
Our analysis of the performance of the modular-analytical model with additional correction parameters showed that when
$ {\mathrm{E}}_{\mathrm{bend}} $
$ < $
$ {\mathrm{E}}_{\mathrm{stored}} $
over the entire depressurization data range of REP (Figure 2(b)),
$ {\alpha}_{bend}>{\beta}_{stored} $
. Also, when the energy is switched between large and small values during the depressurization data range (Figure 2(a) and (c)), the relationship between
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
was confirmed to be variable. Some H-Prediction data showed that
$ {\mathrm{E}}_{\mathrm{bend}} $
$ > $
$ {\mathrm{E}}_{\mathrm{stored}} $
over the entire depressurization range, and in such cases,
$ {\alpha}_{bend}<{\beta}_{stored} $
(e.g., the MCP joint in Figure 6(e)). Adding the correction parameters improves the stiffness estimation accuracy of the modular-analytical model (Figure 6) and confirms the effect of adjusting the imbalance between
$ {\mathrm{E}}_{\mathrm{bend}} $
and
$ {\mathrm{E}}_{\mathrm{stored}} $
. These results indicate that adjusting the energy balance of the modular-analytical model based on the measurement data is effective in improving the analytical model-based stiffness estimation accuracy. The linear relationship between the two correction parameters is a natural consequence of the modular-analytical model being derived based on energy conservation in the Modular-SECA.
We hypothesized that correcting the modular-analytical model would reduce the influence near the singularity (
$ \theta ={\theta}_0 $
) in the range
$ 0.7{\theta}_0<\theta <{\theta}_0 $
, thus extending the range of stiffness estimation. However, at
$ 0.7{\theta}_0<\theta <{\theta}_0 $
in Figure 7(c), the difference from the target stiffness value is still large, although the data near
$ {\theta}_0 $
are improved from the original estimate (
$ {k}_{uncorr} $
in Figure 7(c)). Due to this, even after correction, these data should not be included in the average calculation (2.6) to calculate the final stiffness values. On the other hand, the data close to
$ 0.7{\theta}_0 $
showed considerable improvement in the estimated values due to the correction. For example, in Figure 6(e), good estimation results (error rate
$ < $
10%) were obtained with the corrected modular-analytical model for the REP that did not include angles close to
$ {\theta}_0 $
even for the H-Prediction data for which stiffness values cannot be calculated in
$ 0\le \theta \le 0.7{\theta}_0 $
. Therefore, it was shown that extending the stiffness estimation range beyond
$ 0.7{\theta}_0 $
is possible. As a result, the range of stiffness that the modular-analytical model could not estimate can be estimated by the correction.
4.2. Estimation performance of correction parameters using the ANN
The correction parameters estimation with the ANN performed well only in specific ranges, biasing the estimation results (Figure 8 and Table 3). As a result, the stiffness estimation of the corrected modular-analytical model, although improved over the modular-analytical model, was not as good as the ANN-only model. Since the ANN-only model was constructed for each joint, it could learn the features of each joint. This ability to learn joint-specific features may have a higher stiffness estimation accuracy than the proposed model, which used one ANN model to estimate correction parameters for three joints. Additionally, the proposed method uses only depressurization data, whereas the ANN-only model uses both pressurization and depressurization data. Thus, fewer input variables in the proposed method may also have contributed to its lower accuracy. Another possible reason for this could be the bias in the training data. As can be seen from Figures 6 and 8, the model did not predict well for values outside the range of correction parameters present in the M-Training data. For example, the parameter values for the MCP joint in S-Prediction (Figure 6(c)) contain many values in the range that do not exist in the data for any of the joints (DIP, PIP, and MCP) used in M-Training (Figure 6(a)). Therefore, the MCP joint in S-Prediction (Figure 8(b)) can not estimate parameters in ranges that do not include the M-Training data.
This observation suggests that the training data did not cover the full range of parameters, which may have reduced the prediction performance of the ANN. Since the stiffness estimates by the modular-analytical model incorporating physical information were used as input variables for the ANN, generalization could be achieved to some extent for a range of data not present in the training data. Nevertheless, the reason for the lack of improvement in generalization is the imbalance in the training data. It has been shown that when the training data distribution is not uniform, the model’s performance is inhibited (Gavas et al., Reference Gavas, Das, Ghosh and Pal2023). When trained on imbalanced data, the model preferentially learns from areas where data are concentrated and underestimates in other areas, thereby reducing prediction accuracy (Ghosh et al., Reference Ghosh, Bellinger, Corizzo, Branco, Krawczyk and Japkowicz2022; Scheepens et al., Reference Scheepens, Schicker, Hlaváčková-Schindler and Plant2023). Figure 6(a) and (b) show that the values of the correction parameters for the M-Test are within the range of the M-Training data. However, the PIP joint in the M-Test do not predict values of
$ {\alpha}_{bend}\ge 5 $
(Figure 8(a)). Table 6 presents the percentage of data for each value of the correction parameter
$ {\alpha}_{bend} $
in the M-Training data. Data within the range of
$ {\alpha}_{bend} $
from 0.5 to 2 accounts for more than half of the total data, suggesting that this range is the center of the learning process. On the other hand, there is little data near the upper and lower limits of
$ {\alpha}_{bend} $
in the training data, which may have led to the low prediction performance of this range (Table 6 and Figure 8). Therefore, even if the range of
$ {\alpha}_{bend} $
is increased in the S-Prediction and L-Prediction, the prediction accuracy is likely to have been lower (i.e., generalization performance was lower) in the areas beyond the range of the training data. Measures, such as data augmentation and weighting of the loss function, are needed to improve this.
Table 6. Distribution of data across the correction parameter
$ {\alpha}_{bend} $
ranges
In Figure 10, all data of correction parameters and estimated stiffness values are shown for the PIP joint of the S-Prediction in Figure 8(b) and the MCP joint of the L-Prediction in Figure 8(c). In the S-Prediction’s PIP joint, the stiffness estimation results are not so bad (Figure 8(b)), even though the correction parameter
$ {\alpha}_{bend} $
is not estimated to be more than 2. Also, for the L-Prediction’s MCP joint, there is a twofold difference in the RMSPE of
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
(Table 3), and the error is more significant for stiffness estimates with target stiffness values higher than 1 (Figure 8(c)). From Figure 10, it can be seen that when the target stiffness value is small, the error in the correction parameter has a small effect on the stiffness estimation. In contrast, when the target stiffness value is large, a small error in the correction parameter has a large effect on the stiffness estimation. In the S-Prediction’s PIP joint,
$ {\alpha}_{bend} $
was higher than 2 in the data of low stiffness values, so even if the correction parameter estimation error was large, it did not have much effect on the stiffness estimation results. In the L-Prediction’s MCP joint, the predicted values of
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
did not vary much, even though the actual parameter values varied. As a result, there was a difference in the estimation accuracy of these two parameters. In addition, the MCP joint of the L-Prediction has many data where the actual value of the correction parameter
$ {\alpha}_{bend} $
is <0.5. As shown in Table 6, the small data below 0.5 in the training data may have contributed to lower parameter estimation accuracy in this range. The cause of this issue is considered to be the same as that of the low accuracy in the H-Prediction’s
$ {k}_{corr} $
estimation (Figure 8(d) and Table 4).
Figure 10. Relationship between the correction parameters using the ANN and stiffness estimates by the corrected modular-analytical model.
Overall, when the target stiffness value is small, errors in the correction parameters do not significantly affect stiffness estimation. However, the effect becomes more pronounced when the target stiffness value is large. Future improvements are required to consider the impact of the bias in the dataset on the correction parameters estimation accuracy.
4.3. Stiffness estimation performance of human finger joint
In all three subjects, except H2, the prediction accuracy of the correction parameters was low, and accurate stiffness estimation was not possible (Figure 9). As shown in Figure 9(b), the correction parameters predicted by the proposed method for all subjects, except H2, took values near 1. However, if one wants the proposed method to output the reference value, the correction parameters should take values near 0. From the results of the prototype experiment, the percentage of data in the training data of the ANN to predict the correction parameter that takes a value <0.5 is relatively small, which is <10% (Table 6). In other words, the prediction performance is low when the correction parameter takes values <0.5 due to bias in the training data. As a result, the effect of correcting the analytical model by adding the correction parameters (change in estimated stiffness) was limited (Table 5 and Figure 9(a)). In contrast, the ANN-only model directly predicts stiffness values, and the range of target stiffness values in the training data includes the reference values from the subject experiments. Therefore, the training data obtained from the prototype experiments were also more easily adapted to the subject experiments than the proposed method.
In H3, the estimation error was particularly large, and similarly large errors occurred in the ANN-only model, which achieved small estimation errors for the other subjects. This large error may be due to the palm thickness, especially near the MCP joint (Figure 11). As shown in Table 7, H3’s palm thickness was larger than the other subjects. When a palm thickness is larger, it is more difficult to fixate the Modular-SECA to the hand, resulting in a looser fixation. As a result, the force transmission from the Modular-SECA to the finger is weakened, resulting in a smaller amount of angular change in the REP. This smaller angular change corresponded to a high stiffness in the prototype experiment (training data for the ANN model), so it is assumed that the estimated stiffness in H3 was overestimated.
Figure 11. Palm thickness (thickness of the MCP joint).
Table 7. Thickness of the subjects’ palm
Therefore, the results suggest that palm thickness may be an influencing factor in estimation accuracy. On the other hand, in the baseline method, force is applied only to the fingers, and rotation around the axis is used, so errors in force transmission due to palm thickness are unlikely to occur. Consequently, the palm thickness does not affect the accuracy of the measurement.
In order to remedy this problem that occurred in the real environment, it is necessary to review the attachment method and improve the Modular-SECA or the model itself. Specifically, the following are required:
-
• Development of soft gloves to facilitate robust force transmission: The stability of the contact surface between the Modular-SECA and the hand should be improved by employing a nonslip finish or elastic material to facilitate more efficient force transmission. It is essential to design soft gloves with a proper fit to compensate for loose fixation due to the palm thickness.
-
• Uniformity of force transmission performance: The current Modular-SECA is designed with a fixed size. Hence, the efficiency of force transmission varies with the fixation conditions, which are influenced by physical characteristics such as palm thickness. This effect causes the amount of angular change to vary from subject to subject, which is a problem that degrades the performance of the stiffness estimation model. To solve this problem, it is important to identify the optimal size of the Modular-SECA, which is not affected by human physical characteristics and can obtain the same amount of angle change for any subject. Once the appropriate size is determined, the stiffness estimation model will no longer need to compensate for differences in the angular change due to physical characteristics, and more stable estimation will be possible.
-
• Standardization of a glove wearing method: In this study, the Modular-SECA was attached using a fixture, and the strength of the fixture varied depending on the force applied by the experimenter. Thus, it is always difficult to reproduce the same fixation condition. In addition, if the device is to be applied in telerehabilitation, a method that allows patients themselves to attach and detach the device easily is also needed (Seim et al., Reference Seim, Ritter, Starner, Flavin, Lansberg and Okamura2022). Therefore, it is necessary to automate the application of the device and introduce a mechanism to easily and appropriately adjust the fixation condition according to each subject’s physical characteristics.
-
• Adaptation to human data: Expansion of training data and adaptation of simulation data to real-world data are required (Zhou et al., Reference Zhou, Lu, Tortós, Qin, Kokubu, Matsunaga, Xie and Yu2024). In addition, it is necessary to improve the use of information from the modular-analytical model so that the strengths of both the modular-analytical model and machine-learning can be better utilized.
4.4. Feasibility of better stiffness estimation brought about by the correction parameters
Due to the difference between the prototype and the subject experiments, the prediction performance of the subject data decreased. Therefore, in order to show the generality of the proposed method, we attempted additional training to improve the adaptation of the subject data. Specifically, all parameters were retrained (fine-tuning) for the ANN-only model and the proposed method (the correction parameters prediction model). Data from three REPs of H1 (three datasets) were used as training data, and each model was trained for 4,000 epochs.
The stiffness estimation results for the subject experiments after additional training are shown in Table 8 and Figure 12. The fine-tuning of the proposed method improved the prediction accuracy in the range of actual values of the correction parameters below 0.5 and also improved the stiffness estimation results. On the other hand, unlike the other subjects, the actual values of the correction parameters in H2 are around 1. Therefore, the accuracy of H2 was reduced in the proposed method because the model adapted too much to the correction parameters around 0 by fine-tuning the proposed method. The accuracy of the ANN-only model was also improved by fine-tuning.
Table 8. Subjects’ joint stiffness estimation results before and after additional training
a Mean (± mean absolute deviation) of six stiffness values (
$ {k}_{MCP} $
).
b Mean (± mean absolute deviation) of the three stiffness values estimated from each of the three REPs.
c The stiffness values estimated from the ANN-only model in
$ 0\le \theta <{\theta}_0 $
.
d The stiffness values calculated from the corrected modular-analytical model in
$ 0\le \theta <0.7{\theta}_0 $
.
e The stiffness values estimated from the fine-tuning ANN-only model in
$ 0\le \theta <{\theta}_0 $
.
f The stiffness values calculated from the fine-tuning corrected modular-analytical model in
$ 0\le \theta <0.7{\theta}_0 $
.
Figure 12. (a) Subject’s joint stiffness reference values and results estimated by each model before and after additional training. (b) The reference values and predicted results of the modular-analytical model correction parameters before and after additional training. REP is simplified as R. FT, estimated parameters after additional training (fine-tuning).
Moreover, the fine-tuning results revealed the performance difference between the two models (the ANN-only model and the proposed method). The ANN-only model is a mechanism to predict stiffness values based on the angular change trend of the REP. Therefore, since H3 had a larger palm thickness and smaller angular change than the other subjects, additional training in the H1 data did not significantly improve the accuracy. In contrast, the actual values of the correction parameters in the H3 data take values around 0, which are similar to those of H1. As a result, fine-tuning the proposed method improved the prediction performance of the correction parameters near 0, resulting in improved stiffness estimation results for H3 (Figure 12).
Therefore, it is evident that the proposed method may be easier to adapt to subject data than the ANN-only model by compensating for additional training and training data bias. In particular, the proposed method may absorb variations in the physical characteristics of each subject that the ANN-only model cannot correct. However, further study on the generality of the proposed method is needed in future research.
Furthermore, the correction parameters of the modular-analytical model could potentially be used for data augmentation. More diverse training data are needed to improve the performance of the stiffness estimation model. However, in the stiffness estimation, adding data by conducting additional experiments with dummy fingers or human fingers is not easy. Thus, the proposed method could be used to increase the training data and eliminate bias. From Equation (2.10), the correction parameters
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
can be calculated from
$ {\theta}_0 $
,
$ k $
,
$ P $
, and
$ \theta $
. Since air pressure (
$ P $
) is fixed from 80 to 0 kPa, a new set of angles
$ \theta $
(i.e., depressurization data for one REP) can be obtained from the modular-analytical model (
$ {\alpha}_{bend}=1 $
,
$ {\beta}_{stored}=1 $
) simply by changing the values of
$ {\alpha}_{bend} $
and
$ {\beta}_{stored} $
.
In actual measurements, the angle change does not necessarily follow the analytical values because it contains noise, such as blurring of the measuring hand and errors due to the joint angle measurement method. However, it is possible to create data that roughly captures joint angle changes. Therefore, we expect that the proposed method can generate various training data and construct a model with high generalization.
4.5. Contribution and limitations
In this study, the analytical model-based stiffness estimation accuracy was improved by energy conservation-based online tuning (correction) of the modular-analytical model using the ANN. To the best of our knowledge, this is the first time that the analytical model accuracy has been improved by correcting the stiffness estimation analytical model online based on information from measurement data. An important aspect of the proposed method is that through the prediction of correction parameters using ANN, it can absorb nonlinear and complex effects that are difficult to deal with by the modular-analytical model-only estimation methods. Due to the bias of the training data, the estimation performance was not as good as the ANN-only estimation method. However, if the training data are improved using the analytical model, it could be more generalizable than the ANN-only estimation method. In addition, the correction of the analytical model allows for a wider range of joint angles over which stiffness estimation can be performed and allows for the estimation of higher stiffness values that could not be estimated with the previous analytical model.
This study also verified the effectiveness of the proposed method through the healthy subject experiment. As a result, it was confirmed that the ANN-only model does not provide accurate stiffness estimation for all subjects. However, applying the proposed method is expected to improve the estimation accuracy. In particular, the proposed method may absorb variations in the physical characteristics of each subject that cannot be corrected for by the ANN-only model. Therefore, using the proposed method is crucial in further improving the performance of the stiffness estimation with the Modular-SECA.
However, this study has some limitations:
-
• The neural network cannot know if it outputs over- or underpredicted values because the certainty of the predictions is unknown. Therefore, when the actual stiffness value is unknown, it is difficult to determine whether the value is reliable. Also, with the analytical model, the values may change when uncertain influencing factors are added to the analytical model. In such cases, it is essential to either derive confidence intervals for the estimated stiffness values or find an index derived from the measured data related to the stiffness value to determine the uncertainty of the estimates.
-
• The performance of the proposed method for spasticity patients with high stiffness has not been evaluated in this study. As differences were observed between the prototype and healthy subject experiments, new challenges may arise when estimating stiffness in spasticity patients, such as issues related to the attachment method or model applicability. Therefore, future studies should include patient experiments to verify the effectiveness of the proposed method.
-
• The effectiveness of the proposed stiffness estimation method was demonstrated under controlled experimental conditions. However, further investigations are required for practical applications. In particular, potential errors may arise due to actuator misalignment during repeated use or voluntary movements and external disturbances during rehabilitation. To address these issues in real-world environments, future work should focus on implementing real-time calibration techniques, improving angle-sensing accuracy, and enhancing the stiffness estimation model.
5. Conclusions
This study improves the analytical model-based stiffness estimation accuracy with the Modular-SECA by adding correction parameters to adjust the modular-analytical model to the measurement data. We found that adding the correction parameters can improve the modular-analytical model’s performance by adjusting the energy balance due to external inflow and outflow energy. Moreover, estimating the correction parameters using the ANN could provide a more generalizable model for stiffness estimation with the Modular-SECAs, which is prone to inaccuracy due to complex factors. In particular, using the proposed method is extremely promising in estimating joint stiffness of human hands, which are susceptible to the effects of hand shape and wearing conditions and have significant estimation errors.
In the future, we plan to verify the generalization of the approach using more subject data, including patients, and to expand the scope of application of the proposed method. In addition, we will develop a method that can more adequately compensate for individual differences by further improving the model to consider the effects of palm thickness and wearing conditions and establishing a method for selecting the Modular-SECA size.
Subscript notations
- analytic
-
value calculated from the analytical model
- exp
-
value calculated from the analytical model
- target
-
actual stiffness value
- uncorr
-
stiffness value calculated from the modular-analytical model before correction
- corr
-
stiffness value calculated from the corrected modular-analytical model
- ANN
-
stiffness value estimated by the ANN-only model
- FT
-
value after additional training (fine-tuning)
Data availability statement
The prototype experimental dataset used in this study can be found online: https://github.com/Fuko-Matsunaga/joint-stiffness-estimation-experiment-using-dummy-fingers.
Acknowledgments
The authors would like to thank Zhongchao Zhou for providing insights into neural networks. They would also like to thank Pablo Tortós for his advice on interpreting the results.
Authorship contribution
F.M. and W.Y. designed this study. F.M. analyzed the data, built and validated the models, and wrote the original draft. T.K., M.-T.K., Y.-H.H., S.Y.H., J.G.-T., and W.Y. revised and edited the manuscript. All authors have read and agreed to the published version of the manuscript.
Funding statement
This work was supported by the Grant-in-Aid for Scientific Research (B), JSPS KAKENHI Grant Number 22H03450.
Competing interests
The authors declare none.
Ethical standard
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. This study was approved and registered under the Uekusa Gakuen University Research Ethics Board, study number 24-01.
Open access
