A medical imaging analysis system for trigger finger using an adaptive texture-based active shape model (ATASM) in ultrasound images

Trigger finger has become a prevalent disease that greatly affects occupational activity and daily life. Ultrasound imaging is commonly used for the clinical diagnosis of trigger finger severity. Due to image property variations, traditional methods cannot effectively segment the finger joint’s tendon structure. In this study, an adaptive texture-based active shape model method is used for segmenting the tendon and synovial sheath. Adapted weights are applied in the segmentation process to adjust the contribution of energy terms depending on image characteristics at different positions. The pathology is then determined according to the wavelet and co-occurrence texture features of the segmented tendon area. In the experiments, the segmentation results have fewer errors, with respect to the ground truth, than contours drawn by regular users. The mean values of the absolute segmentation difference of the tendon and synovial sheath are 3.14 and 4.54 pixels, respectively. The average accuracy of pathological determination is 87.14%. The segmentation results are all acceptable in data of both clear and fuzzy boundary cases in 74 images. And the symptom classifications of 42 cases are also a good reference for diagnosis according to the expert clinicians’ opinions.


Introduction
Trigger finger, which is usually caused by repetitive or forceful use of the finger, has become a frequent occupational disease in recent years. It occurs when a nodule is formed in the tendon, causing a mismatch of the volumes of the tendon and pulley. The tendon may become stuck at the mouth of the tendon sheath tunnel so that flexion and extension become ragged [1]. Treatment depends on severity. Corticosteroid injections can cure moderate cases. However, only surgery can resolve the symptoms at later stages.
In clinical settings, ultrasound imaging is used most in trigger finger diagnosis and surgery. alter the tissue's characteristics when it becomes irritated. Some research demonstrated that the A1 pulley or tendon becomes thick and hypoechoic. However, these studies segmented the tendon and pulley manually. Manual assessments may yield inconsistent results due to intraand inter-observer deviations [2][3][4]. An image analysis system that automatically segments the tendon and synovial sheath may address these limitations. As shown in Fig 1(B), the tendon is elliptical. The synovial sheath surrounds it, and the volar plate is attached underneath. The underlying tissues and the acquisition position and angle affect an ultrasound image's quality. The synovial sheath's boundary exhibits good contrast at the top region of Fig 2(A), while it is fuzzy in Fig 2(B). Similarly, the tendon's lower boundaries appear different in Fig 2(A) and 2(B) because the volar plates echo at a wide range of intensities. Such variations may make it difficult to segment the tendon and sheath using only intensity-based clues. However, shape and local texture measures, such as the Gabor texture [5], are commonly used to increase the accuracy of the segmentation process. Several research groups have introduced shape-based methods, such as the active contour model (ACM) and active shape model (ASM), for tissue segmentation in ultrasound images [6][7][8][9][10][11][12][13][14][15]. Hamameh and Gustavsson [6] presented a method that combined the ASM with an ACM to clarify the boundary of the left ventricle in echocardiograms. Tsai et al. [11] and Chen et al. [12] proposed an automatic image analysis system that precisely measures the fetal craniofacial structures using a surface template model.
Many of these studies segmented tissue by applying models to sonograms. However, model-based methods cannot always overcome the noise and deviations in images that prevent accurate segmentation. To increase reliability, many studies incorporated local texture features into model-based methods [16][17][18][19][20][21][22][23][24]. Kim et al. [16] and Huang and Dony [17] used frequencybased texture features for tendon segmentation in equine legs. Christodoulou et al. [18] classified both symptomatic and asymptomatic atherosclerotic carotid plaques using 61 texture features and a self-organized map classifier. In these studies, using the texture features provided good results.
In this paper, we propose a new model-based method called the adaptive texture-based active shape model (ATASM) to segment tendons and synovial sheaths in clinical ultrasound images. We combine the model-based method with the use of texture features to segment the tendon and synovial sheath at the A1 pulley position from the ultrasound image. A new weighting mechanism is designed to adjust the contributions of different features in the objective function. To validate the proposed method, 58 finger ultrasound images are acquired and analyzed. In addition, we adopt the SVM classifier, using tissue size and textural features extracted from the segmented area to classify tendons as symptomatic or asymptomatic.

Ultrasound data acquisition and ethics
The ultrasound image data are acquired from National Cheng Kung University Hospital and Ton-Yen General Hospital using MicroMaxx portable ultrasound system (Sonosite, Bothell, WA, USA) and t3000 ultrasound system (Terason, Burlington, MA, USA) with a 13MHz probe. Prior to participation, all participants were informed about the study's aims and procedures and signed consent forms approved by the Institutional Review Board of National Cheng Kung University Hospital and Ton-Yen General Hospital (IRB number: HR-98-048). Each participant laid the hand on the table palm up for ultrasound acquisition. In all images, each pixel is 0.075x0.075mm 2 . To evaluate the characteristic differences between symptomatic (trigger finger) and asymptomatic samples, 21 images of each hand are acquired from both patients and normal individuals. In the training phase, the model is constructed with several manually drawn contours, which are first aligned using a Procrustes analysis [25]. A principal component analysis (PCA) [26] is then applied to construct the point distribution model:

ATASM overview
where " X is the average shape, P is the set of eigenvectors of shape variability, and b is the set of tunable shape parameters that control the weight of the eigenvectors. Tuning the parameters in b can adjust the shape to fit different targets. To prevent large deformation, each parameter must satisfy the following constraint: where λ i is the ith eigenvalue of the point distribution model. In the segmentation phase, the point distribution model is modified by iteratively adjusting the pose parameters, including translation, scaling, and rotation, and then the shape parameters in b. Tendon images usually have poor quality and complex boundary characteristics. Therefore, to overcome complex tendon segmentation problems, we modify the ASM principle to include texture features as one of the energy terms and adopt the adaptive weighting

Texture profile construction
The model is deformed by maximizing the energy function, which includes curvature, gradient, and texture information. Thus, texture profiles that contain the texture information from the training data are needed. Considering the textural characteristics of the tendon boundary, some of the Gabor [5] and Laws' texture features [28] are also used in the energy function. The Gabor texture is a general texture descriptor that comprises a real component and an imaginary component representing the orthogonal directions. Both real and imaginary components are extracted using the real and imaginary parts of the Gabor filtered results: The real and imaginary components of the Gabor images are extracted using Gabor filters with angles of 0˚, 30˚, 60˚, and 90˚. The tendon contours are longer and stabler on the upper and lower boundaries. The imaginary components of 0˚in the Gabor images shows clear tendon boundary on the upper and lower parts, as shown in Fig 5(B), and thus is adopted as one of the texture features.
The Laws' texture energy measure is another famous texture feature and includes three types of masks: local averaging, an edge detector, and a spot detector. These are applied in both directions to obtain the texture images. Six texture features are extracted using six different 5x5-pixel square kernels. The tendon boundary using the level-edge kernel in Eq (5) has the most significant edge and is adopted as one of the texture features, as shown in Fig 5(C). : ð5Þ The texture features are computed on each control point and its four related points, two inside and two outside the contour every two pixels on the search line perpendicular to the current contour. Because we has 20 control points, each of which forms a 1x5 texture vector, we can concatenate the vectors and obtain a 1x100 vector of 100 texture values as the texture profile for each texture feature from each training image. The texture model is then constructed by averaging the texture profiles from all of the training images.

Energy function setup
In the original ASM, the gradient value along the search line at each control point is defined as the energy function. In tendon images, the gradient provides boundary information because of the high contrast between the hyperechoic tendon area and the hypoechoic area surrounding the synovial fluid. However, the gradient information is not adequate to find the tendon contour in complex ultrasound images. Clinicians also rely on texture. We thus adopt the Gabor and Laws' texture measures as the features for the energy function in the ATASM. Because the tendon contour is usually smooth, the curvature of the control points is considered another feature of the energy function. In the ATASM, the energy function of the model deformation is maximized and defined as where w k i and F k i are the weight and the value of kth energy term on the ith control point, respectively. Due to the different image characteristics of the tendon and synovial sheath, the energy terms are defined distinctively and are described in later sections.

Weight of energy term computation
The weights of the energy terms vary at different contour positions depending on the corresponding image characteristics. When computing the total ATASM energy, the most effective energy is assigned the highest weight. The larger energy value and larger standard deviation for a given energy term at a control point implies the energy term and position are more effective in ASM model modification. Thus, energy value and standard deviation are used to compute the weights. Along the search line, if the energy value on the correct tendon boundary is larger than on the other points and the standard deviation of the energy values is large, the energy term is considered effective in discerning the control point's position. Based on this concept, the weights of energy terms are assigned by the following equation: where F k t;i ,F k t;i , and s k t;i are the value, maximum value, and standard deviation, respectively, of the kth energy term at the ith control point across the search line in the tth training image and T is the total number of training images.

Shape model locating
In the ATASM, the shape model requires an initial location for the subsequent procedure. Due to the different characteristics of the tendon and synovial sheath, we employ different locating methods for the two types of tissue. In tendon segmentation, a coarse-to-fine matching mechanism is applied to locate the contour's position automatically. In the synovial sheath segmentation, the contour is localized by the segmented tendon instead. The locating details are described in the next section.

GA-based energy optimization
Due to the noisy nature of ultrasound images, deformation may be trapped in many local optima. To obtain a near optimal tendon shape and prevent any deformation from being trapped in the local optima, we use the GA to search for the best shape parameter b.
GA-based energy optimization is an efficient method by which to search for the optimal solution in a complex search domain. The chromosomes are initialized with a random generator and then evolve iteratively to produce the global solution. GA's evolution processes include reproduction, crossover, and mutation. In reproduction, the chromosomes with better fitness values are copied to the next generation. In crossover, the partial parameters in a couple of chromosomes are exchanged to produce a new chromosome. In mutation, a new parameter is randomly generated and substitutes for the old one in a chromosome.
In this research, every chromosome represents a set of shape parameters in b. The chromosomes of the initial population are first generated with a random generator, and each parameter b i of b is required to satisfy the constraint of point distribution model in Eq (2). In addition, the fitness function is designed as the energy function of the model deformation in Eq (6). In the GA reproduction step, roulette wheel selection is used to select the chromosomes with better fitness values. The single point crossover and mutation with probabilities of 50% and 1%, respectively, are used in each generation. The mutation process replaces a random shape parameter b i with a random value that satisfies the shape constraint in Eq (2). Elitism is applied to keep the best chromosome during evolution. If the best chromosome does not change for 10 generations, the evolution converges, and the elite chromosome is determined to be the best solution for b.

ATASM for tendon segmentation
In tendon segmentation, an intensity template has to be obtained and used to search for the tendon on the test image. From a given set of training images, a reference template covering the tendon area is first selected from a proper training image. Then, the obtained contours from all the other training images are aligned to the reference tendon contour by Procrustes analysis. The final intensity template is obtained by averaging the intensities of all the aligned training images, as shown in Fig 6. The resulting intensity template cam then be used to locate the tendon area on the test ultrasound image, and the corresponding tendon contour can be used as the initial ATASM contour.
In the model localization step, a coarse-to-fine matching mechanic is applied to initialize the contour position automatically. In the coarse matching, we first reduce the image resolution by two to increase the search speed. The input image is then split into several overlapping blocks. For each block, the sum of the absolute difference (SAD) value with respect to the intensity template is computed: where A(x, y) and B(x, y) are the intensity values of a single block and intensity template at point (x, y) and N is the size of the template. The block with the smallest SAD value is selected to refine the initial position during fine matching. In the fine matching step, the transformation parameters of the minimum SAD value between the transformed intensity template and the selected block in the original image resolution are computed. The parameters are optimized with Powell's method.
To set up the energy function, the segmentation target's image characteristics are considered. Because the tendon boundary is usually smooth, the curvature at each control point on the contour is considered small. The internal energy term, which is the curvature value, is defined as where c(x) is the position of control point x and the dot operator indicates the dot product of two vectors. The gradient properties that are bright inside and dark outside across the tendon boundary are used as two image energy terms for segmentation. Two types of energy terms, area-based gradient  gradient using a rectangular window can reduce the noise effect when computing the gradient. Because the tendon area is light in intensity and the surrounding synovial fluid is dark and has a specific width, the line-based gradient is thus designed with a mask of 5x1 pixels. Both the Gabor and Laws' texture energy terms are defined as Laws' : where G M;d i is the Gabor profiles with d degrees computed from the texture model and G T;d i is computed from the deformed contour. The variables L M i and L T i are the Laws' texture profiles computed from the texture model and the deformed contour, respectively. For these energy terms, 5x20 weights of energy terms for all the control points are computed using Eq (7).

ATASM for synovial sheath segmentation
Although the initial tendon localization is complicated, the localization of the synovial sheath is simple and can be defined based on the detected tendon position. In the synovial sheath segmentation, because the synovial sheath surrounds the tendon, the center position of the tendon is close to the center of the synovial sheath and can be used as the initial guess.
In synovial sheath segmentation, the definitions of the terms for curvature, Gabor, and Laws' textures are the same as those in tendon segmentation. Because the synovial sheath is darker inside than outside, the gradient orientation on the synovial sheath boundary is defined from dark to bright, the opposite of the gradient orientation for the tendon. The area gradient and line gradient are thus modified as Maximizing these energy terms help with the detection of the synovial sheath boundary.

Post-processing of synovial sheath segmentation
Because the lower portion of the synovial sheath connects to the volar plate and the tendon is tightly placed on the volar plate, the lower border of the synovial sheath is defined on the boundary between the tendon and the volar plate, as shown in Fig 7(B). The lower part of the tendon boundary within thirty degrees with respect to the y-axis is adopted as the lower synovial sheath boundary. Although the synovial sheath does have a closed boundary, the boundaries between the upper and lower endpoints of the synovial sheath are usually invisible in an ultrasound image. Therefore, we connect these endpoints using a second-order interpolation to approximate the synovial sheath boundaries on both the right and left sides as two parabolas. A segmentation example is shown in Fig 7.

Experimental design
Due to the characteristics of the bottom side of the tendon boundary, all images are split into clear and fuzzy boundary groups by computing the average intensity difference between the two 15x15 windows above and below the bottom tendon boundary. If the average intensity of the window above is higher than the one below by 30, the image is placed in the clear boundary group. Otherwise, the image placed in the fuzzy boundary group. In each group, twenty images are used in the training phase. A total of thirty-eight images with clear boundaries and thirtysix images with fuzzy boundaries clustered from seventy-four images (S1 Dataset) are used in the experiments to evaluate the accuracy of the segmentation results. In the first experiment, the accuracy of proposed ATASM is evaluated. The ground truth is generated by a medical expert, Dr. T. H. Yang, who is asked to draw the boundaries of the tendon from seventy-four images separately three times in three consecutive weeks (once each week). The ground truth boundaries are then acquired by averaging the three boundaries (S2 Dataset). Three different segmentation methods, ACM, traditional ASM, and texture-based ASM which uses the same energy function as ATASM but with equivalent weights, are applied for comparison. Furthermore, two users who are trained to find the tendon in ultrasound are asked to outline the tendon boundaries from sixteen images. Each trained user is blinded to the ground truths and the results of the proposed method. These outlined results are used to evaluate the difference between the proposed ATASM method and human operation. In the second experiment, we evaluate the segmented synovial sheath using the proposed ATASM compared to the ground truth outlined by Dr. T. H. Yang. Thirty images with clear boundaries and twenty-seven images with fuzzy boundaries clustered from fifty-seven images are used in this experiment. In the last experiment, the segmentation results of the proposed system are used for the symptomatic vs. asymptomatic classification.

Evaluation metrics
To evaluate the segmentation results of the proposed method, two quantitative indexes, the mean of absolute distance (MAD) and the dice similarity coefficient (DSC), are used: where a and b are the control points of contour A and B and d(a, b) is the distance between a and b. MAD calculates the average distance between the ground truth and the segmented contour. DSC evaluates the ratio of the overlapped area to the total area of the two contours. Conventionally, the two contours, A and B, are deemed similar if the DSC is higher than 0.75. In the symptomatic vs. asymptomatic classification experiment, the confusion matrix is constructed using the classification results. Accuracy and precision are computed from the confusion matrix with where TP, TN, FP, and FN denote true positive, true negative, false positive, and false negative, respectively.

Accuracy of tendon segmentation
The segmentation accuracies of tendons with clear and fuzzy boundary groups are shown in Tables 1 and 2

Accuracy of synovial sheath segmentation
As shown in Table 3

Symptomatic vs. asymptomatic classification
In this experiment, we compare the texture in the segmented tendon area to distinguish diseased tendons from normal. Two areas, the tendon area and the area inside the synovial sheath, and two types of texture features, a wavelet transform [29] and a gray level co-occurrence matrix [30], are used for symptomatic and asymptomatic finger classifications. Wavelet transform is a commonly used transformation that decomposes data into highand low-frequency parts in rows and columns by basis function. In this experiment, we use a stationary wavelet transform (SWT) and a wavelet packet (WP) to extract the features. The mean and standard deviation of each transformation image are computed to acquire features.
Due to the large number of features, we use a feature selection method to select the best feature set. An index called divergence value and described by Tsiaparas et al. in [31] is adopted: where s normal;f i and s abnormal;f i are the standard deviations of the feature vector f i for the normal and abnormal data sets. The more important feature shows the smaller variance within classes and the larger variance between classes and results in the larger divergence value. A total of 42 pairs of finger images (S3 Dataset) are acquired in this experiment. Each image pair contains the left and right hand images from a subject. In all these image pairs, there are twenty-one symptomatic pairs and twenty-one asymptomatic pairs. In the symptomatic pairs, one of the finger images has trigger finger disease, and the contralateral one is normal. In the asymptomatic category, both finger images are normal. For each pair of finger images, the tendon and synovial sheath are first segmented. A 41x21 pixels area at the center of the segmented tendon is used to extract the above-mentioned texture features. For each pair of images, the features are extracted, and the differences in the features of both hands are computed. These feature differences are then selected using the above-mentioned feature selection process. The image pairs are then classified as symptomatic or asymptomatic using these selected feature differences. In recent years, the deep learning methods have been widely used for classification [32][33][34]. However, the amount of training data used to train the classification model is large. As the number of training data is not enough, deep learning method will be left for future investigation. Here, we used the support vector machine (SVM) for classification. In this study, a SVM library called LIBSVM [35], which uses second order information in the objective function for approximation [36] to attain a faster convergence than traditional SVM, is applied. The. By computing the divergence value (Eq (20)) of each feature, we choose the most important 20 features from all the texture features for classification. The chosen features are given in Table 4, where haar, db4, db6, and coif are the different basis functions [31]. The confusion matrix is constructed and shown in Table 5. The system accuracy is 87.14%, and the precision is 82.38%. Based on the classification results, the selected geometric and texture features can be properly used for trigger finger identification.

Discussion
For tendon segmentation, the proposed ATASM achieves the lowest MAD values (3.14 and 3.34 pixels for clear and fuzzy boundary groups) and the highest DSC value (0.91 and 0.90 for clear and fuzzy boundary groups) among the adopted automatic methods as shown in Tables   from training images and obtains the more applicable tendon boundary than ACM. However, the energy term which only considers the gradient information is easily affected by the speckle noise. By adding the texture information in energy terms as in the texture-based ASM, the segmentation results are improved and less affected by the noise. Since the top boundary of the tendon surrounded by a narrow band of synovial fluid, and the boundaries on both sides of the tendon are usually blurry, the texture-based ASM using equivalent weights of energy terms is less effective in segmentation as shown in the fourth column of  In the synovial sheath segmentation, all the segmentation results for clear and fuzzy boundary cases are higher than 0.85 in averaged DSC values which imply good correspondence with the expert outlines. As shown in Fig 9, the top side of the synovial sheath can be correctly segmented by using the proposed ATASM. The boundary on the bottom side of the synovial  sheath is also well segmented by using the good tendon segmentation results obtained in the previous step. We also proposed the coarse-to-fine model matching for automatically locating the initial block of tendon as shown in Fig 10. Since the proposed intensity template containing both the tendon and surrounding synovial areas, which provide the necessary information for automatically locating the tendon block, the initial tendon block can be detected automatically. However, for those images with less synovial sheath areas or containing two tendons as in Fig 11(A) and 11(B), the tendon block is outlined manually.  As GA is adopted in ATASM for parameter selection, the segmentation process is repeated three times for different initials of GA and tested with five different images, and the resulting standard deviation of MAD is 0.52 pixels. This result shows that the proposed method can obtain the stable segmentation results with GA structure.
In order to test the stability of the proposed method with different training data sets, we had randomly selected ten training data from the tendon images with clear boundaries for model construction. Each set contains ten training images randomly selected from twenty training data. The constructed model was then applied to the thirty-eight images with clear boundaries to obtain the average MAD and DSC values of all segmentation results. The average MAD values of five trials were 4.07, 4.02, 4.04, 4.01 and 3.84 pixels. And the average DSC values were 0.89, 0.89, 0.88, 0.89, and 0.89. The standard deviation of MAD and DSC values of five trials were 0.08 pixels and 0.002, respectively. These results implied that the proposed ATASM structure with different training data set can achieve the good and similar segmentation results.
Our system also performs well with regard to classifying trigger finger and normal cases in the last experiment. The features selected by divergence value shows that the tissue areas and the low-frequency components of the wavelet transform, e.g., the A1 and A2A1 features, are the important features for classification, which satisfies the results of previous research [2][3][4] that the tendon with trigger finger may become thick and hypoechoic.

Conclusion
In this study, we develop a fully automatic segmentation system for the tendon and synovial sheath (S1 File). Because the characteristics of the tendon and synovial sheath boundaries exhibit large variations in ultrasound images, the conventional ASM cannot always achieve accurate segmentation results. The proposed ATASM can segment the tissue, which can be described using a statistical shape model with some variations in ultrasound images. The ATASM adopts the Gabor and Laws' profiles as the texture features of tendon borders and uses adapted weighting to adjust the influence of energy terms at each control point in the energy function. Coarse-to-fine model matching is adopted for initializing the affine parameters of the ATASM model. GA-based optimization is used to maximize the energy function to determine the final organ shape. In the experiments, the MAD values are 3.14 and 3.34 pixels and the DSC values are 0.91 and 0.90 between our method and ground truth for the two image groups. These results are better than the ones by ACM, traditional ASM, and texture-based ASM, and similar to the human operations. In synovial sheath segmentations, the MAD values are 5.12 and 4.54 pixels and the DSC values are 0.87 and 0.88 that also implies good correspondence with the ground truth for synovial sheath. With the segmented area and the texture features, the proposed system performs well in classifying the normal and trigger fingers in the current clinical experiment.
The proposed study still has some limitations. Because there are acoustic shadows on the bottom two sides of synovial sheath in the axial ultrasound image, the parabolic interpolation is adopted in defining the sheath boundaries. For a better model of synovial sheath, we will acquire the contour shape either from other imaging modalities or with different ultrasound scanning structure. In addition, a larger number of subject images may also be acquired for further improving the tendon symptom classifications. Other finger symptoms, such as the de Quervain's syndrome, can also be included in the future clinical experiments.