Related Work

Various operators are commonly used for image segmentation, such as Sobel [13], [14], Prewitt [13], [14], Laplace of Gaussian function (or Laplacian of Gaussian function: LoG) [15], Canny [16], Kirsch [13], [14], and Roberts Cross operators [13], [14]. The Sobel operator is a simple and effective tool for image segmentation. However, it is not based on gray image processing, so it cannot separate foreground objects from the background of an image, which may make the outline of an image unsatisfactory. The Prewitt operator computes an approximation of the gradient of the image intensity function. The Prewitt operator tends to lose small amplitude boundary points, which lead to some errors. The LoG operator is used frequently in digital image processing for segmentation and binarization. The LoG operator starts by smoothing the original image suppress noise, before detecting the boundary.

The parameters of the Canny operator can be adjusted to specific requirements to identify boundaries with different characteristics. However, the Canny operator is slightly slow during real-time image processing. The Kirsch operator is a nonlinear detector that finds the maximum boundary strength in a few predetermined directions. The Roberts Cross operator is used for image segmentation and it highlight changes in intensity in a diagonal direction. This operator is simple, but it suffers greatly from sensitivity to noise.

Recent studies have shown that machine learning can improve the accuracy when detecting object boundaries in images. Many features associated with boundaries, such as the brightness, color, gray level, and texture, can be utilized [17]. Dollar et al. [18] proposed a supervised learning algorithm for object boundary detection, which selects and combines features at different scales. Some researchers have aimed to improve optimization constraints, e.g., the metric proposed by Jain et al. [10] aimed to minimize topological disagreements rather than disagreements over boundary location and their approach improved the segmentation performance significantly. In addition, the gap elimination approach proposed by Denk et al. [19] can amend an incomplete curve.

Image segmentation related techniques have been applied widely to biological and biomedical images. Accurate and automatic particle detection by EM is very important for the high-resolution reconstruction of large macromolecular structures. Cardona et al. [9] introduced an approach to the comprehensive anatomical reconstruction of neuronal microcircuitry based on transmission EM sections of a small brain, i.e., the early larval brain of Drosophila melanogaster. Yu et al. [20] proposed a method for particle picking based on shape feature detection. Jurrus et al. [21] used Kalman-snakes and optical flow computation to track axons across large distances in volumes acquired by serial block-face scanning EM. Zhang et al. [22] proposed a multi-domain fast-marching method with manual or fit-based multi-seeding to segment meaningful substructures. The dual point decision process developed by Giuly et al. [23] can segment the three-dimensional (3D) structures obtained by 3D microscopy. Seghier et al. [24] described a method for microbleed detection using automated image segmentation, which delivered fairly consistent results compared with the manual method. Plaza et al. [25] aimed to minimize the manual correction and segmentation time by proposing a probabilistic method for reducing manual correction, but without losing the segmentation quality.

Brain magnetic resonance image (MRI)-related information processing is a hotpot for researchers. Gousias et al. [26] segmented neonatal and developed brain MRIs into 50 anatomical regions. Their proposed approach could automatically classify the images into predefined categories, which allowed age-specific brain atlases of neonates to be produced. Yu et al. [27] evaluated the effects of neonatal intensive care and predicted the neurodevelopmental outcomes of high risk newborns using a combination of manual and automated segmentation tools. Wang et al. [28] also argued the importance of infant brain MRI segmentation when quantifying patterns during early brain development. They proposed a longitudinally guided level set method for segmenting serial infant brain MRIs. Keihaninejad et al. [29] utilized a kernel-based class separability criterion to segment structures in brain images, before using a support vector machine to categorize the results. Attique et al. [30] identified tissues using MRIs. Caskey et al. [31] proposed an open-source software tool for tissue segmentation in images. Eggert et al. [32] analyzed and discussed several factors that may affect MRI segmentation in terms of the final segmentation quality and specific adequate performance criteria.

Precision, Recall Rate, and F-score

Many metrics have been used to evaluate the performance of image segmentation task where the goal is to detect boundary. The precision, recall rate, and F-score are used frequently. The precision [33] is the probability that a resulting boundary pixel is a true boundary pixel. The recall rate [34] is the probability that a true boundary pixel is detected. The F-score [35], which is defined for all points on the precision-recall curve, is the harmonic mean of the precision and the recall rate. Definitions of the precision, recall rate, and F-score are as follows.(1)(2)(3)

Pixel Error

The pixel error [10] of the test labeling of an image relative to the standard labeling is based on the number of pixel locations where the two labeling systems disagree. The pixel error is defined as 1 - the maximal F-score of the pixel similarity, as follows:(4)

Rand Index

The Rand index [36] is a measure of the similarity between two data groups. It can also be used to evaluate the performance of image segmentation. Given an image S with n pixels, as well as two segmentations X and Y, let:

a be the number of pixels in S that are both the same in two segmentations;

b be the number of pairs in S that are both different in the two segmentations.

The Rand index [36] is then defined as:(5)

Rand error, used as a measure of disagreement, is the frequency with which the two segmentations disagree over whether a pair of pixels belongs to the same objects or not.(6)

Warping Error

There is a type of disagreement when the sketch of an object has been identified roughly whereas the detected boundaries are incomplete. The pixel error metric is overly sensitive to minor displacements in the location of a boundary, which leads to large quantitative differences in the pixel error, although there are no qualitative differences in the interpretation of the image. This shows that the pixel error and Rand error are not good choices in such cases. Thus, we use the warping error [10] to evaluate this type of disagreement. The warping error penalizes the topological disagreements produced when an object splits and merges. Given the pixel error of T relative to warpings of L*, the warping error between some candidate labeling T and a reference labeling L* can be regarded as the Hamming distance between L* and the best warping of L* onto T, as follows [10]:(7)

Macro Metrics

The aforementioned error rates can be only used to evaluate the performance of a single test. If we want to measure the global performance of a system, we need to use a macro averaging evaluation rate, such as macro precision, which is computed by averaging the labeled precision. Definition of the macro precision rate, macro recall rate, macro F-score, macro Rand error, and macro warping error are as follows:(8)(9)(10)(11)

Criteria for Structure Detection

In general, a good structure detection algorithm should determine the precise location of the boundary. Thus, we propose five criteria that our structure detection algorithm should follow.

Multi-scale Fused Structure Boundary Detection

Two key problems must be solved to segment connectomes. First, each synapse must be identified. Second, the “wires” of the brain, i.e., its axons and dendrites, must be traced in images. The complexity of neuronal structures and the negative effects of noise make it difficult to segment structures from EM images. Furthermore, the absence of useful features such as brightness, color, and texture, add the difficulty of segmentation.

Many operators, such as Sobel, Prewitt, Kirsch, and Roberts cross, require little computation, but they are sensitive to noise, which makes it difficult to obtain a satisfactory result. LoG and Canny are effective in determining the locations of boundary points, but these two methods require an appropriate scale. At a small scale, operators are sensitive to boundary points and noise, whereas they are stable at a large scale, but they tend to filter out incorrect details. Thus, we utilize a multi-scale method to describe the diversity of structures and to determine the boundary. At a low resolution scale, our approach identifies boundary points rapidly by suppressing noise and detail, before identifying the position of boundary points precisely at high resolution, and it finally tracks the actual position of the boundary points using a coarse-to-fine tracking strategy.

The Laplacian operator is usually used in image segmentation. However, it is very sensitive to noise. A tradeoff between computational complexity and noise reduction is to carry out Gaussian blur before detecting the boundary using Laplacian operator, which is known as LoG. However, LoG is not much effective in the case of EM images with too much noise information. At present, Gaussian pyramid [12] is getting widely used in many fields, such as image processing, computer vision and signal processing. A Gaussian pyramid decomposes an image into a set of images at different scales, each of which provides specific boundary information. Gaussian pyramid is essentially the representation of image at different scales. An image is blurred by Gaussian function and down-sampled to generate a series of images at different scales for later processing. Inspired by the principle of Gaussian blur, we generate a Gaussian pyramid of the image to be processed, then apply LoG to each level of Gaussian pyramid image to detect neuron structure boundary, and finally combine the detected results together to attain the structures.

The Gaussian pyramid technique generates a stack of successive images, where each pixel contains a local average that corresponds to a pixel neighborhood at a lower level of the pyramid. The detected boundaries are different at various scales, the fine details can be detected at small scale whereas details are often missed at a large scale, such as, a ramp boundary can be only detected easily at a small scale. The last step is to assemble the detected boundaries. However, multi-scale boundary fusion does not mean that the boundaries detected at different scales are simply merged together. The operator discriminates responses at different scales, thereby locating the detected boundary point in a different position, which would cause boundary redundancy if we simply combined them. In general, the resulting boundary produced at a large scale, which tends to be more stable and noise-resistant but poor in terms of positioning accuracy, corresponds to the outline of the image. By contrast, the results obtained at a small scale maintain the rich details of the image and a high positioning accuracy, but they are susceptible to noise interference. Thus, we obtain boundaries with different positions at different scales. However, the boundary positions at two adjacent scales are close to each other. For example, the positional difference at two adjacent scales is just one pixel. Thus, multi-scale boundary fusion should be carried out between adjacent scales. The algorithm used for multi-scale fused structure boundary detection is as follows.

Algorithm 1: Multi-scale Fused Structure Boundary Detection.

Input: I, EM image

l, the level of Gaussian pyramid

α, predefined value for a boundary point

β, predefined value for a non-boundary point

weight, predefined weight for Gaussian pyramid of image

Output: boundary, array for detected result where boundary(x,y) = 1 denotes point (x,y) is a boundary point and boundary(x,y) = 0 denotes point (x,y) isn’t a boundary point

1: generate an l level Gaussian pyramid of image I and get I₁…I_l.

2: for i = 1 to l do

3: detect boundary of image I_i and get boundary_i

4: end for

5: for all point (x,y) in I

6: fusion_account ← 0

7: for i = 1 to l do

8: if point_i (x,y) is a structure boundary then

9: fusion _i (x,y) ← α _* weight of level i

10: else

11: fusion _i (x,y) ← β

12: end if

13: for i = 1 to l do

14: fusion_account ← fusion_account+fusion _i (x,y)

15: end for

16: if fusion_account<threshold

17: point (x,y) isn’t a boundary

18: boundary(x,y) ← 0

19: else

20: point (x,y) is a boundary

21: boundary(x,y) ← 1

22: end if

23: end for

24: return array boundary

Boundary Amendment

As neuronal EM images are usually of low contrast, of low resolution and with much noise, the structure detection approaches often have incorrect results and leave many gaps in detected boundary curves. The gaps will make it more difficult to determine the outlines of neuronal structures. Connecting incomplete curves improves our comprehension of images. In general, the gaps are not very large and the curves still appear continuous overall. Thus, we can take advantage of the criteria stated earlier, to exploit the continuity of the boundary pixels in the gradient magnitude or gradient direction and to connect the gaps in the detected boundary.

Linking a gap in an incomplete curve can be regarded as connecting the starting point P₀ (x₀, y₀) at one side of the curve with the ending point P₁ (x₁, y₁) at the other side of the curve given some curvilinear trend. It is not appropriate to take the shortest path from point P₀ (x₀, y₀) to the nearest point P_i (x_i, y_i) as the gap connection curve. As shown in Figure 1, point C is the nearest point to starting point A, but it is not the ending point of gap connection curve. Indeed, point B is the correct point. Connecting point A with point C fails to complete the task of closing the opening curve, but it could also destroy the topology of recognized neuronal structures. Therefore, we propose a SARSA (λ)-based curve shape fitting amendment algorithm to connect gaps in open-ended curves.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. A curve with a gap between A and B rather than A and C which is nearer than B.

However connecting A and C is incorrect.

https://doi.org/10.1371/journal.pone.0090873.g001

SARSA (λ)

The reinforcement learning [37] problem is considered to be a straightforward framing of the problem of learning from interaction to achieve a goal. The basic reinforcement learning model comprises a set of controllers, process, actions, rewards, and states [38]. The controller obtains the output from a process and applies an action to the process so the behavior of the process can fit predefined requirements. The flow of the interactions during reinforcement learning is shown in Figure 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. The flow of the interactions in the reinforcement learning framework.

https://doi.org/10.1371/journal.pone.0090873.g002

At each time step t, the controller selects an action a_t ∈A from the action space A. As a result, the state changes to s_t+1∈S from s_t∈S according to a transition function f: S×A→[0,∞):(12)where s_t+1 denotes the state at step t+1, s_t denotes the state at step t, a_t is the action taken at step t, and f is the transition function.

The controller receives a reward r_t+1 according to the reward function :(13)where r_t+1 is the received reward by reward function at step t with taking action a_t and transferring state from state s_t to state s_t+1.

The state-action value function Q^π: S×A→R of some policy π yields the return, a long-term reward, from a starting state:(14)where Q^π is the state-action value function with policy π, and γ∈[0,1) is the discount rate, which shows how far-sighted the controller is when considering the rewards, and it is also a factor for increasing the uncertainty of future rewards.

Temporal difference (TD) learning [39] is an important concept in reinforcement learning, which is a combination of Monte Carlo and dynamic programming concepts. TD learning occurs according to experience in an environment model. For most real-world applications, TD requires less memory and computational time than conventional approaches, but it usually yields better effects.

SARSA, which is defined by the tuples of state, action, reward, next state, and next action, which is denoted by (s_t, a_t, r_t+1, s_t+1), is an online TD control method. The update of Q [40] depends on (15):(15)where Q is the state-action value function, γ∈[0,1) is the discount rate, a_t∈ (0,1] is the learning rate and the r_t+1+γQ_t(s_t+1, a_t+1)-Q_t(s_t, a_t) part is regarded as the TD. The SARSA TD includes the value that Q takes in the next state.

SARSA updates according to the difference between the continuous expected reward values, rather than the difference between the expected value and the true value. Therefore, SARSA learning does not need to wait for the end of the task execution.

The TD part of SARSA demonstrates its online characteristic, which means that the data are not necessarily prepared in advance because the system can find out a solution during processing. In an offline approach, the image data should be available in advance because the offline approach needs to know the image model for processing, which is almost impossible in practice because there may be various sophisticated images that make the image model changeable so we cannot predict the image model. Online characteristics of SARSA are superior to offline approaches when solving our problem.

SARSA (λ) is an eligibility trace version of SARSA. Eligibility trace, a memory mechanism used by cognitive science, is associated with backward states. The eligibility trace is updated by:(16)where e_t denotes the eligibility trace at step t, γ∈[0,1) is the discount rate, λ∈ [0,1] is a weight parameter and Q is updated by:(17)where δ_t = r_t+1+γQ_t(s_t+1, a_t+1)-Q_t(s_t, a_t) is the TD error for state-action prediction.

Reinforced Gradient-descent Curve Shape Fitting

Curve fitting aims to find out a mathematical expression that models the existing data. The distribution of the points on the curve is restricted by the model. In our study, we consider the curve before the amendment and the curve after amendment when fitting the model. Thus, if we consider the untreated curve and the amended curve as a whole, the continuous portion of the curve can be viewed as one with continuous and dense points whereas the incomplete portion of the curve can be treated as that with discrete and sparse points.

Let the curve be given by:(18)where θ is the parameter and f(x) can be any given basis function. A point on the curve can be represented by (x, F(x)).

Let F(x) be the real ordinate value of point (x, y) and is the estimated ordinate value of point (x, y). We can see that a lower difference between F(x) and yields a better fitting result. Thus, our goal is to minimize the mean-squared error . If the derivative of Δ equals zero, takes the maximum or minimum value of the function. In the present study, takes the minimum value.

In gradient-descent [41], is a parameter vector with real valued components. For each θ, we have a representation of a corresponding approximation Q, as follows.(19)where is the approximation value of Q, θ is the parameter and G is the basis function.

Gradient-descent methods minimize the value of the mean-squared error Δ by adjusting the parameter vector gradually with each sampled data:(20)where α is the step-size parameter, and Q_t(s_t, a) is an approximating function:(21)where θ* approximates θ. Then, can be represented as , which is the vector of the partial derivatives:(22)where the derivative vector is the gradient of f with respect to .

Various basis functions [42], such as polynomial functions and radial basis functions, have been used widely for image processing, nonlinear approximation, time sequence analysis, data categorization, pattern recognition, information processing, system modeling, and other tasks. Polynomial fitting functions, which aim at fitting a function using a polynomial function, are simple and easy to use. Given the cost of implementation, we selected a polynomial function as the basis function of f(x). Other types of basis function, such as radial basis functions, trigonometric functions, and 0–1 binary functions, could also be used. A typical polynomial function is defined by:(23)where a₀,a₁,…,a_n are constants.

Therefore, the curve F(x) is given as follows:(24)

We put together the items with the same order of x and we obtain:(25)where

(26)Indeed, equation (25) is a linear basis function version of equation (18). Without loss of generality, we can use equation (27) to represent the curve.(27)where θ is the parameter, and x and y is the coordinate values of the point.

In our study, the current point position (x, y) is referred to as the state. The action set is defined as:

The reward for an action is defined based on the distance between the point with coordinates (x, y) on the curve and the corresponding approximated point with coordinates (x’, y’).(28)

As the objective can be viewed as minimize total space interval between the approximate curve and real curve, the goal of the proposed SARSA (λ) based algorithm is to minimize the total reward of all points along the curve.

From equation (27), we can see that the reward can be rewritten as follows:(29)

Let and , and then Q is defined as:(30)

Correspondingly, the gradient-descent of Q is the difference between the two.

The complete algorithm for online reinforced gradient-descent curve shape fitting is given in algorithm 2.

Algorithm 2: Reinforced gradient-descent curve shape fitting.

Input:the curve C, fitting function F(x), neighborhood Δ, learning rate α.

Output: parameter vector

1:

2:

3: get all coordinates of points on the curve in neighborhood Δ

4: repeat

5: for each point i in Δ of the curve C

6: move ← get move given by π for (x, y)

7: take move

8: observe reward r by (28) from (x, y)

9: observe (x’, y’) for next step

10: observe

11: step counter ← step counter +1 (batch updating version)

12: if reaches predefined step count (batch updating version)

13:

14: step counter ← 0

15: end if

16: (instant updating version)

17:

18:

19: (x, y) ← (x’, y’)

20: move ← move’

21: end for

22: until unchanged

23: return

SARSA (λ)-based Curve Traveling and Amendment

Given a starting point P₀ (x₀, y₀), the ending point P_n (x_n, y_n) could be in any of eight directions, i.e., up, down, left, right, up-left, down-left, up-right, and down-right relative to the point P₀ (x₀, y₀). The neighboring points of P₀ (x₀, y₀) could be on the boundary, which we refer to as a marked point with pixel gray value 1 in the image, or they might not be on the boundary, which we refer to as a blank point with pixel gray value 1 in the image.

If the successive point P₁ (x₁, y₁) of the current point P₀ (x₀, y₀) is a blank point, which indicates that the line that is assumed to fit the curve model should be made up from the current point to the next point to connect them, we will subtract a predefined penalization cost value gap, otherwise if the next point is a marked point, we will add a gain to show that the two points have already been connected and the traveling route of the time is correct. Next, the point P₁ (x₁, y₁) is set as the subsequent starting point. We continue exploring until any one of the termination conditions is satisfied, e.g., going back to point P₀ (x₀, y₀), the sum of the penalization cost is less than a predefined threshold, or maximal number of processing episode steps is reached.

In our study, we know little about the model of the structures hidden in complex images. Furthermore, we cannot obtain a complete image beforehand in actual cases. An online approach does not strictly require a model of the environment, reward, and subsequent state probability [40]. Thus, we utilized SARSA (λ) as the basis of our amendment approach.

The slope of a curve is the tangent slope of some point on the curve. In a small domain, a change in the curve slope can be represented as a derivative value of the point, which is the geometric meaning of the derivative. In a small domain, the curve is related to the curve model of some point within its neighborhood. Thus, if the next point is a blank point, we will obtain a model of part of the curve that is the neighborhood of the current point as the expected curve trend. Moreover, it is very difficult to obtain the whole model, whereas it is relatively easy to obtain a model of a small portion of the curve. Therefore, obtaining a curve model of a small area satisfies the characteristics of changing stability and it is also feasible. In our study, when the next point is blank point, we take the current point as the starting point, trace back and move forward within a small neighborhood, obtain a small portion of the curve, approximate a model of the segment, and connect the curve under the supervision of the obtained model.

The current point position (x, y) is referred to as a state. The action set is defined as:

The reward for traveling is given by:(31)

In algorithm 3, a moving point starts from one end of the incomplete curve and walks randomly through the image where decisions are made by a controller. The point will stop in any of three conditions: (i) returning to the starting point; (ii) the action-value function Q is less than the predefined threshold; (iii) the maximum number of processing step in the whole image is reached, which is called an episode step. The SARSA (λ)-based curve traveling and amendment algorithm is as follows.

Algorithm 3: SARSA(λ)-based curve traveling and amendment

Input:starting point (x₀, y₀), threshold, maximal_step

Output: action-value function Q under moving policy

1: initialize Q((x, y), move) arbitrarily

2: episode_count ← 0

3: for each point (x, y) in image and move_i in move-set do

4: e((x, y),move_i) ← 0

5: end for

6: repeat scanning of image

7: for each episode step

8: initialize (x_t, y_t), move, and e((x_t, y_t), move)

9: move-set ←

10: take move and get new point (x_t+1, y_t+1)

11: r ← get reward by equation (31)

12: if (x_t+1, y_t+1) is a blank point

13: trace backward neighborhood and save points

14: repeat

15: move forward to point (x’, y’)

16: if (x’, y’) is not a blank point

25:

26: δ ← r+ γQ((x_t+1, y_t+1), move_t+1)- Q((x_t, y_t), move)

27: e((x_t, y_t), move) ← e ((x_t, y_t), move)+1

28: for each point (x, y) in image, move_i in move-set do

29: Q((x_t, y_t), move) ← Q((x_t, y_t), move)+ αδe((x_t, y_t), move)

30: e((x_t, y_t), move) ← γλe ((x_t, y_t), move)

31: end for

32: (x, y) ← (x_t+1, y _t+1)

33 move_t+1← move_t

34: episode_count ← episode_count +1

35: until (x, y) = (x₀, y₀) or Q((x, y), move)< threshold or episode_count>maximal_step

36: end for

37: return Q^move