Introduction

Recently, multicore architecture is vastly used and accordingly automatic code parallelization becomes a hot concern. Speculative parallelization is a promising technique that can be used to select potentially parallel region based on runtime analysis [1], parallelize the code, and provide for rolling-back mechanisms for maintaining correctness, in case of incorrect analysis. Another approach is purely static, where the analysis guarantees that no data dependence exits in code regions, which are mainly loops [2]. However, such dependence analysis is a complex task, as it requires proving that all possible execution paths would result in a lack of data dependence. Moreover, the analysis is more complex for the case of binary code, especially for legacy applications and binary code for which the source code is not available. Therefore, the decision of parallelizing of a particular memory word in instruction or data is undecidable. Thus, data dependence in binary-level is challenging [3].

Dependence analysis methods can be broadly categorized into static and dynamic. With static dependence analysis, at compile-time, the compiler conducts conservative analysis to ensure that there is no possible dependence. However, the compiler can incorporate runtime checking for simplifying the static analysis, and decide to execute either serial or parallel accordingly. Nevertheless, once the parallel code executes, correctness is guaranteed.

Dynamic dependence analysis is featured with its for wider applicability especially when dependence violations are rare or when the code complexity is high, thereby making it difficult to prove the lack of dependence. Furthermore, static analysis is generally applicable at the source code level. Since dynamic analysis relies on detecting dependence violations rather than proving their impossibility, it could be applied at binary-level [4].

Various techniques of analysis are used to detect data dependence statically. One of these techniques is Abstract Interpretation (AI), which is an elaborate static analysis method that is generally used in program verification. It simplifies gathering all possible program execution states (collective trace semantics) through abstracting program states and statements. In other words, it provides a set of equations corresponding to each program statement (i.e. the analysis is based on original program semantics). The solution of this equations set generates the program semantics. Abstraction simplifies solving the equations set through a fast converging iterative process. The abstract semantics are a super-set of the actual concrete semantics, provided for sound analysis [5, 6].

In this paper, we consider the case of data dependence analysis of binary code for memory read and write operations using AI. We propose applying AI data dependence technique at runtime for hot binary code regions. In particular, our system identifies the hot function through profiling. Then, the approach interprets each instruction of this function. In doing so, our system could provide corresponding AI equation for each interpreted instruction, and also compute the corresponding current abstract collective semantic value. The interpretation continues until all reachable instructions are executed at least two times with no change in the collected semantics (i.e. reached a fixpoint).

Having the generated semantics, our approach can then compute all possible data dependence through intersecting memory read and write addresses, and generating a corresponding dependence graph. Also, the approach marks all non interpreted region exit instructions, so as they can be used by an underlying speculative execution system to trigger dependence violation and recovering the correct program state program.

To assess the performance and accuracy of our method, we have extended the Padrone binary code modification system to implement the above analysis [7]. The main contributions of this article can be listed as:

• Propose a data dependence analysis technique for the binary code based on AI at runtime;
• Integrate the method into the Padrone binary modification framework;
• Conduct an initial performance investigation on a set of PolyBench kernels to validate the approach.

The remaining part of this paper is organized as follows: Section 1 discusses the related work to our proposed approach. Section 2 provides background on AI, dynamic binary analysis, and the underlying Padrone binary modification framework, Padrone. Section 3 discusses the main concept of the proposed method. Section 4 provides details about the implementation of our system. Section 5 discusses the experimental results. Finally, Section 6 concludes the paper and discusses future work.

1 Related work

Several research papers have studied automatic parallelization and how to improve dependence detection. Nevertheless, most of these studies perform parallelization guided by static data dependence analysis techniques. Also, these techniques require re-execution of the target program, and it is not generally possible to conduct parallelization automatically at runtime.

Notable static analysis work is based on the polyhedral model. The polyhedral model is a mathematical framework that represents loop nests as polyhedra whose facets are computed from the program’s loop bounds. Polyhedral transformations are typically applied by a static compiler on its intermediate representation [8]. Pradelle et al. [9] proposed an automatic static parallelizer for binary code which extracts high-level information by parsing the binary code. According to this information, the system generates a C program which is parallelized via polyhedral parallelizers. Then the semantics of the original source is re-introduced and recompiled by C compiler. Therefore, this system analyzes the program statically. Furthermore, it requires re-generation of high-level program, re-compilation and re-execution.

Jimborean et al. [10] have studied dynamic and speculative polyhedral parallelization based on compiler-generated skeletons. These skeletons are applied at runtime to the original code through polyhedral transformations. Skeletons are generated at compile-time, then they are selected and instantiated at runtime. The model requires all loop bounds and memory access functions to be affine functions of global variables or outer loop induction variables. Appropriate regions are defined as Static Control Parts (SCoPs).

Yukinori et al. [11] have introduced a system to monitor binary code to check the data dependencies between memory references and dynamic loop- or call-contexts. Therefore, the system extracts the dynamic memory data-flow for the program binary code. Then, this data-flow is provided with hints to the programmers in order to re-executing the program with parallelization technique correctly.

Gupta et al. [12] have provided a static analysis technique using divide-and-conquer algorithms. This technique is studied at compile-time which benefits from the symbolic array analysis for dependence detection. The approach is applied with speculative parallelization.

Rugina et. al [13] have introduced a compiler which parallelizes recursive functions. This approach uses pointer and symbolic analysis to extract the independent recursive calls. The symbolic information provides the compiler with the main information needed to extract the procedure calls which should be executed concurrently without any violation. This approach is applied statically and generates a code to be re-executed in parallelization.

Caamano et al. [14] have proposed an automatic speculative polyhedral loop optimizer. On-the-fly polyhedral optimizations are applied on nested loops. These loops include memory access patterns and its control flow which may not computed at runtime. Apollo detects the affine expressions of the memory accesses for nested loops to be applied to speculative transformation [15]. Thus, the technique could be applied on any kind of nested loops which may contain memory accesses through pointers or indirect accesses as well as the unpredictable control. The method constructs a predictive polyhedral model to be utilized by speculative optimization. The system requires preparations to get the computed perfect version. These preparation should lead the system to re-compile. Moreover, Apollo technique is limited to the Polyhedral model.

AI is generally used for dependence checking in functional programming languages [2, 16, 17]. However, it is also considered for imperative languages [18]. Tzolovski et al. [19] have introduced an initial discussion on abstracting dependence. Moreover, they have explained how AI could be utilized for dependence properties such as dependence distance and iteration data dependence graphs. Also Ricci et al. [20] have proposed a static AI dependence technique to parallelize loops. They have used the interval and bisection domains to extract dependencies at compile-time. The system is implemented using PAG, an automatic analyzer [21]. PAG includes a set of codes which define the elements and the domain order during the application of the technique. Therefore, the approach requires special automatic software for implementation. Moreover, this method is applied at compile-time and it is not utilized automatically with the parallelization system.

Our work differs mainly from the related work in that we conduct analysis at runtime. This provides much accurate dependence detection. Moreover, the method is automatic and dynamic which does not need any additional directives which restricts the speculative parallelization to re-execute the code. Furthermore, our technique detects dependence correctly in the case of pointer aliasing which refers to existing of two pointers with the same value. If a pointer aliasing occurs, this may cause a dependence during the parallelized execution. Also, our system has no information about the fact that loop bounds/array indices are linear or not. This fact could be leveraged by any optimization that needs this information. Therefore, our approach does not need to build a polyhedral model. The main focus is on ranges which are determined during runtime by observing the addresses included in the hot trace. The method is able to handle the affine and semi-affine codes because of the dynamic monitoring of the addresses of the program points during runtime. Thus, our proposed technique still proves that a loop is free of dependence.

2 Background

In our method, we rely on conducting dynamic binary analysis (DBA) using AI to run the program using speculative parallelization. Therefore, we provide the necessary background on speculative parallelization, AI, DBA, and the Padrone binary modification framework.

3 Runtime AI approach

This section introduces the main concept of our proposed method, which is conducting AI at runtime. In particular, the code region is defined through interpreting the binary instruction, thereby revealing control-flow among instructions. As mentioned earlier, AI computes the collective semantics of the program using a set of equations. These equations consider the data-flow of the program during runtime. In our work, our system saves relevant predecessor instructions’ collective semantics through a store structure. Therefore, each current instruction could retrieve its predecessors’ previous values.

In the binary-level, each equation declares a program point which accumulates the semantics according to the join of all reaching definition and the corresponding instruction’s operation. The program point here is defined as the arc between every two successive instructions.

We can explain briefly our main approach using the following example:

As shown in the example, the statements S2 and S3 may include dependence because of y and x arrays. This dependence may be RAW or WAR. However, our approach includes the actual values, concrete values, stored in the indices i and j during actual execution. These concrete values are applied to the abstract domain as abstract intervals. The abstract intervals will contains all possible range of memory locations which may be read from or written to by x, y and z arrays. Also, the system computes the data-flow analysis to facilitate the definition of all collective semantics at each instruction at the same run. Therefore the method computes the abstract domain at runtime and reaches the fixpoint after small number of iterations. In the shown example, the approach can compute that index j will be [0, 499] in the first loop, and it will be [500, 999] in the second loop. Thence, the statements S2 and S3 do not contain any type of dependence.

For more clarification, the approach is explained using another example code fragment. The high-level code of the example is presented as follows:

This high-level code presents a loop which include WAR dependence at line 8. This line includes an addition operation between three arrays s[i], a[i], and b[i]. The result is stored in s[i]. Thus, s[i] is used as read and write at the same iteration at high-level. In the following code, we present the extracted equations of AI according to our proposed method.

In line 5, if there is a flag, rare, which is not visited during our executing and analyzing, this condition will not be analyzed. However, our system inserts some guards in the case of existing unseen code parts. If this condition is visited during speculation, there is a guard to inform the speculator that these instructions are not analyzed.

The example shows a loop that contains no loop-carried dependence, however there is a WAR dependence inside the same loop iteration. At each control-flow edge, abstract equations are defined (which are points 1, 2, … 6). We consider the interval domain as the abstract domain.

The following code demonstrates the computed AI equations for the high-level example code:

Generally, we could compute the array abstract intervals for addresses and values. The abstract value interval would be updated as any other variable during runtime till reaching a fixpoint. Although for the addresses intervals, we need to compute the abstract addresses intervals using its index variable. Therefore, the technique extracts the abstract value interval from the index variable at runtime. Like the example, at program point 4, the sia_r4 refers to the address for the array s using index i at point 4. The variable sia_r4 is computed by adding the intervals of addresses of s1 as a base address of the array and i3 as the collective index value interval. Also, sia_w4 is the address interval of array s1 using index i and w is for the write operation. The variable sia_w4 will use address interval of s1 and value interval of i3. Each of the read and write operations are done in separate assembly instructions. Therefore, the method could detect any dependent pairs at runtime.

The following is a portion of assembly instructions corresponding to the equation s[i]+ = a[i]+b[i] (S1−3) and i++ (S4):

This code has a number of operations that access the memory like S1, S2, and S3. The array access is inferred through the instruction format of having a fixed base address, scale, and an index register.

During runtime, there may be branches which contain variables that are affected by AI. Therefore, we need to compute the new abstract intervals that should be true inside each branch. This would be using the intersection operation between the collective semantic of the variable in the condition and the wide interval that achieves its condition. For example, if-then condition is less than 10, so the widened interval would be [-inf, 9]. In our example, for the condition in the while loop, the widened interval would be [-inf, 99]. Furthermore, collective semantics of i are constructed, then the approach evaluates their intersection with the widened interval. Thus, we could get the abstract interval at the condition at the point 3.

The Arithmetic Logic Unit (ALU) is used to perform arithmetic and logic operations like addition, subtraction, etc. [26]. Our method applies AI on ALU operations at binary-level. Therefore, every interval includes its lower bound (LB) value and upper bound (UB) value. For example, for the addition operation between I1, [LB1, UB1], and I2, [LB2, UB2], it could be performed as follows:

LB = LB1 + LB2

UB = UB1 + UB2

I = [LB, UB]

Thus, these intervals could be computed for all ALU operations using the same method.

The binary instruction is mapped into an abstract equations. Each equation includes right-hand side which represents the input abstract intervals and left-hand side representing the output abstract intervals. Furthermore, the abstract semantics are interpreted till convergence. The abstract state reserves the collective semantics for the hot trace of the extracted hot function.

The hot function which contains the hot loop may include function call(s). These function calls are not studied or handled in our method because each function call needs complex requirements which are not solved in purpose of parallelization [32]. Once our system detects the function call, then the analysis would terminate.

The program and its corresponding abstract equations are executed/evaluated dynamically according to the control-flow. If all reachable instructions are visited at least two times with no state change (fixpoint), the analysis is terminated, thereby the approach could compute the solution of the abstract equation and compute the collective semantics for each program point. The example code will be further analyzed in Section 4 showing the fixpoint collective semantics.

4 Our method design and implementation

The system receives a pre-compiled executable binary code. Padrone lightweight profiling computes the hot function for this binary program. Once the hot function entry is reached, Padrone pauses the execution of the program and take a private copy of the execution status (registers and memory values) to let the emulation part to start working on it. The hot function is then fed to our analysis interpreter to interpret, execute and analyze, using the abstract equations. Apparently, the analysis is performed to fetch and execute/analyze each assembly instruction in the hot execution region during runtime. The approach applies the data-flow on the fly to gather the reaching definitions and updates the abstract equations. Thus, the system can compute the abstract collective semantics at each program point. Therefore, our analyzer could compute the data dependence (through intersecting accessed addresses’ intervals). Padrone framework resumes the program execution after receiving the dependence analysis through a speculative parallelization technique.

The proposed system studies the behavior of each address included in the hot trace. This hot trace consists of the hot consecutive program points that are usually executed at runtime. Also, the preserved hot trace would let the analysis compute the loop abstract intervals for bounds. The program points, which are not involved in the current hot trace, will not be analyzed. Therefore, our approach could elicit the dependent instruction pairs which are subsequent of the analyzed function hot trace.

4.2 Interpreting and emulating

As shown in Fig 1, we have implemented an emulation of interpreter on the Padrone framework. This interpreter is used to interpret each assembly instruction as well as applying our dynamic AI technique. In the first iteration, the interpreter maps each variable value and address to the abstract domain. Moreover, the Padrone framework reads the values of the used registers and memory locations at each program point. In the consecutive iterations, the method reads the collective semantics computed in previous iterations to update the abstract domains according to the current program state. Therefore, the interpretation process is done by reading the operand and our approach could execute the operand’s operation to get the result. This result would update the abstract domain for the current program point every iteration till fixpoint is reached.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. System flow-graph.

https://doi.org/10.1371/journal.pone.0230904.g001

Fig 1 illustrates the flow-graph for our system design. The example explained in Section 3 will be discussed in this section more clearly to explain each step in Fig 1.

Our analysis is based on the CFG for the hottest function which is excerpted by the Padrone framework. As presented in Algorithm 1, when the native execution reaches the first instruction in the extracted CFG, our analysis starts to work by reading the current status of the registers and memory. Moreover, with interpreting each instruction in the CFG, the analysis can both perform the native execution of this CFG and apply the AI technique. Our analyzer begins to update the registers/memory locations in a private memory and the AI results in hash maps. By keep going through the interpretation of the instructions flow, the approach can reach to either the exit point from the working CFG and hence exit the current analysis, or reaching a stable fixpoint from the AI analysis. Therefore, our method starts the next step to utilize the AI results to compute the dependence between instructions.

Algorithm 1: Dynamic abstract interpretation analysis algorithm

Input: CFG Hot Function Code (HFC), Program Counter (C)

Output: List of Dependent-Pair Instructions, Analyzed Region

1 C ← HFC.begin();

2 List of Dependent-Pair Instructions ← φ;

3 while C ≠ HFC.end() & AccFixpoint() = false & timeOutIteration(X) = false do

4  FetchAndExecute(C);

5  if Var ∉ VarList then

6   Initialize VarInterval;

7   Store VarInterval in VarList;

8  else

9   for RHSVar ∈ VarList do

10    Retrieve RHSVarInterval From DFAnalysis;

11   OldInterval = VarList(C);

12   NewInterval = ApplyEquations(op);

13   if OldInterval ≠ φ then

14    CheckFixpoint(C);

15  if AccFixpoint() = true then

16   List of Dependent-Pair Instructions = DependenceCheck();

4.5 Intersection computation

When the fixpoint is detected, the analysis stops. In this step, our system begins computing all instructions’ pairs of abstract intervals intersection. If the intersection result is true, so these instructions’ pairs are dependent. If there is no intersection, so these instructions’ pair is independent.

In Table 1 the highlighted rows refer to the dependent pair for two instructions S1 and S3 WAR dependence. The dependent pair means that this loop contains a dependence, so the speculative parallelizer should act in a suitable way with this loop after resuming the original execution by Padrone.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Part of abstract intervals for the example.

https://doi.org/10.1371/journal.pone.0230904.t001

5 Experimental results

In this paper, we implement an emulation of interpreter in Padrone framework which relies mainly on interpreting each binary instruction by our AI technique. The method main results are generated on Intel Core i7-2670QM CPU 2.20,GHz x8 machine. This machine runs Ubuntu 14.04 LTS 64-bit Linux operating system. Furthermore, we study the main results produced from the proposed method on Polybench [35]. Polybench is called from the Polyhedral Benchmark suite. It is used to uniformize the kernels execution and monitoring. Moreover, the kernels of Polybench are used as a single file to perform the kernel instrumentation. Also, the kernels would run with non-null data initialization and live-out data dump. The loop bounds in each kernel are parametric to be utilized for general-purpose implementation. Moreover, the method impact is studied in two selected programs from SPEC CPU 2006 in terms of the overhead.

Our approach is examined by linear algebra kernels for this initial study. All used kernels include dependence in the hottest extracted function in the code by Padrone lightweight profiler. The results are demonstrated in Table 2 and Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overhead between our method and original run of Polybench kernels and SPEC selected kernels which ranges from 2% to 19%.

The overhead is caused because of the analysis of large number of program instructions during runtime.

https://doi.org/10.1371/journal.pone.0230904.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Dependence detected in kernels of polybench by our method and static AI (True and False Positives).

https://doi.org/10.1371/journal.pone.0230904.t002

Fig 2 illustrates the overhead as a main metric for dependence detection techniques. The overhead is computed as the difference between the execution time of the kernels executed through our proposed approach and the execution time of the original running on basic kernels. The overhead is partitioned as Padrone overhead and our proposed method overhead. The Padrone overhead is ranged from about 1% to 14.3%. This overhead includes the Padrone profiling and extracting the hot function during runtime. Also, the overall overhead of 401.gcc and 403.bzip2 kernels are about 18%. On the other hand, our proposed analysis technique overhead is between 0.2% to 4.88%. Therefore, the overall overhead is mostly less than 10% which could be considered comparatively very low percentage. However, we could notice that the overhead increases on the other kernels. This increase is a result of the larger number of program instructions which are checked during runtime. This overhead will be diminished by applying the speculative parallelization in the full speculative system.

The comparison is applied between the kernels binary by native running and the kernels binary running by our AI method. The results are presented in Table 2. The first column elucidates each kernel name. Also, the hottest function exists on the inner loop of one of the main called functions. Moreover, the second column explains the true positives. This column’s presented numbers are the dependent instructions pairs which are correctly dependent. The third column presents the false positives number of produced dependent instructions pairs which may not be dependent. If the false positives’ values equal 0, thus the parallelizer would take the right decision to parallelize or not according to the true positives’ values only. Apparently, the result of false positives is able to reflect the efficiency of any proposed approach. However, the false positives non-zero values will affect this efficiency partly. The speculator would take the decision during execution according to the preserved results of the dependence flags map. Our results prove the accuracy of our proposed approach in detecting data dependence dynamically.

All dependent pairs in Table 2 are between two instructions containing WAR dependence. The true positives, second column, are illustrated as the number of actual dependent pairs. Most of the experimented kernels contain only one dependent pair of instructions with WAR dependence which is included as a C equations in the for-loop. However, the kernels trisolv, bicg and syr2k have two C statements with dependence in each one. Also, the kernels cholesky and symm include three dependencies between each pair of statements. Hence, our approach will provide the speculative system that each kernel contains dependent pair which cannot be parallelized. Moreover, false positives, third column numbers present low values in comparison with static false positives. Therefore, the missed opportunities of speculative parallelization will be lower and the accuracy will be higher than the static version. The parallelization decision would be relied on the current state as expected by our dependence analysis.

Our proposed technique have been examined in comparison with a traditional static analysis technique. This static method is based on computing the static AI intervals at compile-time as well as monitoring the definitions of registers would be used by the binary code. We have implemented a standard static AI technique using Padrone binary modification system. Padrone profiles and extracts the hot function, therefore the extracted code would be applied statically to compute data dependence. In each program point, the system checks mainly three parts. Firstly, the type of operation is checked whether it includes write, read,‥ etc according to the binary instruction type mov, add, sub,‥ etc. Secondly, the types of operators should be determined. Thirdly, the available values, if any, are inspected whether they are immediate or not. According to these three parts, the system could compute the abstract interval. Moreover, the system would use previous definitions at each program point. If there is no available values, the abstract intervals would be assumed as [-inf, inf].

The extracted hot function may include arguments. The static AI analysis is not able to detect the recent values for these arguments which affects the correctness. However, our technique would obtain the values of the arguments during runtime. These values are converted from concrete domain to abstract domain.

The impact of static AI technique is presented in terms of correctness which are shown in Table 2. the fourth and fifth columns include the true and false positives respectively. The false dependent pairs exist in all kernels with at least 1 or 2 pairs. In our method, the existence of false positives are limited to number of kernels most of cases is one pair except two kernels, doitgen and atax. Therefore, Table 2 proofs that our method correctness is higher than the applied static AI method.

For more clarification, the following example is supported to investigate the main difference in correctness between our method and static method.

The loop contains mainly three branches. First branch, S1, contains data dependence at S2 and S3. Second branch, S2 does not contain any data dependence. Last branch, S6 includes data dependence for S7 and S8. The branches rely mainly on the input of variables c1 and c2. One of these branches will be executed during all loop iterations. Static analysis would be applied on the loop for all branches which may be executed or not. If S4 is true, the other two branches would not be executed till the end of loop. Therefore, the loop would continue execution in parallel. However, static analysis would preserve the other two branches dependence. The speculator would resume the execution serially because of dependent program points which will never be executed. In our method, the analysis would detect that there is a branch that is executed at this run. Our dynamic analysis works for the hot trace in the hot function. Therefore, if S4 branch is taken, the analysis would detect that there is no dependence in the current run. Subsequently, if one of the other branches is taken, our analysis would detect only the dependence in the taken branch. The other branches would not be analyzed which increase our technique’s correctness.

The paper mainly achieves that we can implement a full speculative system which can detect the existence of dependence at runtime. This detection occurs at the same run with the speculative parallelization. The proposed approach can extract the dependence during the early iterations of the loops. The method will output a map of flags which will give a hint to the speculator whether the current loop contains any dependence or not. Our technique is implemented at the binary-level which would be applicable in all different architectures.

6 Conclusions and future work

This work proposes conducting dependence analysis at runtime for binary code using AI. During runtime, our system both interprets the seen hot execution code parts and constructs abstract equations to collect program semantics at all these parts’ program arcs. Also, the system uses data-flow analysis to gather reaching definitions. Therefore, the interpretation of the code iteratively solves the equations, generating the collective semantics upon reaching fixpoint in an efficient way. The analysis is valid for a hot sub-region. Speculation recovery can be triggered upon exiting the region from unsafe points (such as not the loop exit) if used with an underlying speculative system. We have implemented the method into Padrone binary modification system using interval widening to speedup analysis. The system is fully automatic, does not require re-execution, and is applied at runtime without any special hardware/software requirements. The results show accurately the dependent pairs. Moreover, the overhead ranges from 2% to 18%.

Future work includes extending the work to consider pointer arithmetic. The work involves also studying more complex kernels with much complicated operations as some of floating point operations and vector instructions. Moreover, our system would be provided with abstracting the dependence distance between iterations. Also, we need to study the function calls handling as well as the dependence analysis during the execution of the function code. The indirections in case of [-inf, inf] would be handled as a future work to correctly get the results. Furthermore, we plan to integrate the dependence analysis work into the sub-system of speculative parallelization environment. Therefore, we would study the full system behavior and investigate the approach validation. Speculative parallelization would highly reduce the per-computed overhead and accordingly speedup the processing.