CN116149917A - Method and device for evaluating processor performance, computing device, and readable storage medium - Google Patents

Abstract

The present application provides a method and device for evaluating the performance of a processor, a computing device, and a readable storage medium. The method includes: obtaining the execution time of multiple program segments in a benchmark test program; Execution time of the program segment, evaluating the performance of the processor. In the embodiment of the present application, the execution time of multiple program segments is considered in the process of evaluating the performance of the processor by using the simulation point tool, thereby helping to reduce errors in processor performance evaluation using the simulation point tool.

Description

Method and device for evaluating processor performance, computing device, and readable storage medium

technical field

The present application relates to the field of computer technology, and more specifically, to a method and device for evaluating processor performance, computing equipment, and a readable storage medium.

Background technique

The simulation point (simpoint) tool can take advantage of the fact that the program has repetitive behaviors that change over time during execution, by identifying these repetitive behaviors, and then taking a sample for each repetitive behavior as a representative to reduce the benchmark The execution time of the test program is used to solve the time-consuming problem of executing the benchmark test program multiple times in the processor performance evaluation process.

However, in practical applications, it is found that there is a large error between the execution time of the benchmark test program obtained based on the simulation point tool and the actual execution time of the complete benchmark test program.

Contents of the invention

The present application provides a method and device for evaluating processor performance, a computing device, and a readable storage medium. Various aspects involved in the embodiments of the present application are introduced below.

In the first aspect, a method for evaluating the performance of a processor is provided, the method comprising: obtaining the execution time of multiple program segments in a benchmark test program; calling a simulation point tool, and evaluating the processor performance.

As a possible implementation manner, the calling the simulation point tool, and evaluating the performance of the processor according to the execution time of the multiple program segments includes: calling the simulation point tool, based on the multiple program segments clustering the multiple program segments; evaluating the performance of the processor according to the clustering results of the multiple program segments.

As a possible implementation manner, the calling the simulation point tool to cluster the multiple program segments based on the execution time of the multiple program segments includes: according to the execution time of the multiple program segments time, divide the plurality of program segments into one or more program segment groups; call the simulation point tool, respectively collect the program segments included in each program segment group of the one or more program segment groups kind.

As a possible implementation manner, the dividing the multiple program segments into one or more program segment groups according to the execution time of the multiple program segments includes: according to the execution time of the multiple program segments, Sorting the plurality of program segments; dividing the plurality of program segments into one or more program segment groups based on the sorting results of the plurality of program segments.

As a possible implementation, the calling the simulation point tool to respectively cluster the program segments included in each program segment group of the one or more program segment groups includes: calling the simulation A point tool for clustering the program segments included in each of the one or more program segment groups based on the execution time of the plurality of program segment groups.

As a possible implementation, the invoking the simulation point tool and clustering the multiple program segments based on the execution time of the multiple program segments includes: constructing each of the multiple program segments The basic block vector of a program segment, the basic block vector includes the time feature value of the program segment, and the time feature value is used to indicate the execution time of the program segment; call the simulation point tool, based on the The basic block vector is used to cluster the multiple program segments.

As a possible implementation, the calling the simulation point tool, and evaluating the performance of the processor according to the execution time of the multiple program segments includes: calling the simulation point tool to obtain the multiple program segments The clustering results, the clustering results include one or more program segment sample representatives; based on the execution time of the plurality of program segments, determine the weight of the one or more program segment sample representatives; according to the one or more A number of program segment samples represent weights for evaluating the performance of the processor.

As a possible implementation, the acquiring the execution time of multiple program segments in the benchmark test program includes: acquiring the cache miss information of each program segment among the multiple program segments in the benchmark test program; based on the The cache miss information determines the execution time of a plurality of program segments in the benchmark test program.

As a possible implementation, each program segment of the plurality of program segments includes instructions corresponding to the cache miss information and other instructions, and the benchmark test program is determined based on the cache miss information The execution time of multiple program segments in the program includes: determining the number of instructions based on the product of the number of instructions corresponding to the cache miss information and the first execution time, and the product of the number of other instructions and the second execution time The execution time of a program segment, wherein the first execution time is greater than the second execution time.

As a possible implementation, the acquiring the cache miss information of each of the multiple program segments in the benchmark test program includes: based on the three-level cache model, counting the multiple programs in the benchmark test program Cache miss information for each program segment in the segment.

In a second aspect, there is provided a device for evaluating the performance of a processor, the device comprising: an acquisition module, configured to acquire the execution time of multiple program segments in a benchmark test program; an evaluation module, configured to call a simulation point tool, according to the multiple The execution time of a program segment is used to evaluate the performance of the processor.

As a possible implementation, the evaluation module is configured to: call the simulation point tool, and cluster the multiple program segments based on the execution time of the multiple program segments; The clustering results of the segments evaluate the performance of the processor.

As a possible implementation manner, the calling the simulation point tool to cluster the multiple program segments based on the execution time of the multiple program segments includes: according to the execution time of the multiple program segments time, divide the plurality of program segments into one or more program segment groups; call the simulation point tool, respectively collect the program segments included in each program segment group of the one or more program segment groups kind.

As a possible implementation manner, the dividing the multiple program segments into one or more program segment groups according to the execution time of the multiple program segments includes: according to the execution time of the multiple program segments, Sorting the plurality of program segments; dividing the plurality of program segments into one or more program segment groups based on the sorting results of the plurality of program segments.

As a possible implementation, the calling the simulation point tool to respectively cluster the program segments included in each program segment group of the one or more program segment groups includes: calling the simulation A point tool for clustering the program segments included in each of the one or more program segment groups based on the execution time of the plurality of program segment groups.

As a possible implementation, the invoking the simulation point tool and clustering the multiple program segments based on the execution time of the multiple program segments includes: constructing each of the multiple program segments The basic block vector of a program segment, the basic block vector includes the time feature value of the program segment, and the time feature value is used to indicate the execution time of the program segment; call the simulation point tool, based on the The basic block vector is used to cluster the multiple program segments.

As a possible implementation, the evaluation module is used to: call the simulation point tool to obtain the clustering results of the multiple program segments, the clustering results include one or more program segment sample representatives; based on The execution time of the plurality of program segments determines the weights represented by the one or more program segment samples; and evaluates the performance of the processor according to the weights represented by the one or more program segment samples.

As a possible implementation, the acquiring the execution time of multiple program segments in the benchmark test program includes: acquiring the cache miss information of each program segment among the multiple program segments in the benchmark test program; based on the The cache miss information determines the execution time of a plurality of program segments in the benchmark test program.

As a possible implementation, each program segment of the plurality of program segments includes instructions corresponding to the cache miss information and other instructions, and the benchmark test program is determined based on the cache miss information The execution time of multiple program segments in the program includes: determining the number of instructions based on the product of the number of instructions corresponding to the cache miss information and the first execution time, and the product of the number of other instructions and the second execution time The execution time of a program segment, wherein the first execution time is greater than the second execution time.

As a possible implementation, the acquiring the cache miss information of each of the multiple program segments in the benchmark test program includes: based on the three-level cache model, counting the multiple programs in the benchmark test program Cache miss information for each program segment in the segment.

In a third aspect, a computing device is provided, including: a memory, used to store codes; a processor, used to execute the codes stored in the memory, to perform any one of the possible operations in the first aspect or the first aspect Implement the method described in the manner.

In a fourth aspect, a computer-readable storage medium is provided, on which codes for executing the method described in the first aspect or any possible implementation manner of the first aspect are stored.

A fifth aspect provides computer program code, including instructions for executing the method described in the first aspect or any possible implementation manner of the first aspect.

The embodiment of the present application considers the execution time of multiple program segments in the process of evaluating the performance of the processor using the simulation point tool, thereby helping to reduce errors in processor performance evaluation using the simulation point tool.

Description of drawings

FIG. 1 is a schematic flowchart of a method for evaluating processor performance provided by an embodiment of the present application.

FIG. 2 is a schematic flowchart of another method for estimating processor performance provided by an embodiment of the present application.

FIG. 3 is a schematic structural diagram of an apparatus for evaluating processor performance provided by an embodiment of the present application.

FIG. 4 is a schematic structural diagram of a computing device provided by an embodiment of the present application.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some, not all, embodiments of the application.

In the design process of modern computer architecture, it is often necessary to evaluate the performance of different architectures in order to improve the performance of the processor.

As an implementation, the performance and correctness of the processor design can be predicted by simulating the execution of a benchmark. For example, the benchmark test program may be the benchmark test program provided in the CPU-intensive benchmark test suite of the Standard Performance Evaluation Corporation (SPEC), referred to as the SPEC CPU program.

In order to understand the periodic behavior of the processor during the execution of the application, it is usually necessary to use a simulator to simulate each cycle in detail, but unfortunately, it takes weeks or even months to fully execute the SPEC CPU program, even in the fastest The same is true on the simulator. To make matters worse, the architectural research process often requires simulating each benchmark on different architectural configurations and designs to find a balance between performance, complexity, area, and power consumption. For example, in order to find out the impact of the cache size on the performance of the architecture, it may be necessary to execute the same application hundreds or thousands of times, and the time spent will be unacceptable for product design.

The simulation point tool is an important tool for architecture research. It can use program analysis and machine learning tools to solve the above-mentioned problem that the simulation time of the architecture is too long. Simpoint can take advantage of the feature that the program has repetitive behaviors that change over time during execution, by identifying these repetitive behaviors, and then taking a sample for each repetitive behavior as a representative, thereby reducing the execution time of the program. That is to say, in the process of executing the benchmark test program, multiple programs with repetitive behavior in the benchmark test program are replaced by the sample representatives selected by simpoint, thereby reducing the execution time of the program.

In order to facilitate understanding, the process and related concepts of using the simpoint tool for architecture simulation are briefly introduced below.

Before adopting the simpoint tool, the instruction stream of the benchmark test program is usually segmented in advance, such as equally divided into segments, to obtain multiple program segments (intervals).

Each program segment may include one or more basic blocks. A basic block refers to a program with only one entry and one exit. A typical feature of a basic block is that as long as the first instruction in the basic block is executed Then all instructions in the basic block will be executed only once in order.

The basic block vector can indicate the information of the basic block in the program segment, wherein, the number of elements included in the basic block vector can be the number of basic blocks in the program segment, and the value of each element can be during the process of running the program segment, The product of the number of executions of each basic block and the number of instructions contained in the basic block.

If the value of each basic block in the basic block vector of two program segments is the same, then running the two program segments is statistically the same as running one of the program segments twice. Based on this principle, the simulation point tool uses the basic block vector as a feature to cluster multiple program segments contained in the instruction stream of the benchmark test program. If the classic K-means clustering algorithm (K-means) is used, many Small simulation points, which are sample representations of program segments. In addition, in some cases, program segments that have not been clustered can be aligned to close cluster centers.

After clustering, the simpoint tool generally outputs two files: a simulation point file and a weights file. The number of the selected interval (that is, the sample representative of the program segment) is saved in the simulation point file, and the weights of the program segment representative are included in the weights. Based on the weight file output by the simpoint tool and the running time represented by the program segment samples, the running time of the benchmark program can be calculated to evaluate the performance of the processor.

However, in the process of practical application, we found that the program segment sample selected by simpoint represents a large difference between the execution time simulated on the field programmable gate array (field programmable gate array, FPGA) and the actual execution time of the complete program. Errors, especially when there are a large number of load/store instructions in the program, such as the gcc.403_2 benchmark test program of Spec2006, the error rate is as high as 21.17%. This kind of error is unacceptable in our actual design. Through analysis, it can be concluded that the reasons for this kind of error may include the following two points.

On the one hand, the reason is that, as mentioned above, the simpoint tool constructs the basic block vector by multiplying the number of code executions in each basic block by the number of codes, that is, the number of instructions in the instruction stream in each basic block as a feature. This method regards the time delay (i.e. execution time) of each instruction as the same (i.e. the time delay of all instructions is one beat), but this is not in line with the actual architecture knowledge and is not in line with the actual situation. .

On the other hand, the reason is that simpoint clustering only considers the logical repetition of the program, although the number of instructions in each program segment is actually the same (in actual use, the instruction stream of the benchmark test program is usually divided into multiple programs section), but ignores that the execution time of different instructions is not the same, and the actual execution time of different program sections is very different, especially when the memory access instruction load/store misses the cache, the time delay is 100 to 100% of other instructions. 400 times, this omission is unacceptable when using simulation point results to estimate performance.

In order to solve the above-mentioned problems, the embodiment of the present application provides a method for evaluating the performance of a processor, which helps to reduce the Error in processor performance evaluation using the analog point tool.

FIG. 1 is a schematic flowchart of a method for evaluating processor performance provided by an embodiment of the present application.

Referring to Fig. 1, the method 100 includes step S110 and step S120.

In step S110, the execution time of multiple program segments in the benchmark test program is acquired.

In order to identify repetitive behaviors in the benchmarking program, the benchmarking program can be divided into multiple units first, for example, the program flow of the benchmarking program is divided into multiple program segments. As an implementation, the instruction stream of the benchmark test program can be segmented according to a fixed length, such as 100M instructions, to obtain multiple program segments.

In actual use, run the binary program of the benchmark test program through the emulator, and then use the pin tool to trace (trace) the instruction flow of the entire benchmark test program, and then divide the instruction stream into fixed sized fragments, to obtain multiple program segments.

There are many methods for obtaining the execution time of multiple program segments. For example, you can run multiple program segments and count the execution time of each program segment.

In order to reduce the complexity of obtaining the execution time of multiple program segments, the execution time of multiple program segments can also be estimated. Through theoretical analysis and experimental verification, it can be found that the execution time of most instructions is one clock cycle (cycle), and when executing memory access instructions, the execution time is mostly longer than one clock cycle, especially in the three-level cache (L3cache) In the case of a miss, it may take about 100 to 400 clock cycles to execute the memory access instruction, which is also the main reason for the error of the simpoint tool. Therefore, as an implementation manner, the execution time of each program segment can be estimated based on the cache miss information of the program segment. Further, since the error caused by the first-level cache miss and the second-level cache miss is small, in order to reduce the complexity of estimation, the execution of each program segment can be estimated based on the third-level cache miss information of the program segment time.

In step S120, the simulation point tool is invoked to evaluate the performance of the processor according to the execution time of multiple program segments.

Combined with the method of estimating the performance of the processor using the simulation point tool described above, the execution time of the program segment can be considered from multiple aspects, for example, multiple program segments can be clustered based on the execution time of multiple program segments, according to multiple The clustering results of a program segment to evaluate the performance of the processor; it is also possible to consider the execution time of multiple program segments in the process of estimating the execution time of the benchmark program according to the execution time represented by the program segment, that is, based on the execution time of all program segments. Describe the execution time of multiple program segments and the clustering results to evaluate the performance of the processor.

In the embodiment of the present application, by considering the execution time of multiple program segments in the process of evaluating the performance of the processor using the simulation point tool, the influence of different execution times of multiple program segments on the performance estimation result of the processor can be reduced, and the simulation point tool can be reduced. estimation error.

As an implementation, according to the execution time of multiple program segments, multiple program segments can be divided into one or more program segment groups, wherein the program segments included in each program segment group of one or more program segment groups The execution time of the program segment is similar; through the simulation point tool, the program segments included in each program segment group of one or more program segment groups are clustered; according to the clustering results of multiple program segments, the performance of the processor can be evaluated performance.

In the process of dividing multiple program segments into one or more program segment groups, the program segments can be sorted according to the execution time of multiple program segments, such as the numerical value of cache misses, and then the multiple program segments can be sorted according to the sorting. The blocks are divided into one or more block groups.

The similar execution time of the program segments in each program segment group may mean that the cache miss ratios of the program segments in each program segment group are similar. If the number of representatives of the program segment selected from the plurality of program segments is too large, the time for estimating the performance of the processor based on the benchmark test program cannot be effectively reduced. Therefore, in order to control the number of program segment representations, as an example, the difference in execution time of the program segments included in each program segment group of the one or more program segment groups is less than or equal to 5%.

Through the above method, the program segments with similar execution time can be grouped into a program segment group. In this way, when the simpoint tool is used to estimate the execution time of the benchmark test program in the program segment group, multiple programs can be excluded to a certain extent. The error caused by the different execution time of the program segment is helpful to reduce the error of estimating the performance of the processor by using the simpoint tool.

In order to further reduce estimation errors, program segments can be clustered within program segment groups based on their execution time.

In some embodiments, the simulation point tool is called to cluster multiple program segments based on the execution time of multiple program segments, and the time feature value of each program segment can be increased during the clustering process, wherein the time feature The value is used to indicate the execution time of the program segment, such as the estimation result of the execution time of the program segment mentioned above. As mentioned above, clustering can be performed based on the basic block vectors of the program segments. Therefore, the basic block vectors of each of the multiple program segments can be constructed, and the basic block vectors include the time feature values of the program segments.

For example, an element may be added to the basic block vector of the program segment, and the element is used to indicate the execution time of the program segment. Usually the basic block vector of the program segment is stored in the *.bb file. Therefore, the value dim_max of the maximum number of digits of the basic block vector can be obtained by traversing the *.bb file, and then the time characteristic is added to the basic block vector as the dim_max dimension.

According to the basic block vector including the time feature value, clustering is performed on multiple program segments, so that the clustering results of the program segments are associated with the execution time of the program segments, and the error of the processor performance estimation result can be reduced.

As mentioned above, the clustering results of multiple program segments can be obtained by calling the simulation point tool, and the clustering results can include one or more program segment sample representatives. As an implementation manner, weights represented by one or more program segment samples may be determined based on the execution times of multiple program segments, so as to evaluate the performance of the processor according to the weights.

In actual use, the clustering results of multiple program segments obtained by calling the simulation point tool may also include initial weights represented by one or more program segment samples. Since the initial weight represented by one or more program segment samples given by the simulation point tool cannot represent the weight of the real execution time of the program segment, one or more program segment samples can be obtained according to the execution time of multiple program segments Represents the updated weights, based on the initial weights and the updated weights, the performance of the processor can be evaluated. For example, weighted summation may be performed on the initial weight represented by the program segment sample and the updated weight, and the result may be used as the final weight to estimate the performance of the processor.

As an example, the update weight of a program segment sample representative can be determined according to the ratio of the sum of the execution times of the program segments in the cluster where the program segment sample representative is located to the sum of the execution times of all program segments of the benchmark program .

The accuracy of estimating processor performance can be further improved by determining the weight of program segment sample representatives based on execution time.

As mentioned above, according to the cache miss information of the program segment, the execution time of the program segment can be calculated. In actual use, the cache miss information may include one or more of information such as the number of cache misses (miss_num), the total number of memory accesses (visit_num), or the miss rate (miss_rate) in the program segment.

Based on the cache miss information, the number of cache miss instructions in the program segment can be determined, thereby calculating the execution time of the program segment. For example, the execution time of the program segment is determined based on the product of the number of cache miss instructions and the first execution time, and the product of the number of other instructions in the program segment and the second execution time. Wherein, the first execution time may be the time required to execute a cache miss instruction, and the second execution time may be the time required to execute another instruction, such as estimated according to one clock cycle. Usually, the execution time of the cache miss instruction is longer than the execution time of other instructions, that is to say, the first execution time is longer than the second execution time.

Take the cache miss information as the third-level cache miss information as an example. Under different simulation architectures, such as different FPGAs, the instruction execution time of the third-level cache miss, that is, the first execution time may be different, which can be based on the actual configuration. Circumstances determine the first execution time.

As an example, the first execution time may be 400 clock cycles, and the second execution time may be 1 clock cycle. Based on this, taking the instruction quantity of the program segment as 100M (100×1024×1024) as an example, the calculation formula of the execution time (interval_time) of the program segment can be obtained: interval_time=miss_num×400+(100×1024×1024-miss_num) ×1.

In order to obtain the cache miss information of the program segment, the script can be used to simulate the cache behavior in actual execution. In order to maintain the irrelevant characteristics of the simpoint architecture, a common cache model can be selected for simulation. Since the L3 cache model logically includes L1 cache and L2 cache, and in the case of a L3 cache miss, the time delay for executing instructions is the largest, so we can use the L3 cache model for the above simulation. A general three-level cache model is given below:

(cache_size, line_size, [line_num, way_num], index_method, replacement_method)

(2048,64,[2048,16],'TRA','PLRU')

Among them, cache_size is the cache size, line_size is the size of the cache line, line_num is the number of cache lines, way_num is the number of group association ways, index_method is the index method, and replacement_method is the replacement algorithm.

In the above model, the size of the cache is 2048, the size of the cache line is 64, the number of group connection ways is 16, the indexing method is TRA, and the replacement algorithm is PLRU algorithm.

In order to simulate the cache behavior of the program segment during execution, it is also necessary to obtain the memory access information of the program segment. For example, the pin tool can be used to trace the instruction flow of the entire benchmark test program, record the PC information of the jump instruction, and obtain the memory access file of the program segment. Information such as PC value, virtual address, value recorded in memory, memory length, instruction type (such as access instruction, storage instruction) can be recorded in the access file.

Using the information in the memory access file of the program segment, the replacement process of the instruction in the cache is simulated in the cache model, so as to count the number of misses, the total number of memory accesses, and the ratio of memory access misses in each program segment.

It should be noted that the methods for reducing the simpoint estimation error mentioned above can be used alone or in combination.

FIG. 2 is a schematic flowchart of another method for estimating processor performance provided by an embodiment of the present application. The method for estimating the operating performance of the processor will be described in detail below with reference to FIG. 2 .

The method 200 can help improve the accuracy of the simpoint by introducing the execution time of the program segment, in the scenario where the simpoint tool is used to shorten the binary program execution time of the SPEC CPU benchmark test suite.

Referring to FIG. 2 , the method 200 may include steps S210 to S280.

In step S210, a benchmark test program is executed.

Before using the simpoint tool to optimize the processor performance estimation method, a benchmark test program is usually executed first, such as running the binary program of the SPEC CPU benchmark test program on the emulator to obtain relevant information of the benchmark test program.

In order to facilitate the use of the simpoint tool, in the process of executing the benchmark test program, the pin tool can be used to trace the instruction stream of the entire benchmark test program, and according to the fixed instruction stream size, the instruction stream is divided into fixed-size fragments, which is the previous article Multiple program segments mentioned.

In step S220, the access file is obtained.

In order to obtain the cache miss information of the program segment, it is first necessary to obtain the memory access information of each program segment, such as the memory access file mem_interval. As an implementation, in the process of using the pin tool to trace the instruction flow of the benchmark test program, the PC information of each actual jump instruction can be recorded to obtain the access file. Information such as PC value, virtual address, value recorded in memory, memory length, instruction type (such as access instruction, storage instruction) can be recorded in the access file.

In step S230, a program segment file is obtained.

In order to construct the basic block vector of the program segment, it is first necessary to obtain the program segment file of the program segment. As an implementation, in the process of tracing the instruction flow of the benchmark test program, information such as the type of the jump instruction, the source address of the jump instruction, and the destination address of the jump instruction can be recorded to form a program segment file.

In step S240, the time characteristic data of the program segment is obtained.

As mentioned above, according to the cache miss information of the program segment, the time characteristic data of the program segment can be obtained. As an implementation, based on the memory access information of the program segment, the time characteristic data of the program segment can be obtained by simulating the cache behavior of the program segment during actual execution through the cache model. Wherein, the access information can be obtained through the access file of the program segment in step S220.

In order to maintain the irrelevant characteristics of the simpoint architecture, a common cache model can be selected for simulation. Since the L3 cache model logically includes L1 cache and L2 cache, and in the case of a L3 cache miss, the time delay for executing instructions is the largest, so we can use the L3 cache model for the above simulation.

A general three-level cache model was given above, and for the sake of brevity, details will not be repeated here.

Use the information in the access file of the program segment to simulate the replacement process of instructions in the cache in the cache model, so that the number of misses, the total number of memory accesses, and the ratio of memory access misses in each program segment can be counted. Cache miss information.

The execution time of the cache miss instruction can be flexibly determined according to the actual situation, for example, it can be determined according to the configuration of the adopted FPGA. Considering the worst case of the configuration, when the L3 cache misses, the execution time of the instruction (that is, the first execution time mentioned above) may be 400 clock cycles.

Taking the first execution time as 400 clock cycles, the execution time of other instructions in the program segment except for the L3 cache miss instruction is 1 clock cycle as an example, the time characteristic data of the program segment, such as the execution time of the program segment (interval_time) can be obtained by the following formula: interval_time=miss_num×400+(100×1024×1024−miss_num)×1.

In some embodiments, the execution time of a program segment may be used to determine a weight file for a program segment.

In step S250, a *.bb file is generated.

First, get the initial basic block vector. Usually, the bbv_build tool can be used to traverse the entire jump instruction flow and complete the entire dynamic instruction flow of the benchmark test program. At the same time, the control flow graph (control flow graph, cfg) can be used to obtain all basic block information (such as the index of the basic block and size). According to these information, the initial basic block vector of the program segment can be obtained, and the initial basic block vector can be generated as a *.bb file.

Secondly, by adding the time characteristic data in the initial basic block vector, the *.bb file including the time characteristic is obtained. For example, the *.bb file of the initial basic block vector of the program segment may be traversed to obtain the value dim_max of the maximum number of bits of the basic block vector of the program segment. Then, add the time characteristic data of the program segment as the dim_maxth dimension of each basic block vector to the last dimension of the basic block vector as an additional feature value.

Finally, build a new *.bb file according to the grouping information of the program segment.

In order to further reduce the estimation error of simpoint, before clustering multiple program segments, the program segments can be divided into one or more program segment groups. For example, the program segments may be sorted according to the numerical value of cache_miss, and the sorted program segments are divided into one or more large parts (parts), that is, one or more program segment groups mentioned above.

If the number of program segment representatives selected from multiple program segments (that is, the selected checkpoint image) is too large, the time for estimating processor performance based on the benchmark test program cannot be effectively reduced. Therefore, according to the experimental results, when grouping multiple program segments, the difference of cache_miss in each program segment group can be controlled within 5%.

According to the division result of one or more program segment groups, a new *.bb file can be reconstructed. For example, the *.bb files of multiple program segments are written into the *.bb files of program segment groups according to the grouping of program segments, that is to say, the number of final *.bb files is the same as the number of program segment groups.

In step S260, the simulation point file and the weight file are obtained.

By calling the simpoint tool, cluster and sample the *.bb files of each program segment group to realize the clustering of multiple program segments within the program segment group. The sampling parameters can include "-saveSimpoints", "-saveSimpointWeights", "-saveLabels" and "-maxK". Among them, "-saveSimpoints" is the number of the program segment sample representative of the cluster center of the cluster sampling, "-saveSimpointWeights" is the weight of the program segment sample representative, and "-saveLabels" is the cluster to which each program segment belongs The number of the center, that is, to obtain which program segments are clustered in the same cluster, "-maxK" is to set the maximum number of cluster centers.

If the K-means algorithm is used to sample multiple program segments, the K value (the number of cluster centers is within the K value, that is, the number of cluster centers is less than or equal to the K value) defaults to 10, and the K value can also be based on your own needs. Adjust with the actual situation.

Based on the above sampling results, the simulation point file (*.simpoints file), weight file (*.weights file) and *.lables file of each block group can be obtained. That is to say, the program segment sample representatives in each program segment group and the weight of the program segment sample representatives in the program segment group can be obtained.

However, since the above sampling is carried out inside the *.bb file corresponding to each block group, that is to say, the number in the sampling result is the number inside the block group, therefore, it is necessary to map the number in the above sampling result to For the global number, get a new *.simpoints file.

Considering the time characteristic of the program segment, the weight directly generated by the simpoint tool cannot represent the real execution time weight of the program segment, therefore, the weight can be recalculated based on the execution time of the program segment obtained in step S240. Taking m program segments in a program segment group as an example, the simpoint tool can select n cluster centers (corresponding to n cluster clusters) from the program segment group, and each cluster center represents the program A cluster of clusters within a segment group. Among them, the time characteristic data of each program segment is ti (i=1,2,...,m), if the i-th cluster has xi program segments, each program in the i-th cluster The time feature of a segment is tj (j=1,2,...,xi), then the weights _i of the i-th cluster center point can be calculated by the following formula.

In order to further improve the accuracy of the processor performance estimation results, the initial weight represented by the program segment sample (that is, the weight directly given by the simpoint tool) and the update weight (that is, the above-mentioned recalculated weight) can be weighted and summed, and the The result is used as the final weight to estimate the performance of the processor.

In step S270, data preparation.

In order to improve the performance of the simpoint tool, one or more of the following data can be prepared before the program segment sample representative (that is, the program segment corresponding to the cluster center) runs: checkpoint (checkpoint) image file, the last 1000 items of the program segment Instructions, 5M warm-up data. Wherein, the number of instructions can be flexibly set according to actual conditions.

The checkpoint image file is the specific content of the register, CPU and other hardware at the starting point of the program segment executed on the FPGA. Based on the checkpoint image file, the sample representative of the program segment can be quickly run during the verification process.

In step S280, FPGA verification.

According to the checkpoint image file, the program segment sample representative is executed on the FPGA, and the execution time of each program segment is recorded. Further, according to the recording result and the final weight obtained in step S260, the execution time of the complete benchmark test program can be estimated. Processor performance can be evaluated based on the execution time of the complete benchmark program.

In order to verify the correctness of the checkpoint image file, the last 1000 instructions of each program segment can be obtained, and the FPGA also obtains 1000 instructions. The two sides compare to ensure the correctness of our execution.

In order to improve the accuracy of the program segment execution time estimation results, the first 5M instructions of the program segment can be obtained for warm-up, so that the overall hardware is in a normal execution state. Experiments show that this method can improve the evaluation effect very well.

In order to verify the accuracy of the method for estimating the operating performance of the processor provided by the embodiment of the present application, a complete benchmark test program can be executed on the FPGA to obtain the complete benchmark test program in the case of bare metal and the case of adding an operating system. execution time. Through experimental verification, directly adopting the weight given by simpoint, the estimated runtime of the benchmark test program is closer to the execution time of the bare metal, and the estimation result of the weight based on the execution time of the program segment provided by the embodiment of the application is closer to that in the existing The execution time in the case of the operating system, that is, by using the method provided in the embodiment of the present application, the estimation result of the processor performance is closer to the actual usage situation.

According to the experimental results, using the processor performance evaluation method in the embodiment of this application, the error of the simpoint tool estimation time under SPEC CPU2006 can be reduced from 21% to less than 3%, and the effect is good.

The method embodiment of the present application is described in detail above with reference to FIG. 1 to FIG. 2 , and the device embodiment of the present application is described in detail below in conjunction with FIG. 3 to FIG. 4 . It should be understood that the descriptions of the device embodiments correspond to the descriptions of the method embodiments, therefore, for parts that are not described in detail, reference may be made to the foregoing method embodiments.

FIG. 3 is a schematic structural diagram of an apparatus for evaluating processor performance provided by an embodiment of the present application. The apparatus 300 includes an acquisition module 310 and an evaluation module 320 .

An acquisition module 310, configured to acquire the execution time of multiple program segments in the benchmark test program;

The evaluation module 320 is configured to call a simulation point tool to evaluate the performance of the processor according to the execution time of the multiple program segments.

Optionally, the evaluation module is configured to: call the simulation point tool, and cluster the multiple program segments based on the execution time of the multiple program segments; according to the clustering of the multiple program segments As a result, the performance of the processor is evaluated.

Optionally, the calling the simulation point tool, and clustering the multiple program segments based on the execution time of the multiple program segments includes: clustering the multiple program segments according to the execution time of the multiple program segments The multiple program segments are divided into one or more program segment groups; the simulation point tool is called to cluster the program segments included in each program segment group of the one or more program segment groups.

Optionally, the dividing the plurality of program segments into one or more program segment groups according to the execution time of the plurality of program segments includes: The program segments are sorted; based on the sorting results of the multiple program segments, the multiple program segments are divided into one or more program segment groups.

Optionally, calling the simulation point tool to cluster the program segments included in each program segment group of the one or more program segment groups includes: calling the simulation point tool based on The execution time of the plurality of program segments clusters the program segments included in each program segment group of the one or more program segment groups respectively.

Optionally, the invoking the simulation point tool, and clustering the multiple program segments based on the execution time of the multiple program segments includes: constructing the A basic block vector, the basic block vector includes the time feature value of the program segment, the time feature value is used to indicate the execution time of the program segment; calling the simulation point tool, based on the basic block vector, clustering the plurality of program segments.

Optionally, the evaluation module is configured to: call the simulation point tool to obtain the clustering results of the multiple program segments, the clustering results include one or more program segment sample representatives; based on the multiple The execution time of the program segment determines the weight represented by the one or more program segment samples; and evaluates the performance of the processor according to the weight represented by the one or more program segment samples.

Optionally, the acquiring the execution time of multiple program segments in the benchmark test program includes: acquiring cache miss information of each program segment among the multiple program segments in the benchmark test program; based on the cache miss information , to determine execution times of multiple program segments in the benchmark test program.

Optionally, each program segment of the plurality of program segments includes instructions corresponding to the cache miss information and other instructions, and based on the cache miss information, determine the number of programs in the benchmark test program The execution time of the segment includes: based on the product of the number of instructions corresponding to the cache miss information and the first execution time, and the product of the number of other instructions and the second execution time, determining the time of the plurality of program segments execution time, wherein the first execution time is greater than the second execution time.

Optionally, the acquiring the cache miss information of each of the multiple program segments in the benchmark test program includes: based on the three-level cache model, counting each of the multiple program segments in the benchmark test program Cache miss information for the program segment.

FIG. 4 is a schematic structural diagram of a computing device provided by an embodiment of the present application. The computing device 400 shown in FIG. 4 may include a memory 410 and a processor 420 . In some embodiments, the computing device 400 shown in FIG. 4 may further include an input/output interface 430 . The memory 410 is used to store instructions, and the processor 420 is used to execute the instructions stored in the memory 410 to execute the method described in any of the foregoing embodiments.

It should be understood that, in the embodiment of the present application, the processor 420 may adopt a general-purpose central processing unit (central processing unit, CPU), a microprocessor, an application specific integrated circuit (application specific integrated circuit, ASIC), or one or more integrated The circuit is used to execute related programs, so as to realize the technical solutions provided by the embodiments of the present application.

The memory 410 may include read-only memory and random-access memory, and provides instructions and data to the processor 420 . A portion of processor 420 may also include non-volatile random access memory. For example, processor 420 may also store device type information.

In the implementation process, each step of the above method may be implemented by an integrated logic circuit of hardware in the processor 420 or instructions in the form of software. The methods disclosed in the embodiments of the present application may be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register. The storage medium is located in the memory 410, and the processor 420 reads the information in the memory 410, and completes the steps of the above method in combination with its hardware. To avoid repetition, no detailed description is given here.

It should be understood that in the embodiment of the present application, the processor may be a central processing unit (central processing unit, CPU), and the processor may also be other general processors, digital signal processors (digital signal processor, DSP), application specific integrated circuits ( application specific integrated circuit, ASIC), FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

An embodiment of the present application further provides a computer-readable storage medium for storing a program, so that a computer executes the method described in any of the foregoing embodiments.

It should be understood that in this embodiment of the present application, "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B may also be determined according to A and/or other information.

It should be understood that the term "and/or" in this article is only an association relationship describing associated objects, which means that there may be three relationships, for example, A and/or B may mean: A exists alone, and A and B exist at the same time , there are three cases of B alone. In addition, the character "/" in this article generally indicates that the contextual objects are an "or" relationship.

It should be understood that, in various embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the order of execution, and the execution order of the processes should be determined by their functions and internal logic, and should not be used in the embodiments of the present application. The implementation process constitutes any limitation.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be read by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital versatile disc (digital video disc, DVD)) or a semiconductor medium (for example, a solid state disk (solid state disk, SSD) )wait.

The above is only a specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the application. Should be covered within the protection scope of this application. Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

Claims (22)

1. A method of evaluating performance of a processor, comprising:

acquiring execution time of a plurality of program segments in a benchmark test program;

and calling a simulation point tool, and evaluating the performance of the processor according to the execution time of the program segments.

2. The method of claim 1, wherein the invoking the simulation site tool evaluates performance of the processor based on execution time of the plurality of program segments, comprising:

invoking the simulation point tool, and clustering the program segments based on the execution time of the program segments;

and evaluating the performance of the processor according to the clustering result of the program segments.

3. The method of claim 2, wherein the invoking the simulated point tool to cluster the plurality of program segments based on execution times of the plurality of program segments comprises:

Dividing the plurality of program segments into one or more program segment groups according to the execution time of the plurality of program segments;

and calling the simulation point tool to cluster the program segments included in each program segment group of the one or more program segment groups respectively.

4. A method according to claim 3, wherein said dividing the plurality of program segments into one or more program segment groups according to execution times of the plurality of program segments comprises:

sequencing the program segments according to the execution time of the program segments;

the plurality of program segments are divided into one or more program segment groups based on a result of the ordering of the plurality of program segments.

5. A method according to claim 3, wherein said invoking the simulated point tool to separately cluster the program segments included in each of the one or more program segment groups comprises:

and calling the simulation point tool, and clustering the program segments included in each program segment group of the one or more program segment groups based on the execution time of the program segments.

6. The method of claim 2, wherein the invoking the simulated point tool to cluster the plurality of program segments based on execution times of the plurality of program segments comprises:

Constructing a basic block vector of each program segment in the plurality of program segments, wherein the basic block vector comprises a time characteristic value of the program segment, and the time characteristic value is used for indicating the execution time of the program segment;

and calling the simulation point tool, and clustering the program segments based on the basic block vector.

7. The method of claim 1, wherein the invoking the simulation site tool evaluates performance of the processor based on execution time of the plurality of program segments, comprising:

invoking the simulation point tool to obtain a clustering result of the program segments, wherein the clustering result comprises one or more program segment sample representations;

determining weights represented by the one or more program segment samples based on execution times of the plurality of program segments;

the performance of the processor is evaluated based on the weights represented by the one or more program segment samples.

8. The method of claim 1, wherein the obtaining the execution time of the plurality of program segments in the benchmark program comprises:

obtaining cache miss information of each program segment in a plurality of program segments in the benchmark test program;

And determining the execution time of a plurality of program segments in the benchmark test program based on the cache miss information.

9. The method of claim 8, wherein each of the plurality of program segments includes instructions corresponding to the cache miss information and other instructions therein, wherein determining execution time of the plurality of program segments in the benchmark test program based on the cache miss information comprises:

and determining the execution time of the program segments based on the product of the number of the instructions corresponding to the cache miss information and the first execution time and the product of the number of the other instructions and the second execution time, wherein the first execution time is larger than the second execution time.

10. The method of claim 8, wherein the obtaining cache miss information for each of the plurality of segments in the benchmark program comprises:

and based on the three-level cache model, counting the cache miss information of each program segment in the program segments in the benchmark test program.

11. An apparatus for evaluating performance of a processor, comprising:

the acquisition module is used for acquiring the execution time of a plurality of program segments in the benchmark test program;

And the evaluation module is used for calling a simulation point tool and evaluating the performance of the processor according to the execution time of the program segments.

12. The apparatus of claim 11, wherein the evaluation module is to:

invoking the simulation point tool, and clustering the program segments based on the execution time of the program segments;

and evaluating the performance of the processor according to the clustering result of the program segments.

13. The apparatus of claim 12, wherein the invoking the simulated point tool to cluster the plurality of program segments based on execution times of the plurality of program segments comprises:

dividing the plurality of program segments into one or more program segment groups according to the execution time of the plurality of program segments;

and calling the simulation point tool to cluster the program segments included in each program segment group of the one or more program segment groups respectively.

14. The apparatus of claim 13, wherein the dividing the plurality of program segments into one or more program segment groups according to execution times of the plurality of program segments comprises:

sequencing the program segments according to the execution time of the program segments;

The plurality of program segments are divided into one or more program segment groups based on a result of the ordering of the plurality of program segments.

15. The apparatus of claim 13, wherein the invoking the simulated point tool to separately cluster the program segments included in each of the one or more program segment groups comprises:

and calling the simulation point tool, and clustering the program segments included in each program segment group of the one or more program segment groups based on the execution time of the program segments.

16. The apparatus of claim 12, wherein the invoking the simulated point tool to cluster the plurality of program segments based on execution times of the plurality of program segments comprises:

constructing a basic block vector of each program segment in the plurality of program segments, wherein the basic block vector comprises a time characteristic value of the program segment, and the time characteristic value is used for indicating the execution time of the program segment;

and calling the simulation point tool, and clustering the program segments based on the basic block vector.

17. The apparatus of claim 11, wherein the evaluation module is to:

Invoking the simulation point tool to obtain a clustering result of the program segments, wherein the clustering result comprises one or more program segment sample representations;

determining weights represented by the one or more program segment samples based on execution times of the plurality of program segments;

the performance of the processor is evaluated based on the weights represented by the one or more program segment samples.

18. The apparatus of claim 11, wherein the obtaining the execution time of the plurality of program segments in the benchmark program comprises:

obtaining cache miss information of each program segment in a plurality of program segments in the benchmark test program;

and determining the execution time of a plurality of program segments in the benchmark test program based on the cache miss information.

19. The apparatus of claim 18, wherein each of the plurality of program segments includes instructions corresponding to the cache miss information and other instructions therein, wherein the determining the execution time of the plurality of program segments in the benchmark test program based on the cache miss information comprises:

and determining the execution time of the program segments based on the product of the number of the instructions corresponding to the cache miss information and the first execution time and the product of the number of the other instructions and the second execution time, wherein the first execution time is larger than the second execution time.

20. The apparatus of claim 18, wherein the obtaining cache miss information for each of a plurality of program segments in the benchmark program comprises:

and based on the three-level cache model, counting the cache miss information of each program segment in the program segments in the benchmark test program.

21. A computing device, comprising:

a memory for storing codes;

a processor for executing code stored in the memory to perform the method of any one of claims 1-10.

22. A computer readable storage medium having stored thereon code for performing the method of any of claims 1-10.

Images