Information retrieval algorithm of industrial cluster based on vector space

2.1 Purification of industrial cluster information database

In order to improve the precision and recall ratio of information retrieval in industrial clusters, reduce the retrieval delay and energy consumption, the information database of industrial clusters needs to be processed. Noise removal module is indispensable[9]. In this paper, the improved VIPS algorithm is used to debase information blocks. Through a large number of statistics and analysis, noise semantic blocks are identified by using the number of text and links, the relative position of the page blocks and the content attributes of the page blocks. The origin coordinates of the database web page window are defined as the top left corner of the web page, the abscissa coordinate of the web page block center X is the abscissa coordinate of the center point of the web page block in the window, the ordinate coordinate of the web page block center is Y, the width of the web page is W, and the height of the web page is H. The spatial position of the web page is defined by the relative space position K of the web block, and the expression of K is:

Where u, d, l, r, and mid represent the top, bottom, left, right and middle positions of the database pages.

According to the definition of equation (1), VIPS algorithm is used to partition the web pages, and the rules are used to purify the web pages. The optimized sorting algorithm is as follows:

Input: database web page set P and the keyword set Q of industry cluster related information.

Output: a good set S_Pof database pages.

The detailed process is as follows:

Optimize the illegal labels in the database network.

Use VIPS algorithm to segment web pages.

Calculate the scores of key information related to industrial clusters in a web page:

Where S_j represents the score of web page j corresponding to relevant information keywords of industrial clusters, c_j represents the number of entries containing relevant information keywords of industrial clusters, t_ij represents the occurrence frequency of relevant information keywords of industrial clusters in web page j, f_i represents the frequency of inverted words in web pages of relevant information keyword i of industrial clusters. b represents the field parameter, and l_j represents the length of page j.

According to the equation (2), it can get a good set of database page S_P:

Where S represents the set of original database page. The result of equation (3) is the result of purifying industrial cluster information database.

2.2 Feature extraction of industry cluster information

According to the purification of industrial cluster information database in Section 2.1, RFD algorithm is used to extract the page data features of the purified industrial cluster information database.

Usually, if a feature item becomes a representative feature of a category, most samples of that category have this feature; if a feature item becomes a discriminant feature of a category, then most samples of other categories do not have this feature. In feature extraction, the representative and discriminant features should be selected as vector representations of a class [10, 11].

Supposing that p x ′ c ′ can approximate the ratio of the number of information containing feature item x̕ in training set category ć to the total number of information containing ć in training set, then p x ′ c ¯ ′ can approximate the ratio of the number of information not belonging to category ć and containing feature item x̕ to the total number of information not belonging to category ć in training set. The similarity measure of characteristic RFD (x̕, ć) can be expressed as:

Where A represents the number of training information belonging to category ^ć and containing characteristic item x̕. B represents the number of training information that does not fall into category ć and contains characteristic item x̕. C represents the number of training information data that belong to category ć and does not contain characteristic item x̕. D represents the number of training information that does not fall into category ć and does not contain characteristic item x̕. M represents the number of information belonging to category ć. N represents the total number of training data.

Since both N and N − M in the equation (4) are constants, the equation (4) can be simplified to:

In order to reduce the error of feature extraction, the equation (5) is improved.

Based on the above considerations, the calculated feature items of equation (6) have more classification discrimination ability, which is mainly to remove the information data features of industrial clusters which do not have the classification ability.

The main idea of improved feature extraction based on RFD is that for a feature to become a representative feature of a certain category, it must have the following two characteristics: representative and discriminant [12]. The absolute value of the sum of the representativeness measure of feature item x̕ and the discriminability measure of feature item x̕ are used to measure the correlation between features and categories, which is called ARFD. Conditional probability p x ′ c ′ is a representative measure of characteristic item , while − p x ′ c ¯ ′ is a discriminant measure of characteristic item x ⁰. The improved feature extraction is to calculate by using equation (7):

equation (7) can be approximated to:

Where the greater the value of ARFD(x̕, ć) is, the more the relevant information of feature item x̕ and class ć is.

2.3 Information retrieval algorithm based on vector space for industrial clusters

In order to improve the precision and recall ratio of information retrieval in industrial clusters, information retrieval is realized on the basis of information feature extraction of industrial clusters. The retrieval pattern of vector space is a relatively easy to understand retrieval pattern, and is a widely used information retrieval algorithm model in the field of information retrieval[13, 14]. The basic idea is that information and query are made up of words, and each query can be described by a vector composed of retrieval units. When searching, the correlation between information and query is calculated, and the higher the correlation with a specific query is considered the more relevant information.

The common way to describe information and retrieval vectors is that the retrieval space is composed of all retrieval units contained in information and retrieval, and the information and retrieval are represented as vectors in this space.

It is assumed that the information retrieval space of industrial clusters is = 〈ť₁ , ť₂ , . · · , ť_ń〉. Among them, ťí = (í = 1, 2, · · · , ń) is the different retrieval units con_itained in information and query, ^ń is the size of the whole retrieval space Ω, that is, the total number of different retrieval units contained in information query.

In retrieval space Ω, all information can be represented by vectors: d ′ ω d ′ 1 , ω d ′ 2 , ⋯ , ω d ′ n ′ . Among them, ω d ′ n ′ ( i ′ = 1 , 2 , ⋯ , n ′ ) is a series of descriptions of the information meaning, when the retrieval unit ť_í appears in the information type, ω d ′ n ′ is 1, conversely, when the retrieval unit t i ′ ′ does not appear in the information, ω_ďń is 0. Usually, most of the items in ω_ďń are zero because the size of search space is much larger than the length of each industry information file.

Combining the above information, we can approximately understand that in search space , all queries can also be represented by vectors: q = ω q 1 , ω q 2 , ⋯ , ω q n ′ . Among them, ω q n ′ = ( i ′ = 1 , 2 , ⋯ , n ′ ) is a series of descriptions of the query meaning, when retrieval unit ť_í appears in the query, ω q n ′ is 1, conversely, when retrieval unit t i ′ ′ does not appear in the query, ω_qń is 0. In general, because the length of queries is shorter than that of industrial clusters, more entries will be zero in ω_qń.

According to the above analysis, not every retrieval unit is equally important in information retrieval of industrial clusters (for example, keywords should be more important than non-keywords). So, how to embody such information in vectors needs to be solved urgently [15, 16, 17, 18, 19, 20, 21]. One of the feasible schemes is to adjust the weight of vectors manually, which enlarges the weight of retrieval units that users care about. However, manual intervention is difficult to achieve because of the huge workload. Therefore, another method is more commonly used in information retrieval: the weights based on the statistical frequency of the information file set, also known as TF-IDF weights.

TF-IDF weights consist of two parts, one is the frequency of the retrieval unit appearing in the information file, that is, TF, the other is called inverted file frequency, that is, IDF. TF-IDF weight is usually the product of TF and IDF for a given retrieval unit.

For the convenience of illustrating the problem, the following definition is made: T F i ′ j ′ represents the frequency of the retrieval unit ť_í appearing in the industrial cluster information database, DF_j represents the amount of information containing the retrieval unit ť_í in the entire industrial cluster information database.

By defining the above definition, the frequency of inversion information can be defined as:

Where, IDF_j represents the frequency of reversal information.

For a given information file, the vector describing the information file is composed of ń elements, which correspond to ^ń retrieval units in the information file set. The weights of each element are determined by the frequency of the corresponding retrieval unit appearing in the industrial cluster information database and the frequency of the retrieval unit appearing in the entire industrial cluster information database, as shown in Eq. (10):

Using as the weight of each element in the vector, the vector of information and retrieval is further adjusted. When the value range is [0.25, 0.30], the vector of information and retrieval can be adjusted best. This vector can describe the information and query more accurately.

For vector space retrieval model, it not only needs to define vectors to represent information and retrieval, but also needs to choose an appropriate method to calculate the relevance of information and query to determine whether information and query are related. The cosine of vector angle is used as the basis for judging the relevance of industrial cluster information.

According to the above, the similarity between information ď and retrieval q is defined in retrieval space . The retrieval matching process can be expressed as follows:

Where SC(ď, q) calculated by equation (11) is the result of information retrieval based on vector space.