A knowledge graph approach for recommending patents to companies

Abstract

Online platforms have emerged to facilitate patent transfer between academia and industry, but a recommendation method that matches patents with company needs is missing in the literature. Previous patent recommendation methods were designed mainly for query-driven patent search contexts, where user needs are given. However, company needs are implicit in the patent transfer context. The problem of profiling the needs and recommending patents accordingly remains unsolved. This research proposes a knowledge graph approach to address the problem. The proposed approach defines and constructs a patent knowledge graph to capture the semantic information between keywords in the patent domain. Then, it profiles patents and companies as weighted graphs based on the patent knowledge graph. Finally, it generates recommendations by comparing the weighted graphs based on the graph edit distance measure. During the recommendation process, three recommendation strategies (i.e., supplementary, complementary, and hybrid recommendation strategies) are proposed to profile different company needs and make recommendations accordingly. The proposed approach has been implemented and tested on a knowledge transfer platform in Jiangxi province, R.P. China. A pretest experiment shows that the proposed approach outperforms several baseline methods in terms of precision, recall, F-score, and mean average precision. User feedback from an online experiment further demonstrates the usability and the effectiveness of the proposed approach for recommending patents to companies.

Appendices

Appendix 1. Algorithm for computing the graph edit distance

Appendix 2. Algorithm for constructing the complementary profile

Appendix 3. Algorithm for learning the preference weights

Appendix 4. Introduction of the VSM-based recommendation method

The VSM-based recommendation method uses TF-IDF to extract keywords and their weights from documents. TF-IDF assumes that a document is represented by a collection of words and a word is important for the document if it repeatedly appears in the document but rarely appears in other documents. Therefore, TF-IDF is defined as follows given a document corpus:

$$TF{-}IDF_{ij} = tf_{ij} \times \log \left( {\frac{N}{{df_{i} + 1}}} \right)$$

(19)

where $tf_{ij}$ is the frequency of word $i$ appears in document $j$, $df_{i}$ is the number of documents that contain word $i$, and $N$ is the number of documents in the document corpus. Word $i$ is considered to be important for document $j$ only when $tf_{ij}$ is large and $df_{i}$ is small.

After profiling patents and companies as vectors of keyword weights, the cosine similarity between a target company $C$ and a candidate patent $P$ is calculated as below:

$$Sim_{VSM} \left( {C,P} \right) = \frac{{\mathop \sum \nolimits_{k = 1}^{n} TF{-}IDF_{k,C} \times TF{-}IDF_{k,P} }}{{\sqrt {\mathop \sum \nolimits_{k = 1}^{n} TF{-}IDF_{k,C}^{2} } \times \sqrt {\mathop \sum \nolimits_{k = 1}^{n} TF{-}IDF_{k,P}^{2} } }}$$

(20)

where $n$ is the number of keywords extracted from the corpus and $Sim_{VSM} \left( {C,P} \right)$ is the similarity between $C$ and $P$ based on the VSM-based recommendation method.

Appendix 5. Introduction of the LDA-based recommendation method

The basic assumption of LDA is that a document is generated by picking a set of topics and each topic is generated by picking a set of words. Therefore, a document is represented by the distribution of topics, and a topic is described by the distribution of words in LDA. The structure of the LDA model is presented in Fig. 

Fig. 8

The structure of the LDA model

8. Given a corpus consisting of $M$ documents each of length $N_{d}$, the topic distribution per document and the word distribution per topic are estimated according to the following generation process: First, choose a topic distribution $\theta_{d} \sim Dir\left( \alpha \right)$ for each document, where $d \in \left\{ {1, \ldots ,M} \right\}$ and $Dir\left( \alpha \right)$ is a Dirichlet distribution with the hyperparameter $\alpha$. Second, choose a word distribution $\varphi_{k} \sim Dir\left( \beta \right)$ for each topic, where $k \in \left\{ {1, \ldots ,K} \right\}$ is the $k$-th topic and $Dir\left( \beta \right)$ is a Dirichlet distribution with the hyperparameter $\beta$. Third, for each word position in the corpus, choose a topic $z_{d,l} \sim Multinomial\left( {\theta_{d} } \right)$ and a word $w_{d,l} \sim Multinomial\left( {\varphi_{{z_{d,l} }} } \right)$, where $d \in \left\{ {1, \ldots ,M} \right\}$, $l \in \left\{ {1, \ldots ,N_{d} } \right\}$, $Multinomial\left( {\theta_{d} } \right)$ is a multinomial distribution with the parameter $\theta_{d}$, and $Multinomial\left( {\varphi_{{z_{d,l} }} } \right)$ is the multinomial distribution with the parameter $\varphi_{{z_{d,l} }}$.

The parameters $\theta_{d}$ and $\varphi_{k}$ can be obtained by collapsed Gibbs sampling [56] and the number of latent topics $K$ is experimentally determined based on the perplexity-based approach [57]. The perplexity measure is defined as $perp\left( D \right) = exp\left( { - \frac{{\mathop \sum \nolimits_{d \in D} \ln P\left( {d{|}\theta ,\varphi } \right)}}{{\mathop \sum \nolimits_{d \in D} \left| d \right|}}} \right)$, where $D$ is a corpus, $d$ is a document in the corpus, and $P\left( {d{|}\theta ,\varphi } \right) = \mathop \prod \limits_{{w_{l} \in d}} \mathop \sum \limits_{k = 1}^{K} P\left( {w_{l} {|}z_{k} } \right) \cdot P\left( {z_{k} {|}d} \right)$. The smaller the perplexity value is, the better the model is trained.

After characterizing patents and companies as topic distributions, the similarity between a target company $C$ and a candidate patent $P$ is calculated based on the Jensen-Shannon divergence measure and is defined as follows:

$$Sim_{LDA} \left( {C,P} \right) = 1 - JSD\left( {\theta_{C} \parallel \theta_{P} } \right)$$

(21)

$$JSD\left( {\theta_{C} \parallel \theta_{P} } \right) = \frac{1}{2}\left[ {\mathop \sum \limits_{k = 1}^{K} \theta_{C,k} \log \frac{{\theta_{C,k} }}{{\theta_{C,P,k} }} + \mathop \sum \limits_{k = 1}^{K} \theta_{P,k} \log \frac{{\theta_{P,k} }}{{\theta_{C,P,k} }}} \right]$$

(22)

where $Sim_{LDA} \left( {C,P} \right)$ is the similarity between $C$ and $P$ based on the LDA-based recommendation method, $\theta_{C}$ is the topic distribution of $C$, $\theta_{P}$ is the topic distribution of $P$, $JSD\left( {\theta_{C} \parallel \theta_{P} } \right)$ denotes the Jensen-Shannon divergence between $C$ and $P$, $\theta_{C,P} = \frac{1}{2}\left( {\theta_{C} + \theta_{P} } \right)$ is the average of the two probability distributions, $K$ is the number of latent topics, and $\theta_{C,k}$ is the probability of the $k$-the topic in $C$.

Appendix 6. Introduction of the doc2vec-based recommendation method

Doc2vec is an unsupervised method that learns an embedding model from a corpus and then can be used to generate embedding vectors for new documents. It is an extension to word2vec, a word embedding technique that predicts a word given the other words in a context. Given a sequence of words $word_{1} ,word_{2} , \ldots ,word_{T}$, word2vec learns a word embedding model by maximizing the log probability $\frac{1}{T}\mathop \sum \limits_{t = 1}^{T - k} \log p\left( {word_{t} {|}word_{t - k} , \ldots ,word_{t + k} } \right)$, where $k$ is the number of surrounding words for a target word. Then the softmax function $p\left( {word_{t} {|}word_{t - k} , \ldots ,word_{t + k} } \right) = \frac{{e^{{y_{{word_{t} }} }} }}{{\mathop \sum \nolimits_{i} e^{{y_{i} }} }}$ is used to conduct the prediction task. In the softmax function, $y_{i}$ is the $i$-th output value of a feed-forward neural network and is calculated as follows: $y = b + Uh\left( {word_{t - k} , \ldots ,word_{t + k} ;WE} \right)$, where $U$ and $b$ are the softmax parameters, $h$ is the concatenation or average for context words, and $WE$ is the word embedding matrix. Doc2vec extends word2vec by considering a document as another word in the context when predicting words. The only difference between doc2vec and word2vec is the addition of a document embedding matrix $DE$ in $y$: $y = b + Uh\left( {word_{t - k} , \ldots ,word_{t + k} ;WE,DE} \right)$. Therefore, both documents and words are embedded in the same latent space and both embedding vectors contribute to the prediction task. The number of surrounding words and the dimension of the latent space are determined experimentally.

After training, the doc2vec model can be used to infer the embedding vectors for companies and patents. The similarity between a target company $C$ and a candidate patent $P$ is then calculated based on the cosine similarity shown below:

$$Sim_{doc2vec} \left( {C,P} \right) = \frac{{DE_{C} \cdot DE_{P} }}{{\|{DE_{C}} \|}\cdot{\| {DE_{P} }\|}}$$

(23)

where $DE_{C}$ is the embedding vector of $C$, $DE_{P}$ is the embedding vector of $P$, and $Sim_{doc2vec} \left( {C,P} \right)$ is the similarity between $C$ and $P$ based on the doc2vec-based recommendation method.