Privacy Preserving Multi-server k-means Computation over Horizontally Partitioned Data

Abstract

The k-means clustering is one of the most popular clustering algorithms in data mining. Recently a lot of research has been concentrated on the algorithm when the data-set is divided into multiple parties or when the data-set is too large to be handled by the data owner. In the latter case, usually some servers are hired to perform the task of clustering. The data set is divided by the data owner among the servers who together compute the k-means and return the cluster labels to the owner. The major challenge in this method is to prevent the servers from gaining substantial information about the actual data of the owner. Several algorithms have been designed in the past that provide cryptographic solutions to perform privacy preserving k-means. We propose a new method to perform k-means over a large set of data using multiple servers. Our technique avoids heavy cryptographic computations and instead we use a simple randomization technique to preserve the privacy of the data. The k-means computed has essentially the same efficiency and accuracy as the k-means computed over the original data-set without any randomization. We argue that our algorithm is secure against honest-but-curious and non-colluding adversary.

Appendices

A Detailed Computations

1.1 A.1 Lower Bound for $r_i$, $1 \le i \le 2d$

$$\begin{aligned}&\text{ max }\root 2 \of {\sum _{i=1}^d({r_{2i-1}^2(x_{1i}-x_{2i})^2+(\epsilon _1x_{1i}-\epsilon _2x_{2i})^2+2r_{2i-1}(x_{1i}-x_{2i})(\epsilon _1x_{1i}-\epsilon _2x_{2i}))}} \\< & {} \text{ min } \root 2 \of {\sum _{i=1}^d({r_{2i-1}^2(x_{1i}-x_{3i})^2+(\epsilon _1x_{1i}-\epsilon _3x_{3i})^2+2r_{2i-1}(x_{1i}-x_{3i})(\epsilon _1x_{1i}-\epsilon _3x_{3i}))}} \end{aligned}$$

To maximize LHS and minimize RHS, we take $\epsilon _1$=$\epsilon$, $\epsilon _2$=0, $\epsilon _3=\epsilon$ and thus,

$$\begin{aligned}&\sum _{i=1}^d({r_{2i-1}^2(x_{1i}-x_{2i})^2+(\epsilon ^2x_{1i}^2+2r_{2i-1}(x_{1i}-x_{2i})(\epsilon x_{1i}))}\\< & {} \sum _{i=1}^d({r_{2i-1}^2(x_{1i}-x_{3i})^2+(\epsilon ^2(x_{1i}-x_{3i})^2+2r_{2i-1} \epsilon (x_{1i}-x_{3i})^2)} \end{aligned}$$

Using (10), we further get,

$$\begin{aligned}&\epsilon \sum _{i=1}^d{[x_{1i}^2-(x_{1i}-x_{3i})^2]}+2 \sum _{i=1}^d{r_{2i-1}[x_{1i}(x_{1i}-x_{2i})-(x_{1i}-x_{3i})^2]}<0\nonumber \\\Rightarrow & {} 2 \sum _{i=1}^d{r_{2i-1}[(x_{1i}-x_{3i})^2-x_{1i}(x_{1i}-x_{2i})]}>\epsilon \sum _{i=1}^d{[x_{1i}^2-(x_{1i}-x_{3i})^2]}\nonumber \\\Rightarrow & {} r> \text{ max } (\epsilon \frac{\sum _{l=1}^d{(x_{il}^2-(x_{il}-x_{kl})^2)}}{2 \sum _{l=1}^d{(x_{il}-x_{kl})^2-x_{il}(x_{ik}-x_{jl})}}), \forall i, j, k. \end{aligned}$$

(20)

Given that $\epsilon$ is sufficiently small, it can be safely assumed to be less than 1. Hence if (12) is satisfied, (20) is satisfied as well. Although (20) is a better bound, we use (12) to make it independent of $\epsilon$.

1.2 A.2 Kullback Leibler Distance

The Kullback Leibler Distance (KD) is defined to be $-\sum _i{P(i)\log {\frac{Q(i)}{P(i)}}}$ where

$$P(i)=\frac{x_{1i}-x_{2i}}{x_{4i}-x_{3i}}~~~\text{ and } Q(i)=\frac{r_{2i-1}(x_{1i}-x_{2i})+\epsilon _1x_{1i}-\epsilon _2x_{2i}}{r_{2i-1}(x_{4i}-x_{3i})+\epsilon _4x_{4i}-\epsilon _3x_{3i}}.$$

$$\begin{aligned} KD= & {} -\sum _i{\frac{x_{1i}-x_{2i}}{x_{4i}-x_{3i}}\log {\frac{r_{2i-1}+\frac{\epsilon _1x_{1i}-\epsilon _2x_{2i}}{x_{1i}-x_{2i}}}{r_{2i-1}+\frac{\epsilon _4x_{4i}-\epsilon _3x_{3i}}{x_{4i}-x_{3i}}}}} \nonumber \\= & {} \sum _i{\frac{x_{2i}-x_{1i}}{x_{4i}-x_{3i}}\log {\frac{r_{2i-1}+\frac{\epsilon _1x_{1i}-\epsilon _2x_{2i}}{x_{1i}-x_{2i}}}{r_{2i-1}+\frac{\epsilon _4x_{4i}-\epsilon _3x_{3i}}{x_{4i}-x_{3i}}}}}. \end{aligned}$$

(21)

Without loss of generality, we assume that for $i=1$, the above expression attains minima,

$$\ge d\frac{x_{21}-x_{11}}{x_{41}-x_{31}}\log {\frac{r_{1}+\frac{\epsilon _1x_{11}-\epsilon _2x_{21}}{x_{11}-x_{21}}}{r_{1}+\frac{\epsilon _4x_{41}-\epsilon _3x_{31}}{x_{41}-x_{31}}}}$$

Let,

$$D_1=\frac{KD}{ d\frac{x_{21}-x_{11}}{x_{41}-x_{31}}}\ge \log {\frac{r_{1}+\frac{\epsilon _1x_{11}-\epsilon _2x_{21}}{x_{11}-x_{21}}}{r_{1}+\frac{\epsilon _4x_{41}-\epsilon _3x_{31}}{x_{41}-x_{31}}}}.$$

Hence,

$$e^{D_1} \ge \frac{r_{1}+\frac{\epsilon _1x_{11}-\epsilon _2x_{21}}{x_{11}-x_{21}}}{r_{1}+\frac{\epsilon _4x_{41}-\epsilon _3x_{31}}{x_{41}-x_{31}}} \ge \frac{r_1-\frac{\epsilon x_{21}}{x_{11}-x_{21}}}{r_1+\frac{\epsilon x_{41}}{x_{41}-x_{31}}}.$$

Finally,

$$KD \ge d\frac{x_{11}-x_{21}}{x_{41}-x_{31}}\log { \frac{r_1+\frac{\epsilon x_{41}}{x_{41}-x_{31}}}{r_1-\frac{\epsilon x_{21}}{x_{11}-x_{21}}}}.$$

B Range of Bit Length of the Parameters

The probability of correctly guessing the random numbers from Eq. (1) is computed as follows. The adversary may arbitrarily fix the choice of two indices from $\{1,\ldots ,2d\}$ for the $r_i$s and the corresponding index from $\{1,\ldots ,n\}$ for the choice of $\epsilon$. Fixing the two $r_i$ from 2d many $r_i$’s can be done in ${2d\atopwithdelims ()2}$ ways. Similarly choosing one $\epsilon _i$ from n many $\epsilon _i$’s can be done in n ways. Hence the probability is:

$${2d \atopwithdelims ()2}{n \atopwithdelims ()1} \frac{1}{2^{2\ell _1}} \frac{1}{2^{\ell _2}}.$$

Similarly, the probability of correctly guessing from Eq. (13) is:

$${2d \atopwithdelims ()1}{n \atopwithdelims ()3}\frac{1}{2^{\ell _1}} \frac{1}{2^{3\ell _2}}.$$

Fixing n and d as chosen, for the probability to be less than $2^{-80}$, the following two equations must be satisfied,

$$\begin{aligned} 2\ell _1 +\ell _2 \ge 103, \end{aligned}$$

(22)

and

$$\begin{aligned} \ell _1+3\ell _2 \ge 130. \end{aligned}$$

(23)

Hence the above two equations give us the range for the bit length of the parameters.