The Complexity of Vector Partition

Abstract

We consider the vector partition problem, where n agents, each with a d-dimensional attribute vector, are to be partitioned into p parts so as to minimize cost which is a given function on the sums of attribute vectors in each part. The problem has applications in a variety of areas including clustering, logistics and health care. We consider the complexity and parameterized complexity of the problem under various assumptions on the natural parameters p,d,a,t of the problem where a is the maximum absolute value of any attribute and t is the number of agent types, and raise some of the many remaining open problems.

1 Introduction

The vector partition problem is the following. We are given a set $[n]:=\{1,\dots ,n\}$ of n agents, a d × n attribute matrix A whose j-th column A^j is a d-dimensional attribute vector associated with agent j, a positive integer p, subsets $B_{1},\dots ,B_{p}\subseteq [n]$, and a function $f:\mathbb {R}^{d\times p}\rightarrow \mathbb {R}$. A p-partition of [n] is a p-tuple $\pi =(\pi _{1},\dots ,\pi _{p})$ of sets with $[n]=\uplus _{k=1}^{p}\pi _{k}$. A partition is admissible if |π_k|∈ B_k for all k, and its cost is $f\left ({\sum }_{j\in \pi _{1}}A^{j},\dots ,{\sum }_{j\in \pi _{p}}A^{j}\right )$, that is, the value of f on the d × p matrix whose k-th column is the sum of the attribute vectors of the agents in part π_k of π. The problem is to find a p-partition of minimum cost, satisfying the bounds.

Vector Partition Problem

Given $A\in \mathbb {Z}^{d\times n}$, $B_{1},\dots ,B_{p}\subseteq [n]$, and $f:\mathbb {R}^{d\times p}\rightarrow \mathbb {R}$, solve

$$ \min\left\{f\!\left( \sum\limits_{j\in\pi_{1}}A^{j},\dots,\sum\limits_{j\in\pi_{p}}A^{j}\!\right)\!: \pi=(\pi_{1},\dots,\pi_{p}), \uplus_{k=1}^{p}\pi_{k}=[n], |\pi_{k}|\in B_{k}, k\in[p]\right\}. $$

The problem is called bounded-shape if all B_k are intervals, that is, $B_{k}=[l_{k},u_{k}]\cap \mathbb {Z}$ with 0 ≤ l_k ≤ u_k ≤ n given lower and upper bounds for all k, so π is admissible if l_k ≤|π_k|≤ u_k for all k. In particular, it is single-shape if l_k = u_k for all k, so π is admissible if |π_k| = l_k = u_k for all k, and free if l_k = 0 and u_k = n for all k, so all p-partitions π are admissible.

A special case is the following type partition problem. We are now given a t-partition $\tau =(\tau _{1},\dots ,\tau _{t})$ of [n] where τ_i is the set of agents of type i, $B_{1},\dots ,B_{p}\subseteq [n]$, and a function $g:\mathbb {R}^{t\times p}\rightarrow \mathbb {R}$. The cost of π is $g\left (|\pi _{1}\cap \tau |,\dots ,|\pi _{p}\cap \tau |\right )$ where $|\pi _{k}\cap \tau |:=(|\pi _{k}\cap \tau _{1}|,\dots ,|\pi _{k}\cap \tau _{t}|)$, that is, the value of g on the t × p matrix whose k-th column counts the agents of each type in part π_k of π. To encode it as a vector partition problem set d := t, f := g, and define an attribute matrix A by $A^{j}:=\textbf {1}_{i}$ the i-th unit vector in $\mathbb {R}^{d}$ for $i=1,\dots ,t$ and j ∈ τ_i, giving

$$ g\left( |\pi_{1}\cap\tau|,\dots,|\pi_{p}\cap\tau|\right) = f\left( \sum\limits_{j\in\pi_{1}}A^{j},\dots,\sum\limits_{j\in\pi_{p}}A^{j}\right). $$

Both problems have applications in a variety of areas including logistics, health care, and clustering, see for instance [2, 6, 8, 15] and the references therein. In particular, [6] describes a very recent application to group testing for COVID-19, briefly discussed in Section 3. They also bare relations to constructing universal Gröbner bases, see [1, 16] and [13, Chapter 6].

In the vector partition problem we also say that two agents are of the same type if they have the same attribute vector, and we let t := |{A^j : j ∈ [n]}| denote the number of agent types. We also let $a:=\|A\|_{\infty }=\max \limits _{i,j}|{A^{j}_{i}}|$ be the maximum absolute value of any attribute.

The vector partition problem is called separable if $f(x^{1},\dots ,x^{p})={\sum }_{k=1}^{p} f_{k}(x^{k})$ for some given functions $f_{k}:\mathbb {R}^{d}\rightarrow \mathbb {R}$, and symmetric separable if these functions satisfy f₁ = ⋯ = f_p. It is called completely separable if $f(x^{1},\dots ,x^{p})={\sum }_{k=1}^{p}{\sum }_{i=1}^{d} f_{k,i}({x^{k}_{i}})$ for some given functions $f_{k,i}:\mathbb {R}\rightarrow \mathbb {R}$, and symmetric completely separable if these functions satisfy f_1,1 = ⋯ = f_p,d.

The corresponding specializations of the type partition problem are similarly defined. The functions are given by oracles that return the function values on queried arguments.

In this article we study the computational complexity of these problems under various assumptions on their natural parameters p,d,a,t. In particular, we discuss the parameterized complexity of the problem, see [3]. Scaling up rational data if necessary, we assume for simplicity that all data is integer, that is, the attribute matrix has integer entries and all functions take on integer values on integer arguments. The running times of our algorithms are in terms of the number of arithmetic operations and oracle queries they perform.

First we note that with d variable, even with p = 2 and $a=\|A\|_{\infty }=1$, or p variable, even with d = 1 and a given in unary, the problem requires exponential time, see Proposition 2.1.

Second we note that when a is large, that is, the attribute matrix A is binary encoded, even with fixed d = 1 and p = 2, even the symmetric completely separable free problem with quadratic convex functions of the form f_k,1(x) = (x − b)², is NP-hard, see Proposition 2.4.

In contrast, if the function $f:\mathbb {R}^{d\times p}\rightarrow \mathbb {R}$ is concave, then even the general non-separable problem, even with A binary encoded, is polynomial time solvable for every fixed p,d. This follows from a geometric theory developed in [8, 13,14,15], briefly discussed in Section 4.

Here we establish the following. First, in Theorem 2.2, we show that for any function $f:\mathbb {R}^{d\times p}\rightarrow \mathbb {R}$, even the general non-separable problem is solvable in polynomial time for every fixed p,d when A is encoded in unary. Second, in Theorem 2.3, we show that for every fixed number t of agent types, the separable problem is solvable in polynomial time even if the number p of parts and attribute dimension d are variable. Third, in Theorem 2.7, we show that the completely separable bounded-shape problem with convex functions, is fixed-parameter tractable on p,d,a (see [3]), that is, solvable in time which is a computable function of p,d,a times a polynomial in n. The proof of this result uses recent results on sparse integer programming in variable dimension from [5, 9, 10] which are the culmination of a theory initiated in [4] and developed in [13], and whose origins could be traced back to [18]. We also establish analogous, usually stronger, results for the simpler type partition problem. Table 1 above outlines some of our results (the precise statements are given in Section 2).

Table 1 Our results on the complexity of vector and type partition

In Section 2 we give the precise statements of the above theorems and their proofs. In Section 3 we provide some motivation for the problem by briefly discussing a recent application to testing for COVID-19. Finally, in Section 4, we discuss the geometric method of solving the problem for concave objective functions, and some of the many remaining open problems.

2 Statements and Proofs

2.1 The General Problem: Fixed Number of Parts and Attribute Dimension

First we note that with d variable, even with p = 2 and $a=\|A\|_{\infty }=1$, or p variable, even with d = 1 and $a=\|A\|_{\infty }$ given in unary, the problem requires exponential time.

Proposition 2.1

The vector partition problem requires exponential time as follows: (1) with variable d, fixed p = 2, and 0 − 1 matrix A, so with fixed $a=\|A\|_{\infty }=1$, the free problem, with $B_{1}=B_{2}=\{0,1,\dots ,n\}$, needs time Ω(2ⁿ); (2) with variable p and fixed d = 1, and a given in unary, the single-shape problem, with B_k = {1} for $k=1,\dots ,p$, needs time Ω(n!).

Proof

For (1), let d := n and let $A^{j}:=\textbf {1}_{j}$ be the j-th unit vector in $\mathbb {R}^{d}$ for $j=1,\dots ,n$. (Note that the number of agent types is t = n hence also variable.) Then, for a 2-partition π, the first column of the matrix $A(\pi ):=\left ({\sum }_{j\in \pi _{1}}A^{j},{\sum }_{j\in \pi _{2}}A^{j}\right )\in \{0,1\}^{n\times 2}$ is the indicating vector $A^{1}(\pi )={\sum }_{j\in \pi _{1}}\textbf {1}_{j}$ of $\pi _{1}\subseteq [n]$. Thus, the 2-partitions π of [n] give 2ⁿ distinct matrices A(π) with first columns all vectors in {0,1}ⁿ. So the function f can have arbitrary distinct values $f\left ({\sum }_{j\in \pi _{1}}A^{j},{\sum }_{j\in \pi _{2}}A^{j}\right )=f(A(\pi ))$ on the matrices A(π) of all distinct 2-partitions.

For (2), let p := n and let $A:=[1,2,\dots ,n]$. (Again, t = n is variable). Then, the n-partitions with |π_k| = 1 for all k correspond to the n! permutations $\pi =(\pi _{1},\dots ,\pi _{n})$ of A, so the function f can have arbitrary distinct values $f(\pi _{1},\dots ,\pi _{n})$ on these distinct permutations.

So if an algorithm trying to solve the partition problem fails to query the oracle of f on one of the 2ⁿ matrices A(π) in the first case, or one of the n! permutations in the second case, it cannot tell whether the minimum value of f is attained at the missing matrix or the missing permutation or not, so it may fail to correctly solve the problem. So the algorithm must query on all 2ⁿ matrices A(π) in the first case and on all n! permutations in the second case. □

In contrast, when both p,d are fixed, and A is encoded in unary, we have the following.

Theorem 2.2

For the general partition problems with fixed number p of parts we have:

• 1. For any fixed d, the vector partition problem with matrix $A\in \mathbb {Z}^{d\times n}$ with $a:=\|A\|_{\infty }$, $B_{1},\dots ,B_{p}\subseteq [n]$, and any function $f:\mathbb {Z}^{d\times p}\rightarrow \mathbb {Z}$, is solvable in time $O\left (n^{(d+1)p+1}a^{dp}\right )$, that is, in time polynomial in the number n of agents and in the unary encoding of a.

• 2. For any fixed t, the type partition problem with t types and with an arbitrary function $g:\mathbb {Z}^{t\times p}\rightarrow \mathbb {Z}$, can be solved in time $O\left (n^{(t+1)p+1}\right )$ polynomial in the number n of agents.

Proof

We begin with part 1. We augment the given attribute matrix A with an additional row indexed 0, and we set ${A_{0}^{1}}:=\cdots := {A_{0}^{n}}:=1$. We let $V\subset \mathbb {Z}^{(d+1)\times p}$ be the set of integer (d + 1) × p matrices v which satisfy 0 ≤ v_0,k ≤ n and 0 ≤|v_i,k|≤ na for all i ∈ [d] and k ∈ [p]. We denote by ${\bar A}\in \mathbb {Z}^{d\times n}$ and ${\bar v}\in \mathbb {Z}^{d\times p}$ the submatrices of A and v, respectively, obtained by deleting their 0-th row. In particular, $\bar A$ is the original attribute matrix. We now reduce the vector partition problem to that of finding a shortest directed path in an acyclic directed graph D where each edge has a length. We construct D as follows. There is one special vertex s, and all other vertices are labeled by pairs [j,v] with $j\in \{0,1,\dots ,n\}$ and v ∈ V. For $j=1,\dots ,n$ and v,w ∈ V there is an edge, of length 0, from [j − 1,v] to [j,w], if and only if $w=v+A^{j}\otimes \textbf {1}_{k}$ for some k ∈ [p], with 1_k the k-th unit vector in $\mathbb {R}^{p}$, that is, w is obtained by adding A^j to the k-th column of v. For each v ∈ V with v_0,k ∈ B_k for all k ∈ [p] there is an edge from [n,v] to s of (possibly negative) length $f({\bar v})$. Let $\textbf {0}\in \mathbb {Z}^{(d+1)\times p}$ be the 0 matrix.

For any p-partition π with |π_k|∈ B_k for all k ∈ [p], we attach a path from [0,0] to s in D,

$$ [0,v^{0}=\textbf{0}]\rightarrow[1,v^{1}] \rightarrow\cdots\rightarrow[n,v^{n}]\rightarrow s, $$

(1)

where, for $j=1,\dots ,n$, letting k(j) be such that j ∈ π_k(j), we set $v^{j}:=v^{j-1}+A^{j}\otimes \textbf {1}_{k(j)}$. Note that each entry $v^{j}_{i,k}$ of each v^j is obtained by summing at most n entries of A and hence has nonnegative value bounded by n for i = 0 and absolute value bounded by $n\|A\|_{\infty }=na$ for $i=1,\dots ,d$. So v^j ∈ V for all j. Moreover, for $k=1,\dots ,p$, the k-th column of vⁿ is equal to ${\sum }_{j\in \pi _{k}}A^{j}$. In particular, $v^{n}_{0,k}={\sum }_{j\in \pi _{k}}{A^{j}_{0}}=|\pi _{k}|\in B_{k}$ and hence there is an edge from [n,vⁿ] to s. So this is indeed a path in D. Furthermore, the length of the path, which is the length $f({\bar v^{n}})$ of the edge from [n,vⁿ] to s, is $f({\sum }_{j\in \pi _{1}}{\bar A}^{j},\dots ,{\sum }_{j\in \pi _{p}}{\bar A}^{j})$, which is equal to the objective value of the partition π in the vector partition problem.

Conversely, from each path as in (1) we read off a partition $\pi =(\pi _{1},\dots ,\pi _{p})$ as follows. For $j=1,\dots ,n$ there is a k(j) ∈ [p] such that $v^{j}:=v^{j-1}+A^{j}\otimes \textbf {1}_{k(j)}$. We let π_k := {j : k(j) = k}. Since there is an edge from [n,vⁿ] to s we have |π_k|∈ B_k for k ∈ [p]. Moreover, the objective value of π in the vector partition problem is equal to the length $f({\bar v}^{n})$ of the path.

So finding a minimum cost vector partition reduces to finding a shortest directed path in D from [0,0] to s and reading off an optimal partition as explained above.

Now, a shortest path in an acyclic directed graph can be found in time which is linear in the number of edges of D (see e.g. [17]), which, as claimed, is $O(np|V|)=O\left (n^{(d+1)p+1}a^{dp}\right )$.

For part 2, as explained in the introduction, the type partition problem with t types can be encoded as a vector partition problem with attribute dimension d = t and 0 − 1 attribute matrix A, so with a = 1, implying part 2 about the type partition problem as well. □

2.2 The Separable Problem: Fixed Number of Agent Types

Next we consider fixed number t of agent types in both vector and type partition problems.

Theorem 2.3

The following statements hold even with a variable number p of parts:

• 1. For any fixed t, the separable type problem with t types is solvable in time O(pn^2t);

• 2. For any fixed t, the separable vector problem with t types is solvable in time O(dpn^2t);

• 3. For any fixed d and a, the separable vector partition problem with an integer d × p attribute matrix A satisfying $\|A\|_{\infty }\leq a$ is solvable in polynomial time $O(pn^{2(2a+1)^{d}})$.

Proof

We begin with part 1 about the type partition problem. Let $\tau =(\tau _{1},\dots ,\tau _{t})$ be the given partition into types and let n_i := |τ_i|. Let $g_{k}:\mathbb {R}^{t}\rightarrow \mathbb {R}$ be the given functions for $k=1,\dots ,p$ so that the cost of a p-partition $\pi =(\pi _{1},\dots ,\pi _{p})$ is ${\sum }_{k=1}^{p} g_{k}\left (|\pi _{k}\cap \tau |\right )$. Define

$$ V := \left\{(v_{1},\dots,v_{t})~:~v_{i}\in\{0,1,\dots,n_{i}\},~i=1,\dots,t\right\} \subseteq \{0,1,\dots,n\}^{t} \subset \mathbb{R}^{t} . $$

We again reduce the problem to that of finding a shortest directed path in an acyclic directed graph D. We construct this graph as follows. The vertices are labeled by pairs [k,v] with $k\in \{0,1,\dots ,p\}$ and v ∈ V. There is an edge from [k − 1,v] to [k,w] if and only if for $i=1,\dots ,t$ we have w_i − v_i ≥ 0, and ${\sum }_{i=1}^{t}(w_{i}-v_{i})\in B_{k}$, where $B_{1},\dots ,B_{p}$ are the input subsets in the partition problem. The (possibly negative) length of this edge is g_k(w − v). Consider any p-partition π with |π_k|∈ B_k for k ∈ [p]. For $k=1,\dots ,p$ let x^k := |π_k ∩ τ|. This partition is encoded as the following directed path in D from $[0,(0,\dots ,0)]$ to $[p,(n_{1},\dots ,n_{t})]$,

$$ \begin{array}{@{}rcl@{}} [0,v^{0}=(0,\dots,0)] \rightarrow [1,v^{1}=x^{1}] &\rightarrow& [2,v^{2}=x^{1}+x^{2}] \\ &\rightarrow& \cdots\rightarrow[p,v^{p}=x^{1}+\cdots+x^{p}=(n_{1},\dots,n_{t})]. \end{array} $$

All edges in the path exist, since for all k ∈ [p] we have ${\sum }_{i=1}^{t}({v^{k}_{i}}-v^{k-1}_{i})\in B_{k}$, because

$$ \sum\limits_{i=1}^{t}\left( {v^{k}_{i}}-v^{k-1}_{i}\right)=\sum\limits_{i=1}^{t} {x^{k}_{i}}=\sum\limits_{i=1}^{t}|\pi_{k}\cap\tau_{i}|=|\pi_{k}|. $$

Further, the length of this path is the following expression, which is precisely the cost of π,

$$ \sum\limits_{k=1}^{p} g_{k}(v^{k}-v^{k-1}) = \sum\limits_{k=1}^{p} g_{k}(x^{k}) = \sum\limits_{k=1}^{p} g_{k}\left( |\pi_{k}\cap\tau|\right). $$

Conversely, consider any directed path in D from $[0,(0,\dots ,0)]$ to $[p,(n_{1},\dots ,n_{t})]$,

$$ [0,v^{0}=(0,\dots,0)]\rightarrow[1,v^{1}]\rightarrow[2,v^{2}] \rightarrow\cdots\rightarrow[p,v^{p}=(n_{1},\dots,n_{t})]. $$

For $k=1,\dots ,p$ let x^k := v^k − v^k− 1. Then x^k ≥ 0 for all k and $x^{1}+\cdots +x^{p}=v^{p}=(n_{1},\dots ,n_{t})$, so for $i=1,\dots ,t$ we have ${x^{1}_{i}}+\cdots +{x^{p}_{i}}=n_{i}$. Letting $\pi =(\pi _{1},\dots ,\pi _{p})$ be a p-partition of [n] where for $k=1,\dots ,p$ we let π_k consist of ${x^{k}_{i}}$ agents of type i for each $i=1,\dots ,t$, we have that |π_k ∩ τ| = x^k, $|\pi _{k}|={\sum }_{i=1}^{t} {x^{k}_{i}}\in B_{k}$, and the cost of π is precisely the path length,

$$ \sum\limits_{k=1}^{p} g_{k}\left( |\pi_{k}\cap\tau|\right) = \sum\limits_{k=1}^{p} g_{k}(x^{k}) = \sum\limits_{k=1}^{p} g_{k}(v^{k}-v^{k-1}). $$

So, finding a minimum cost type partition reduces to finding a shortest directed path in D from $[0,(0,\dots ,0)]$ to $[p,(n_{1},\dots ,n_{t})]$ and reading off an optimal partition as explained above.

Now, a shortest path in an acyclic directed graph can be found in time linear in the number of edges of D (see again e.g. [17]), which is O(p|V |²) = O(pn^2t), proving part 1.

We proceed with part 2 about the vector partition problem with t types. We now show that it can be encoded as a suitable type partition problem. Let $A\in \mathbb {Z}^{d\times n}$ be the attribute matrix, $B_{1},\dots ,B_{p}\subseteq [n]$, and $f_{k}:\mathbb {R}^{d}\rightarrow \mathbb {R}$ for $k=1,\dots ,p$ be the functions, forming the data for the vector partition problem. Relabeling agents if necessary, assume $A^{1},\dots ,A^{t}$ are the t distinct attribute vectors, and for $i=1,\dots ,t$ let $\tau _{i}\subseteq [n]$ be the set of agents of type i, that is, with A^j = Aⁱ for every j ∈ τ_i. Also let n_i := |τ_i|. For $k=1,\dots ,p$ define a function $g_{k}:V\rightarrow \mathbb {R}$ by $g_{k}(v_{1},\dots ,v_{t}):=f_{k}\left ({\sum }_{i=1}^{t} v_{i}A^{i}\right )$. Then the cost of a p-partition π satisfies

$$ \sum\limits_{k=1}^{p} f_{k}\!\left( \sum\limits_{j\in\pi_{k}}A^{j}\!\right) \! = \sum\limits_{k=1}^{p} f_{k}\!\left( \!\sum\limits_{i=1}^{t}\sum\limits_{j\in\pi_{k}\cap\tau_{i}}A^{j}\!\right) = \sum\limits_{k=1}^{p} f_{k}\!\left( \!\sum\limits_{i=1}^{t}|\pi_{k}\cap\tau_{i}|A^{i}\!\right) = \sum\limits_{k=1}^{p} g_{k}\left( |\pi_{k}\cap\tau|\right). $$

This encodes the vector problem as a type problem. Now, evaluating any $g_{k}(v_{1},\dots ,v_{t})$ takes (2t − 1)d = O(d) arithmetic operations to compute ${\sum }_{i=1}^{t} v_{i}A^{i}$ plus one call to the oracle of f_k, and we need such an evaluation to compute the length of each edge of the graph D. Using part 1, the total time for solving the vector partition problem is O(dpn^2t), proving part 2.

For part 3, note that if d and a are fixed then the number of possible distinct columns of A satisfies t ≤ (2a + 1)^d, so part 3 follows from part 2 on fixed number t of types. □

2.3 The Completely Separable Problem: Fixed-parameter Tractability

We begin by noting that when a is large, that is, the attribute matrix A is binary encoded, even with fixed d = 1 and fixed p = 2, even the symmetric completely separable free partition problem with quadratic convex functions of the simple form f_k,1(x) = (x − b)², is NP-hard.

Proposition 2.4

The symmetric completely separable problem is NP-hard even with d = 1, p = 2, convex functions f_k,1(x) = (x − b)², and the free case, with $B_{1}=B_{2}=\{0,1,\dots ,n\}$.

Proof

It is well known to be NP-complete to decide if given numbers $a_{1},\dots ,a_{n}\in \mathbb {Z}_{+}$ can be partitioned to two parts of equal sums. Let $A:=[a_{1},\dots ,a_{n}]$, let $b:=\frac {1}{2}{\sum }_{j=1}^{n}a_{j}$, and let the functions be f_1,1(x) := f_2,1(x) := (x − b)². Then the cost of 2-partition π = (π₁,π₂) is $({\sum }_{j\in \pi _{1}}a_{j}-b)^{2}+({\sum }_{j\in \pi _{2}}a_{j}-b)^{2}$, and is nonnegative and equals to 0 if and only if ${\sum }_{j\in \pi _{1}}a_{j}={\sum }_{j\in \pi _{2}}a_{j}$. Thus, the optimal value of the symmetric completely separable free partition problem is 0 if and only if $a_{1},\dots ,a_{n}$ can be partitioned to two parts of equal sums. □

In what follows we make use of nonlinear integer programming in standard form,

$$ \min\{f(x): Ax=b,~l\leq x\leq u,~x\in\mathbb{Z}^{n}\}, $$

(2)

with $A\in \mathbb {Z}^{m\times n}$, $b\in \mathbb {Z}^{m}$, $l,u\in \mathbb {Z}^{n}$, and $f:\mathbb {R}^{n}\rightarrow \mathbb {R}$ a separable convex function, that is, given by $f(x)={\sum }_{j=1}^{n} f_{j}(x_{j})$ with $f_{j}:\mathbb {R}\rightarrow \mathbb {R}$ a univariate convex function for all j. As mentioned in the introduction, we assume that all f_j take on integer values on integer arguments. We also denote by $L:=\log (\|u-l\|_{\infty }+1)$ the bit complexity of the lower and upper bounds.

Unfortunately, as is well known, the problem is generally NP-hard even for linear objective functions. Moreover, the classical result of Lenstra [11] that the problem with linear objectives, parameterized by the number of variables, is fixed-parameter tractable, cannot help us, since our integer programs below involve a variable number of variables. So we need to review and use one early and one recent result on integer programming in variable dimension.

The first result we need, which is quite well known by now, from [7], concerns the case when the matrix A is totally unimodular, that is every subdeterminant of A is − 1,0,1.

Proposition 2.5

[7] Program (2) with A totally unimodular is solvable in time poly(n)L.

Next, we need to discuss recent results of [5, 9, 10] on fixed-parameter tractability of sparse integer programming. For this we need some more terminology. The tree-depth of a graph G = (V,E) is defined as follows. The height of a rooted tree is the maximum number of vertices on a path from the root to a leaf. A rooted tree on V is valid for G if for each edge {i,j}∈ E one of i,j lies on the path from the root to the other. The tree-depthtd(G) of G is the smallest height of a rooted tree which is valid for G, see also [12]. For instance, if G = ([2m],E) is a perfect matching with E = {{i,m + i} : i ∈ [m]} then its tree-depth is 3 where a tree validating it rooted at 1 has edge set $E\uplus \{\{1,i\}:i=2,\dots ,m\}$. Next, the graph of an m × n matrix A is the graph G(A) on [n] where j,k is an edge if and only if there is an i ∈ [m] such that ${A_{i}^{j}}{A_{i}^{k}}\neq 0$. The tree-depth of A is the tree-depth td(A) := td(G(A)) of its graph. The following result asserts that the integer programming problem (2) (with separable convex and in particular linear objective) is solvable in fixed-parameter tractable time (see [3]) parameterized by the numeric measure $a:=\|A\|_{\infty }$ and sparsity measure d := td(A^T) of the transpose A^T of the matrix defining the program. Moreover, for any fixed d it is solvable in polynomial time even when a is a variable part of the input which is encoded in unary. In the proposition and theorem below, g and h are some computable functions of the parameters.

Proposition 2.6

[10] Program (2) is solvable in fixed-parameter tractable time

$$ (a+1)^{g(d)}\text{poly}(n)L. $$

We are now is position to show that the completely separable bounded-shape vector partition problem with convex functions, parameterized by p,d,a, is fixed parameter-tractable. The subsets are intervals $B_{k}:=\{z\in \mathbb {Z}:l_{k}\leq z\leq u_{k}\}$ defined by given bounds $l,u\in \mathbb {Z}_{+}^{p}$. We denote $|\pi |:=(|\pi _{1}|,\dots ,|\pi _{p}|)$, so the admissible partitions are those satisfying l ≤|π|≤ u.

Theorem 2.7

For the completely separable partition problems with convex functions we have:

• 1. The completely separable vector problem, with matrix $A\in \mathbb {Z}^{d\times n}$ with $a:=\|A\|_{\infty }$, bounds $l,u\in \mathbb {Z}_{+}^{p}$, and convex functions $f_{k,i}:\mathbb {Z}\rightarrow \mathbb {Z}$, is solvable in time (a + 1)^h(d,p)poly(n). Hence, it is fixed-parameter tractable on p,d,a, and for fixed p,d it is solvable in time polynomial in the unary encoding of a and polynomial in n of degree independent of p,d.

• 2. The completely separable type problem, with convex functions $g_{k,i}:\mathbb {Z}\rightarrow \mathbb {Z}$, is solvable in time $\text {poly}(pt)\log (n)$, even with variable the number p of parts and number t of types.

Proof

We begin with part 1 about the vector partition problem. We augment again the given attribute matrix A with an additional row indexed 0, and we set ${A_{0}^{1}}:=\cdots := {A_{0}^{n}}:=1$. We now construct an integer program as follows. There are binary variables x_k,j for $k=1,\dots ,p$ and $j=1,\dots ,n$ with the interpretation that x_k,j = 1 indicates that agent j goes to part π_k of the sought partition. There are integer variables y_k,i for $k=1,\dots ,p$ and $i=0,\dots ,d$ with the interpretation that in the partition π determined by the x_k,j, the attribute sums satisfy $y_{k,i}={\sum }_{j\in \pi _{k}}{A^{j}_{i}}$. Here is the program, followed by further explanation:

$$ \begin{array}{@{}rcl@{}} &&\min\sum\limits_{k=1}^{p}\sum\limits_{i=1}^{d}f_{k,i}(y_{k,i})\\ &&\sum\limits_{j=1}^{n} {A^{j}_{i}} x_{k,j} - y_{k,i} = 0,\quad k=1,\dots,p,~i=0,\dots,d, \end{array} $$

(3)

$$ \begin{array}{@{}rcl@{}} &&\sum\limits_{k=1}^{p} x_{k,j} = 1,\quad j=1,\dots,n, \end{array} $$

(4)

$$ \begin{array}{@{}rcl@{}} &&0\leq x_{k,j}\leq 1,~~l_{k}\leq y_{k,0}\leq u_{k},~~ -na\leq y_{k,i}\leq na,\\ &&\qquad x_{k,j},y_{k,0},y_{k,i}\in\mathbb{Z},~~k\in[p],~~i\in[d],~~ j\in[n]. \end{array} $$

On the one hand, consider any feasible solution (x,y) of this program and define a tuple $\pi =(\pi _{1},\dots ,\pi _{p})$ by π_k := {j : x_k,j = 1} for all k. Equations (4) guarantee that π is indeed a p-partition of [n] and (3) guarantee that indeed $y_{k,i}={\sum }_{j\in \pi _{k}}{A^{j}_{i}}$ for all k,i. In particular, $|\pi _{k}|={\sum }_{j\in \pi _{k}}{A^{j}_{0}}=y_{k,0}$ for all k, so l ≤|π|≤ u and π is an admissible partition. Moreover the objective function value of this solution is equal to ${\sum }_{k=1}^{p}{\sum }_{i=1}^{d}f_{k,i}({\sum }_{j\in \pi _{k}}{A^{j}_{i}})$ and is indeed the objective function value of the partition π in the vector partition problem.

On the other hand, consider any p-partition $\pi =(\pi _{1},\dots ,\pi _{p})$ of [n] with l ≤|π|≤ u. Define (x,y) as follows. For all k,j define x_k,j := 1 if j ∈ π_k and x_k,j := 0 otherwise. Clearly x satisfies (4). Now define y_k,i for all k,i by (3), which then automatically hold. Further, for all k we have that $|y_{k,0}|={\sum }_{j\in \pi _{k}}{A^{j}_{0}}=|\pi _{k}|$ so l_k ≤ y_k,0 ≤ u_k, and for $i=1,\dots ,d$ we have that $|y_{k,i}|\leq {\sum }_{j\in \pi _{k}}|{A^{j}_{i}}|\leq na$, so all y_k,i satisfy the bounds. Therefore (x,y) is a feasible solution of the integer program. Moreover, the objective function value of the partition π in the vector partition problem is equal to ${\sum }_{k=1}^{p}{\sum }_{i=1}^{d}f_{k,i}(y_{k,i})$ and is indeed the objective function value of the solution (x,y) in the integer program. This shows that the vector partition problem reduces to the above integer program.

Now consider the matrix B expressing equations (3), (4) and its transpose B^T. The columns of B are indexed by the variables. Let us index the equations and the rows of B by $r_{1,0},\dots ,r_{p,d}$ corresponding to (3) and $s_{1},\dots ,s_{n}$ corresponding to (4). Let T be the tree on the r_k,i and s_j which is rooted at r_1,0 and consists of the path $(r_{1,0},\dots ,r_{p,d})$ together with the leaves $s_{1},\dots ,s_{n}$, each connected to r_p,d. The height of T is Clearly (d + 1)p + 1. We now show that T is valid for G(B^T). For this we need to show that if two equations share a variable then one lies on the path in T from its root to the other. Since all the r_k,i lie on a common path, it is clear that this condition holds for any $r_{k,i},r_{{\bar k},{\bar i}}$. It is also clear that this condition holds for any $s_{j},s_{\bar j}$ since any two such distinct equations involve disjoint sets of variables. Finally, the condition also holds for any two equations r_k,i, s_j since r_k,i lies on the path from the root r_1,0 to s_j. So the tree T is indeed valid for G(B^T).

Therefore the tree-depth of B^T satisfies td(B^T) ≤ (d + 1)p + 1. Also, it is clear that the matrix B satisfies $\|B\|_{\infty }=\|A\|_{\infty }=a$. Furthermore, all variables are bounded below by − na and above by na so $L=\log (\|u-l\|_{\infty }+1)=\log (2na+1)$. Finally, the number of variables is p(n + d + 1). Substituting these in Proposition 2.6, we find that, for some suitable computable function h, the running time is bounded by (a + 1)^h(d,p)poly(n), proving part 1.

We proceed with part 2 about the type partition problem. As explained in the introduction, the problem with t types can be encoded as a vector partition problem with attribute dimension d = t and 0 − 1 attribute matrix A, so with a = 1. Plugging this into the time bound for the vector partition problem gives 2^h(t,p)poly(n). However, for the type partition problem, which is easier, we can do much better. Let then $\tau =(\tau _{1},\dots ,\tau _{t})$ be the given type partition with n_i := |τ_i| for $i=1,\dots ,t$. We construct again a suitable integer program, reminiscent of but different than the one above. In particular, the number of variables is independent of n, and the variables are not binary. For $k=1,\dots ,p$ and $i=1,\dots ,t$ we now have variables x_k,i and variables y_k with the interpretation that x_k,i = |π_k ∩ τ_i| and y_k = |π_k| in the sought partition π. Here is the program, followed by further explanation:

$$ \begin{array}{@{}rcl@{}} &&\min{\sum}_{k=1}^{p}{\sum}_{i=1}^{t} g_{k,i}(x_{k,i})\\ &&{\sum}_{i=1}^{t} x_{k,i}-y_{k} = 0 ,\quad k=1,\dots,p, \end{array} $$

(5)

$$ \begin{array}{@{}rcl@{}} &&{\sum}_{k=1}^{p} x_{k,i} = n_{i},\quad i=1,\dots,t, \end{array} $$

(6)

$$ \begin{array}{@{}rcl@{}} &&0\leq x_{k,i}\leq\min\{u_{k},n_{i}\},\quad l_{k}\leq y_{k}\leq u_{k},\quad x_{k,i},y_{k}\in\mathbb{Z},\quad k\in[p],\quad i\in[d]. \end{array} $$

On the one hand, consider any feasible solution (x,y). Then for all i, (6) assert that ${\sum }_{k=1}^{p} x_{k,i}=n_{i}=|\tau _{i}|$, so we can define a p-partition $\pi =(\pi _{1},\dots ,\pi _{p})$ where for all k we let π_k consist of x_k,i agents of type i, so that indeed |π_k ∩ τ_i| = x_k,i. Then for all k, (5) imply $y_{k}={\sum }_{i=1}^{t}|\pi _{k}\cap \tau _{i}|=|\pi _{k}|$, so the bounds on y_k imply l_k ≤|π_k|≤ u_k and π is an admissible partition. Moreover, the objective function value of this solution is equal to ${\sum }_{k=1}^{p}{\sum }_{i=1}^{t} g_{k,i}(|\pi _{k}\cap \tau _{i}|)$ and is indeed the value of π in the type partition problem.

On the other hand, consider any p-partition $\pi =(\pi _{1},\dots ,\pi _{p})$ satisfying l ≤|π|≤ u. Define (x,y) as follows. For all k,i let $x_{k,i}:=|\pi _{k}\cap \tau _{i}|\leq \min \limits \{u_{k},n_{i}\}$ which implies that ${\sum }_{k=1}^{p} x_{k,i}={\sum }_{k=1}^{p}|\pi _{k}\cap \tau _{i}|=|\tau _{i}|=n_{i}$ so (6) hold, and define y_k by (5) which then hold and give $y_{k}={\sum }_{i=1}^{t}|\pi _{k}\cap \tau _{i}|=|\pi _{k}|$, so the bounds on |π_k| imply l_k ≤ y_k ≤ u_k. Therefore, (x,y) is a feasible solution of the integer program. Moreover, the objective function value of the partition π in the type partition problem is equal to ${\sum }_{k=1}^{p}{\sum }_{i=1}^{t} g_{k,i}(x_{k,i})$ and is indeed the objective function value of the solution (x,y) in the integer program. This shows that the type partition problem reduces to the above integer program.

Now consider the matrix C expressing equations (5), (6). We use the following well known sufficient condition for total unimodularity (see e.g. [17]). Suppose a matrix has − 1,0,1 entries, at most two nonzero entries per column, and the rows can be 2-partitioned so that, for every column with two nonzero entries, if these entries have the same sign then they lie in rows in opposite parts, whereas if they have opposite signs then they lie in rows in the same part. Then the matrix is totally unimodular. Now, all entries of C are − 1,0,1. Each variable y_k appears in one equation so the corresponding column has one nonzero entry. Each variable x_k,i appears in two equations, one from (5) and one from (6), so the corresponding column has exactly two nonzero entries of the same sign, one from the set of rows corresponding to (5) and one from the set of rows corresponding to (6), which form the desired 2-partition of rows. So by the above sufficient condition, C is totally unimodular.

Also, all variables are bounded below by 0 and above by n so $L=\log (\|u-l\|_{\infty }+1)\leq \log (n+1)$. Finally, the number of variables is p(t + 1). Substituting these in Proposition 2.5, the running time is found to be bounded by $\text {poly}(pt)\log (n)$, proving part 2. □

Two remarks are in order here. First, note that for fixed p,d, the running time for the vector partition problem in both Theorem 2.2 part 1 and Theorem 2.7 part 1 is polynomial in n and a. The former is stronger in that it holds for the general, non-separable non-convex problem, and the degree of the polynomial in a is dp which is much smaller than the degree h(d,p) in the latter. But the latter is stronger in that the degree of the polynomial in n is independent of p,d whereas it is (d + 1)p + 1 in the former. Second, it is tempting, in the proof of Theorem 2.7, to try and encode the completely separable vector partition problem as a completely separable type partition problem, as done for the (non-completely) separable problems in the proof of Theorem 2.3 part 2, to aim at a running time which is polynomial also in p,d and independent of a. However, this does not work, as can be expected from the NP-hardness of the completely separable vector partition problem asserted in Proposition 2.4.

3 Application

Here we very briefly discuss the interesting application to testing for COVID-19 from [6]. Patients (agents) exposed to infected individuals are to be tested for the virus in a facility with limited testing capacity. They are to be partitioned into groups: the first group consists of patients not to be tested at all; the second group consists of patients to be each tested individually; the other groups consist of patients to be tested according to the so-called Dorfman testing scheme: all patients in each such group are tested together (by mixing their specimens); if the result is negative then they are all declared negative, whereas if it is positive then each is tested again individually. The patient population is assumed heterogeneous: with each patient are associated two parameters: the positivity risk which depends on the nature of exposure to an infected individual, such as the exposing activity, its duration, proximity, and environment; and the gap between pre-intervention harm and post-intervention harm which estimate, respectively, the potential to spread the infection if undetected or detected and treated (such as by isolation, quarantine, or symptom monitoring), and which depend on the professional network (such as students, teachers, health care professionals, grocery store workers) which affects the potential to spread the infection. The authors indicate that it suffices to approximate these parameters by three levels (low, average, high). Thus, the agents come in nine types. Using these parameters and the tests true-positive and true-negative probabilities, they are able to formulate the problem of grouping patients for testing so as to minimize expected harm, with expected number of tests not exceeding capacity, as a suitable type partition problem, with t = 9 agent types and a variable number p of parts (as in our Theorem 2.3), and describe a heuristic for solving it. See [6] for the precise details.

4 Discussion

First, we very briefly discuss the geometric theory developed in [13, Chapter 2] and [8, 14, 15], which in particular enables to solve the vector partition problem with large, binary encoded attributes, and concave functions, for every fixed p,d. This theory considers minimization over sets $S\subset \mathbb {Z}^{N}$ where the objective is composite of the form f(Wx) where $W\in \mathbb {R}^{D\times N}$ and $f:\mathbb {R}^{D}\rightarrow \mathbb {R}$ is concave. It is assumed that f is given by an oracle, and that we are given an oracle for linear optimization over S and a set $E\subset \mathbb {R}^{N}$ of edge-directions of the convex hull conv(S) of S in $\mathbb {R}^{N}$. The framework involves the construction of the zonotope $Z\subset \mathbb {R}^{D}$ generated by the projection WE of E by W into $\mathbb {R}^{D}$, and the enumeration of its vertices, which can be done in polynomial time for fixed D. Then a vertex y of Z minimizing f is obtained, and finally, a point x ∈ S projecting down to y = Wx is determined. To specialize this framework to the vector partition problem, we let N = pn and let S ⊂{0,1}^p×n be the set of 0 − 1 matrices with each column sum 1, which correspond to p-partitions of [n]. Linear optimization over S is equivalent to solving a transportation problem which is easy to do, and a set of edge-directions of polynomial size can also be determined. We let D = pd and then the attribute matrix A for the partition problem induces the matrix W for the above framework. See [13, Chapter 2] for the precise details of this framework, for the proof that these problems can be solved in polynomial time even in the binary encoding of W, and for the description of the specialization of this framework to the vector partition problem.

Second, we discuss some of the many remaining open problems here. First, the results of [8] solve the problem with general concave function f and binary encoded attribute matrix A for every fixed p,d in strongly polynomial time $O(n^{dp^{2}})$. It would be very interesting to know if the problem with a concave function is in fact fixed-parameter tractable or W[1]-hard (see [3]) when d is fixed and p is the parameter or when p is fixed and d is the parameter.

Similarly, Theorem 2.2 solves the problem with an arbitrary function f and unary encoded attribute matrix A for every fixed p,d in polynomial time O(n^{(d+ 1)p+ 1}a^dp). It would again be very interesting to know if the problem is in fact fixed-parameter tractable or W[1]-hard when d is fixed and p is the parameter or when p is fixed and d is the parameter.

Another interesting question, related to Theorem 2.3, is if the separable type partition problem with variable p, or the separable vector partition problem with variable d and p, are fixed-parameter tractable or W[1]-hard when parameterized by the number t of types.

Last, it would be interesting to know if Theorem 2.7 can be extended to show that the (non-completely) separable problem is fixed-parameter tractable parameterized by p,d,a.