Minimum information about tolerogenic antigen-presenting cells (MITAP): a first step towards reproducibility and standardisation of cellular therapies

Introduction

Immunotherapy with whole living cells shows great promise for the treatment of a wide variety of complex diseases. A recent innovation in this field is the use of myeloid antigen-presenting cells (APC) with immunoregulatory function to treat autoimmune diseases, or to prevent destructive immune reactions after organ—or haematopoietic stem cell-transplantation (Amodio & Gregori, 2012; Hutchinson, Riquelme & Geissler, 2012; Hilkens & Isaacs, 2013; Van Brussel et al., 2014; Creusot et al., 2014; Thomas, 2014; Morelli & Thomson, 2014; Ten Brinke et al., 2015). Examples of these immunoregulatory APC include tolerogenic dendritic cells (tolDC) and regulatory macrophages (Mreg). The main advantage of these tolerogenic/regulatory APC (hereafter referred to as tolAPC) therapies over conventional immunosuppressive drugs is that these cells are uniquely equipped to specifically target harmful T-cell responses to auto- or allo-antigens. Thus, tolAPC act in an antigen-specific manner and do not cause general immunosuppression after exposure to antigen or in the allogeneic transplant setting—they are, therefore, less likely to affect protective immune responses to invading pathogens. More remarkably, as tolAPC induce or restore immune tolerance, and immune tolerance is a self-reinforcing state, the beneficial effects of tolAPC treatment are expected to continue beyond the lifespan of these therapeutic cells. Thus, tolAPC therapy holds the promise of a potentially curative treatment with low toxicity.

A variety of therapeutic tolAPC products have been developed over the last decade, and a number of these have recently been tested, or are undergoing testing in the clinic. Phase I trials with autologous tolAPC have been completed for type I diabetes (Giannoukakis et al., 2011), rheumatoid arthritis (Benham et al., 2015; Bell et al., 2016) and Crohn’s disease (Jauregui-Amezaga et al., 2015), and further phase I trials for multiple sclerosis, allergic asthma and kidney transplantation are in progress (Ten Brinke et al., 2015). So far the results are highly encouraging from a safety standpoint, and further phase II clinical trials to test the efficacy of tolAPC therapies are under development, with the first tolDC phase II trial in type I diabetes recruiting imminently.

An outstanding and important issue within the tolAPC field is the variation in the methods used for cell isolation and culture. Current protocols to produce tolDC or Mreg involve the isolation of peripheral blood CD14⁺ monocytes followed by cell culture under specific conditions to generate the desired type of tolAPC. For example, monocytes can be differentiated into DC by culturing them in the presence of the cytokines IL-4 and GM-CSF, and stable tolerogenic function can be induced in these DC by treatment with immunosuppressive compounds (e.g., dexamethasone, IL-10) or genetic modification to silence immunostimulatory molecules. In addition to compound variety, the concentration used may vary, as well as the timing of administration to the tolAPC cultures. These almost endless variations in the methodology make comparison of different tolAPC products difficult, therefore bringing real uncertainty when comparing safety and efficacy results from different clinical trials testing different tolAPC products in different diseases/conditions.

Minimum Information Models (MIMs)

Minimum Information Models (MIMs) are now being recognised as an important tool for the scientific community to use and interpret reported data, enabling comparison between data from different studies and facilitating experimental reproducibility. While it is simple for different laboratories to share and exchange their data, for this sharing to be meaningful, allowing reuse and repurposing, it must share some commonality of reporting, methodology or interpretation. This process has recently raised much interest, for example with the 5-star open data initiative (http://5stardata.info). A MIM is the first step in this process, providing a mechanism for the broadest level of commonality, ensuring that different laboratories report at least the key facts about their analysis in a clear and consistent manner, by providing a document and checklist describing the necessary information. They are simple and easy-to-use, so are supportive rather than burdensome for investigators. Examples of MIMs are MIAME (minimum information about a microarray experiment) (Knudsen, Daston & Teratology, 2005) and the more recent MIATA (minimal information about T-cell assays) (Janetzki et al., 2009; Britten et al., 2011), the latter now actively being endorsed by seven international scientific journals. As well as allowing sharing of data between individual laboratories, when widely used, they also provide an excellent basis for building large integrated resources of data that can be searched efficiently and comprehensively.

Setting up MITAP: Community Building and Initial Analysis

Our goal was to provide a reporting framework that will make differences and similarities of tolAPC transparent. We therefore set out to create minimum reporting guidelines for the production process of tolAPC used in pre-clinical and/or clinical studies. We call this MITAP (Minimum Information about Tolerogenic Antigen-Presenting cells).

A critical part of building the MITAP document was community building; to enable data sharing and commonality, the document must be broadly reflective of the opinions and current practices of many laboratories within the discipline. For this, a community is needed to provide information, feedback and tests subjects. Helpfully, we recently established a consortium of investigators working in the field of tolerogenic cellular therapies, with the aim of jointly addressing issues related to the translation and clinical application of these new treatments. This consortium is called AFACTT (action to focus and accelerate cell-based tolerance-inducing therapies—http://www.afactt.eu). It convenes every 6 months at interactive workshops. This consortium provided an ideal vehicle for the process of developing MITAP, which we achieved through a series of “exercises” that introduced AFACTT members to the general notion of MIMs, as well as providing some initial data, which we used to scope the MITAP document. We used three main forms of exercise both of which aimed at gathering “terms”—basic vocabulary in use within the community. The first is called the “sticky-note” exercise which we performed at several AFACTT meetings: for this, each participant writes a term on a sticky-note; these are then collated and clustered on a wall by the whole group, identifying synonyms and related terms. Secondly, we gathered bibliographic databases from all willing AFACTT participants; abstracts were gathered from PubMed, and analysed computationally to find words that were over-represented in these abstracts compared to the bulk of PubMed. Finally, we built some bespoke software to crowd-source community feedback on these terms using a simple web interface.

The conception of MITAP took place in April 2014, at the first AFACTT meeting; the overall process of creating the MITAP document took 18 months (Fig. 1), which reflects the nature of the process i.e., the active involvement of a diverse group of investigators and the absolute need for incorporating feedback and suggestions from this group.

Overview of the MITAP Document

The MITAP document is divided into 4 sections that encapsulate the most important parts of the tolAPC production process in a chronological manner. The sections describe the cells at the start of the production process, during the differentiation/induction of tolerogenicity process, and at the end of the production process, prior to injection into the recipient or use in experimental assays. A final section is dedicated to aspects that are important for understanding the application and purpose of tolAPC. The sections are described in more detail below; for all the sections and subsections the document states clearly whether the investigator must (required), should (if available) or may (optional) provide the relevant information. It also advises on the use of existing taxonomies and databases to provide the information in a uniform manner, and it suggests the use of other MIMs where appropriate. The full MITAP document can be found on http://w3id.org/ontolink/mitap (see also ‘Sustainability’ below).

Testing MITAP: the “20 Questions” Experiment

The MITAP document was developed collaboratively by scientists in the AFACTT consortium and should, therefore, be usable by this community. However, we wished to test directly whether the document as it stood was comprehensible by individuals from the discipline, and that data could be entered in a consistent manner. We therefore performed a “question-and-answer” experiment, a variation on the spoken parlour game “20 questions,” popular in the 19th and 20th centuries. We collected a list of terms—that is short descriptive phrases that might be found in a description of a procedure to generate tolAPC, each of which would fit into one of the categories defined by the MITAP document. We then asked a number of scientists from the discipline, our test subjects, to place these terms into the relevant category from MITAP, which we call the test task. The basic premise of the experiment is simple: if the document is clear and comprehensible, then different test subjects should independently perform this task consistently.

The experiment was designed following a number of preconditions. First, the test task could not be too onerous for the test subjects, so that we could maximise completion rates; in particular, we felt that the task should take less than 30 min (including reading documentation). Our initial trials suggested that a subject could place two phrases for each category in MITAP in this time. Second, our terms should be representative of the discipline, rather than a single group, as far as was possible. Therefore, we asked members of the AFACTT group to contribute these terms, limited to two for each section in MITAP; for the experiment, we selected a subset of these terms randomly, with minor alterations for length. Third, our test subjects should also be representative of the discipline. Therefore, we asked those who contributed terms to also nominate a test subject. Finally, the test subjects were people who had not directly contributed to MITAP, nor seen the document, nor were involved in contributing terms.

For the test itself, test subjects were presented with a set of terms and for each asked to assess into which category it should be placed. Test subjects were given no guidance on how to categorise terms except for the MITAP document. In fact, they were given a slightly reduced MITAP document; the non-normative (i.e., explanatory and not a strict part of the standard) examples were removed for the purpose of this experiment. Subjects were also asked for some basic metadata (level of experience, discipline and so forth). The test was conducted using an online survey system (Google forms). Strictly, this means that we could not enforce lack of collaboration; we considered this compromise to be appropriate, as it increased the number of test subjects and their spread across the discipline.

“20 Questions” Results

The question and answer experiment was conducted over a two-week period in July 2015. We invited 24 test subjects to take part and received results from 23, representing a 96% completion rate. Each subject categorized 44 terms from which example terms are “xenogeneic,” “Rattus norvegicus” or “immunomagnetic separation.”

For the test task, subjects could ascribe each term to any number of MITAP categories. A number of categories in MITAP are repetitive (cell numbers before and after the differentiation/induction of tolerogenicity process, for example). These categories were normalized for most of the results presented below. Answers from test subjects were compared against our “Gold Standard”—the answers defined by primary authors of the MITAP document. As subjects could give multiple answers, we assessed answers as either being exactly the same as the gold standard, or containing at least the gold standard (Fig. 2).

Broadly, the results showed a very high-level of test subject agreement, averaging at around 73% for Gold Standard only and 84% for at least Gold Standard. Terms that were put in the right category by all test subjects included ‘immunomagnetic separation’ (Section 1. Cell separation process) and ‘Spanish regulatory agency (AEMPS)’ (Section 4. Regulatory Authority). Terms that test subjects were struggling with (0–50% agreement) included “cytomegalovirus (CMV)” for which the Gold Standard was antigen (see Section 2. Differentiation and Induction of Tolerogenicity), but that could also correctly be put under ‘characteristics of the organism’ (Section 1). Another example of a term often placed under the wrong subheading was: “mice were obtained from the Jackson Laboratory,” which was often placed under ‘species and strain’ (Section 1), but for which the Gold Standard was ‘characteristics of the organism’ (Section 1). For these terms, where the document was shown to lack in clarity, we adjusted the MITAP document accordingly; for example, we now state explicitly under ‘characteristics of the organism’ that authors should indicate the source of purchase.

Prevalence of MITAP Data in Extant Papers

The purpose of the MITAP document is to ensure that authors provide sufficient basic information about their tolAPC production protocol. An implicit assumption is that currently some or all of this information is not being routinely described. To test this assumption, we reviewed a number of papers about tolAPC, and for each, we determined whether the data required by the MITAP document was present or not.

In detail, forty-two tolAPC papers were selected (predominantly from members of AFACTT or from researchers wellknown in the field), randomized and the first twenty were read in detail. For each section of MITAP, we determined whether the information required was either: directly stated in the paper or reference (Fig. 3: green squares), whether it was possible to infer from other information given in the paper (Fig. 3: yellow triangles), whether all details were present or only partly (Fig. 3: yellow triangles) or whether the information was not present at all (Fig. 3: red circles). For example, Section 1-ai of MITAP describes the species of the experimental organism. A paper with the phrase “human” or “Homo sapiens” would fall into the first category (included in the publication) one referring to “patients” would fall into the second category (included but some details missing). Many papers do not describe their experimental methodology, but instead refer to another paper (“as described previously”); in this case, we checked the paper up to two references away and if found the information was considered as ‘present’ (Fig. 3: green squares), if not it was considered as ‘not present’ (Fig. 3: red circles). This work was performed by one of us (RS) who is a post-doctoral scientist with 6 years experience in the discipline.

Results are shown in Fig. 3. This shows that no papers have all the information required by MITAP, and that most have substantial gaps. We can also see that while there is a relatively high degree of conformity between different papers: e.g., species and growth media are commonly reported; cell number, viability and culture container are rarely documented.

Sustainability

One significant issue with resources such as MITAP is their sustainability. Previous research has shown that uniform resource identifiers (URIs; i.e., URLs or Web addresses) given in papers have short half-lives (around a 25% loss three years after publication (Wren, 2008)), and either move or become inaccessible, in many cases before the papers have been published. This can happen either because the web hosting is not stable, or fails, or because the domain providing the URI is not maintained. To alleviate these difficulties with MITAP, neither the URIs nor the hosting of the documents require active maintenance by the authors of this paper. We have used a permanent identifier (http://w3id.org/ontolink/mitap) thereby providing a re-direction step. The primary hosting of the resource is provided by Internet Archive (http://archive.org). Both of these organisations have sustainability as their key aim. Maintenance of MITAP documents as live, web-resources therefore requires no active contribution from the authors of this paper. Either archive.org or w3id.org need to fail before MITAP cannot be maintained at the stated location.

We have also given explicit consideration to data formats for our documents. Resources are available as Word docs and PDFs, providing typographical elegance and familiarity to the community, but also an extremely simple HTML representation, ensuring vendor-neutrality and future-proofing of the knowledge contained within them.

Discussion

The tolAPC community has advanced significantly in the last 15 years, from the initial discoveries of how to culture APC with potent immune-regulatory properties, through to clinical trials. At the current time, however, most investigators are operating as a “cottage industry”: individual labs working with different cell isolation techniques, procedures to induce stable tolerogenic properties in the cells, and investigating different therapeutic outcomes. Against this heterogeneous background, it becomes difficult to compare investigations between different laboratories, raising concerns about reproducibility (Ioannidis, 2005; Boulton, 2016).

The MITAP document that we have described here will, of course, not address all of these issues, however as a minimum guidelines tool for reporting of protocols, it should at least address the issue of data heterogeneity; different laboratories may still use different protocols to generate tolAPC, but at least we will be clear about this diversity, and have a greater understanding of what these differences are.

We have been careful in the process of building MITAP not to produce a document that is overly complex. Unlike many other minimum information documents, we have actively tested it with independent, essentially untrained annotators (test subjects). We have shown that it is possible, under these circumstances, to provide the information required in a clear and consistent manner. We believe that it should be possible for authors to complete the MITAP document in approximately 30 min. We also believe that this process should not be a burden to researchers; in our experience, we have adopted use of the MITAP document in writing our own papers, and believe that having a clear checklist to follow simplifies rather than confuses.

The process of building the MITAP document has been a highly collaborative one which has been a useful exercise in and of its own right: the degree of heterogeneity in the community has come as a surprise to many of those involved in this work. By analyzing extant papers, we have also shown that there is a discrepancy between the information that the community considers important and wants to receive about a tolAPC production protocol and the information that they actually report. For example, very few papers describe the viability of tolAPC at the end of the production process, although we consider it extremely likely that most researchers do actually measure this. Moreover, we can see no a priori reason why culture media largely is reported while culture container largely is not.

We believe that wide adoption could also contribute toward greater consideration of data management in the tolerogenic cell therapies community in general. For example, the MIATA project was recently extended to include NK cells (called MIANKA), and a similar approach could be taken by MITAP to include other regulatory cells, such as myeloid-derived suppressor cells, mesenchymal stem cells, and even T regulatory cells. Furthermore, unlike many minimum information efforts, we have considered upfront long-term sustainability of the documents and the URIs used to refer to them. Our sustainability plan is robust and does not require active involvement of any of the authors.

Of course, there is much work remaining. The production of tolAPC is only a small part of the overall process. Moreover, we are only attempting to standardize the reporting of protocols, and not the standardisation of the protocols themselves. Finally, we currently have no common location or database which can store the experimental data, such as exists for instance with microarray or genomic data. MITAP is a first step, laying the groundwork for further collaboration.

Supplemental Information