Active living research: A six-year report
Evaluation of Active Living Research: Progress and Lessons in Building a New Field

https://doi.org/10.1016/j.amepre.2008.10.009Get rights and content

Background

An evaluation was undertaken of the initial 6 years of the Active Living Research (ALR) program. Conducted in 2006 and analyzed in 2007, the evaluation was designed to assess productivity and progress on all three program goals and to inform consideration of program re-authorization.

Methods

The evaluation was a retrospective, in-depth, descriptive study utilizing multiple methods, both qualitative and quantitative. Qualitative data were derived mainly from 88 interviews with key informants. Quantitative data were derived primarily from a web-based survey of grantee investigators.

Results

Examples of key indicators of ALR's progress in addressing its goals were (1) in building a knowledge base: 40% of grantee investigators reported producing at least one scientific publication based on their ALR study, averaging two papers per principal investigator who had published; (2) in growing a transdisciplinary field: investigators funded in the first five rounds of grants reported representing more than 20 disciplines; and (3) in contributing to policy change: ten examples were reported of contribution to specific policy changes. In addition, more than one-third (37%) of principal investigators had leveraged additional funds, averaging $275,000 per ALR grant, suggesting that ALR also had made progress in growing financial resources for the field.

Conclusions

Overall, ALR made strides during 6 years in addressing its mission to develop a transdisciplinary field of research on environmental and policy factors that promote physical activity. The evaluation provided insight into useful approaches and strategies for building a nascent research field and suggested how to enhance the contribution of research to policy.

Introduction

In 2000, the Robert Wood Johnson Foundation (RWJF) expanded its efforts to improve healthy lifestyles by targeting increased physical activity in daily life. RWJF staff selected environmental and policy change as the primary approach to this health behavior issue, building on their decade-long work on tobacco control. RWJF staff posited that progress on policy and practice in this nascent field would require both the development of an evidence base and capacity among researchers to produce needed knowledge and the development and testing of policies and interventions, with support for successful policies and interventions coming from professionals and policymakers. Consequently, the RWJF took a multi-pronged grantmaking approach, developing a suite of programs to address the various necessary pieces of the paradigm: research (Active Living Research [ALR]); community demonstrations (Active Living by Design); support from elected officials (Active Living Leadership); and support and coordination from the media and other “influentials” (Active Living Network). This suite of programs was designed to increase physical activity across the entire population; see Orleans et al.1 in this supplement to the American Journal of Preventive Medicine.

Authorized in 2001 for $12.5 million, ALR was designed “to stimulate and support research that will identify environmental factors and public and private policies that influence physical activity among Americans” (www.activelivingresearch.org). Program activities were designed to achieve three goals: (1) establishing a strong research base regarding the environment as well as the policy correlates and determinants of physical activity, (2) helping to build a transdisciplinary field of physical activity policy and environmental researchers, and (3) facilitating the use of research to support policy change (www.activelivingresearch.org). During the subsequent 6 years, the program, led by a national program office (NPO) and a national advisory committee, issued seven calls for proposals (CFPs) starting in 2002 and awarded 121 grants. Sallis et al.2 in this supplement provides more specifics about the program.

In June 2006, the RWJF supported evaluation both to assess the initial 6-year stage of ALR's efforts to stimulate and develop research on physical activity in daily life and to inform consideration of the program's re-authorization, including potential ways to enhance the program's function and its contribution to the more recently adopted goal of preventing childhood obesity. Two complementary assessments were instituted: (1) the Gutman Research Associates' (GRA) study to examine mainly the program's productivity and its progress in achieving its original goals and objectives during the past 6 years; and (2) the Group Health Community Foundation's study to focus more on the future of ALR within the context of childhood obesity prevention. Orleans et al.1 in this supplement provides more information on the decision to support two complementary evaluations. Although 6 years have passed since the program was authorized and 5 years since its first CFP, it is important to note that only 16% of the competitively awarded grants had been completed prior to 2006.

To achieve these purposes, the GRA study addressed four research questions that derived from the initiative's goals and the RWJF's information needs:

  • 1

    To what extent is ALR working to build the knowledge base on policy and environmental factors conducive to physical activity?

  • 2

    To what extent is ALR working to build human resources—in this case, a dynamic, transdisciplinary research community?

  • 3

    To what extent is ALR working to build additional financial resources for active living research?

  • 4

    To what extent are the research findings from ALR studies contributing to policy discussions on how to promote physical activity through policy and environmental change?

This paper presents the findings from the GRA study that address the four evaluation questions above, describing the evaluation's focus and methods and presenting the results generated as well as conclusions and recommendations.

Section snippets

Logic Model

The conceptual, or logic, model for the ALR program, developed by the NPO, was used to guide the evaluation as it addressed the four research questions (see Figure 1 in Sallis et al.4 in this supplement). While the model was generally useful for this purpose, three aspects of it needed refinement to provide optimal guidance for the evaluation. First, the model did not give explicit recognition to the program's context, notably the RWJF organization and the set of related Active Living programs

Methods

The evaluation was a retrospective, in-depth, descriptive study utilizing multiple methods and both qualitative and quantitative data. Most of the data collection took place from July 2006 through December 2006, with a few interviews and some of the quantitative analyses implemented thereafter, ending in February 2007; a final report was submitted in June 2007.

Building the Knowledge Base

An important goal of ALR was to build a knowledge base regarding environmental and policy correlates of physical activity that could inform policy change. The program implemented several major activities to build the knowledge base, notably developing a research agenda; developing and issuing a series of CFPs; peer review and the awarding of grants on a competitive basis; and the awarding of grants on a noncompetitive basis for commissioned studies (e.g., systematic literature reviews, White

Summary and Conclusion

Gutman Research Associates conducted a retrospective, in-depth, descriptive evaluation of ALR 6 years after the program's authorization and 5 years after the program issued its first of seven CFPs. The evaluation was intended to assess progress on program goals in order to inform re-authorization discussions at the RWJF and to guide future grantmaking efforts within the program itself. At the time of the evaluation, only 16% of 83 ALR studies awarded on a competitive basis had been completed,

Cited by (20)

  • The characteristics and reporting quality of research impact case studies: A systematic review

    2019, Evaluation and Program Planning
    Citation Excerpt :

    Milat et al. (2013) reported their research to have made contributions to organisational development and informed program planning (n = 1, 4%). More than half of the studies (n = 14, 56%) reported small to medium-sized contributions to, and/or changes to either local, national, or international policy (Ottoson et al., 2009, Bunn & Kendall, 2011; Caddell et al., 2010; Donovan et al., 2014; Greenhalgh & Fahy, 2015; Gutman et al., 2009; Harris, 2010; Jarman & Bryan, 2015; Kingwell et al., 2006; Klautzer et al., 2011; Milat et al., 2013; Raftery et al., 2009; Tulloch et al., 2011; Walker et al., 2010). However, some of the contributions to policy did not have any consequential effects or practical application (n = 3, 12%) (Bunn & Kendall, 2011; Ottoson et al., 2009; Tulloch et al., 2011).

  • Building the field of population health intervention research: The development and use of an initial set of competencies

    2015, Preventive Medicine Reports
    Citation Excerpt :

    This orientation for the competencies is consistent with field building for applied research. For example, the competencies are intended to strengthen both academic and professional disciplines (Isaacs and Knickman, 2005), and they aim to link research and action, which is essential in applied fields with a focus on policy and practice interventions (Gutman et al., 2009). Timely use of the PHIR competencies will help to capitalize on early successes in defining and building PHIR as a field.

  • Evaluation of active living research: Ten years of progress in building a new field

    2014, American Journal of Preventive Medicine
    Citation Excerpt :

    Considerable progress also was noted in its contribution to policy and practice, its third goal. As recommended by the ALR-1 evaluation,3 ALR-2 improved its outreach to the policy and practitioner community, and in doing so, helped to accelerate ALR research’s contribution to policy and practice. Of the 62 examples of policy contribution and/or impact from 2001 to 2011, slighty more than one half of these examples changed policy or practice, mainly at the local level.

View all citing articles on Scopus
View full text