The Coastal Imaging Research Network (CIRN)

Abstract

The Coastal Imaging Research Network (CIRN) is an international group of researchers who exploit signatures of phenomena in imagery of coastal, estuarine, and riverine environments. CIRN participants develop and implement new coastal imaging methodologies. The research objective of the group is to use imagery to gain a better fundamental understanding of the processes shaping those environments. Coastal imaging data may also be used to derive inputs for model boundary and initial conditions through assimilation, to validate models, and to make management decisions. CIRN was officially formed in 2016 to provide an integrative, multi-institutional group to collaborate on remotely sensed data techniques. As of 2021, the network is a collaboration between researchers from approximately 16 countries and includes investigators from universities, government laboratories and agencies, non-profits, and private companies. CIRN has a strong emphasis on education, exemplified by hosting annual “boot camps” to teach photogrammetry fundamentals and toolboxes from the CIRN code repository, as well as hosting an annual meeting for its members to present coastal imaging research. In this review article, we provide context for the development of CIRN as well as describe the goals and accomplishments of the CIRN community. We highlight components of CIRN’s resources for researchers worldwide including an open-source GitHub repository and coding boot camps. Finally, we provide CIRN’s perspective on the future of coastal imaging.

1. Introduction

Coasts are located at the intersection of a dynamic landform, a changing climate, and high population pressure. A recent assessment of world-wide coastal environments indicates that more than half of sandy coastlines are actively changing [1] and three quarters of all coastlines have the potential for flooding [2]. Physical processes drive the morphologic evolution of our coastlines on timescales ranging from seconds to decades and centimeters to hundreds of kilometers [3]. Within this dynamic environment, climate change, rising sea levels (e.g., [4,5]), and coastal management actions have economic and ecologic impacts (e.g., [6,7,8]) that create complex feedbacks and alter coastal evolution (e.g., [9]). These pressures highlight the need for improved management, forecasting, and understanding of coastal change.

Practical and research needs for coastal information require long-term observations of nearshore hydrodynamics, bathymetry, and topography to provide context for current events. Management applications may require hourly to weekly estimates of shoreline position or estimates of bathymetry at a particular location [10]. A recent survey of Australian coastal geoscience and engineering researchers and practitioners found that observations of shoreline change over a range of spatial and temporal scales along with observations of hydrodynamic processes contributing to inundation are the top priorities [11]. Operational forecasts require observations, which may include information about waves, currents, or total water level elevation, to assimilate and assess model skill. However, operational assimilation is not yet common practice in coastal change forecasts, due largely to a lack of ubiquitous coastal data. Research models require data to improve physical or empirical parameterizations of processes. Yet, extensive observing networks in the coastal environment have lagged deep-water networks [3]. As climate change impacts become more prevalent along the world’s coastlines, the need for local coastal data will accelerate.

Collecting imagery of the coasts from fixed (long-term video monitoring stations) and moving (Unoccupied Aerial Systems (UAS), satellites) platforms provides an abundant source of high spatial and temporal resolution hydrodynamic and morphologic data in the nearshore environment that can satisfy many of the above needs. The signatures of nearshore processes, including shoaling and breaking waves, foam advecting on currents, surface-roughness patterns, and turbidity are all sensible using different imaging modalities including optical imagery, lidar, or radar [12], and similar processing algorithms may be used across multiple modalities.

Coincident with needs for coastal monitoring via remote sensing is the need to develop the next generation of coastal scientists and engineers who can exploit these data. However, recent reviews have identified the imbalance between societal needs for coastal information and the development of new scientists and engineers who have the skills to address those needs [3,13]. The U.S. National Research Council suggested building a coastal science and engineering consortium able to leverage the resources across a range of organizations to address research and educational needs [13]. A more recent report highlighted the need for community support, collaborations, leveraging, and avoiding redundancy to foster rapid advancement in the fields of coastal science and engineering [3]. Similar approaches may be applied to the specialized study of coastal imaging.

The purpose of this review paper is to describe the development of the Coastal Imaging Research Network (CIRN) within the context of remote sensing of nearshore physical processes and coastal geomorphology. The manuscript reviews the community supported resources that CIRN provides in the context of addressing gaps in methods to process coastal imaging data and providing educational opportunities for the community. In the remainder of Section 1 we summarize history and methodology of optical coastal imaging, as well as the challenges for the coastal imaging community. In Section 2, Section 3 and Section 4, we describe the development of the Coastal Imaging Research Network and contributions to the coastal imaging community. In Section 5, we describe future research and directions expected within the coastal imaging community.

1.1. History of Coastal Imaging and Argus

Aerial photography as a means to map the coastline and observe waves began as early as the 1930s and 1940s (e.g., [14,15]). However, close-range coastal imaging from a fixed platform as a means for understanding nearshore physical processes began in the 1980s, most notably by the Argus program developed within the Coastal Imaging Lab (CIL) at Oregon State University (OSU) [16]. Over the course of four decades, laboratory members and collaborators built extensive technical capabilities, knowledge base, and repositories of code to derive quantitative observations from optical imagery. The novel observations made possible through coastal imaging were characterized by high spatial resolution (compared to instrument arrays) observations of the water surface and subaerial beach at timescales ranging from seconds to multiple decades. The observations resulted in a new understanding of the coastal environment.

Underpinning the methodological and scientific developments over the course of the history of coastal imaging was a drastic evolution in imaging technology. Technological improvements were characterized by the decreasing cost of equipment, the increasing sophistication of electronics, and the increasing quality and quantity of image data. The first coastal images were snapshots and time-exposure images collected with Single-Lens Reflex (SLR) film cameras [16]. Time-exposure images represent the average image intensity over a user-defined period. With the development of video cassette recorders and image processors, time series of waves and currents became readily obtainable, further expanding possibilities for image analysis. Electronics continued to advance with digital video and high-quality industrial cameras. Fast internet and cellular technology have enabled transfer of data more rapidly and from more remote locations. Early coastal imaging systems, like those developed at Oregon State University, required a fairly expert level of knowledge to collect and process imagery. Over the last decade, the advent of smart phones [17,18,19], action [20,21,22], web-based beach and security cameras [23,24,25,26,27], as well as low-cost, high-quality optical imaging from UAS have made coastal imaging more accessible [21,28]. The ubiquity of UAS technology has also made low-cost infra-red sensors more accessible for coastal imaging applications [29,30].

Beginning in the 1990s, an important component of the Oregon State University-based coastal imaging community was workshops. The workshops were designed to share technical knowledge about imaging systems, report on recent scientific advances, as well as provide time for the community to work together on coastal imaging problems and code development.

Throughout the 1990s and early 2000s, the community grew to include collaborators in Australia and Europe and a commercial version of Argus. Work at the University of New South Wales (UNSW) led to the development of extensive remote sensing capabilities for research purposes, which fundamentally shaped our understanding of nearshore coastal behavior as techniques evolved from qualitative observations and traditional survey methods to remote sensing methods using optical and lidar imaging from fixed-platforms and UAS [31]. UNSW was also responsible for the development of practical beach management tools. The core management tools included shoreline positions and proxies for bathymetry from time-exposure images. Applications included management of sand-by-passing system, beach nourishment, and monitoring of an artificial surfing reef [10,32]. Collaborations with the European research community led to the CoastView Project, which brought together researchers and managers from five countries to test new methods for coastal management using fixed-platform remote sensing at four European beaches representing different coasts [33,34]. The CoastView project relied on video-derived coastal state indicators including estimates of beach volume and shoreline positions [35]. A third collaboration focused on fixed-platform remote sensing, called BeachWizard. The project developed a data-assimilation approach for video and radar images to estimate nearshore bathymetry from intertidal shorelines, wave celerity, and wave roller dissipation [36]. With this project, the first nowcast estimates of nearshore bathymetry were prototyped [37]. During this time, a commercial version of Argus also became available, which helped to expand the community of scientists applying coastal imaging methods.

Simultaneous to these advances within the Argus community, additional research groups around the world also began exploring fixed-platform coastal imaging techniques and developed their own research directions. By 2015, a number of coastal imaging systems were documented in the literature [38,39,40,41,42] or available commercially [43,44]. These systems produced similar image products to Argus, and some offered additional features, in particular, an open-source approach. In addition to the single-view camera approach that dominated the Argus community, other groups explored the use of multiple camera views to reproduce 3-dimensional scenes of waves (e.g., [45]).

Developments within the land-based coastal imaging community were largely independent of developments within the satellite-based earth imaging community (see [46] for a broad overview of coastal satellite applications). The spatial scales of nearshore hydrodynamic processes of interest to the coastal imaging community were on the order of meters to 100s of meters and timescales of seconds to hours. Additionally, many of the coastal imaging algorithms required dwell times of 10 minutes or longer to adequately sample the processes of interest. For much of the history of coastal imaging, these scales were much smaller than the pixel resolution of satellite imagery. With the advent of satellite imagery data available through application programming interfaces (e.g., [47]) and higher spatial resolution, algorithms first developed for land-based coastal imaging have begun to be applied to satellite imagery (see [48] for a review).

1.2. Coastal Imaging Methodology

The development of land-based coastal imaging described in Section 1.1 relies on two primary concepts. The first is the processing of a time series of images of the coastal environment to produce image products that can be analyzed to extract information about oceanographic phenomena. The second is that images can be projected into world coordinates using geometric principles and linear algebra. Here, we summarize both concepts briefly to provide an introduction to coastal imaging techniques.

The first methodology commonly applied in coastal imaging research is the processing of a time series of images of waves, currents, and beaches. Composite images produced by taking the mean, standard deviation, minimum, or maximum of image intensity at each pixel in an image over a (typically) 10-min period are known as time exposure (or timex), variance, darkest brightness-filtered, and brightest images, respectively. The composite images can reveal useful information, not obvious in a single snapshot (compare snapshot Figure 1a to timex in Figure 1b). For example, a timex image can be used to map the location of wave breaking and residual foam, indicating the location of the shoreline or nearshore sandbar [49,50]. Variance images can be used to identify persistent variations in an image, for example the edge of wave breaking zones. The darkest images have most utility in removing bright foam from the image, and have been used to better image nearshore morphology [51]. The brightest images have been used to map the extent of the surf zone [52] and may be used to map the highest extent of wave run up. In addition to the creation of composite images, pixel time series may be sampled from the video and can be a useful way to quantify time-varying coastal processes without needing to retain a full-frame time-series. Pixel time series are most commonly extracted in cross-shore transects, alongshore transects, grids, or at specific points of interest (e.g., above an in-situ sensor). Cross-shore transects, particularly near the shoreline, are frequently used to measure wave run up time-series [53,54]. For example, the position of the observed total water level (green dot, Figure 1) was estimated by tracking the edge of waves running up the beach along a shore perpendicular transect of pixels through time (blue line, Figure 1b). Alongshore transects, particularly within the surf-zone, can be used to track advecting foam and measure alongshore currents [55]. Cross-correlation or cross-spectral analysis between pixels extracted in grids, can be used to produce quantitative information about waves, including their celerity, period, and angle, and thus to estimate bathymetry [56]. CIRN software tools for producing image products and time series analyses of pixels are described in Section 3.

In order to make the observations of coastal processes described above quantitative, we must have knowledge of where each pixel in the image is located in the real world. The relationships for mapping between images and the real world are widely understood in photogrammetric and computer vision applications (e.g., [57,58]). In order to describe the projective matrix for mapping between the real-world and image coordinate systems, two sets of parameters, intrinsic and extrinsic, must be established through camera calibration. The intrinsic calibration finds coefficients that are functions of the camera and lens, including the focal length, image center, pixel skewness, and lens distortion. The extrinsic calibration finds six coefficients describing the location and orientation (heading, roll, and pitch). Once the intrinsic and extrinsic coefficients are used to establish the projection matrix, image coordinates maybe projected into map space using known beach and water level elevations, and quantitative measurements can be made from coastal images. For example, the image shown in Figure 1b was mapped into a local coordinate system aligned with the orientation of the shoreline (Figure 1c). Complete details on camera calibrations and projections for coastal imaging applications are available in the literature (e.g., [27,57,59,60]), and CIRN software tools for conducting calibrations and projections are described in Section 3.

1.3. Scientific Advancements

Coastal imaging methodology, developed over the previous four decades, has contributed to a new understanding of nearshore physical processes by complementing in-situ observations and surveys. Splinter et al. [31], Holman and Stanley [16], and Holman and Haller [61] provide comprehensive reviews of fixed-platform coastal imaging contributions, so we only summarize the scope of contributions here. The first two decades of coastal imaging were dominated by improved observations and understanding of nearshore morphologic behavior. Research focused on developing analysis approaches to quantify the evolution of shorelines, sand bars, rip current channels, and inter-tidal topography at a range of timescales. The high-spatial and temporal resolution data sets, which could resolve the morphologic evolution at the same timescale as forcing observations (i.e., hourly measurements), enabled new insights into surf-zone morphodynamics. In the second two decades, recognition of the similar time and space scales of remotely sensed data and numerical simulations led to the exploration of the application of data assimilation in nearshore hydrodynamics. The complementary technologies enabled researchers to investigate wave kinematics, complex circulation patterns, and related morphologic behavior. Long-timeseries observations also allowed for the development of data-driven models of morphologic features including sandbars and shorelines. Additionally, coastal imagery provided an excellent source of observations for model hindcast and forecast skill assessment (transparent red band and red line in Figure 1b,c).

1.4. Challenges for the Coastal Imaging Community

A successful multi-decadal collection of Argus optical coastal imaging data resulted from the automated, quantitative tools developed in the OSU CIL. However, by the early 2010s, few academic institutions were teaching coastal imaging to new students, resulting in a decreasing number of contributors to the coastal imaging community. By 2016, members of the coastal imaging community recognized the need to develop a broader community with open-source tools and educational materials in order to sustain the continued advancement of these tools. At the same time, new commercial off-the-shelf imaging systems were becoming readily available, making image data collection less challenging than in early years. The need to decouple data collection, which now could happen in many ways, from data processing became apparent.

2. Coastal Imaging Research Network

2.1. CIRN Creation

The Coastal Imaging Research Network (CIRN) was founded during the 2016 Argus Workshop which took place at the U.S. Army Engineer Research and Development Center’s Field Research Facility (FRF) in Duck, NC, USA. The meeting commemorated 30 years of Argus coastal imaging at the FRF [62] and was attended by 30 researchers. During the breakout sessions, meeting attendees recognized the value of the automated, quantitative tools that had been developed out of OSU’s CIL, but also agreed that for coastal imaging to expand its footprint in the scientific, engineering, and practical application communities, we needed to address sustainability issues, to move from a centralized to a decentralized network, and to increase the overall visibility of our coastal imaging community. To progress towards these goals, CIRN was established, and an initial set of priorities was developed.

• Increased growth, visibility, and knowledge of the community;
• Development of modular, application-based data products;
• Development of a new organizational structure;
• Establishment of an International Coastline Observatory Network (ICON);
• Development of measures for success.

2.2. CIRN Goals and Mission

CIRN’s first goal focuses on developing a lively and inspiring network of researchers and users that promotes and facilitates open collaboration, interaction, and education revolving around coastal imaging advances worldwide. Researchers who attended the 2016 Argus workshop established a commitment to open and accessible collaboration through sharing of innovations in software and hardware developments. Participants identified initial steps towards promoting collaboration. These steps included a focus on developing capabilities for use beyond the single user, establishing open-source codes, improving documentation, creating a modern web presence, decreasing the interval between community meetings, and training students and new users. Central to accomplishing this goal was a focus on the development of an open-source code repository, establishment of a coastal imaging coding boot camp curriculum, and development of a social media presence.

CIRN’s second goal focuses on the development of modular, application-based data products. In 2016, a few long-term coastal imaging stations existed for research purposes, and their data collection and associated automated processing codes were largely based on legacy CIL software, which required specific Argus hardware, directory structures, and database tables. While Argus stations are very robust and represent cornerstones of the community’s research and development progress, the single platform strategy was not sustainable for wide-spread use. Participants recognized that the path forward for data collection may include a range of optical sensing platforms, including video from satellites or UAS, cameras of opportunity, or simple imaging systems (e.g., action cameras made by manufacturers such as GoPro, Garmin), in addition to complex research-grade camera installations. Thus, CIRN’s second goal focused on transitioning to the development of independent toolboxes for data processing that would decouple data from the acquisition process and the Argus/CIL database and directory structure. Development of new software would focus on robust, automated, and modular toolboxes applicable to any imagery collected by any method, similar to the existing cBathy codebase (https://github.com/Coastal-Imaging-Research-Network/cBathy-Toolbox, accessed on 30 September 2021). Figure 2 was created during the 2016 workshop to visually describe the paradigm shift. Central to accomplishing this goal was the establishment of an open-source code repository where modular toolboxes could be developed, tested, and shared with the community.

CIRN’s third goal focuses on the development of a new organizational structure thaat included a distributed volunteer leadership committee spread across the international academic research community, including long-standing Argus community collaborators from OSU, TU Delft, and the University of New South Wales; government research agencies (U.S. Geological Survey, U.S. Army Engineer Research and Development Center, and the U.S. Naval Research Laboratory), and private industry. A smaller leadership group was also established to ensure the maintenance of the enabling technologies (e.g., code repository, boot camp material), and to plan and schedule frequent workshops, keeping CIRN sustainable into the future.

CIRN’s fourth goal, the development of an International Coastline Observatory Network (ICON), has received less initial attention in the first five years of CIRN, but remains supported in the community. The goal of ICON is to establish long-term coastal monitoring stations, which are research driven and combine remotely sensed observations, in-situ observations, and models. ICON will cover a broad range of hydrodynamic/sedimentary environments, having world-wide representation, and data would be publicly available. More information on ICON is provided in the future work section, and a description of efforts towards establishing ICON in the Netherlands can be found in [63].

CIRN’s final goal, to develop the measures of success of the network, was included to help ensure the network was making meaningful contributions to nearshore science. CIRN collaborators defined the impact as the number of publications associated with CIRN, involvement of new principal investigators, and the number of new students and young professionals utilizing CIRN tools. Initial CIRN accomplishments over the first 5 years are discussed in the following section, and quantify the impact CIRN has had on developing a thriving coastal imaging community.

2.3. CIRN Accomplishments

Since its inception in 2016, CIRN held annual in-person meetings until the pandemic began in 2020. In 2017, the community assembled to teach the first “boot camp” focused on coastal imaging fundamentals (described in Section 4) with 25 attendees hosted by the U.S. Army Engineer Research and Development Center. In 2018 and 2019, the in-person meetings became known as “CIRN Week” and included a 2-day boot camp followed by a 3-day workshop, which featured scientific presentations as well as small group discussions on the development of the coastal imaging research community. The 2018 CIRN Week was hosted by the U.S. Geological Survey. In 2019, more than 90 people attended CIRN Week, hosted at the Laboratoire d’Etudes en Géophysique et Océanographie Spatiales. CoastSNAP [19], a new citizen science effort held their user’s meeting as part of CIRN Week, and leadership participants met to define the scope of the ICON effort. The 2020 and 2021 meetings were postponed due to the pandemic.

In addition to CIRN Week boot camps, a mini-CIRN boot camp was taught as a Coastal Imaging Short Course at the Coastal Sediments 2019 meeting in Saint Petersburg, FL, USA and as part of the Young Coastal Scientists and Engineers of Australia Conference in 2019. A new session was also established at the American Geophysical Union’s Ocean Sciences Meeting in 2018, 2020, and upcoming 2022 meeting, called “Close-range Remote Sensing of Nearshore Processes and Coastal Morphology”, which has consistently had over 50 abstract submissions. CIRN also established a LinkedIn group (https://www.linkedin.com/groups/12010767/, accessed on 30 September 2021), which now has 271 members. Internship opportunities, new journal articles, updates on field efforts, training opportunities, and conference sessions are all shared through the LinkedIn group, addressing the CIRN goal of community development.

Integral to community growth and development was the creation of an open-source code repository and boot camp teaching curriculum. In five years, the open access code repository has grown to include almost 40 members, more than 20 repositories have been developed, and CIRN toolboxes have been taught to more than 100 new people. These efforts are described in detail in the following two sections.

3. CIRN Code Repository

CIRN placed high priority on the development of open-access code repositories to enable community development and scientific advancement through sharing. To that end, CIRN established an open-access organization on GitHub (https://github.com/Coastal-Imaging-Research-Network, accessed on 30 September 2021) where contributors can upload, download, and develop codes collaboratively. The development of the CIRN repositories enabled the community to use modern coding collaboration tools and encourage open development and sharing of algorithms. Algorithms are grouped into modular toolboxes and updated using proper version control. Codes are shared under the GNU General Public License v3.0.

CIRN selected GitHub to host and share the repository because of its front-end web presence, which provided a central access point, enabled easy communication between users, had a built-in Wiki for documentation, and helped wrap the Git version control functionality in an accessible and flexible manner. For example, users may access repositories through a command line interface, graphical user interface (GUI), or download the entire repository through a web-interface. CIRN uses the Wiki capabilities to document all of the functionality of the repositories, and contribution of new repositories requires the development of a user manual. In addition, CIRN developed an over-arching Wiki and Forum for general discussions about coastal imaging (https://github.com/Coastal-Imaging-Research-Network/Forum-Wiki, accessed on 30 September 2021) to improve knowledge transfer and establish continuity. The Wiki and Forum helps introduce CIRN to new users and documents historic innovations. Naming conventions and standard data structures are documented in the Wiki as well. Background information on designing coastal imaging station and data processing techniques are also described on the Wiki. To date, the general CIRN Wiki has over 60 different pages of information.

The transition to version control for CIRN codes marked a large step forward in maintaining continuity of algorithms, preventing code loss, and improving reproducibility of results within the coastal imaging community. For example, for popular codes like cBathy, different versions of the code are now documented with official releases. Users can clearly cite which version of the code they are running in their publications, improving interpretability of results. In addition, version control has enabled seamless collaborative development without compromising the integrity of the code base. Any new update is now submitted as a pull request with documentation and testing. Each pull request is reviewed by at least two CIRN administrators, who are intimately familiar with the code base, before the code is approved for incorporation in the repository. Efforts are underway to develop a series of clear test scripts that can be run to ensure any new developments improve code accuracy, speed, functionalities, or readability. Once a new repository is contributed, it is maintained and curated by the developer and a group of volunteer CIRN developers.

The organization of CIRN code into repositories provided the foundational structure to help meet the CIRN goal of developing easy-to-use, modular, application-based products. CIRN repositories consist of CIRN-developed Core Repositories, based on legacy CIL code, Community Contributed Repositories, and Forked Repositories from other related organizations or coastal imaging efforts. There are five core CIRN repositories: cBathy, the Quantitative Coastal Imaging Toolbox, the PiXel-Toolbox, the argusDB0, and the Support-Routines toolbox. The latter three toolboxes descend directly from CIL codes and are largely used for data handling and organization, as well as support photogrammetry functions. The Quantitative Coastal Imaging Toolbox [60] provides the foundational codes for initial processing and rectification of imagery data. The Toolbox includes codes to produce geo-rectified images of oblique imagery from land-based multi/single-camera stations or stationary UAS. In addition to rectified frames, the toolbox produces ensemble products such as statistical or subsampled pixel collections for optical wave, current, and bathymetric inversion analyses, as well as the original Argus image products: Timex, variance, brightest, and darkest images [16]. The toolbox includes demos that teach fundamentals but also provides the backbone code infrastructure to utilize in turnkey applications for operational data processing. The Quantitative Coastal Imaging Toolbox codes were developed from legacy CIL codes and the original UAV-Processing-Toolbox [28] (still available in the CIRN repository), and are used as the foundational codes for the coding boot camps discussed in the following section. The cBathy Toolbox is a widely used frequency-based linear depth inversion method and is taught in advanced boot camp sessions [56].

In addition to CIRN-developed repositories, a number of community members have contributed their own algorithms as repositories on the CIRN GitHub. Many of these repositories have associated publications documenting their development, like the Station-Design-Toolbox, a helpful planning tool for installing coastal imaging observation stations [64], the CoastSnap-Toolbox, a unique citizen science effort to use imagery from smart phones to monitor shoreline change [19], and the Video-Currents toolbox, an algorithm to extract longshore current magnitudes based off of the techniques described in [55]. Additional toolboxes include algorithms to digitize wave runup from timestacks of video imagery (runupTool-Toolbox), propagate uncertainty through photogrammetry equations (Photogrammetry Propagated Uncertainty), extract the wet/dry shoreline from imagery (Shoreline-Mapping Toolbox) [65], and to digitize emergent sandbars [66]. Development on the CIRN repositories is active, and CIRN expects to see additions of new satellite-based depth inversion algorithms in the coming year (e.g., [67]).

Finally, the CIRN Github also contains forked repositories from other coastal imaging repositories, including CoastSAT, SurfRCaT, and guiBathy. CoastSAT [68,69] contains algorithms for quantifying coastal change from shorelines extracted from Landsat and Sentinel-2 imagery, accessed through the Google Earth Engine. SurfRCaT contains algorithms to transform pre-existing cameras on U.S. coastlines into coastal monitoring tools by facilitating remote camera calibrations and image rectification [27]. guiBathy is a downloadable Matlab executable that packages codes from the Quantitative Coastal Imaging Toolbox and cBathy within an easy-to-use GUI to enable users less familiar with code to be able to process imagery from a UAS into bathymetry data [70]. The GUI is similar in concept to a GUI developed for the original UAV-Processing-Toolbox [71], but adds the functionality to run cBathy as well as a parametric approximation of depth from timex imagery [72,73]. The source code for guiBathy is also hosted separately in the CIRN Repository.

4. Coding Boot Camps

CIRN coding boot camps were developed to advance the CIRN goal of increasing community knowledge and creating a cadre of new developers for the CIRN repository. At the time that CIRN was established, few academic programs were focused on coastal imaging. The boot camp curriculum was designed to teach coastal imaging techniques to novices. The target audience for the boot camp has coding skills and a graduate-school level knowledge of nearshore physical processes. The goals for boot camp attendees are:

• Learn to use optical or infrared cameras to quantify nearshore physical processes;
• Develop an understanding of optical cameras and basic photogrammetry;
• Use the CIRN repository to process data from a UAS-mounted camera and multiple fixed cameras.

The CIRN boot camp format includes lectures on photogrammetry fundamentals as well as practical applications (Figure 3). The introductory lectures are coupled to the Quantitative Coastal Imaging Toolbox. Advanced lectures are focused on other toolboxes (e.g., the cBathy Toolbox). A series of online video lectures are available on YouTube (https://www.youtube.com/channel/UC-WEMuYqe81TmuV9lA7W9YA, accessed on 30 September 2021) to make the boot camp material more widely accessible, and the Quantitative Coastal Imaging Toolbox may be used for independent study.

The progression of boot camp lectures is described here. The first lecture in the series covers an introduction to coastal imaging including the role of imaging in understanding the nearshore system and the need for data to effectively hindcast, nowcast, and forecast the nearshore system. The additional lectures focus on developing an understanding of photogrammetry to conduct basic coastal imaging analyses. Topics include the basics of the camera projection matrix, intrinsic and extrinsic calibration needed to rectify imagery, and the production of statistical image data products created from full-frame imagery, and spatial subsampling of imagery for use in time series analysis. Lectures on photogrammetry are complemented by lectures providing practical insight into understanding image vocabulary, camera hardware, and considerations and tools useful when designing a new coastal imaging station or UAS data collection.

Boot camp attendees participate in a series of practical exercises designed to prepare them to conduct their own coastal imaging analyses. Students learn to perform a laboratory-based intrinsic lens calibration, including collecting images and conducting a calibration using the CalTech lens calibration toolbox for Matlab [74]. Students also perform an extrinsic camera calibration using the Quantitative Coastal Imaging Toolbox using previously collected imagery from a UAS-mounted optical camera and survey ground control points. In the final practical exercise, students produce rectified image products and pixel time series generated from spatially sub-sampled videos. These products can be used for quantitative analyses or further processing with other CIRN toolboxes.

Additional boot camps sessions are focused on advanced CIRN Toolboxes and developing practical skills. Previous boot camps have included lectures and practical exercises introducing the cBathy Toolbox, CIRN coding standards, naming conventions, and guidelines for contribution to the CIRN GitHub repository. Many boot camp attendees are not familiar with contributing to a large code base using version control, so one session introduces GitHub. Future boot camps will include sessions on a wave runup time series (RunupTool), alongshore current velocities (Video Currents Toolbox), SurfRCaT, and other algorithms as volunteered by community members and available in the CIRN GitHub repository.

5. Future of Coastal Imaging

5.1. Technological Advancements and Challenges

Digital imagery permeates all aspects of our world. Traffic cameras, beach and surf cameras, security cameras, smartphones, and perception sensors on self-driving cars are all examples of “cameras of opportunity” that can be exploited for the quantification of environmental processes. The volume of imagery and variety of image data sources are constantly expanding at both small and large spatial scales–smartphone cameras are now coupled with lidar to enable 3-dimensional depth perception and sensing, whereas satellite resolution and re-visit rates are increasing rapidly while large satellite image data sets are becoming more easily accessible [47], enabling sensing of a range of coastal processes from space [48], including the collection of high-frame-rate videos. These new imagery sources only exacerbate the known problem of the “proverbial fire hose” of coastal imaging data [61].

In this subsection, we present four approaches to addressing the increasingly data-rich problem in coastal imaging. Specifically, we envision leveraging edge computing, cloud computing, machine learning, as well as leveraging already mature coastal imaging algorithms. Approaching the data problem with this combination of methodologies will enable the coastal imaging community to fully exploit real-time analysis and address challenges with volumes of new data.

As the “internet of things” continues to develop, access to real-time data will expand, allowing for a decrease in time between data collection and data exploitation and laying the groundwork for real-time analysis. One path to address rapid analysis is an edge computing approach, where data needed for real-time application are processed at the “edge of the network” [75]. A primary advantage of pursuing edge computing is the efficiency gained in processing data where it is collected and reduced to a bandwidth relative to sending full data sets to a centralized location for processing [75]. However, constraints to this approach include more processing power and memory at the collection computer and limitations on battery life [75] for remote imaging stations. An additional trade-off for edge computing approach is that imagery cannot be reprocessed for new information once the original data are lost. Already, some imaging systems, including the original Argus hardware [16,61], implement an edge computing approach to produce statistical image products at the computer where data is collected, saving full time-series at only a handful of pre-determined pixels.

As an alternative, a cloud computing approach may be taken. Cloud computing, which came to prominence in the early 2010s [76], is defined as a model allowing access to shared computing resources, the scale of which can be adjusted on demand [77]. Using this approach for coastal imaging, a volume of data from a variety of sources may be pushed to the cloud and processed efficiently. Cloud computing has the advantage of being scalable and cost effective [76]. Pushing large volumes of data is bandwidth-limited, however, and comes with the drawback of the potential for slow response times for real-time data products [75].

In the five years since CIRN’s inception, the use of machine learning algorithms to exploit coastal imagery has grown. Data-driven approaches have been used to extract nearshore parameters from imagery including bathymetry [78,79], wave heights [80], wave breaking type [81] and occurrence [82,83,84,85], shoreline position [86], and land cover [87]; to classify nearshore morphology [88]; and to identify dangerous flows, like rip currents [89]. These data-driven approaches can increase analysis capabilities, enabling rapid extraction of data and improving ease of use in engineering and science applications. The linking of our physics-based understanding of nearshore processing with data-driven algorithms will be crucial for developing the next generation of analysis techniques.

Machine learning methods described above can be augmented by well-established algorithms for coastal image processing that are well established and available for implementation from open code repositories like CIRN. Efforts to develop algorithms that can be applied independently of collection hardware have been fruitful. For example, shorelines can be extracted using similar algorithms from both satellites and smartphones, and built off of algorithms originally developed for fixed-platform optical imaging systems e.g., [19,65,69,90]. Bathymetry can be estimated using the cBathy algorithm applied to optical, IR, or radar imagery from fixed or mobile platforms, e.g., [56,91]. Data are plentiful, but there are still relatively few sites where techniques have been applied, and the ever-growing volume of data collected requires new processing approaches.

In practice, a hybrid edge-cloud approach may be most efficient, where initial processing, using machine-learning or traditional image processing algorithms, occurs at the location where imagery is collected. Data initially processed at the collection site can be pushed to the cloud for further processing. In the case of informing numerical models with imagery products, the assimilation of image data into models will likely be completed most efficiently in the cloud. Finally, data and visualizations can be shared from the cloud to distributed end-users through web portals and cloud-based storage of processed results.

Ultimately, the coastal community is undergoing a paradigm shift, where coastal management and engineering decisions will no-longer have to be made in “data-starved” or “processing-limited” environments. CIRN envisions a future, within the next decade, where coastal imaging data from a variety of sources–localized continuous imaging stations, satellites, cameras of opportunity, citizen’s smart phones–will be combined with robust, automated algorithms and real-time modeling frameworks to provide an omnipresent “now-state” understanding of our coasts. This “now-state” will be able to be combined with coastal hazard forecasts (e.g., https://coastal.er.usgs.gov/hurricanes/research/twlviewer/, accessed on 30 September 2021) to enable informed coastal management and engineering decisions at global scales. CIRN endeavors to support the community during this exciting time by facilitating collaboration and knowledge exchange to support scientific advancement.

5.2. Understanding Physical Processes

Integration of technologies and algorithms from other disciplines will allow observations of the physical processes shaping our coasts at an unprecedented resolution. Up to this point, the majority of fixed-platform coastal imaging has been conducted in the optical band. Exploration into the use of wavelengths other than the visible band (e.g., infrared, hyperspectral) and more active sensing approaches (e.g., lidar, x-band radar, synthetic aperture radar (SAR)) must continue, and will likely accelerate with decreasing costs of more sophisticated sensors. For example, 3-dimensional observations of individual breaking waves using lidar have revealed new information on the complex kinematics and geometry of spilling and plunging breakers and propagating bores [29,30,92,93]. Fusion between data sources will likely lead to an improved resolution of processes and better parameterization of those processes in numerical models [29,85]. A multi-modal approach camera calibration has been shown to be effective as well [27]. In addition to data fusion, applying multiple processing techniques across one or more wavelengths has been shown to improve the resolution of processes, with each bandwidth and methodology responding to a different aspect of the process of interest [94].

The exploitation of image processing and computer vision techniques from other fields can aid in the extraction of nearshore processes data. The coastal imaging community has a history of augmenting 2-dimensional image products with stereo imaging [57,95,96]. Now, commercial structure-from-motion photogrammetry software, typically used to reconstruct stationary terrain data, makes sea-surface reconstruction without bespoke software possible, enabling better, more accessible processing of 3-dimensional sea-surface elevations in the laboratory and field environments [97,98]. Likewise, optical flow algorithms developed in the computer vision discipline can be applied to extract two-dimensional surf-zone flows [99].

Pairing in situ measurements, multiple modalities of remote sensing, and the processing described above, along with high-resolution research models will allow for better understanding of the spatial and temporal variability of surf-zone hydrodynamic processes. The new insights gained from these observations may help close the gap in our understanding of the complex sediment transport processes and feedback driving morphology evolution.

5.3. CIRN Community

To continue to address community needs for education of new coastal scientists, engineers, and practitioners [3], CIRN will continue to pursue the primary priorities of increasing knowledge of coastal imaging within the community and developing and curating new image processing tools. To lower the barriers to entry, future CIRN Week activities will include a hybrid component, allowing researchers from around the world to participate without the need to travel. CIRN organizers and developers are committed to continuing to add new tools to the CIRN GitHub repository and curate existing repositories. Additionally, new tools will be encouraged to be developed in Python rather than Matlab, and existing tools will be translated to Python to provide a fully open-source code base.

The coastal community recently identified the need to expand the capability to make long-term nearshore observations to improve our understanding of coastal change and the impacts of extreme events, particularly in the face of a changing climate [3]. CIRN will take a grass-roots approach to addressing this need by prioritizing the development of ICON sites and research-grade sites which can be used as the proving ground for new technologies. Open-access, routine monitoring, and support data at ICON sites, exemplified by data collection at the FRF, will be critical ground-truth to rigorously evaluate new methodologies for the coastal engineering and science community (e.g., [100]). Owners of ICON sites will strive to assure data collected will be Findable, Accessible, Interoperable, and Reusable (FAIR) [101]. In addition, the plethora of data at ICON sites will enable the development of robust real-time model-data assimilation frameworks and model skill assessment that exploit advanced simulation capabilities and synoptic remotely sensed data.

6. Conclusions

This article documents the inception and accomplishments of the Coastal Imaging Research Network (CIRN) in the first five years of its existence. CIRN was established to improve education and access to open resources within the coastal imaging community needed to maximize rapid developments in remote sensing technology. The resulting code repository, boot camp, and workshops contribute to sustaining coastal imaging expertise and the development of new scientific understanding of nearshore physical processes in a data-rich future.