Ciprian Docan
ABSTRACT
Large scale simulations of complex physics phenomena have long run times and generate massive amounts of data. Saving this data to external storage systems or transferring it to remote locations for analysis is a costly operation that quickly becomes a performance bottleneck. In this paper, we present DART (Decoupled and Asynchronous Remote Transfers), an efficient data transfer substrate that effectively minimizes the data I/O overhead on the running simulations. DART is a thin software layer built on RDMA technology to enable fast, low-overhead and asynchronous access to data from a running simulation, and support high-throughput, low-latency data transfers.
- R. Brightwell, T. Hudson, K. Pedretti, R. Riesen, and K. Underwood. Implementation and Performance of Portals 3.3 on the Cray XT3. IEEE International Conference on Cluster Computing, September 2005.Google Scholar
Cross Ref
- K. S. Hasan Abbasi, Matthew Wolf. Live data workspace: A flexible, dynamic and extensible platform for petascale applications. In Cluster Computing, Austin, TX, September 2007. IEEE International. Google ScholarDigital Library
- C. F. S. Inc. Lustre: A Scalable, High Performance File System. http://lustre.org/docs/whitepaper.pdf.Google Scholar
- I. Inc. Gpfs: A Shared-Disk File System for Large Computing Clusters. www.almaden.ibm.com/StorageSystems/projects/gpfs/Fast02.pdf.Google Scholar
- N. Stone, D. Balog, B. Gill, B. Johanson, J. Marsteller, P. Nowoczynski, D. Porter, R. Reddy, J. R. Scott, D. Simmel, J. Sommerfield, K. Vargo, and C. Vizino. Pdio: High-performance remote file i/o for portals-enabled compute nodes. In H. R. Arabnia, editor, PDPTA, pages 925--930. CSREA Press, 2006.Google Scholar
- R. Thakur, W. Gropp, and E. Lusk. Data sieving and collective I/O in ROMIO. In Proceedings of the Seventh Symposium on the Frontiers of Massively Parallel Computation, pages 182--189. IEEE Computer Society Press, 1999. Google ScholarDigital Library
Index Terms
- DART: a substrate for high speed asynchronous data IO
Recommendations
DART: a programmable architecture for NoC simulation on FPGAs
NOCS '11: Proceedings of the Fifth ACM/IEEE International Symposium on Networks-on-ChipThe increased demand for on-chip communication bandwidth as a result of the multi-core trend has made networks on-chip (NoCs) a compelling choice for the communication backbone in next-generation systems [3]. However, NoC designs have many power, area, ...
DART: A Programmable Architecture for NoC Simulation on FPGAs
The increased demand for on-chip communication bandwidth as a result of the multicore trend has made packet-switched networks-on-chip (NoCs) a more compelling choice for the communication backbone in next-generation systems . However, NoC designs have ...
Dart: A Geographic Information System on Hadoop
CLOUD '15: Proceedings of the 2015 IEEE 8th International Conference on Cloud ComputingIn the field of big data research, analytics on spatio-temporal data from social media is one of the fastest growing areas and poses a major challenge on research and application. An efficient and flexible computing and storage platform is needed for ...
Comments