You are here: Home User Information Facility Services Data Archiving

Data Archiving

by admin last modified Jun 01, 2007 04:16 PM
Archival storage for user files

The RACF facility provides a data archiving service for users that uses the HPSS mass storage system as a back end. The procedure for archiving data into HPSS differs between RHIC/LSST users and US Atlas users. RHIC/LSST users should use the native HPSS archiving tools to put files into HPSS. US Atlas users should use Dcache to archive files in HPSS. Link to HPSS Archiving Tools [RHIC/LSST Users only]

Users should note that the HPSS system is specifically designed to handle large files (> 1GB) files, small files will result in extremely poor HPSS performance. For reference, small file performance of HPSS is measured at a FEW FILES PER MINUTE per tape drive. In contrast, for large files, HPSS performance is measured in excess of 50MB per second per tape drive. Since HPSS is a shared facility that handles high bandwidth data streams from the RHIC and LHC accelerators, in addition to archived user files, users are strongly encouraged to aggregate small files into larger files using tools like tar before archiving files.

Document Actions