Contributing Storage using the Transparent File System

TFS

A key barrier to the adoption of contributory storage systems is that contributing a large quantity of local storage interferes with the principal user of the machine. To overcome this barrier, the Transparent File System (TFS) is introduced which provides background tasks with large amounts of unreliable storage without impacting the performance of ordinary file access operations. TFS is a method for contributing disk space in peer-to-peer storage systems.

Problems found:

  • Existing contributory storage systems degrade normal application performance. They has ignored their local performance impact; in particular, as more storage is allocated, the performance of the user’s file system operations quickly degrade.
  • Even with the fact that end-user hard drives are often half empty, users are generally reluctant to relinquish their free space. Besides, users view storage space as a limited resource.

Transparent File System (TFS):

  • A file system that can contribute 100% of the idle space on a disk while imposing a negligible performance penalty on the local user.
  • Operates by storing files in the free space of the file system so that they are invisible to ordinary files. Normal file allocation proceeds as if the system were not contributing any space at all.
  • Ensure the transparency of contributed data: the presence of contributed storage should have no measurable effect on the file system, either in performance, or in capacity.
  • Transparent files: files that are transparent in the local file system (local file system does not see these files).
  • Transparent data or transparent blocks: data belonging to such transparent files

Treats transparent blocks as if they were free, overwriting whatever data might be currently stored in them. An ordinary file can be allocated over a transparent file. Allocation Policy. A storage block can have different states:

  • in a typical file system (ext2): free and allocated.
  • in TFS can be in one of five states: free, allocated, transparent, free-and-overwritten, and allocated-and-overwritten.

TFS imposes nearly no overhead on the local user.

  • Minimizes the interference with the file system’s block allocation policy
  • Sacrifices persistence for contributed space: normal files may overwrite contributed space at any time

TFS is specifically designed for contributory applications. The key benefit is that it leaves the allocation for local files intact, avoiding issues of fragmentation

Drawbacks

  • Sacrifices file persistence. When TFS allocates a block for an ordinary file, it treats free blocks and transparent blocks the same, and thus may overwrite transparent data. Files marked transparent may be overwritten and deleted at any time.
  • Applications using transparent files must ensure the correctness of all file data.
  • Sensitive information stored in ordinary files must be protected from applications trying to read transparent files.

The unreliability of files in TFS produces a bandwidth consumption due to the replication that is needed to handle deleted files.

Other features:

  • Especially useful for replicated storage systems executing across relatively stable machines with plentiful bandwidth (university or corporate networks).
  • This environment is the same one targeted by distributed storage systems such as FARSITE

For high-failure modes, such as wide-area Internet-based systems, the key limitation is the bandwidth between nodes, not the total storage.

  • The bandwidth needed to replicate data after failures essentially limits the amount of storage the network can use. In a stable network, TFS offers substantially more storage than dynamic.

P.S.:

TFS is neither a Distributed File System nor a P2P data file sharing. Instead, it is a technique for contributing storage to a distributed system which focuses on the performance of the users’ local storage management.

Advantages:

  • A file system that can contribute 100% of the idle space on a disk while imposing a negligible performance penalty on the local user.
  • Operates by storing files in the free space of the file system so that they are invisible to ordinary files. Normal file allocation proceeds as if the system were not contributing any space at all.
  • Ensure the transparency of contributed data: the presence of contributed storage should have no measurable effect on the file system, either in performance, or in capacity.
    • Transparent files: files that are transparent in the local file system (local file system does not see these files).
    • Transparent data or transparent blocks: data belonging to such transparent files
  • Treats transparent blocks as if they were free, overwriting whatever data might be currently stored in them. An ordinary file can be allocated over a transparent file.
  • TFS imposes nearly no overhead on the local user.
  • TFS is specifically designed for contributory applications. The key benefit is that it leaves the allocation for local files intact, avoiding issues of fragmentation

Disadvantage:

  • Sacrifices file persistence. When TFS allocates a block for an ordinary file, it treats free blocks and transparent blocks the same, and thus may overwrite transparent data. Files marked transparent may be overwritten and deleted at any time.
    • Applications using transparent files must ensure the correctness of all file data.
    • Sensitive information stored in ordinary files must be protected from applications trying to read transparent files.
  • The unreliability of files in TFS produces a bandwidth consumption due to the replication that is needed to handle deleted files.

Volunteer Systems:

This technique cannot be used for storage in a volunteer system because it requires changes in the local file systems of the volunteers. It means that the all the local file systems, such as Ext3, NTFS and others must be modified.

decentralized_storage_systems/transparent_file_system.txt · Last modified: 2012/04/23 00:59 by julia
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki