Skip to content

Latest commit

 

History

History
83 lines (62 loc) · 3.11 KB

Hyrax_MDS.adoc

File metadata and controls

83 lines (62 loc) · 3.11 KB
Raw

MetaData Store (MDS)

A new cache, the MetaData Store (MDS), has been added to the BES for metadata responses. This cache is unlike the other BES caches in that it is intended to be operated as either a "cache" or a "store." In the latter case, added items will never be removed. It is an open-ended place where metadata response objects are kept indefinitely. The contents of the MDS (as a cache or a store) will persist through Hyrax restarts.

Tip
 The MDS is especially important for scenarios where data is not as close as you need it to be. By using the MDS, you can reduce the time it takes to look through files and make quick decisions based on the metadata that the MDS has saved; however, because the underlying data in the MDS does not update automatically, the metadata may be out of sync with the actual data.

Enable or Disable the MDS

To enable or disable the MDS, access dap.conf in the /etc/bes/modules directory, and remove the comment before the following line of code:

DAP.GlobalMetadataStore.path = /usr/share/mds

See the code block below for the MDS section of dap.conf:

#-----------------------------------------------------------------------#
# Metadata Store parameters                                             #
#-----------------------------------------------------------------------#

# Control the Metadata Response Store. Here, DAP metadata responses
# are stored/cached so that they can be returned by the server w/o
# having to touch the data files/objects. Setting the 'path' to null
# disables uses of the MDS. Setting 'size' to zero makes the MDS
# hold objects forever; setting a positive non-zero size makes the
# MDS behave like a cache, purging responses when the size is exceeded.

#DAP.GlobalMetadataStore.path = /usr/share/mds
DAP.GlobalMetadataStore.prefix = mds

# Size in MB
DAP.GlobalMetadataStore.size = 200

# The MDS writes a ledger of additions and removals. By default the
# ledger is kept in 'mds_ledger.txt' in the directory used to start
# the BES.

DAP.GlobalMetadataStore.ledger = /usr/share/mds_ledger.txt

# This tells the BES Framework's DAP module to use the DMR++
# handler for data requests if it find a DMR++ response in the MDS
# for a given granule.

# DAP.Use.Dmrpp = yes

Configure the MDS

The MDS' parameters should be configured in site.conf. For more information, please see the site.conf section.

To configure the MDS to work as a store, rather than a cache, set the Dap.GlobalMetadataSote.size to 0. To configure the MDS as a cache, set it to your desired size.

Cache Ejection Strategy

The MDS caches complete metadata responses and serves those directly.

The MDS can be configured to not exceed a certain size. When the configured size is met, the MDS ejects the metadata files in the cache that were least recently accessed.

By default the metadata files are stored in usr/share/mds. You can change this location by modifying the DAP.GlobalMetadataStore.path. You can modify this parameter in dap.conf, but you should modify it in site.conf. For more information, please see the site.conf section.