Minor updates and doc changes

docs/source/developer-notes.rst

@@ -333,6 +333,11 @@ Server Nonblocking Control
 
 By design, the region transfer request start does not guarantee the finish of data transfer or server I/O. In fact, this function should return to the application as soon as possible. Data transfer and server I/O can occur in the background so that client applications can take advantage of overlapping timings between application computations and PDC data management.
 
+Server Data Cache
+---------------------------------------------
+
+PDC supports server-side write data cache and is enabled in the CMake option ``PDC_SERVER_CACHE`` by default. Each time the server receives a region writerequest, it will cache the data in the server's memory without writing it to the file system. The server monitors both the total amount of cached data and how long it has not received any I/O requests to determine when to flush the data from cache to the file system. Two additional CMake options ``PDC_SERVER_CACHE_MAX_GB`` and ``PDC_SERVER_IDLE_CACHE_FLUSH_TIME`` can be set to affect the cache flush behavior. When the cached data size reaches the limit or the server is idle longer than the idle time, the flush operation is triggered. With the idle time trigger, when a new I/O request is received during the flush, PDC will stop flushng the next region and reset the timer to avoid interfering with the client's I/O. Setting ``export PDC_SERVER_CACHE_NO_FLUSH=0`` can disable the flush operation and keep the data in cache.
+
 Server Region Transfer Request Start
 ---------------------------------------------
 
@@ -343,6 +348,11 @@ Then, ``PDC_commit_request`` is called for request registration. This operation
 
 Finally, the server RPC returns a finished code to the client so that the client can return to the application immediately.
 
+Server Region Transfer Data Sieving
+---------------------------------------------
+When reading a 2D/3D region, PDC server uses data sieving if a subset of a storage region is requested, which would improve the read performance. The entire region is read as a contiguous chunk and the request subset will be extracted before sending the data to the client. Setting ``export PDC_DATA_SIEVING=0`` before running the server will disable this feature.
+
+
 Server Region Transfer Request Wait
 ---------------------------------------------
 
@@ -373,6 +383,11 @@ However, when a new region is written to an object, it is necessary to scan all
 
 I/O by region will store repeated bytes when write requests contain overlapping parts. In addition, the region update mechanism generates extra I/O operations. This is one of its disadvantages. Optimization for region search (as R trees) in the future can relieve this problem.
 
+Storage Compression (Prototype)
+---------------------------------------------
+
+PDC has partial support for storing the compressed data for each storage regions with the ZFP compression library. Currently the compression is hard-coded to the ZFP accuracy mode.
+
 +++++++++++++++++++++++++++++++++++++++++++++
 Contributing to PDC project
 +++++++++++++++++++++++++++++++++++++++++++++
@@ -560,4 +575,4 @@ But if you need to debug the server, you can prepend ``srun`` with ``ddt --conne
     rm -rf ./pdc_tmp # optional if you need to clean up the PDC tmp directory
     ddt --connect srun -N 1 -n  4 -c 2 --mem=25600 --cpu_bind=cores ./bin/pdc_server.exe &
 
-We recommend to use 1 node when debugging PDC, but if memory is not sufficient, you can use more nodes. 
+We recommend to use 1 node when debugging PDC, but if memory is not sufficient, you can use more nodes. 

src/api/pdc_region/pdc_region_transfer.c

@@ -1787,29 +1787,13 @@ release_region_buffer(char *buf, uint64_t *obj_dims, int local_ndim, uint64_t *l
     if (local_ndim == 2) {
         if (access_type == PDC_READ) {
             ptr = new_buf;
-            // Tang
             memcpy(buf, ptr, local_size[0] * local_size[1] * unit);
-            /* for (i = 0; i < local_size[0]; ++i) { */
-            /*     memcpy(buf + ((local_offset[0] + i) * obj_dims[1] + local_offset[1]) * unit, ptr, */
-            /*            local_size[1] * unit); */
-            /*     ptr += local_size[1] * unit; */
-            /* } */
         }
     }
     else if (local_ndim == 3) {
         if (access_type == PDC_READ) {
             ptr = new_buf;
-            // Tang
             memcpy(buf, ptr, local_size[0] * local_size[1] * local_size[2] * unit);
-            /* for (i = 0; i < local_size[0]; ++i) { */
-            /*     for (j = 0; j < local_size[1]; ++j) { */
-            /*         memcpy(buf + ((local_offset[0] + i) * obj_dims[1] * obj_dims[2] + */
-            /*                       (local_offset[1] + j) * obj_dims[2] + local_offset[2]) * */
-            /*                          unit, */
-            /*                ptr, local_size[2] * unit); */
-            /*         ptr += local_size[2] * unit; */
-            /*     } */
-            /* } */
         }
     }
     if (bulk_buf_ref) {

src/server/pdc_server_region/pdc_server_region_cache.c

@@ -12,12 +12,6 @@
 #define MAX_CACHE_SIZE_GB 32
 #endif
 
-#ifdef PDC_SERVER_CACHE_FLUSH_TIME
-#define PDC_CACHE_FLUSH_TIME_INT PDC_SERVER_CACHE_FLUSH_TIME
-#else
-#define PDC_CACHE_FLUSH_TIME_INT 30
-#endif
-
 #ifdef PDC_SERVER_IDLE_CACHE_FLUSH_TIME
 #define PDC_IDLE_CACHE_FLUSH_TIME_INT PDC_SERVER_IDLE_CACHE_FLUSH_TIME
 #else
@@ -928,15 +922,11 @@ PDC_region_cache_clock_cycle(void *ptr)
     struct timeval current_time;
     struct timeval finish_time;
     int            nflush            = 0;
-    double         flush_frequency_s = PDC_CACHE_FLUSH_TIME_INT, elapsed_time;
+    double         elapsed_time;
     time_t         t;
     struct tm *    tm;
     char           cur_time[64];
 
-    char *p = getenv("PDC_SERVER_CACHE_FLUSH_FREQUENCY_S");
-    if (p != NULL)
-        flush_frequency_s = atoi(p);
-
     if (ptr == NULL) {
         obj_cache_iter = NULL;
     }