Optimization Middleware

Some applications require global file system access because files are dynamically selected at runtime, or because files are shared by processes. In those cases, a file staging mechanism cannot work, and thus file I/O issues remain.
In order to reduce the global file system access ratio, we have designed and implemented a mechanism for caching files and reducing file I/O access requests†.

Figure 1. File I/O Statistics

The x-axis an y-axis represent time step and number of bytes issued in file I/O, respectively.

†Kazuki Ohta, Hiroya Matsuba, and Yutaka Ishikawa, “Improving Parallel Write by Node-Level Request Scheduling,” IEEE CCGrid2009, 2009.

File I/O Tracer

Understanding file I/O access patterns in applications is important for designing a global file system and network between compute nodes and the file system. In order to capture how an application issues file I/O requests, a file I/O tracer has been implemented. It does not require recompilation of the source code, but dynamically changes the file I/O library at runtime, and thus captures all file I/O access patterns.