Fig 1.
Two example workflows (simplified, read from left to right) that a cross-center DDI as in LEXIS should accommodate. a) stent-placement simulation [56]; b) regular weather and wildfire-risk predictions [13].
Yellow and grey boxes indicate interactive and regularly-executed actions, respectively, while the actual workflow steps are marked green. In case a), runs are attempted on three HPC sites to warrant success in an urgent-computing [55] setting. Functional and non-functional requirements (FR/NR, see text) are related to example functionalities of the workflows. NR2, FR4 and FR5 (see blue boxes) make sure that the DDI offers uniform APIs and authentication across all relevant backends. NR1 and NR3 warrant reliability and performance, most importantly in the urgent-computing case a). FR1 warrants cross-center functionality in both workflows. FR2 facilitates automated cross-center data replication (case a)) and FR3 warrants FAIR Research Data Management (case b)).
Table 1.
Requirements on a DDI as in LEXIS vs. characteristics of middleware solutions.
“Y” indicates that the solution fulfils the requirement, “N” that it does not. “C” indicates conditional or partial fulfilment.
Fig 2.
Both iRODS provider servers and database servers as potential points of failure are redundantly provisioned, following [44].
Fig 3.
The role of B2SAFE and B2HANDLE for data replication in the LEXIS DDI.
The left part (a) of the figure sketches the system, where B2HANDLE is run on a server separate from the DDI (“external”). Part (b) illustrates the replication and replica-registration process.
Table 2.
Overview of performance tests, grouped by figures where the results are shown (second column).
Details on the tests and further explanations can be found in the respective sections in the text.
Fig 4.
Data transfer rates in the LEXIS iRODS federation (between LRZ and IT4I) with icp.
The figure shows average transfer rates and sample standard deviations (from 20 transfer tests per data point shown) in MB/s, using icp on files of various sizes.
Fig 5.
Data transfer rates in the LEXIS iRODS federation (from LRZ to IT4I) with various methods.
The figure shows average transfer rates and sample standard deviations (from 20 transfer tests per data point shown) in MB/s, using icp, B2SAFE or the Python iRODS Client on files of various sizes.
Fig 6.
Data transfer rates from LRZ to IT4I in a test iRODS federation, using the Python iRODS Client put routine and various buffer/chunk sizes.
The figure shows average transfer rates for various file sizes and sample standard deviations (from 20 transfer tests per data point shown) in MB/s, depending on the buffer size setting in the modified put routine. For this experiment, we have used virtual machines in OpenStack infrastructure on both sides.