workflow

  1"""QuakeCoRE Scientific Software Stack.
  2
  3## Overview
  4The QuakeCoRE scientific workflow is composed of a number of independent components
  5
  6- `workflow` :: This module, which implements the Cybershake workflow. It defines a cylc workflow and scripts that researchers can use to build their own automated research pipeline using scientifically validated, regularly tested, well-documented components built from the software team.
  7- `source_modelling` :: A collection of modules for modelling sources.
  8- `velocity_modelling` :: The next-generation velocity model.
  9- `qcore` :: Utilities and common functions.
 10
 11## Using the Workflow
 12
 13To deploy the workflow on NeSI, first clone the repository and then install the cylc workflow
 14
 15```
 16$ git clone https://github.com/ucgmsim/workflow
 17$ # add inputs into workflow/cylc/inputs
 18$ cp -r workflow/cylc ~/cylc-src/cybershake # OR a suitable name for your custom workflow
 19$ cylc install cybershake
 20$ cylc play cybershake
 21```
 22
 23And that's it! You can monitor the progress of your workflow with `cylc log cybershake` and `squeue -u $USER` to see the slurm job queue for your user.
 24
 25## The Stages
 26
 27The workflow repository contains all the tools necessary to build a
 28workflow for ground motion model simulations. We build and maintain a
 29Cybershake workflow, but researchers may wish to build their own. For
 30this reason, the workflow is build out of composable parts we intend
 31for anyone to reuse to build their own workflow based on the
 32Cybershake workflow. Below you will find documentation for all the
 33workflow stages, their inputs, outputs and environments.
 34
 35```mermaid
 36flowchart LR
 37    A[NSHM To Realisation] --> B[SRF Generation]
 38    A --> C[Domain Generation]
 39    B --> D[Stoch Generation]
 40    C --> E[Velocity Model Generation]
 41    C --> F[Station Selection]
 42    C --> G[Write Model Coordinates]
 43    B --> H[Create EMOD3D Parameters]
 44    E --> H
 45    F --> H
 46    G --> H
 47    H --> I[EMOD3D]
 48    D --> J[High Frequency Simulation]
 49    I --> M[Broadband Simulation]
 50    J --> M
 51    M --> N[Intensity Measure Calculation]
 52    I -->|Optionally| K[Merge Timeslices]
 53    K --> L[Create Simulation Video]
 54```
 55
 56
 57Many of the stages will run in a _container_. A container is a self-contained execution environment with all the system and Python libraries required to execute workflow stages. It is also isolated from the host system and may not be able to access certain directories. We maintain a [cybershake container](https://hub.docker.com/r/earthquakesuc/runner) that has a copy of the latest validated workflow, and environment. Use this container as much as possible in your own scripts to run your code.
 58
 59To find the documentation for a given stage, find the module that runs this stage in the above flow diagram and then click the corresponding link
 60
 61| Stage                         | Module                                                |
 62|:------------------------------|:------------------------------------------------------|
 63| NSHM to Realisation           | `workflow.scripts.nshm2022_to_realisation`            |
 64| SRF Generation                | `workflow.scripts.realisation_to_srf`                 |
 65| Domain Generation             | `workflow.scripts.generate_velocity_model_parameters` |
 66| Stoch Generation              | `workflow.scripts.generate_stoch`                     |
 67| Velocity Model Generation     | `workflow.scripts.generate_velocity_model`            |
 68| Station Selection             | `workflow.scripts.generate_station_coordinates`       |
 69| Write Model Coordinates       | `workflow.scripts.generate_model_coordinates`         |
 70| EMOD3D                        | See below.                                            |
 71| High Frequency Simulation     | `workflow.scripts.hf_sim`                             |
 72| Broadband Simulation          | `workflow.scripts.bb_sim`                             |
 73| Merge Timeslices              | `merge_ts.merge_ts`                                   |
 74| Create Simulation Video       | `workflow.scripts.plot_ts`                            |
 75| Intensity Measure Calculation | `workflow.scripts.im_calc`                            |
 76
 77
 78## EMOD3D
 79
 80Most stages are documented in their corresponding python module
 81
 82### Description
 83Run a low frequency ground motion simulation using EMOD3D.
 84
 85### Inputs
 861. A parameter file in "key=value" format,
 872. An SRF file,
 883. A station file list (latitude, longitude, and x, y), see .
 89
 90### Outputs
 911. Ground acceleration timeslice files, one per core.
 922. Seismograms, one per station.
 93
 94### Environment
 95This stage must be run on a system with MPI installed. Typically, we run this stage in Maui on NeSI HPCs or Kisti. Due to high computational requirements, this stage usually cannot be run locally.
 96
 97### Usage
 98On an HPC with slurm enabled `srun emod3d-mpi_v3.0.8 -args "par=$CYLC_WORKFLOW_SHARE_DIR/LF/e3d.par"` will run EMOD3D. Depending on the number of cores rerequested, this may invoke multiple proesses on different compute nodes (for Maui, this will occur when the number of cores exceeds 40). EMOD3D has support for checkpointing, so repeat invocations will continue from their previous checkpointed stage.
 99
100### For More Help
101See Graves, 1996[^1] for a description of the mathematical and technical details of EMOD3D's implementation.
102
103[^1]: Graves, Robert W. "Simulating seismic wave propagation in 3D elastic media using staggered-grid finite differences." Bulletin of the seismological society of America 86.4 (1996): 1091-1106.
104"""