carputils Overview

The increasing complexity of in silico experiments as they have become feasible over the past few years prompted for the development of a framework that simplifies the definition and execution of such experiments which become quickly intractable otherwise even for expert users. The major design goals were

  • Encapsulate the technical complexity to enable users to perform experiments at a more abstract level
  • Once built, limited technical expertise should be required to carry out in silico experiments, thus relieving users from undergoing months/years of training and thus save time to be spent on actual research
  • Expose only steering parameters required for a given experiment
  • Enable the easy sharing of experiments for collaborative research, re-use of used/published experiments or simply for the sake of remote debugging/advising
  • Provide built-in capabilities for generating high quality documentation of experiments

Simplified definition of in silico experiments within the CARPentry modeling framework. Discrete representations of hearts are stored in repositories from where they are loaded when needed. Not only the meshes themselves, but also derived discrete entities such as element/vertex lists or surfaces are stored there to be used for defining electrodes or boundary conditions. The location of all tools for performing the experiments and for analyzing experimental data are encapsulated in the python-based carputils package which is also used to describe the experiment, that is, link the exposed input parameters used to steer the experiment to create appropriate sets of commands which are used to execute the experiment. To share an experiment only the CARPentry experiment ID and the exact experimental protocol are needed to be replicated elsewhere.

carputils is a Python package containing a lot of CARPentry-related functionality and a framework for running CARPentry simulations and testing their outputs en masse. Provided functionality includes:

  • Scripting environment for designing complex in silico experiments

  • Generation of CARP command line inputs with:

    • Automatic adding of MPI launcher
    • Debugger and profiler attachment
    • Automatic batch script generation and submission with platform-specific hardware profiles

  • Compact model setup using Python classes

  • Efficient IO of common CARPentry file formats to numpy arrays

  • Failure-compliant mass running of tests

  • Automatic comparison of test results against reference solutions

Structure

The CARP examples framework encompasses a number of repositories, the purpose of which can be initially confusing. This section aims to explain the structure.

A typical installation might look like:

software
├── carputils
├── carp-examples
│   ├── benchmarks
│   └── devtests
├── carp-meshes
└── carp-reference
    ├── benchmarks
    └── devtests

carputils

A repository containing most of the Python functionality used in setting up and postprocessing CARP simulations. The bin subdirectory should be added to your PATH environment variable, and the top level directory should be added to your PYTHONPATH. On first run of an experiment, or when using the automatic clone script, the settings.yaml file will be generated, which must be tailored to your setup.

carp-examples

This is a normal directory which contains 2 (or more) Python packages whose modules define experiments. Each run.py script in these packages runs an experiment. In fact, as long as carputils is set up correctly, you can move these run scripts anywhere on your system (along with any input files they depend on) and they will work, but in order to run regression tests they must be inside these packages and carp-examples must be added to the PYTHONPATH.

carp-meshes

This is an SVN repository containing meshes used by examples. As it is very large, you may instead use the --get-mesh flag with run scripts to get only their needed meshes. Note that most run scripts generate meshes programmatically, in which case this is not needed. The location of this repository is set in carputils/settings.yaml with MESH_DIR.

carp-reference

These directories contain the reference solutions for regression tests. The repositories under carp-reference must have the same name as those under carp-examples. These repositories are only needed for running regression tests. The location of carp-reference is set by the REGRESSION_REF settings in carputils/settings.yaml 

The actual structure above need not be conformed to. It only matters that `carputils` and 
`carp-examples` are in the `PYTHONPATH`, `carputils/bin` is in the `PATH`, and `carp-meshes` 
and `carp-reference` are at the locations specified in `settings.yaml`. Other than that you
can put them where you want.

[\notice]

Concepts

The following concepts are used in the carputils framework:

sim

A single CARP execution - consecutive, related runs are considered to be multiple sims

job

One or more related sims launched by a single execution of a carputils run script

example

A carputils run script, capable of launching a single job per execution, though different jobs can be launched by changing the command line arguments

test

A fixed (according to the command line arguments) run configuration of an example plus several checks to determine the accuracy of the output

check

A method to determine the accuracy of the output of a test - most commonly by comparing with a previous reference solution but also may be by comparison with an analytic solution and other custom methods

Generate HTML Page

This documentation is based on Sphinx and can be generated using the Makefile in the carputils/doc folder.

    cd carputils/doc
    make clean
    make html

The html webpages are found then in the folder carputils/doc/build/html. To view, point your favorite browser to

    firefox ${CARPUTILS_DIR}/doc/build/html/index.html

Constructing Models First Run