Welcome to MUSCLE3’s documentation!

MUSCLE3 is the third incarnation of the Multiscale Coupling Library and Environment.

With MUSCLE3, you can connect multiple simulation models together into a multiscale simulation. Simulation models can be as simple as a single Python file, or as complex as a combination of multiple separate simulation codes written in C++ or Fortran, and running on an HPC machine.

Contributors

University of Amsterdam Computational Science Lab

Original concept, Multiscale Modelling and Simulation Framework (MMSF) coupling theory, original MUSCLE, MUSCLE2.

Netherlands eScience Center

MUSCLE3 implementation, teaching materials.

Ignition Computing

Checkpointing implementation and additional development.

This work was supported by the Netherlands eScience Center and NWO under grant number 27015G01.

We would like to acknowledge the contribution by The ITER Organization of results of work carried out within the framework of ITER contract IO/22/CT/4300002587. The views and opinions expressed herein do not necessarily reflect those of the ITER Organization.

Academic collaboration

Please contact prof. Alfons Hoekstra (UvA CSL) and/or Lourens Veen (NLeSC).

Commercial support

Please contact Ignition Computing.

Citing MUSCLE3

If you use MUSCLE3 for scientific work, please cite the version of the MUSCLE3 software you used and the following paper:

Veen L.E., Hoekstra A.G. (2020) Easing Multiscale Model Design and Coupling with MUSCLE3. In: Krzhizhanovskaya V. et al. (eds) Computational Science – ICCS 2020. ICCS 2020. Lecture Notes in Computer Science, vol 12142. Springer, Cham. https://doi.org/10.1007/978-3-030-50433-5_33

Contents:

• Introduction
• Current status
  • Coupling different submodel instances
  • Spatial and temporal scale separation and overlap
  • Settings management
  • Combining features
  • Ongoing development
    • Language support
    • Distributed execution
    • Dynamic instantiation
    • Profiling
• Installing
  • Python
  • C++ and Fortran
    • Prerequisites
    • (No) dependencies
    • Downloading
    • Building
    • Getting help
    • Installing
  • Building models with libmuscle
    • Making libmuscle available
    • Compiling with libmuscle
    • Linking with libmuscle
  • Running models
• Tutorial with Python
  • Importing headers
  • A simple submodel
  • The reuse loop
  • Initialisation: Settings and receiving messages
  • The state update loop
  • Sending the final result
  • Message timestamps
  • Receiving messages with a default
  • Connecting it all together
  • Launching the simulation
  • Log output
• Distributed execution
  • Making separate implementations
  • yMMSL files
  • Starting the simulation
  • High-Performance Computing
    • Determining resource requirements
    • Loading modules
    • Hyperthreading
• Profiling coupled simulations
  • Configuring profiling
  • The profiling database
  • Plotting statistics from the command line
    • Per-instance time spent
    • Resource usage
    • Event timeline
  • Analysis with Python
    • Per-instance statistics
    • Per-core statistics
    • Analysing events
  • SQL data model
    • Database format version
    • Formatted events
    • Instances
    • Resources
    • Other tables
• Coupling your model
  • Multicast
    • Example
• MUSCLE and C++
  • Building and running the examples
  • Log output
  • Describing C++ implementations
  • Reaction-diffusion in C++
    • Reaction model
    • Headers
    • Creating an Instance
    • Reuse loop and settings
    • Receiving messages and DataConstRef
    • Sending messages and Data
    • Diffusion model
• MUSCLE and Fortran
  • Building and running the examples
  • Log output
  • Describing Fortran implementations
  • Reaction-diffusion in Fortran
    • Reaction model
    • Modules
    • Variables
    • Creating an Instance
    • Reuse loop and settings
    • Receiving messages and DataConstRef
    • Sending messages and Data
    • Handling errors
    • Diffusion model
• MUSCLE and MPI
  • Building and running the examples
  • MPI Reaction model
    • Headers and modules
    • Creating an Instance
    • Reuse loop and settings
    • Distributed state
    • Receiving messages
• Running containerized applications
  • How to run containerized components
    • Running the reaction-diffusion example in Singularity
    • Running the reaction-diffusion example in Docker
• Uncertainty Quantification
  • Simulation design
  • Implementation
  • The quasi-Monte Carlo element
  • The round-robin load balancer
  • Discussion
  • Examples in C++ and Fortran
• Simulation checkpoints
  • Glossary
  • User tutorial
    • Defining checkpoints
      • Simulation time checkpoints
      • Wallclock time checkpoints
    • Running a simulation with checkpoints
      • Example: running the reaction-diffusion model with checkpoints
    • Resuming a simulation
      • Example: resuming the reaction-diffusion model
      • Making changes to your simulation
      • Resuming from at_end snapshots
    • Snapshot consistency
    • Troubleshooting
  • Developer tutorial
    • Start situation: components without checkpointing
    • Step 1: Set USES_CHECKPOINT_API on instance creation
    • Step 2: Implement checkpoint hooks
      • Intermediate snapshots
      • Final snapshots
      • Example: implemented checkpoint hooks
    • Step 3: Implement resume
      • Reload settings when resuming
    • Example: implemented checkpoint hooks and resume
    • Components that do not keep state between reuse
    • Builtin validation
    • Checkpointing in MPI-enabled components
• Tips & tricks
  • Running simulation components interactively
  • Debugging simulation components

API documentation

• API Documentation for Python
  • Grid
    • Grid.array
    • Grid.indexes
    • Grid.__init__()
  • Instance
    • Instance.__init__()
    • Instance.error_shutdown()
    • Instance.get_port_length()
    • Instance.get_setting()
    • Instance.is_connected()
    • Instance.is_resizable()
    • Instance.is_vector_port()
    • Instance.list_ports()
    • Instance.list_settings()
    • Instance.load_snapshot()
    • Instance.receive()
    • Instance.receive_with_settings()
    • Instance.resuming()
    • Instance.reuse_instance()
    • Instance.save_final_snapshot()
    • Instance.save_snapshot()
    • Instance.send()
    • Instance.set_port_length()
    • Instance.should_init()
    • Instance.should_save_final_snapshot()
    • Instance.should_save_snapshot()
  • InstanceFlags
    • InstanceFlags.DONT_APPLY_OVERLAY
    • InstanceFlags.KEEPS_NO_STATE_FOR_NEXT_USE
    • InstanceFlags.STATE_NOT_REQUIRED_FOR_NEXT_USE
    • InstanceFlags.USES_CHECKPOINT_API
  • Message
    • Message.timestamp
    • Message.next_timestamp
    • Message.data
    • Message.settings
    • Message.__init__()
  • ProfileDatabase
    • ProfileDatabase.__init__()
    • ProfileDatabase.close()
    • ProfileDatabase.instance_stats()
    • ProfileDatabase.resource_stats()
    • ProfileDatabase.time_taken()
  • run_simulation()
• API Documentation for C++
  • Namespace libmuscle
    • Data
    • DataConstRef
    • Instance
    • InstanceFlags
    • Message
    • PortsDescription
  • Namespace ymmsl
    • allows_sending
    • allows_receiving
    • Conduit
    • Identifier
    • Operator
    • Port
    • Reference
    • Settings
    • SettingValue
• API Documentation for Fortran
  • A note on types
  • Namespace LIBMUSCLE
    • LIBMUSCLE_Data
    • LIBMUSCLE_Message
    • LIBMUSCLE_PortsDescription
    • LIBMUSCLE_Instance
    • LIBMUSCLE_InstanceFlags
  • Namespace YMMSL
    • YMMSL_Operator
    • YMMSL_Settings
• Command line tools
  • muscle_manager
  • muscle3
    • profile
    • resources
    • snapshot

MUSCLE3 development

• For Developers
  • Contributing to MUSCLE3
  • Development tools
  • Releasing
  • Checkpointing deep-dive

Indices and tables

• Index

• Module Index

• Search Page