Version v1.7.0 of Damaris adds support for asynchronous Python based processing. This Damaris server side plug-in has support for Dask which is a Python distributed runtime engine and opens up a large number of analytic methods that can use Dask data types, these include SciKit-Learn and Tensorflow/Keras.
In the Damaris XML configuration file, the <variables> XML elements can be exposed to the Python <pyscript> XML element by including its name i.e. “script=MyPyAction” in the following example:
1 2 3 4 5 6 7 | <variable name="cube_i" type="scalar" layout="cells_whd_wf" mesh="mesh" centering="nodal" script="MyPyAction" /> <scripts> <pyscript name="MyPyAction" file="python_script.py" language="python" frequency="1" scheduler-file="/home/user/dask_file.json" nthreads="1" keep-workers="no" timeout="4" /> </scripts> |
The above example excerpt will allow the Damaris server processes to publish the cube_i variable to a Python interpreter that will be run on each iteration (as frequency=1) and be accessible via the script python_script.py. The scheduler-file value, if pointing to a valid Dask scheduler file will allow the python_script.py code to use Dask functionality. This functionality recreates an exposed simulation array as a Dask dask.array, which can be used in various analytic frameworks that have been ported to use Dask distributed objects.
For more details on Python support in Damaris, please see Damaris-Python_Support Wiki Page and also look at the examples in the examples/python directory of the Damaris distribution (or examples/damaris/python of a Damaris installation).
This capability is a large enhancement to Damaris server side processing. A Python based Damaris client side module is also hoped to be available soon, thus allowing pure Python MPI based simulations with asynchronous I/O via Damaris.
Download is available from the Damaris download page.
