Damaris Plug-Ins

Writing data to Damaris is cool, but what should the server do with it? Yes, you can use the built-in plugins for writing HDF5 and integrating with ParaView, Visit or Python, however, if you have specific processing needing to be done with your data (without using Python) you will need to write a plugin. The role of processing data (and removing them from the shared memory – see the example at the end of this page) is delegated to plug-ins, which are triggered by events sent by the simulation to the servers. This page explains how to program a plug-in for Damaris and how to trigger it from the simulation.

Describing Actions

The interactions between the simulation processes and the dedicated cores are based on events. These events are sent by simulation processes to the servers. Like other Damaris objects, these events have to be described in the XML configuration file, more specifically within the <event> section. The listing below presents the description of such an event.

1
2
3
<actions>
    <event name="my_event" action="my_function" library="libsomething.so" scope="core"/>
</actions>

An event is described by a unique name, and an action, which corresponds to the name of a C++ function. If this function is located in a shared library (.so file), the name of the library has to be provided. Damaris will look for shared libraries within the directories specified by the LD_LIBRARY_PATH environment variable. If no library is specified, Damaris will look for the function within its own binary code.
Another important characteristic of an event is its scope:

  • “core” indicates that every time a simulation process sends the event, the corresponding action is triggered by the the server receiving the event. If 15 clients in a 16-cores node send the same event to their server running on a dedicated core, the corresponding action is triggered 15 times within the server.
  • “group” indicates that a server has to wait for all its clients to have sent the same event (at the same iteration) before triggering the action. If the 15 clients send the event, the action is triggered only once. If one of the processes does not send the event, the corresponding action is never triggered.
  • “bcast” indicates that if one client sends this signal, then all the servers will perform the corresponding action, not only the server responsible for this client. MPI communications can safely be used within an action triggered by such an event. It is thus the right type of event to use if the plug-in is collective across servers.

The semantics of these scopes with respect to the order of triggered actions is the following.

  • Two “core” events sent by the same process will be received in the same order by the dedicated core.
  • If an event (whatever the scope) is sent after writing or committing a variable, it is ensured that the server has received the variable before triggering the event’s action.
  • Two “group” events sent by all the clients of a group in the same order will trigger their actions in the same order in the server. No particular order should be assumed if different processes send the same event in a different order.
  • Two “bcast” events sent by the same client will trigger their actions in the same order in all the servers. No particular order should be assumed if different processes send the “bcast” event.
  • If a client sends two events e1 and e2, if e1 has a “core” scope then it is ensured that its corresponding action will be executed before the action of e2.
  • However, if e1 has another scope than “core”, no ordering can be assumed.
  • To force an ordered execution of e1 and e2 where the scope of e1 is not “core”, global synchronization barriers can be used in the simulation before sending e2.

Hello World Plug-In Example

Now let’s take a look at a C++ function that can be called by Damaris. An example of such a function is provided in the below listing. Note the use of extern “C” to prevent C++ name mangling.

1
2
3
4
5
6
7
8
#include <iostream>
 
extern "C" {
void my_function(const std::string& event, int32_t src, int32_t step, const char* args)
{
    std::cout << "--- hello world from Damaris ---" << std::endl;
}
}

The first argument of such a function is a string representing the name of the event that triggered the action (since several events can be connected to the same action). The second parameter is the rank of the client that sent the event. For “group” actions, this source is set to -1. The third parameter corresponds to the iteration in which the event has been sent. Finally the last argument is only used by external plug-ins to send their own commands.
This first example only helps you start with plugins, more consideration on how to access the data written by clients will be provided later in this document.

Compiling and Linking C/C++ Plug-Ins

To compile a plugin within a shared library, you need to create an object file from your source file with the -fPIC option, then create a shared library:

1
2
3
g++ -fPIC -c something.cpp -I/path/to/damaris/include \
    -I/path/to/damaris/dependencies/include
gcc -shared -o libsomething.so something.o

If you want to integrate your plug-in in the application’s code (for instance to avoid loading the .so file from many processes at the same time, which could result in performance degradation at large scale), simply compile the source of your plug-in with your application. You must dynamically link your simulation when compiling it.

Sending Events from Simulation

To send events from the simulation, use the damaris_signal function (in C/C++) or damaris_signal_f  (in Fortran) as shown below. As you can see, these functions take the name of an event as parameter; this name should correspond to an event described in the configuration file. The returned value (or the ierr value in Fortran) is 0 in case of success, -1 in case of failure when sending the event through the shared memory, and -2 if the name does not correspond to a defined event.

1
damaris_signal("my_event");
1
2
 integer::ierr
 call damaris_signal_f("my_event" , ierr)

Binding simulation’s functions to events

On some platforms, the dlopen function does not work properly, which prevents Damaris from retrieving the plugin. In this case, another solution is to provide the function’s pointer at runtime. To do so, all the processes must call damaris_bind_function(“event_name”,function_ptr) before calling damaris_start (so that both clients and servers are aware of the event), and in the same order if multiple events are defined this way. The first argument to this function is the name of the event to define, the second one is the pointer to a function that has the following signature, defined in Damaris.h:

1
typedef (*signal_t)(const char*,int32_t,int32_t,const char*);

This method is experimental, currently only available in C, and allows only a “core” scope. In addition, the event should not be already defined in the configuration file.

The internal Damaris API

In order to access Damaris data within a plug-in, you have to use internal data structures of Damaris. The main structure is the VariableManager, which can be used to access Variable instances and, from there, Block instances (which correspond to our notion of “domains” as set in the Damaris XML file, or blocks via damaris_write_block() API call), and DataSpace instances, which hold data. We invite the reader to take a look at the HDF5 data storage backend (under /src/storage/HDF4Store.cpp) to understand how to retrieve data.
Note: To use the internal structures and objects of Damaris, it is necessary to include the proper header files. These header files have been copied in the damaris folder in the include folder where Damaris is installed.

Error Handling through Events

If the simulation cannot write all of its data to shared memory for a given iteration, a special signal is sent to all the servers to ask them not to update visualization backends and also to erase all remaining data for this iteration and all past iterations. This behavior can be overridden by attaching an error handler to an event, as exemplified in the following XML codes. In this case, whenever the simulation is not able to write its data, the event pointed by the error tag is executed. This event must have a “core” scope, and the event or script must be defined somewhere in the configuration file.

1
2
3
4
<actions>
    <event name="my_event" action="my_function" library="libsomething.so" scope="core"/>
    <error event="my_event" />
</actions>

Garbage collection example

The below code listing, shows an example plug-in for freeing those Damaris shared memory data that belongs to old iterations. This plugin is currently used by Damaris to keep shared memory use to a workable level. The plug-in does nothing for the first two iterations (iterations 0 and 1), but removes the old data after the third iteration (i.e. in iteration 2 and afterwards). This means that at iteration 2, the data from iteration 0 are removed from memory. Given the fact that non-time-varying data such as coordinate variables should always remain in memory, the plug-in only removes time-varying data.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
#include <iostream>
#include <VariableManager.hpp>
 
using namespace damaris;
 
extern "C" {
void damaris_gc(const std::string& event, int32_t src, int32_t step, const char* args)
{
    if (step < 2 ) return;
 
    VariableManager::iterator it = VariableManager::Begin();
    VariableManager::iterator end = VariableManager::End();
 
    while (it != end) {
        // Non-time-varying data are not removed 
        if (it->get()->IsTimeVarying()) {
            (*it)->ClearUpToIteration(step - 2);
        }
        it++;
    }
}
}

Comments are closed.