{"id":229,"date":"2017-10-24T10:50:27","date_gmt":"2017-10-24T08:50:27","guid":{"rendered":"https:\/\/project.inria.fr\/damaris\/?page_id=229"},"modified":"2023-08-29T10:31:11","modified_gmt":"2023-08-29T08:31:11","slug":"data-description","status":"publish","type":"page","link":"https:\/\/project.inria.fr\/damaris\/data-description\/","title":{"rendered":"Data Description"},"content":{"rendered":"\n

In previous pages, you learnt how to instrument a simulation to benefit from Damaris. Now, it is time to describe and then write simulation data to Damaris. This page explains how to describe data in the Damaris XML file and how to access the values at run time. All the data items must be described in the XML configuration within the <data> section.<\/p>\n\n\n\n

Parameters<\/h4>\n\n\n\n

Parameters are simple values associated with a name and a type. They allow us to easily change some important values of the simulation automatically at runtime and Damaris updates all variables and objects that depend on them. In the following, you can see how to define a parameter with a default value. Parameters must be defined at the root of the <data> section, not within nested groups (explained later).<\/p>\n\n\n\n

\n \n    \n<\/data>\n<\/pre>\n\n\n\n
Parameters must have a valid name, i.e., they should not contain any space or special symbols, and should not start with a digit. Basically any name that is valid as a variable name in C or Fortran is a valid parameter name in Damaris. The type attribute must be one of the types listed in Damaris data types<\/a>. The provided value must be acceptable for the specified type. For instance value=”xyz” is not valid for an integer parameter.<\/p>\n\n\n\n
The below listings show how to get and set parameters at run time from the simulation. Please note that when a process modifies a parameter, this modification is local and not visible to other processes or to dedicated cores\/nodes. The user is responsible for synchronizing all processes when a parameter has to be changed globally.<\/p>\n\n\n\n
\n int w_in ;\n int err = damaris_parameter_get (\"w\", &w_in , sizeof (int)) ;\n\n int w_out = 4 ;\n int err = damaris_parameter_set(\"w\", &w_out , sizeof (int)) ;\n<\/pre>\n\n\n\n
And for Fortran:<\/p>\n\n\n\n
 integer: : w_in , w_out\n integer :: ierr\n ...\n\n call damaris_parameter_get_f (\"w\" , w_in , SIZEOF(w_in) , ierr )\n\n w_out = 4\n call damaris_parameter_set_f (\"w\" , w_out , SIZEOF(w_out) , ierr )\n<\/pre>\n\n\n\n
Parameters are especially useful in the definition of layouts, which are described in the following part of this document.<\/p>\n\n\n\n
Layouts<\/h4>\n\n\n\n
Layouts describe the shape of the data. Damaris<\/em> separates the description from the description of the variables themselves since several variables can have the same layout. A layout is characterized by a name, a base type and a list of dimensions. The description allows Damaris to allocate the correct amount of shared memory needed for a variable. Two examples are given in the following listing.<\/p>\n\n\n\n
 \n    \n    \n<\/data> <\/pre>\n\n\n\n
Note:<\/strong> string and label data types are variable-length types, thus the dimension of the layout corresponds to the number of characters that the variable can store.<\/p>\n\n\n\n
Variables and groups<\/h4>\n\n\n\n
XML description<\/h5>\n\n\n\n
Actual data is described through the <variable> and <group> nodes, which allow to build a hierarchy of variables. An example is given as depicted below. Each variable must be given a name, and be associated with a defined layout. These are the two mandatory attributes. The time-varying attribute indicates whether a variable is expected to be written at every iteration, or just once at the beginning of the simulation (i.e., before the first call to damaris_end_iteration). The visualizable attribute indicates that a variable is visualizable by visualization backends (for instance, coordinate arrays
are not visualizable). In the following example, we expect the x coordinates variable to be the coordinate of points in a rectilinear grid. This data is not itself visualizable, however a rectilinear grid (later described) will be a visualizable object. Note that one can use a basic type instead of a layout, as for the simple var variable: basic types are already interpreted as layouts.<\/p>\n\n\n\n
  \n    \n    ;\n        \n    <\/group>\n    \n<\/data>\n<\/pre>\n\n\n\n
Relative and absolute names<\/h5>\n\n\n\n
Variables and layouts can be defined within groups. In Listing 6, the relative name of the temperature variable is “temperature<\/em>“, while its absolute name is “my group\/temperature<\/em>“. The same goes for the
name of layouts.
When Damaris<\/em> searches for a layout associated with a variable, it first looks inside the group where the variable is defined, then in the parent group, and so on. It is thus possible to refer to a layout either with its absolute name (if the layout is located in a different group) or with a relative name if the layout can be found in the same group hierarchy.<\/p>\n\n\n\n
Writing full variables<\/h5>\n\n\n\n
Now that the data is described, we can write it from the simulation. The two samples codes that are shown below present how to write a variable. The full name of the variable should be provided.<\/p>\n\n\n\n
 \nfloat data ;\n ...\n int err = damaris_write(\"my group\/temperature\" , data) ;\n<\/pre>\n\n\n\n
And for Fortran:<\/p>\n\n\n\n
 \n real , dimension ( : ) :: mydata\n integer :: ierr\n ...\n call damaris_write_f (\"my group\/temperature\" , mydata , ierr )\n<\/pre>\n\n\n\n
Writing multiple domains<\/h5>\n\n\n\n
By default, Damaris expects one block of data per client and per variable. It may be necessary for a client to write multiple blocks of a single variable. To do so, the number of domains have to be specified in the domains section of the XML file, as shown here.<\/p>\n\n\n\n
\n\n<\/pre>\n\n\n\n
This number is a maximum, a client may write less domains, but not more. Besides, if a client writes N domains, it must identify them from 0 to N-1. The below sample code shows how to write multiple blocks. Each block is expected to have the size and shape defined in the layout associated with the variable.<\/p>\n\n\n\n
float data[4];\n ...\n for (i=0; i < 4; i++)\n int err = damaris_write_block(\"my group\/temperature\" , i , data);\n<\/pre>\n\n\n\n
And for Fortran:<\/p>\n\n\n\n
 real , dimension(0:3,:) :: mydata\n integer :: i , ierr\n ...\n call damaris_write_block_f(\"my group\/temperature\" , i , mydata (i) , ierr )\n<\/pre>\n\n\n\n
A Visual Example<\/h4>\n\n\n\n
A more complete example of a simple decomposition of a 2D matrix into a distributed variable is as follows and is similar to what is present in the Damaris examples directory (providing Damaris was installed with HDF5 support)<\/p>\n\n\n\n
\ncd \/examples\/damaris\/storage \n\n# 4 Damaris clients and 1 Damaris server\nmpirun -np 5 .\/2dmesh 2dmesh.xml\n<\/pre>\n\n\n\n
An excerpt of the 2dmesh.xml file is as follows:<\/p>\n\n\n\n
 \n\n\n \n \n\n   \n<\/data> <\/pre>\n\n\n\n
The full dataset can be visualised like this:<\/p>\n\n\n\n
\"\"<\/figure><\/div>\n\n\n\n
Next we show the decomposition into MPI ranks and the use of damaris_set_position() to specify the offsets of the global decomposition: <\/p>\n\n\n\n
 \n\/\/ 2D domain\nint dims = 2 ;   \n\/\/ array for offset values    \nint64_t pos[dims]; \n\/\/ values dependent on MPI process \n\/\/ This is for Damaris client process 0\npos[0] = 0 ;         \npos[1] = 0 ;\ndamaris_set_position(\"var_array\", pos)\n<\/pre>\n\n\n\n
\"\"<\/figure><\/div>\n\n\n\n
In the following we see the final result after the damaris_write() call that places a copy of the data into shared memory and is now available for the Damaris server rank to process<\/p>\n\n\n\n
\"\"<\/figure><\/div>\n\n\n\n
Layouts may also specify a ghosts<\/code>  attribute that describes a constant padding on the boundary edges of a data array. Ghost attributes may also use parameters to allow for a changing amount of padding. Each dimension of an array specifies 2 ghost values, one for each edge of the data along a particular axis.<\/p>\n\n\n\n
An example of the layout XML with the ghosts attribute added and a diagram that visualizes the layout for gc0 and gc1 = 2 and gr0 and gr1 = 1 is presented below:<\/p>\n\n\n\n
 \n\n\n\n\n...\n \n<\/pre>\n\n\n\n
The variable that supports the var_layout_with_ghosts above includes the extra padding of the ghost zones, however HDF5 and Visit plugins will automatically remove these extra areas to save the unique data only (ghost zones are usually shared data across ranks who share boundaries. <\/p>\n\n\n\n
\"\"<\/figure><\/div>\n\n\n\n
Meshes<\/h5>\n\n\n\n
Please see the documentation on in-situ vizulisation for descriptions and examples of the mesh xml<\/p>\n\n\n\n
The name of a layout can be any string not including the “\/” character (it can include special characters, white spaces, numbers, etc. even though we advise to keep it simple and use classic C identifiers). The name cannot be the name of a basic type (such as int). The type should be one of the basic types listed in Damaris data types<\/a>. The list of dimensions is a comma-separated list of arithmetical expression featuring +, -, *, \/, %, parenthesis, numbers and any defined int parameters (as of today, only int and integer parameters are allowed, short and long will produce an error). The global attribute is specified so that output functionality that needs a global size for the data can efficiently allocated it (such as HDF5 Collective output) In the example above, the first layout has three dimensions and the second one has two dimensions.
When a layout depends on parameters, any modification of the parameters will automatically modify the layout. Like parameters, the layout is only locally affected and the modification is not propagated to other processes. Making a layout depend on a parameter and changing the parameter at run time can be very useful for particle-based simulations, where the number of particles is different at each process, or simply to make the XML file independent of the simulation size and avoid changing the parameters every time the size changes.
The description of a layout will be used when writing a variable. Note that a modification in a layout at run time (through a modification of a parameter) will not affect previously-written iterations of a variable. It only affects the upcoming ones.<\/p>\n\n\n\n