TOFRaw

NXTOFRAW - A proposal for a NeXus Time-of-Flight Raw Data File Format

Introduction

NeXus is moving onto the idea of inherited incremental definitions as discussed at the last meeting of the NeXus International Advisory Committee (NIAC) – for example with powder diffractometers there is a definition for both time focussing and total scattering with one being a subset of the other; a file can conform to one or both. The initial work on this definition comes from discussions between SNS, J-PARC and ISIS - the three facilities are interested in having a unified base for all instruments to allow for low level instrument debugging tools to be used, without change, in a given facility. Further discussions occured in the TOF Breakout Group at NIAC 2006

An example file is available in HDF4 (2MB), HDF5 (2MB) and XML(16MB) format as well as a basic DTD.

For historical information see the draft proposal for an ISIS NeXus based RAW data file format.

Goal of the Definition

The definition/format should:

• Be general i.e. not specific to any particular instrument type and so can be used as a common root/parent format across all instruments at a facility
• allow the sharing of diagnostic and “first look” data/detector display programs between the facilities.
• provide a common input format to metadata capture programs (such as the ISIS ICAT search interface)

With the above in mind the instrument components of most importance are the ones related to the detector, data, user, sample and sample conditions; other instrument components are, of course, needed for analysis but will be covered by specific NeXus instrument definitions. The NeXus classes we will ultimately consider are then:

NXroot
NXdata
NXdetector
NXentry
NXgeometry
NXinstrument
NXlog
NXmoderator
NXmonitor
NXsample
NXuser
NXevent_data 
NXsource
NXdetector_group (proposed)

Some of these classes, such as NXgeometry, are taken directly from what was ratified by the NIAC.

Conventions Used in this Document

A tabular format is used for ease of viewing and printing rather than the NeXus XML meta-DTD format. The Name column in a table identifies an item in an instance of a NeXus class. Items can have extra “meta data” associated with them, which are called attributes – these, if any, are listed in the next few lines in the attributes column. Any variables in the attributes column are always attached to the previous variable in the Name column above them; if the Name of the variable is the same as the class (e.g. NXfile), then the attributes are associated with an instance of that class (global) and not to any of its specific members.

Identifying Mandatory and Optional Components

The following convention will be used:

• Variables in bold in the Name column of tables are mandatory – they must be present in ALL NeXus files; otherwise they are optional and their inclusion will depend on the instrument, experiment or presence of other items in the class (see the class description of usage)
• Variables in {italics} in the Name column are examples of names and any variable name can in fact be used; variable names in normal type mean that exact name must be used
• Anything in red is currently an extension to NeXus

This information is also included in a RE column (the name derives from the fact that a “Regular Expression” is used here in the XML DTD format). Thus:

The above convention dictates that the name for any item that occurs only 0 or 1 times is fixed; this is not required by the current NeXus standard, but would add clarity and ease of location if followed.

Naming

We will try and name logged variables (type NXlog) such that they end in the _log suffix

NeXus Classes

NXroot

The root is not a real class in the NeXus file, it is a convenient name under which to group the global attributes of the file. This is taken directly from the NeXus technical reference without change.

NXentry

This is the top level group in a file that contains a complete set of information (e.g. a “run”) - raw, reduced, and analyzed data can occur in the same file, each as a separate NXentry . The definition below is taken from the NeXus technical reference changing some elements to be required rather an optional. Additional items are highlighted in

.

This definition covers a single run experiment - extensions are proposed for scan type experiments

NXentry

NXuser

As denoted in NXentry, there can be multiple NXuser, one for each person involved with an experiment. This definition of user requires only a name and a facility identifier and this is taken directly from the NeXus technical reference changing some elements to be required rather an optional.

NXsample

This list is limited to items that were desired by the group. See the NeXus technical reference for a full list of possible items.

Sample environment parameters

By these we mean “temperature”, “magnetic_field” etc. which may be considered to be outside of the remit of this document, but we will just add a reminder that if the file represents a scan then these values will be annotated as described in the NXentry section.

NXinstrument

This is the class that contains all information about instrument components except the monitors and sample (which are just inside the NXentry). This is taken directly from the NeXus technical reference changing some elements to be required rather an optional.

NXmonitor

Note that for a position sensitive monitor detector_number etc. will need to be an array and NXmonitor will have other fields and look more like NXdetector.

NXdetector

We will now look at possible representations of the detector – we will start with a general one and then consider the special case of an area detector. Though the general (point) detector representation would cover all cases, if the detector is physically “rectangular” in nature there are advantages in using this symmetry in the representation. Which representation is used is recorded in the layout attribute

Point Detector

The general representation is to consider a detector as just a group of pixels arranged in no particular order. Each pixel will be identified by a unique single index i and then the following information will be stored:

The detector data would be plotted with axes (detector number, tof) by any program. An NXgeometry object included in the detector contains arrays that store the position and orientation of each pixel. As this detector representation imposes no constraint on the relationship between pixels, a single NXdetector could represent the entire instrument (so long as all detectors have the same time_of_flight) – however in practice an NXdetector and NXdata would be created for each bank. The “origin” object provides a reference point for the pixel geometries – the “shape” part of origin is the bounding box of the entire detector/detector bank.

Linear Detector

Here we mean a collection of linear straight strips e.g. tubes. We have two indicies: j will label the strip/tube and i the position along the tube. All tubes must have the same number of pixels; if not, you must use the point detector representation above. The tubes do not need to be parallel - they just need to be straight. Thus:

By specifying both size and offset “dead space” between pixels can be accounted for.

This looks similar to a point detector, but with two array indices rather than one. However note the geometry information is different - as the tubes are straight we need only specify a location of the tube centre and an offset along the tube. Thus:

• NXgeometry geometry[i] # defines tube/strip centre; each NXshape member give the tube size and shape; each NXorientation member rotates the axes such that x points along each tube.
• pixel_offset[j] # offset from tube centre of each pixel centre
• pixel_size[j] # size of each pixel

Area Detector

A flat rectangular area detector could be described by the “general” representation above, but taking account of the two dimensional symmetry of the detector allows several potential savings in the calculation of angles and in plotting time of the data. An area detector will have indices (i,j) indexing each pixel with i along the local detector “x” axis and j along the local detector “y”. In the case of curved detectors the offsets and sizes are to be considered as arc lengths along the face of the detector. An offset of “0” is the origin of the detector and the NXgeometry named “origin” describes the geometry of the entire detector: the NXtranslation part describes the position of the detector, the NXorientation part defines the local coordinates (local x and y axes) with respect to the global position, and the NXshape describe the size (bounding box) and topology of the detector as a whole. The NXgeometry named “geometry” describes the pixels and their shape (assuming that they are uniform). The necessary shapes are: rectangular prism, cylindrical slice, and spherical slice.

Below are the three cases for describing the pixels on a detector.

You can either specify an NXgeometry[i,j] for the pixels or instead use the x_pixel_* arrays. By specifying both size and offset “dead space” between pixels can be accounted for.

azimuthal_angle, polar_angle and distance can be left out of NXdetector as they can be calculated from the detector geometry

Hardware ganging of detector elements

In some cases individual detector elements are ganged together by the acquisition system for symmetry reasons or to create a smaller data files. In these cases the above formalisms can still be used, but the “detector number” does not correspond to a real physical detector and so the values of “polar_angle”, “distance”, “azimuthal_angle” are some sort of average over the ganged elements. When analysis and simulation of the data is performed, it is sometimes necessary to know the details of the individual detectors that have been ganged together. An initial proposal was that these additional arrays would be stored with the “_unganged” suffix e.g. “Polar_angle_unganged”, “distance_unganged”, “detector_number_unganged”. However after discussions of TOF Group if was decided to move these arrays into a substructure of NXdetector so we would have NXdetector.polar_angle and NXdetector.distance for the ganed values; NXdetector.unganged.polar_angle and NXdetector.unganged.distance for the raw values.

To relate the ganged and unganged arrays, a simple grouping scheme can usually be used: detector.unganged.grouping[j] give the value [i] detector.polar_angle[i] that this detector contributes to. This covers most cases, except for when a detector may have its signal fed into more than one place; in which case a more complex mapping scheme is needed.

To cover the general case the “unganged” arrays are arranged so that elements that are ganged together appear sequentially and information to relate these arrays to the hardware ganged “polar_angle” etc arrays are provided by

Detector_number[i] is ganged from gang_count[i] elements. The values of polar_angle[i] was obtained by average the gang_count[i] values of polar_angle_unganged[gang_index[i]], polar_angle_unganged[gang_index[i]+1], … , polar_angle[gang_index[i]+gang_count[i]-1]

NXdata

The exact format of this will depend on the NXdetector definition used.

NXmoderator

The moderator is the effective source for all time-of-flight instruments. This is taken directly from the NeXus technical reference changing some elements to be required rather an optional. Additional items are in red.

NXgeometry

This group describes the shape, position, and orientation of a component. Almost all of the information is actually stored in subgroups. This is taken directly from the NeXus technical reference without change.

The sample has a component_index of 0, components upstream have negative component_index.

NXlog

Contains log information monitored during the run in a timed fashion. This can contain the time-stamped values, or the average (with standard deviation), minimum, maximum and total time log was taken. This is taken directly from the NeXus technical reference without change.

NXorientation

NXshape

The interpretation of the “shapepar” depends on the “shape”

NXtranslation

NXevent_data

This requires that a Pixel_number field is provided in the NXdetector for determining geometry information. While normally this takes the place of the NXdata in a NXentry, there is no reason that the two cannot coexist. The index I runs over events - the index j runs counts pulses.

NXsource

NXdetector_groups

This class is used to allow a logical grouping of detector elements (e.g. which tube, bank or group of banks) to be recorded in the file. As well as allowing you to e.g just select the “left” or “east” detectors, it may also be useful for determining which elements belong to the same PSD tube and hence have e.g. the same dead time.

For example of we had “bank1” composed of “tube1”, “tube2” and “tube3” then Group_names would be the string “bank1, bank1/tube1, bank1/tube2,bank1/tube3” Group_index would be {1,2,3,4} Group_parent would be {-1,1,1,1}

The mapping array is interpreted as group 1 is a top level group containing groups 2, 3 and 4

A group_index array in NXdetector give the base group for a detector element.